title
Coding Challenge #92: XOR Problem
description
In this coding challenge, I use my Toy Neural Networks library to solve the XOR problem. Code: https://thecodingtrain.com/challenges/92-xor-problem
đšī¸ p5.js Web Editor Sketch: https://editor.p5js.org/codingtrain/sketches/_QGR5d0Bi
đĨ Previous video: https://youtu.be/JrRO3OnWs5s?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
đĨ Next video: https://youtu.be/uWzPe_S-RVE?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
đĨ All videos: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
References:
đģ Toy-Neural-Network-JS: https://github.com/CodingTrain/Toy-Neural-Network-JS
đģ Deeplearn.js: https://deeplearnjs.org
đģ ml5.js: https://ml5js.org/
Videos:
đ My Neural Networks series: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6aCibgK1PTWWu9by6XFdCfh
đ My Perceptron video: https://codingtrain.github.io/website-archive/more/archive/nature-of-code/10-Neural-Networks/10.2-Neural-Networks-Perceptron-Part-1
đĨ Neural Networks: https://www.youtube.com/playlist?list=PLZHQObOWTQDNU6R1_67000Dx_ZCJB-3pi
Live Stream Archive:
đ´ Live Stream Archive #118: https://youtu.be/e8H588SXu6U?t=8090s
đ´ Live Stream Archive #119: https://youtu.be/f9vaiHoq-Fk?t=3171s
Timestamps:
0:00 Introduction
1:15 Machine learning
3:00 Supervised learning
4:20 XOR problem
7:07 Hidden layer
11:49 Training data
13:14 nn.train()
15:14 nn.predict()
16:26 Visualize results
19:30 Initializing weights
20:15 Learning rate
22:33 Add more hidden nodes
23:40 What's next?
Editing by Mathieu Blanchette
Animations by Jason Heglund
Music from Epidemic Sound
đ Website: http://thecodingtrain.com/
đž Share Your Creation! https://thecodingtrain.com/guides/passenger-showcase-guide
đŠ Suggest Topics: https://github.com/CodingTrain/Suggestion-Box
đĄ GitHub: https://github.com/CodingTrain
đŦ Discord: https://thecodingtrain.com/discord
đ Membership: http://youtube.com/thecodingtrain/join
đ Store: https://standard.tv/codingtrain
đī¸ Twitter: https://twitter.com/thecodingtrain
đ¸ Instagram: https://www.instagram.com/the.coding.train/
đĨ Coding Challenges: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
đĨ Intro to Programming: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6Zy51Q-x9tMWIv9cueOFTFA
đ p5.js: https://p5js.org
đ p5.js Web Editor: https://editor.p5js.org/
đ Processing: https://processing.org
đ Code of Conduct: https://github.com/CodingTrain/Code-of-Conduct
This description was auto-generated. If you see a problem, please open an issue: https://github.com/CodingTrain/thecodingtrain.com/issues/new
#neuralnetwork #machinelearning #gradientdescent #xorproblem #javascript #p5js
detail
{'title': 'Coding Challenge #92: XOR Problem', 'heatmap': [{'end': 1066.931, 'start': 1049.403, 'weight': 0.717}, {'end': 1143.509, 'start': 1123.386, 'weight': 1}], 'summary': 'Explores solving the xor problem using a javascript neural network library, covering machine learning basics, supervised learning with labeled data, constructing network architecture, training, visualization, and optimization techniques.', 'chapters': [{'end': 333.1, 'segs': [{'end': 44.631, 'src': 'embed', 'start': 21.528, 'weight': 4, 'content': [{'end': 30.337, 'text': 'And, in fact, if you want to follow along with that process of building it, there is a giant playlist here, starting with 10.1,', 'start': 21.528, 'duration': 8.809}, {'end': 31.478, 'text': 'Introduction to Neural Networks.', 'start': 30.337, 'duration': 1.141}, {'end': 35.382, 'text': "And I start building that library actually around, oops, don't play the video.", 'start': 31.738, 'duration': 3.644}, {'end': 38.526, 'text': 'around 10.4.', 'start': 37.064, 'duration': 1.462}, {'end': 44.631, 'text': "So this video is a standalone video where I'm going to make use of that library in a coding challenge.", 'start': 38.526, 'duration': 6.105}], 'summary': 'A neural network library was developed, starting at video 10.1 and completed around 10.4.', 'duration': 23.103, 'max_score': 21.528, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU21528.jpg'}, {'end': 87.685, 'src': 'embed', 'start': 62.983, 'weight': 2, 'content': [{'end': 68.951, 'text': 'So this is an example, a very beginning, basic, rather trivial example of machine learning.', 'start': 62.983, 'duration': 5.968}, {'end': 76.46, 'text': "And what do I mean by that? I mean, I'm going to draw a circle here, and I'm going to write ML, short for machine learning.", 'start': 68.991, 'duration': 7.469}, {'end': 82.703, 'text': 'And you can think of this as a place where there exists some machine learning recipe, some algorithm.', 'start': 76.5, 'duration': 6.203}, {'end': 86.545, 'text': 'You might have heard of k-nearest neighbor or neural network.', 'start': 82.723, 'duration': 3.822}, {'end': 87.685, 'text': 'Fill in the blank.', 'start': 87.045, 'duration': 0.64}], 'summary': 'Basic example of machine learning, mentioning k-nearest neighbor and neural network.', 'duration': 24.702, 'max_score': 62.983, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU62983.jpg'}, {'end': 222.089, 'src': 'embed', 'start': 156.166, 'weight': 0, 'content': [{'end': 157.067, 'text': "It's written in JavaScript.", 'start': 156.166, 'duration': 0.901}, {'end': 158.448, 'text': "There's nothing about it that's optimized.", 'start': 157.087, 'duration': 1.361}, {'end': 160.569, 'text': 'But just to kind of understand the pieces of how it works.', 'start': 158.668, 'duration': 1.901}, {'end': 165.131, 'text': 'So what I want to do is have my recipe be in there, be a neural network.', 'start': 160.909, 'duration': 4.222}, {'end': 172.736, 'text': 'And I want to create some scenario where I have some training data.', 'start': 166.752, 'duration': 5.984}, {'end': 178.699, 'text': "So what I'm going to do, this is going to be a demonstration also of the machine learning process called supervised learning.", 'start': 172.836, 'duration': 5.863}, {'end': 182.755, 'text': 'supervised learning,', 'start': 180.532, 'duration': 2.223}, {'end': 192.408, 'text': 'meaning I am the supervisor and I am going to teach this machine learning recipe to produce that appropriate output for a given input.', 'start': 182.755, 'duration': 9.653}, {'end': 200.498, 'text': "Now, if I just know all the answers, why would I even do this? Well, likely there's a scenario.", 'start': 194.355, 'duration': 6.143}, {'end': 205.061, 'text': 'Machine learning is going to be used with a scenario where I have a lot of labeled data.', 'start': 201.259, 'duration': 3.802}, {'end': 213.645, 'text': 'I have a known data set that I can use to train this system so that if I give it some unknown data, it will give me an output, a relevant output.', 'start': 205.081, 'duration': 8.564}, {'end': 217.947, 'text': 'A classic example of this, of course, is I have a whole bunch of images.', 'start': 213.785, 'duration': 4.162}, {'end': 219.128, 'text': "Here's a bunch of cats.", 'start': 218.327, 'duration': 0.801}, {'end': 220.529, 'text': "Here's a bunch of dogs.", 'start': 219.488, 'duration': 1.041}, {'end': 222.089, 'text': "Now here's a new image.", 'start': 220.849, 'duration': 1.24}], 'summary': 'Using javascript to build a neural network for supervised learning with labeled data.', 'duration': 65.923, 'max_score': 156.166, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU156166.jpg'}, {'end': 276.608, 'src': 'embed', 'start': 247.227, 'weight': 1, 'content': [{'end': 251.489, 'text': 'And then, of course, we have the actual unknown new data that we want to use.', 'start': 247.227, 'duration': 4.262}, {'end': 262.502, 'text': "OK, so what's the scenario that I'm going to use? The trivial, almost rather ridiculous example that I'm going to use in this video is the XOR.", 'start': 253.098, 'duration': 9.404}, {'end': 265.403, 'text': 'XOR stands for exclusive or.', 'start': 262.942, 'duration': 2.461}, {'end': 270.085, 'text': "It's a Boolean operation that resolves to true only if.", 'start': 265.804, 'duration': 4.281}, {'end': 276.608, 'text': 'So you can think of true and if true XOR true is actually false.', 'start': 271.106, 'duration': 5.502}], 'summary': 'Using xor as a trivial example to demonstrate boolean operations with new data.', 'duration': 29.381, 'max_score': 247.227, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU247227.jpg'}], 'start': 1.718, 'title': 'Solving xor problem with neural networks', 'summary': 'Covers using a javascript neural network library to solve the xor problem, emphasizing basics of machine learning, data input, processing, and output. it also explains supervised learning, use of labeled data for training and testing, and demonstrates solving the xor problem using a neural network.', 'chapters': [{'end': 172.736, 'start': 1.718, 'title': 'Neural network xor challenge', 'summary': 'Discusses using a javascript neural network library to solve the xor problem, emphasizing the basics of machine learning and the process of data input, processing, and output.', 'duration': 171.018, 'highlights': ['The video is focused on testing a JavaScript neural network library, which is available in a playlist starting from 10.1, Introduction to Neural Networks.', 'Explains the basics of machine learning, emphasizing the input of data, the application of a machine learning recipe, and the generation of an output, illustrating the versatility of machine learning applications.', 'Discusses the development of a toy neural network library in JavaScript, highlighting its educational purpose and the intention to use it for a neural network recipe and training data.']}, {'end': 333.1, 'start': 172.836, 'title': 'Supervised learning and xor problem', 'summary': 'Explains the concept of supervised learning, the use of labeled data for training, testing data for evaluating outcomes, and a demonstration of solving the xor problem using a neural network.', 'duration': 160.264, 'highlights': ['The chapter demonstrates the machine learning process called supervised learning, which involves teaching a system to produce the appropriate output for a given input, typically using labeled training data and testing data for evaluation.', 'The XOR problem, a Boolean operation that is not linearly separable, is used as a demonstration of the need for a neural network with a hidden layer in machine learning.', 'Machine learning involves training data, testing data, and unknown new data, with the XOR problem serving as an example of a scenario where supervised learning and neural networks are applied.']}], 'duration': 331.382, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU1718.jpg', 'highlights': ['The chapter demonstrates the machine learning process called supervised learning, which involves teaching a system to produce the appropriate output for a given input, typically using labeled training data and testing data for evaluation.', 'The XOR problem, a Boolean operation that is not linearly separable, is used as a demonstration of the need for a neural network with a hidden layer in machine learning.', 'Explains the basics of machine learning, emphasizing the input of data, the application of a machine learning recipe, and the generation of an output, illustrating the versatility of machine learning applications.', 'Machine learning involves training data, testing data, and unknown new data, with the XOR problem serving as an example of a scenario where supervised learning and neural networks are applied.', 'The video is focused on testing a JavaScript neural network library, which is available in a playlist starting from 10.1, Introduction to Neural Networks.', 'Discusses the development of a toy neural network library in JavaScript, highlighting its educational purpose and the intention to use it for a neural network recipe and training data.']}, {'end': 545.154, 'segs': [{'end': 366.804, 'src': 'embed', 'start': 333.4, 'weight': 1, 'content': [{'end': 344.485, 'text': 'My training set is going to be 0001 one, zero, one, one.', 'start': 333.4, 'duration': 11.085}, {'end': 349.769, 'text': 'I only have four elements in my training set, and each one of these is labeled.', 'start': 344.505, 'duration': 5.264}, {'end': 359.038, 'text': 'This is labeled with a zero, this is labeled with a one, this is labeled with a one, and this is labeled with a zero.', 'start': 350.09, 'duration': 8.948}, {'end': 361.28, 'text': 'this is the known data.', 'start': 359.398, 'duration': 1.882}, {'end': 365.082, 'text': 'right again, this is a sort of trivial you know I could build.', 'start': 361.28, 'duration': 3.802}, {'end': 366.804, 'text': 'you could build a circuit right.', 'start': 365.082, 'duration': 1.722}], 'summary': 'Training set: 0001, 4 labeled elements. known data for building circuits.', 'duration': 33.404, 'max_score': 333.4, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU333400.jpg'}, {'end': 414.889, 'src': 'embed', 'start': 384.154, 'weight': 0, 'content': [{'end': 398.678, 'text': "So the way that it's going to look, and let me erase this here, is we can see here in my labeled data set that there are two inputs and one output.", 'start': 384.154, 'duration': 14.524}, {'end': 403.974, 'text': 'So the neural network structure has to have two inputs and one output.', 'start': 399.588, 'duration': 4.386}, {'end': 407.399, 'text': "I'm going to send in x1 and x2.", 'start': 404.275, 'duration': 3.124}, {'end': 413.427, 'text': 'These are true and true, true and false, false or true, and somehow out of this I should get a y.', 'start': 407.699, 'duration': 5.728}, {'end': 414.889, 'text': 'I should get a true or false.', 'start': 413.427, 'duration': 1.462}], 'summary': 'Neural network has 2 inputs, 1 output; inputs include true and false values to produce true or false output.', 'duration': 30.735, 'max_score': 384.154, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU384154.jpg'}, {'end': 483.452, 'src': 'embed', 'start': 457.914, 'weight': 2, 'content': [{'end': 466.1, 'text': "So internally in the training process, what it does is if I give it 0, 0 with a 0 and it outputs a 1, it's like I tell it, hey, you got that wrong.", 'start': 457.914, 'duration': 8.186}, {'end': 470.123, 'text': 'Adjust all those weights to try to see if you can get it more correct.', 'start': 466.4, 'duration': 3.723}, {'end': 474.546, 'text': "So what we're going to do is over and over again to train it to see if we can then later.", 'start': 470.403, 'duration': 4.143}, {'end': 477.788, 'text': "we don't have a separate training and testing set in this scenario.", 'start': 474.546, 'duration': 3.242}, {'end': 483.452, 'text': "We're just going to say, I'm going to train it and see if you can start producing the correct answers for all of these numbers.", 'start': 477.968, 'duration': 5.484}], 'summary': 'Training process adjusts weights to improve correctness of output. no separate training and testing set used.', 'duration': 25.538, 'max_score': 457.914, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU457914.jpg'}, {'end': 545.154, 'src': 'embed', 'start': 496.034, 'weight': 3, 'content': [{'end': 503.68, 'text': 'So this is actually an example that I made in the processing Java-based programming environment probably like almost eight years ago at this point.', 'start': 496.034, 'duration': 7.646}, {'end': 508.103, 'text': "So it's not using the same code base, but it's actually solved the XOR problem.", 'start': 504.12, 'duration': 3.983}, {'end': 513.546, 'text': "And what's interesting about this, and I'm going to come back to the whiteboard just for one more moment here.", 'start': 508.463, 'duration': 5.083}, {'end': 515.006, 'text': 'Let me find some room here.', 'start': 513.566, 'duration': 1.44}, {'end': 526.522, 'text': 'If I draw a plane and I consider this to be 0, 0 and this to be 1, 1, So this is 1 comma 0, and this is 0 comma 1, right?', 'start': 515.587, 'duration': 10.935}, {'end': 537.611, 'text': "Notice that if I take this idea of XOR and map it to here, I've got false here, false here, true here and true here.", 'start': 527.987, 'duration': 9.624}, {'end': 541.453, 'text': 'And by the way, why did I write a 1 there? True here and true here.', 'start': 537.731, 'duration': 3.722}, {'end': 543.834, 'text': "This, by the way, is why it's not linearly separable.", 'start': 541.753, 'duration': 2.081}, {'end': 545.154, 'text': "You can't draw a line.", 'start': 544.054, 'duration': 1.1}], 'summary': 'Example solved the xor problem in a non-linearly separable manner.', 'duration': 49.12, 'max_score': 496.034, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU496034.jpg'}], 'start': 333.4, 'title': 'Neural network structure, architecture, and xor problem', 'summary': 'Covers the construction of a neural network with a training set of four elements, the key components of a neural network, and the xor problem, including an example code and graphical representation of non-linearity.', 'chapters': [{'end': 414.889, 'start': 333.4, 'title': 'Neural network structure and training set', 'summary': 'Discusses the construction of a neural network with a training set of four elements, labeled with zero and one, in order to achieve a specific circuit behavior with two input and one output, aiming to obtain a true or false value.', 'duration': 81.489, 'highlights': ['The training set consists of four elements labeled with zero and one. The training set contains four elements, each labeled with either zero or one.', 'The neural network structure requires two inputs and one output to achieve a specific circuit behavior. In order to achieve a specific circuit behavior, the neural network structure must have two inputs and one output.', 'The goal is to obtain a true or false value from the input combinations. The objective is to obtain a true or false value from the input combinations of true and true, true and false, and false and true.']}, {'end': 477.788, 'start': 415.771, 'title': 'Neural network architecture', 'summary': 'Explains the key components of a neural network, including the role of hidden layers and the training process, emphasizing the importance of a fully connected network and continual weight adjustments during training.', 'duration': 62.017, 'highlights': ['The crucial component in architecting a neural network system is the presence of a hidden layer, which is part of a fully connected network, allowing for the processing of inputs and generating outputs.', "During the training process, continual adjustments to the weights occur based on the network's performance, aiming to improve accuracy and correct errors, without the need for separate training and testing sets."]}, {'end': 545.154, 'start': 477.968, 'title': 'Xor problem and non-linearity', 'summary': 'Discusses the xor problem, presenting an example of code solving it and explaining the non-linearity of the problem with graphical representation.', 'duration': 67.186, 'highlights': ['The XOR problem is discussed, and an example of code solving it is presented.', 'The non-linearity of the XOR problem is explained with a graphical representation.']}], 'duration': 211.754, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU333400.jpg', 'highlights': ['The neural network structure requires two inputs and one output to achieve a specific circuit behavior.', 'The training set consists of four elements labeled with zero and one.', "During the training process, continual adjustments to the weights occur based on the network's performance.", 'The XOR problem is discussed, and an example of code solving it is presented.', 'The non-linearity of the XOR problem is explained with a graphical representation.']}, {'end': 928.28, 'segs': [{'end': 614.562, 'src': 'embed', 'start': 585.976, 'weight': 0, 'content': [{'end': 591.042, 'text': "So what you're going to need first of all is, if you're writing this code along with me,", 'start': 585.976, 'duration': 5.066}, {'end': 594.806, 'text': "you're going to need to have the toy neural network JS library.", 'start': 591.042, 'duration': 3.764}, {'end': 597.509, 'text': 'It has really just two files that you need.', 'start': 595.427, 'duration': 2.082}, {'end': 601.814, 'text': 'It has neuralnetwork.js and nn.js and matrix.js.', 'start': 597.93, 'duration': 3.884}, {'end': 602.695, 'text': 'You can ignore this.', 'start': 601.834, 'duration': 0.861}, {'end': 604.077, 'text': 'This is for testing.', 'start': 603.136, 'duration': 0.941}, {'end': 606.64, 'text': "That's something I cover in some other videos that you could probably find.", 'start': 604.097, 'duration': 2.543}, {'end': 614.562, 'text': "And you'll see that in my index.html file, I am referencing those two JavaScript files.", 'start': 608.06, 'duration': 6.502}], 'summary': 'To code along, obtain the toy neural network js library with neuralnetwork.js and nn.js and matrix.js files.', 'duration': 28.586, 'max_score': 585.976, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU585976.jpg'}, {'end': 732.677, 'src': 'embed', 'start': 704.771, 'weight': 1, 'content': [{'end': 712.682, 'text': "So I'm going to say let training equal, and I want to have it be in an array, and each element of the array, I want it to be an object.", 'start': 704.771, 'duration': 7.911}, {'end': 718.008, 'text': "And it's going to have the inputs, which would be like this, and the outputs.", 'start': 713.023, 'duration': 4.985}, {'end': 723.271, 'text': 'Now, even though I only have one output, right, one output, outputs can be an array.', 'start': 718.048, 'duration': 5.223}, {'end': 726.453, 'text': "They're often referred to as a vector, so a list of numbers.", 'start': 723.331, 'duration': 3.122}, {'end': 732.677, 'text': "But in this case, 00 gives me just a single, I have single outputs, but I don't put the array, the library expects an array.", 'start': 726.653, 'duration': 6.024}], 'summary': "Training data will be in an array with object elements, including inputs and outputs. although there's only one output, it will be in an array format.", 'duration': 27.906, 'max_score': 704.771, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU704771.jpg'}, {'end': 802.438, 'src': 'embed', 'start': 778.243, 'weight': 2, 'content': [{'end': 787.809, 'text': "you'll see something called you'll have a batch training process where you give it a large batch of data and then you adjust the weights and do the training in another batch.", 'start': 778.243, 'duration': 9.566}, {'end': 790.05, 'text': "I'm going to do everything just one at a time.", 'start': 788.129, 'duration': 1.921}, {'end': 796.054, 'text': "And the way I'm going to do that is with a function called train that's in the neural network library.", 'start': 790.331, 'duration': 5.723}, {'end': 802.438, 'text': 'So if I say neural network train, This expects two arguments.', 'start': 796.255, 'duration': 6.183}], 'summary': 'Neural network uses train function for one-at-a-time training.', 'duration': 24.195, 'max_score': 778.243, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU778243.jpg'}], 'start': 545.774, 'title': 'Neural network programming', 'summary': 'Introduces the concept of creating a neural network using the toy neural network js library, featuring the process of setting up the network, defining training data, and the training process.', 'chapters': [{'end': 928.28, 'start': 545.774, 'title': 'Neural network programming', 'summary': 'Introduces the concept of creating a neural network using the toy neural network js library, featuring the process of setting up the network, defining training data, and the training process.', 'duration': 382.506, 'highlights': ['The chapter introduces the concept of creating a neural network using the toy neural network JS library. It explains the process of setting up the network, defining the number of inputs, hidden neurons, and the output.', 'The process of defining training data is explained, including the format of the input and output data. It demonstrates the creation of training data in an array format, with examples of input and output pairs.', "The training process is detailed, showcasing the use of the 'train' function and the method of adjusting the weights of the neural network. It explains the process of training the network using the 'train' function and highlights the adjustment of weights in the library."]}], 'duration': 382.506, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU545774.jpg', 'highlights': ['The chapter introduces creating a neural network using the toy neural network JS library.', 'The process of defining training data is explained, including the format of the input and output data.', "The training process is detailed, showcasing the use of the 'train' function and the method of adjusting the weights."]}, {'end': 1205.013, 'segs': [{'end': 994.98, 'src': 'embed', 'start': 949.721, 'weight': 0, 'content': [{'end': 956.266, 'text': "I'm going to have it do, every time through draw, 1,000 of these training points.", 'start': 949.721, 'duration': 6.545}, {'end': 959.808, 'text': 'So this will hopefully get us there a little faster.', 'start': 957.006, 'duration': 2.802}, {'end': 961.81, 'text': 'And I can look at this now.', 'start': 959.828, 'duration': 1.982}, {'end': 963.371, 'text': 'And we can see, ah, ooh.', 'start': 961.83, 'duration': 1.541}, {'end': 966.614, 'text': "Now that's wrong, right? Oh no, that's correct.", 'start': 964.571, 'duration': 2.043}, {'end': 969.457, 'text': 'Oh no, I should be getting false, yeah.', 'start': 966.634, 'duration': 2.823}, {'end': 971.68, 'text': "So that's right, I got it backwards for a second.", 'start': 969.718, 'duration': 1.962}, {'end': 973.543, 'text': "I shouldn't be getting true, I should be getting false.", 'start': 971.72, 'duration': 1.823}, {'end': 977.608, 'text': "Zero, zero, so let's try one, zero, and I've got something that's close to one.", 'start': 973.783, 'duration': 3.825}, {'end': 979.41, 'text': 'So you can see that this worked, okay.', 'start': 977.768, 'duration': 1.642}, {'end': 980.912, 'text': 'So the library is working.', 'start': 979.651, 'duration': 1.261}, {'end': 984.995, 'text': "But let's say I want to visualize it.", 'start': 981.553, 'duration': 3.442}, {'end': 986.455, 'text': 'I want to visualize it.', 'start': 985.295, 'duration': 1.16}, {'end': 987.616, 'text': 'I want to see it working.', 'start': 986.475, 'duration': 1.141}, {'end': 989.477, 'text': 'I want to see it animating as it works.', 'start': 987.636, 'duration': 1.841}, {'end': 994.98, 'text': "So one way I could do that is I could basically, let's create some variables like resolution.", 'start': 989.497, 'duration': 5.483}], 'summary': 'Training model with 1,000 points, testing and visualizing its performance.', 'duration': 45.259, 'max_score': 949.721, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU949721.jpg'}, {'end': 1086.787, 'src': 'heatmap', 'start': 1049.403, 'weight': 3, 'content': [{'end': 1055.145, 'text': 'So what this should do is this is a nice little nested loop to just draw a grid of rectangles, 10 by 10.', 'start': 1049.403, 'duration': 5.742}, {'end': 1057.967, 'text': 'And a little warning, this will be kind of flashy.', 'start': 1055.145, 'duration': 2.822}, {'end': 1062.649, 'text': 'So you can see all of these elements are flashing out with a random color.', 'start': 1058.767, 'duration': 3.882}, {'end': 1064.049, 'text': "But I don't want a random color.", 'start': 1062.669, 'duration': 1.38}, {'end': 1066.931, 'text': 'What I want to do is I want to create some inputs.', 'start': 1064.49, 'duration': 2.441}, {'end': 1075.417, 'text': "So I'm going to say let x1 equal i divided by columns from 0 to 1.", 'start': 1067.671, 'duration': 7.746}, {'end': 1079.26, 'text': 'Let x2 equals j divided by rows.', 'start': 1075.417, 'duration': 3.843}, {'end': 1084.305, 'text': 'And the inputs then is an array x1, x2.', 'start': 1079.921, 'duration': 4.384}, {'end': 1086.787, 'text': 'So what I want to do is I want to create a scenario.', 'start': 1084.685, 'duration': 2.102}], 'summary': 'Code creates a nested loop to draw a 10x10 grid of flashing rectangles with random colors, but the intention is to use specific inputs for x1 and x2 in an array.', 'duration': 58.602, 'max_score': 1049.403, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU1049403.jpg'}, {'end': 1152.277, 'src': 'heatmap', 'start': 1123.386, 'weight': 1, 'content': [{'end': 1127.39, 'text': "I should get a y value, That's between 0 and 1, and I should get that a brightness value.", 'start': 1123.386, 'duration': 4.004}, {'end': 1129.412, 'text': 'I can both buy my 255 to get a brightness value.', 'start': 1127.43, 'duration': 1.982}, {'end': 1140.283, 'text': "Okay, let's run this Boom, so we can see now, and let's, let's, let's, let's be a little more thoughtful about this and say no stroke,", 'start': 1129.412, 'duration': 10.871}, {'end': 1143.509, 'text': "and And I'm going to refresh it.", 'start': 1140.283, 'duration': 3.226}, {'end': 1146.992, 'text': "And very quickly, ah, so I'm so glad this happened.", 'start': 1144.249, 'duration': 2.743}, {'end': 1147.632, 'text': "So here's the thing.", 'start': 1147.112, 'duration': 0.52}, {'end': 1149.294, 'text': 'It worked for me a bunch of times.', 'start': 1147.713, 'duration': 1.581}, {'end': 1152.277, 'text': "But actually here, you can see it's totally getting stuck.", 'start': 1150.075, 'duration': 2.202}], 'summary': 'Obtained y value between 0 and 1 for brightness, but encountered issues.', 'duration': 28.891, 'max_score': 1123.386, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU1123386.jpg'}, {'end': 1191.785, 'src': 'embed', 'start': 1165.871, 'weight': 4, 'content': [{'end': 1172.695, 'text': 'Remember how I said, a neural network is an interconnected set of nodes, with each connection being a weight?', 'start': 1165.871, 'duration': 6.824}, {'end': 1175.116, 'text': "So there's a couple important factors here.", 'start': 1173.415, 'duration': 1.701}, {'end': 1177.417, 'text': 'One is those weights are initialized randomly.', 'start': 1175.436, 'duration': 1.981}, {'end': 1181.76, 'text': 'And there are thoughtful ways that you can initialize those weights.', 'start': 1177.778, 'duration': 3.982}, {'end': 1186.342, 'text': "But if you initialize them in a certain way, the problem is there's multiple solutions here.", 'start': 1182.06, 'duration': 4.282}, {'end': 1191.785, 'text': "There's one solution here where there's white all the way along this way.", 'start': 1188.043, 'duration': 3.742}], 'summary': 'Neural network weights are initialized randomly, allowing for multiple solutions.', 'duration': 25.914, 'max_score': 1165.871, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU1165871.jpg'}], 'start': 928.381, 'title': 'Model training and neural network visualization', 'summary': 'Involves training a model with 1,000 points to enhance accuracy and visualize library operations, as well as creating a 10x10 grid for neural network inputs while addressing weight initialization issues and providing multiple solutions for visualization.', 'chapters': [{'end': 994.98, 'start': 928.381, 'title': 'Training model for visualization', 'summary': 'Involves the process of training a model with 1,000 points in order to improve accuracy and visualizing the working of a library.', 'duration': 66.599, 'highlights': ['Training the model with 1,000 points for faster convergence and improved accuracy.', 'Visualizing the working of the library to observe its animation and functioning.', "Using variables like 'resolution' to support the visualization process."]}, {'end': 1205.013, 'start': 995, 'title': 'Neural network grid visualization', 'summary': 'Discusses creating a grid of 10x10 rectangles, using nested loops to draw them with random colors, creating inputs for a neural network, and encountering issues with weight initialization resulting in multiple solutions for the grid visualization.', 'duration': 210.013, 'highlights': ['Creating a grid of 10x10 rectangles The speaker discusses creating a grid of rectangles by using nested loops and drawing them on the canvas.', 'Encountering issues with weight initialization The speaker explains the issues with weight initialization in a neural network, resulting in multiple solutions for the grid visualization.', 'Drawing rectangles with random colors The speaker demonstrates the use of a nested loop to draw rectangles with random colors on the canvas.', 'Creating inputs for a neural network The speaker explains the process of creating inputs for a neural network by defining x1, x2, and using them to predict and fill brightness values on the grid.']}], 'duration': 276.632, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU928381.jpg', 'highlights': ['Training the model with 1,000 points for faster convergence and improved accuracy.', 'Visualizing the working of the library to observe its animation and functioning.', "Using variables like 'resolution' to support the visualization process.", 'Creating a grid of 10x10 rectangles using nested loops and drawing them on the canvas.', 'Encountering issues with weight initialization in a neural network, resulting in multiple solutions for the grid visualization.', 'Drawing rectangles with random colors using a nested loop on the canvas.', 'Explaining the process of creating inputs for a neural network by defining x1, x2, and using them to predict and fill brightness values on the grid.']}, {'end': 1490.291, 'segs': [{'end': 1250.167, 'src': 'embed', 'start': 1205.313, 'weight': 0, 'content': [{'end': 1208.475, 'text': "So there's a bunch of different solutions here, and it sometimes can get stuck in that middle.", 'start': 1205.313, 'duration': 3.162}, {'end': 1212.217, 'text': 'So one thing I might be able to do is play with something called the learning rate.', 'start': 1208.875, 'duration': 3.342}, {'end': 1219.102, 'text': 'So there is a variable in the library called learning rate.', 'start': 1212.717, 'duration': 6.385}, {'end': 1223.226, 'text': 'And I could say neural network learning rate equals.', 'start': 1219.463, 'duration': 3.763}, {'end': 1225.908, 'text': 'So I forgot.', 'start': 1224.907, 'duration': 1.001}, {'end': 1229.39, 'text': 'The library actually has a function called set learning rate.', 'start': 1225.948, 'duration': 3.442}, {'end': 1233.5, 'text': 'And so what I can do is I can set this to some value.', 'start': 1230.379, 'duration': 3.121}, {'end': 1235.541, 'text': 'Now, what should that learning rate be?', 'start': 1233.54, 'duration': 2.001}, {'end': 1237.982, 'text': 'The learning rate is like how big are these steps?', 'start': 1235.601, 'duration': 2.381}, {'end': 1239.423, 'text': 'How big are these adjustments?', 'start': 1238.062, 'duration': 1.361}, {'end': 1246.206, 'text': "And so maybe what I'll do here is I will create a LR slider for learning rate slider.", 'start': 1239.843, 'duration': 6.363}, {'end': 1250.167, 'text': "And I'm going to say LR slider equals create slider.", 'start': 1246.226, 'duration': 3.941}], 'summary': 'Adjusting learning rate in neural network for better optimization and convergence.', 'duration': 44.854, 'max_score': 1205.313, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU1205313.jpg'}, {'end': 1305.282, 'src': 'embed', 'start': 1277.147, 'weight': 1, 'content': [{'end': 1284.198, 'text': "So this should, if I'm right, always set the learning rate according to this.", 'start': 1277.147, 'duration': 7.051}, {'end': 1284.919, 'text': "So let's see.", 'start': 1284.238, 'duration': 0.681}, {'end': 1288.499, 'text': "If I put it up here and I say neural network learning rate, it's 0.5.", 'start': 1285.119, 'duration': 3.38}, {'end': 1292.6, 'text': 'And if I go down to here, it is 0.', 'start': 1288.499, 'duration': 4.101}, {'end': 1299.661, 'text': "Now, I shouldn't let it be 0, right? So the lowest the learning rate should be is probably 0.01.", 'start': 1292.6, 'duration': 7.061}, {'end': 1301.241, 'text': "So let's see if we can get it stuck.", 'start': 1299.661, 'duration': 1.58}, {'end': 1304.542, 'text': "Now, of course, I'm going to hit Refresh.", 'start': 1303.062, 'duration': 1.48}, {'end': 1305.282, 'text': "OK, it's stuck.", 'start': 1304.642, 'duration': 0.64}], 'summary': 'Testing neural network learning rate at 0.5 and 0.01, resulting in getting stuck.', 'duration': 28.135, 'max_score': 1277.147, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU1277147.jpg'}, {'end': 1474.185, 'src': 'embed', 'start': 1446.008, 'weight': 3, 'content': [{'end': 1454.214, 'text': 'But could you one come up with your own data set, try to train the neural network with your own data set, and then how does it perform?', 'start': 1446.008, 'duration': 8.206}, {'end': 1459.237, 'text': 'You also might think about visualizing this output using color in some way.', 'start': 1454.574, 'duration': 4.663}, {'end': 1465.24, 'text': 'You could use 3D, as I showed you with this processing example, which is no longer open.', 'start': 1460.057, 'duration': 5.183}, {'end': 1474.185, 'text': 'So you could try a variety of different ways of visualizing this, or animating it, or changing the way you build an interface to it.', 'start': 1465.54, 'duration': 8.645}], 'summary': 'Train neural network with custom data, visualize output in various ways.', 'duration': 28.177, 'max_score': 1446.008, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU1446008.jpg'}], 'start': 1205.313, 'title': 'Neural network optimization', 'summary': "Covers optimizing neural networks through learning rate adjustments, emphasizing lr slider's impact on step size, and visualization techniques, including testing libraries and adding hidden nodes.", 'chapters': [{'end': 1250.167, 'start': 1205.313, 'title': 'Neural network learning rate adjustment', 'summary': 'Discusses adjusting the learning rate in a neural network to optimize performance, with a focus on the lr slider and its impact on step size and adjustments.', 'duration': 44.854, 'highlights': ['Creating a LR slider to adjust the learning rate in the neural network, which impacts the size of steps and adjustments.', 'Exploring the use of the learning rate variable in the library to optimize neural network performance.']}, {'end': 1490.291, 'start': 1250.507, 'title': 'Neural network learning and visualization', 'summary': "Discusses the process of creating a neural network, adjusting learning rates, and visualizing the network's performance, with an emphasis on testing a neural network library, adding hidden nodes, and encouraging viewers to explore different visualization and training options.", 'duration': 239.784, 'highlights': ['The chapter covers adjusting learning rates, with a range from 0 to 0.5 and an incremental step of 0.01, to set the learning rate for a neural network.', "The process of testing a neural network library, adding hidden nodes, and visualizing the network's performance is demonstrated, showcasing the ability to recognize handwritten digits and encouraging viewers to explore their own datasets and visualization methods.", 'Encouragement is provided to explore the neural network library code and other related videos, promoting further learning and understanding of the topic.']}], 'duration': 284.978, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/188B6k_F9jU/pics/188B6k_F9jU1205313.jpg', 'highlights': ['Creating a LR slider to adjust the learning rate in the neural network, impacting step size and adjustments.', 'The chapter covers adjusting learning rates from 0 to 0.5 with an incremental step of 0.01 for a neural network.', 'Exploring the use of the learning rate variable in the library to optimize neural network performance.', "Demonstrating the process of testing a neural network library, adding hidden nodes, and visualizing the network's performance."]}], 'highlights': ['The XOR problem, a Boolean operation that is not linearly separable, is used as a demonstration of the need for a neural network with a hidden layer in machine learning.', 'The chapter demonstrates the machine learning process called supervised learning, which involves teaching a system to produce the appropriate output for a given input, typically using labeled training data and testing data for evaluation.', 'Explains the basics of machine learning, emphasizing the input of data, the application of a machine learning recipe, and the generation of an output, illustrating the versatility of machine learning applications.', 'The video is focused on testing a JavaScript neural network library, which is available in a playlist starting from 10.1, Introduction to Neural Networks.', 'The neural network structure requires two inputs and one output to achieve a specific circuit behavior.', 'The training set consists of four elements labeled with zero and one.', "During the training process, continual adjustments to the weights occur based on the network's performance.", 'The XOR problem is discussed, and an example of code solving it is presented.', 'The chapter introduces creating a neural network using the toy neural network JS library.', 'The process of defining training data is explained, including the format of the input and output data.', "The training process is detailed, showcasing the use of the 'train' function and the method of adjusting the weights.", 'Training the model with 1,000 points for faster convergence and improved accuracy.', 'Visualizing the working of the library to observe its animation and functioning.', 'Creating a LR slider to adjust the learning rate in the neural network, impacting step size and adjustments.', 'The chapter covers adjusting learning rates from 0 to 0.5 with an incremental step of 0.01 for a neural network.', 'Exploring the use of the learning rate variable in the library to optimize neural network performance.', "Demonstrating the process of testing a neural network library, adding hidden nodes, and visualizing the network's performance."]}