title
10.5: Neural Networks: Multilayer Perceptron Part 2 - The Nature of Code

description
This video follows up on the previous Multilayer Perceptron video (https://youtu.be/u5GAVdLQyIg). Here I begin the long process of coding a simple neural network library in JavaScript. Next video: https://youtu.be/uSzGdfdOoG8 This video is part of Chapter 10 of The Nature of Code (http://natureofcode.com/book/chapter-10-neural-networks/) This video is also part of session 4 of my Spring 2017 ITP "Intelligence and Learning" course (https://github.com/shiffman/NOC-S17-2-Intelligence-Learning/tree/master/week4-neural-networks) Support this channel on Patreon: https://patreon.com/codingtrain To buy Coding Train merchandise: https://www.designbyhumans.com/shop/codingtrain/ To donate to the Processing Foundation: https://processingfoundation.org/ Send me your questions and coding challenges!: https://github.com/CodingTrain/Rainbow-Topics Contact: Twitter: https://twitter.com/shiffman The Coding Train website: http://thecodingtrain.com/ Links discussed in this video: The Nature of Code: http://natureofcode.com/ Session 4 of Intelligence and Learning: https://github.com/shiffman/NOC-S17-2-Intelligence-Learning/tree/master/week4-neural-networks Perceptron on Wikipedia: https://en.wikipedia.org/wiki/Perceptron My Simple Artificial Neural Network JavaScript Library: https://github.com/shiffman/Neural-Network-p5 My video on Perceptrons: https://youtu.be/ntKn5TPHHAk Books discussed in this video: Tariq Rashid's Make Your Own Neural Network: http://amzn.to/2tcVeFS Marvin Minsky's Perceptrons: http://amzn.to/2u8Jv8f Source Code for the all Video Lessons: https://github.com/CodingTrain/Rainbow-Code p5.js: https://p5js.org/ Processing: https://processing.org The Nature of Code playlist: https://www.youtube.com/user/shiffman/playlists?view_as=subscriber&shelf_id=6&view=50&sort=dd For More Coding Challenges: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH For More Intelligence and Learning: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6YJ3XfHhT2Mm4Y5I99nrIKX Help us caption & translate this video! http://amara.org/v/8OIg/ 📄 Code of Conduct: https://github.com/CodingTrain/Code-of-Conduct

detail
{'title': '10.5: Neural Networks: Multilayer Perceptron Part 2 - The Nature of Code', 'heatmap': [{'end': 456.871, 'start': 435.626, 'weight': 0.787}, {'end': 966.735, 'start': 931.714, 'weight': 1}], 'summary': 'Explores the creation of a versatile javascript neural network library, providing an overview of a multi-layered perceptron with 3 input neurons, 4 hidden neurons, and 2 output neurons, and delves into network architecture, weighted sum, initialization, and the use of matrices for neural network operations.', 'chapters': [{'end': 136.945, 'segs': [{'end': 119.272, 'src': 'embed', 'start': 25.578, 'weight': 0, 'content': [{'end': 26.538, 'text': "I've actually already done this.", 'start': 25.578, 'duration': 0.96}, {'end': 32.72, 'text': 'It exists here at this repository, github-schiffman-neuron-network-p5.', 'start': 27.179, 'duration': 5.541}, {'end': 40.183, 'text': "I'm designing this library to be used with a set of p5.js examples with a JavaScript library called p5.", 'start': 33.161, 'duration': 7.022}, {'end': 44.425, 'text': 'Although ultimately, this library stands alone by itself.', 'start': 40.203, 'duration': 4.222}, {'end': 45.865, 'text': "You don't have to use it with just p5.", 'start': 44.445, 'duration': 1.42}, {'end': 51.527, 'text': 'So before I can write the code, let me come over here to the whiteboard.', 'start': 46.685, 'duration': 4.842}, {'end': 61.054, 'text': 'And this is where I last left off talking about how the general structure of a neural network system works.', 'start': 51.747, 'duration': 9.307}, {'end': 76.105, 'text': 'And so what I need to do here is when in the code I create a neural network, I want to create three things.', 'start': 62.655, 'duration': 13.45}, {'end': 80.368, 'text': 'I want to create an input layer.', 'start': 78.166, 'duration': 2.202}, {'end': 86.321, 'text': 'I want to create a hidden layer.', 'start': 83.999, 'duration': 2.322}, {'end': 91.506, 'text': 'And I want to create an output layer.', 'start': 88.843, 'duration': 2.663}, {'end': 101.174, 'text': 'So when I create a new, the way I want to design this library is I want to say new neural network.', 'start': 92.907, 'duration': 8.267}, {'end': 104.136, 'text': 'And I want to give it, can you see this? I think you can.', 'start': 101.794, 'duration': 2.342}, {'end': 105.738, 'text': 'I want to give it three arguments.', 'start': 104.156, 'duration': 1.582}, {'end': 115.93, 'text': "the number of input neurons, let's just use the word neurons, the number of hidden neurons, and the number of output neurons.", 'start': 106.803, 'duration': 9.127}, {'end': 119.272, 'text': "So I'm doing something which I typically don't do,", 'start': 116.27, 'duration': 3.002}], 'summary': 'Developing a neural network library with p5.js, creating input, hidden, and output layers, and allowing standalone use.', 'duration': 93.694, 'max_score': 25.578, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w25578.jpg'}], 'start': 0.409, 'title': 'Creating a javascript neural network library', 'summary': 'Delves into the design and structure of the network, aiming for versatility and applicability in various contexts.', 'chapters': [{'end': 136.945, 'start': 0.409, 'title': 'Creating javascript neural network library', 'summary': 'Discusses the creation of a javascript neural network library, covering the design and structure of the network, and the goal to make it versatile and applicable in various contexts.', 'duration': 136.536, 'highlights': ['The library is designed to be used with a set of p5.js examples with a JavaScript library called p5, but it can also stand alone (GitHub repository: github-schiffman-neuron-network-p5).', 'The neural network system is structured to have an input layer, a hidden layer, and an output layer, with the ability to create a new neural network by specifying the number of input neurons, hidden neurons, and output neurons.', 'The goal of the library creation is to develop a versatile and applicable neural network library that can be used in a variety of contexts.']}], 'duration': 136.536, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w409.jpg', 'highlights': ['The goal of the library creation is to develop a versatile and applicable neural network library that can be used in a variety of contexts.', 'The neural network system is structured to have an input layer, a hidden layer, and an output layer, with the ability to create a new neural network by specifying the number of input neurons, hidden neurons, and output neurons.', 'The library is designed to be used with a set of p5.js examples with a JavaScript library called p5, but it can also stand alone (GitHub repository: github-schiffman-neuron-network-p5).']}, {'end': 294.211, 'segs': [{'end': 163.341, 'src': 'embed', 'start': 137.045, 'weight': 0, 'content': [{'end': 141.57, 'text': "I'm just kind of working on the skeleton, the structure of the library before I start to apply it to things.", 'start': 137.045, 'duration': 4.525}, {'end': 144.112, 'text': "So let's just make up some numbers.", 'start': 142.53, 'duration': 1.582}, {'end': 156.803, 'text': "Let's say there's going to be three input neurons, four hidden neurons, and two output neurons.", 'start': 144.512, 'duration': 12.291}, {'end': 163.341, 'text': 'What this means now is in a feed-forward neural network, There are three inputs.', 'start': 157.084, 'duration': 6.257}], 'summary': 'Working on library structure with 3 input, 4 hidden, and 2 output neurons.', 'duration': 26.296, 'max_score': 137.045, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w137045.jpg'}, {'end': 250.656, 'src': 'embed', 'start': 223.106, 'weight': 1, 'content': [{'end': 226.707, 'text': 'And now there are variations to this that we might see in future examples.', 'start': 223.106, 'duration': 3.601}, {'end': 232.969, 'text': 'But the idea of a fully connected network is that every input is connected to every hidden.', 'start': 226.767, 'duration': 6.202}, {'end': 235.73, 'text': 'Every hidden is connected to every output.', 'start': 233.71, 'duration': 2.02}, {'end': 237.931, 'text': 'So I can draw all those connections.', 'start': 236.231, 'duration': 1.7}, {'end': 244.173, 'text': "And it's not so many that if I were doing some kind of post-production, I would speed this up.", 'start': 238.011, 'duration': 6.162}, {'end': 248.115, 'text': "But I'm going to just draw this web of all these connections.", 'start': 244.674, 'duration': 3.441}, {'end': 250.656, 'text': 'So every input is connected to every hidden.', 'start': 248.335, 'duration': 2.321}], 'summary': 'Fully connected network connects every input to every hidden and every hidden to every output.', 'duration': 27.55, 'max_score': 223.106, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w223106.jpg'}], 'start': 137.045, 'title': 'Neural network structure overview', 'summary': 'Provides an overview of a multi-layered perceptron neural network with 3 input neurons, 4 hidden neurons, and 2 output neurons, forming a fully connected network with extensive interconnections.', 'chapters': [{'end': 294.211, 'start': 137.045, 'title': 'Neural network structure overview', 'summary': 'Explains the configuration of a multi-layered perceptron neural network with 3 input neurons, 4 hidden neurons, and 2 output neurons, forming a fully connected network where every input is connected to every hidden and every hidden is connected to every output.', 'duration': 157.166, 'highlights': ['The neural network consists of 3 input neurons, 4 hidden neurons, and 2 output neurons, forming a multi-layered perceptron. The chapter outlines the specific configuration of the neural network, including the number of input, hidden, and output neurons.', 'The network is a fully connected network where every input is connected to every hidden and every hidden is connected to every output. The chapter describes the network as a fully connected one, emphasizing that every input is linked to every hidden and every hidden is linked to every output.', 'The chapter provides an overview of the feed-forward structure of the neural network. The chapter explains the flow of data in the neural network, from the input layer to the hidden layer and then to the output layer.']}], 'duration': 157.166, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w137045.jpg', 'highlights': ['The neural network consists of 3 input neurons, 4 hidden neurons, and 2 output neurons, forming a multi-layered perceptron.', 'The network is a fully connected network where every input is connected to every hidden and every hidden is connected to every output.', 'The chapter provides an overview of the feed-forward structure of the neural network.']}, {'end': 566.061, 'segs': [{'end': 344.311, 'src': 'embed', 'start': 296.492, 'weight': 1, 'content': [{'end': 300.255, 'text': "And now let's actually try to write a little bit of this library, very, very little.", 'start': 296.492, 'duration': 3.763}, {'end': 305.179, 'text': 'So where am I going here? OK, so this is my code.', 'start': 302.836, 'duration': 2.343}, {'end': 305.779, 'text': "There's nothing yet.", 'start': 305.259, 'duration': 0.52}, {'end': 311.005, 'text': "I'm going to create a new file, and I'm going to call this nn.js.", 'start': 305.899, 'duration': 5.106}, {'end': 312.947, 'text': 'So this is now going to be my.', 'start': 311.726, 'duration': 1.221}, {'end': 315.39, 'text': "So here's the thing.", 'start': 314.108, 'duration': 1.282}, {'end': 321.15, 'text': 'Ultimately, I want this to be like a proper JavaScript library.', 'start': 315.67, 'duration': 5.48}, {'end': 325.334, 'text': 'But ultimately, what is a JavaScript library, but a file with some JavaScript in it?', 'start': 321.57, 'duration': 3.764}, {'end': 333.641, 'text': 'So I might later, as this gets more sophisticated, optimize it and do some sort of build process or break it up into multiple files.', 'start': 325.434, 'duration': 8.207}, {'end': 335.743, 'text': 'But right now, I just want to get the pieces going.', 'start': 333.801, 'duration': 1.942}, {'end': 338.045, 'text': "So I'm also going to use ES5 syntax.", 'start': 336.104, 'duration': 1.941}, {'end': 344.311, 'text': "trajectory that I've been on.", 'start': 343.01, 'duration': 1.301}], 'summary': 'Creating a new javascript library with es5 syntax.', 'duration': 47.819, 'max_score': 296.492, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w296492.jpg'}, {'end': 396.142, 'src': 'embed', 'start': 365.996, 'weight': 4, 'content': [{'end': 372.077, 'text': "Okay, and I should also mention again, while we're here, that I built this library already, and when I built it,", 'start': 365.996, 'duration': 6.081}, {'end': 376.359, 'text': 'I based just about everything out of this book called Make Your Own Neural Network by Tariq Rashid.', 'start': 372.077, 'duration': 4.282}, {'end': 379.919, 'text': "And so, while I'm doing this now kind of a bit more on the fly,", 'start': 376.639, 'duration': 3.28}, {'end': 384.54, 'text': "I'm sure everything that's in my brain ultimately came from here and probably some other sources too.", 'start': 379.919, 'duration': 4.621}, {'end': 386.261, 'text': 'Okay, so.', 'start': 385.461, 'duration': 0.8}, {'end': 388.738, 'text': 'What do I want to do?', 'start': 387.918, 'duration': 0.82}, {'end': 396.142, 'text': 'The core thing that I want to do is I want to create the neural network with a certain number of input nodes number, hidden number of output.', 'start': 388.839, 'duration': 7.303}], 'summary': 'The speaker built a neural network based on make your own neural network by tariq rashid and is now planning to create a neural network with a specific number of input, hidden, and output nodes.', 'duration': 30.146, 'max_score': 365.996, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w365996.jpg'}, {'end': 468.665, 'src': 'heatmap', 'start': 435.626, 'weight': 0, 'content': [{'end': 438.406, 'text': "OK? So this is, we've actually written some code.", 'start': 435.626, 'duration': 2.78}, {'end': 445.068, 'text': 'The idea being that what I want to do is say things like var.', 'start': 438.727, 'duration': 6.341}, {'end': 456.871, 'text': 'brain and brain is a new neural network that has three inputs, with three hidden and one output.', 'start': 445.068, 'duration': 11.803}, {'end': 457.692, 'text': 'This is the idea.', 'start': 456.951, 'duration': 0.741}, {'end': 464.781, 'text': "So I need to figure out what shape, and I'm using the word shape very specifically, does the data come in.", 'start': 457.952, 'duration': 6.829}, {'end': 468.665, 'text': "That's how many input nodes I want.", 'start': 465.401, 'duration': 3.264}], 'summary': 'Developed code for a new neural network with 3 inputs, 3 hidden layers, and 1 output.', 'duration': 29.938, 'max_score': 435.626, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w435626.jpg'}, {'end': 520.219, 'src': 'embed', 'start': 490.207, 'weight': 3, 'content': [{'end': 491.368, 'text': "And we'll come back to that later.", 'start': 490.207, 'duration': 1.161}, {'end': 498.877, 'text': 'And I should also note that this is a oversimplification of how neural network architectures can be.', 'start': 492.208, 'duration': 6.669}, {'end': 501.781, 'text': 'This is, by definition, a three-layer network.', 'start': 499.238, 'duration': 2.543}, {'end': 508.11, 'text': 'And this library is only going to allow for a three-layer network, an input, a single hidden, and an output.', 'start': 502.002, 'duration': 6.108}, {'end': 510.672, 'text': 'As something you might think about for the future.', 'start': 509.111, 'duration': 1.561}, {'end': 513.214, 'text': 'how would you write the code to have multiple hidden layers?', 'start': 510.672, 'duration': 2.542}, {'end': 520.219, 'text': 'Because a lot of neural network-based learning systems need multiple hidden layers to be able to perform optimally.', 'start': 513.494, 'duration': 6.725}], 'summary': 'Neural network architecture is a three-layer network, but multiple hidden layers are needed for optimal performance.', 'duration': 30.012, 'max_score': 490.207, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w490207.jpg'}], 'start': 296.492, 'title': 'Creating a neural network library and architecture', 'summary': 'Details the creation of a neural network library in javascript using es5 syntax and later adopting es6 syntax, and also covers the process of creating a neural network with a specific architecture of 3 input nodes, a variable number of hidden nodes, and 1 output node, discussing network shape considerations and the oversimplification of a three-layer network architecture.', 'chapters': [{'end': 386.261, 'start': 296.492, 'title': 'Creating a neural network library in javascript', 'summary': 'Involves the process of creating a neural network library in javascript, starting with the creation of a constructor function and using es5 syntax with the intention to later adopt es6 syntax.', 'duration': 89.769, 'highlights': ['The process begins with creating a new file nn.js to initiate the development of a JavaScript library.', "The intention is to start with ES5 syntax and later transition to ES6 syntax for the library's development.", "The constructor function 'neural network' is mentioned as a key component in the creation of the library.", "The source of inspiration for the library's creation is credited to the book 'Make Your Own Neural Network' by Tariq Rashid."]}, {'end': 566.061, 'start': 387.918, 'title': 'Creating a neural network architecture', 'summary': "Focuses on creating a neural network with a specific architecture of 3 input nodes, a variable number of hidden nodes, and 1 output node, discussing the considerations for determining the network's shape and the oversimplification of a three-layer network architecture.", 'duration': 178.143, 'highlights': ['The chapter discusses creating a neural network with a specific architecture of 3 input nodes, a variable number of hidden nodes, and 1 output node, emphasizing the importance of determining the shape of the data, the desired output, and the number of hidden neurons (Relevance: 5)', 'It mentions the oversimplification of the neural network architecture, noting that the library being used only allows for a three-layer network with an input, a single hidden, and an output layer, and hints at the potential need for multiple hidden layers in the future (Relevance: 4)', 'The chapter briefly touches on the idea of writing code for multiple hidden layers in the future, acknowledging the necessity for such layers in neural network-based learning systems (Relevance: 3)', 'It introduces the concept of feed forward in neural networks and hints at the complexity and excitement of the upcoming processes (Relevance: 2)']}], 'duration': 269.569, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w296492.jpg', 'highlights': ['The chapter discusses creating a neural network with a specific architecture of 3 input nodes, a variable number of hidden nodes, and 1 output node, emphasizing the importance of determining the shape of the data, the desired output, and the number of hidden neurons (Relevance: 5)', 'The process begins with creating a new file nn.js to initiate the development of a JavaScript library (Relevance: 4)', "The intention is to start with ES5 syntax and later transition to ES6 syntax for the library's development (Relevance: 3)", "The constructor function 'neural network' is mentioned as a key component in the creation of the library (Relevance: 2)", "The source of inspiration for the library's creation is credited to the book 'Make Your Own Neural Network' by Tariq Rashid (Relevance: 1)"]}, {'end': 767.805, 'segs': [{'end': 638.698, 'src': 'embed', 'start': 597.721, 'weight': 0, 'content': [{'end': 600.544, 'text': 'Each one of those connections has a weight to it.', 'start': 597.721, 'duration': 2.823}, {'end': 608.775, 'text': 'Now, ultimately, the whole point of doing this neural network-based learning system is we want to tweak those weights.', 'start': 601.411, 'duration': 7.364}, {'end': 616.119, 'text': 'We want to train the brain, train the neural network to have optimal weights to get good results, results that make sense.', 'start': 608.915, 'duration': 7.204}, {'end': 618.881, 'text': "And that training process is something that I'm going to get to.", 'start': 616.68, 'duration': 2.201}, {'end': 622.023, 'text': "I don't have any videos down the road from now, but not too far away.", 'start': 618.901, 'duration': 3.122}, {'end': 630.675, 'text': "These weights will typically, to start, one way of thinking about them is they're going to just have random values between negative 1 and 1.", 'start': 623.052, 'duration': 7.623}, {'end': 638.698, 'text': "And there's a wide variety of techniques and strategies for initializing random weights that are not just random to a neural network.", 'start': 630.675, 'duration': 8.023}], 'summary': 'Neural network weights are tweaked for optimal results, starting with random values between -1 and 1.', 'duration': 40.977, 'max_score': 597.721, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w597721.jpg'}, {'end': 715.388, 'src': 'embed', 'start': 683.37, 'weight': 3, 'content': [{'end': 692.252, 'text': 'So it takes the input multiplied by the weight and adds that to the other input multiplied by the weight and adds that to the other input multiplied by the weight.', 'start': 683.37, 'duration': 8.882}, {'end': 693.232, 'text': 'So we could actually do this.', 'start': 692.272, 'duration': 0.96}, {'end': 697.506, 'text': '3 times 0.5 is 1.5.', 'start': 693.252, 'duration': 4.254}, {'end': 701.067, 'text': 'plus 2 times negative 0.5 is negative 1, plus 1, 000 times 1 is plus 1, 000.', 'start': 697.506, 'duration': 3.561}, {'end': 712.184, 'text': 'So this value now is 1, 000.', 'start': 701.067, 'duration': 11.117}, {'end': 715.388, 'text': "0.5. Now we can see there's a huge flaw here,", 'start': 712.185, 'duration': 3.203}], 'summary': 'Multiplying inputs by weights yields a value of 1000, indicating a significant flaw.', 'duration': 32.018, 'max_score': 683.37, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w683370.jpg'}, {'end': 758.58, 'src': 'embed', 'start': 732.964, 'weight': 2, 'content': [{'end': 739.568, 'text': 'A lot of time in working with a machine learning or neural network based system, we need to do some type of cleaning or normalizing of the data.', 'start': 732.964, 'duration': 6.604}, {'end': 743.991, 'text': 'And we might do something where we sample this down.', 'start': 739.608, 'duration': 4.383}, {'end': 748.454, 'text': 'So we actually do the number of bedrooms between 0 and 5 as a value between 0 and 1.', 'start': 744.031, 'duration': 4.423}, {'end': 751.456, 'text': 'And the number of bathrooms always is a value between 0 and 1 in square footage.', 'start': 748.454, 'duration': 3.002}, {'end': 755.759, 'text': 'This would actually turn into 0.1, because the range is between 0 and 10, 000 square feet or something.', 'start': 751.756, 'duration': 4.003}, {'end': 758.58, 'text': 'So we would do some kind of normalization of these values.', 'start': 755.979, 'duration': 2.601}], 'summary': 'Data cleaning and normalization is needed for ml, e.g. bedrooms 0-5 mapped to 0-1.', 'duration': 25.616, 'max_score': 732.964, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w732964.jpg'}], 'start': 566.741, 'title': 'Neural network weighted sum and initialization', 'summary': 'Covers the concept and importance of weighted sum in a neural network, along with the initialization of random weights, weighted sum calculation process, and the significance of data normalization in machine learning, addressing potential issues and solutions.', 'chapters': [{'end': 618.881, 'start': 566.741, 'title': 'Neural network weighted sum', 'summary': 'Describes the concept of a weighted sum in a neural network, where data flows through connections with different weights to be trained for optimal results.', 'duration': 52.14, 'highlights': ['The data flows through connections with different weights in a neural network to perform a weighted sum, influencing the prediction results.', 'The training process involves tweaking the weights of the connections to achieve optimal results in a neural network-based learning system.']}, {'end': 767.805, 'start': 618.901, 'title': 'Neural network weight initialization', 'summary': 'Discusses the initialization of random weights in a neural network, the process of weighted sum calculation, and the need for data normalization in machine learning, highlighting the potential issues and solutions.', 'duration': 148.904, 'highlights': ['The process of initialization of random weights in a neural network is crucial, and there are various techniques and strategies for this, but initially, random weights between -1 and 1 are used for simplicity.', 'Explaining the weighted sum calculation process, where each input is multiplied by its weight and then summed, showcasing the potential flaw of this method due to the varying magnitude of input values.', 'Emphasizing the need for data normalization in machine learning to address issues caused by varying input magnitude, suggesting the possibility of normalizing values such as the number of bedrooms and bathrooms to a range between 0 and 1.']}], 'duration': 201.064, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w566741.jpg', 'highlights': ['The training process involves tweaking the weights of the connections to achieve optimal results in a neural network-based learning system.', 'The data flows through connections with different weights in a neural network to perform a weighted sum, influencing the prediction results.', 'Emphasizing the need for data normalization in machine learning to address issues caused by varying input magnitude, suggesting the possibility of normalizing values such as the number of bedrooms and bathrooms to a range between 0 and 1.', 'Explaining the weighted sum calculation process, where each input is multiplied by its weight and then summed, showcasing the potential flaw of this method due to the varying magnitude of input values.', 'The process of initialization of random weights in a neural network is crucial, and there are various techniques and strategies for this, but initially, random weights between -1 and 1 are used for simplicity.']}, {'end': 1275.775, 'segs': [{'end': 844.657, 'src': 'embed', 'start': 809.781, 'weight': 2, 'content': [{'end': 820.487, 'text': 'But the classic and standard approach is actually to look at storing all of these weighted connections in something called a matrix.', 'start': 809.781, 'duration': 10.706}, {'end': 833.947, 'text': 'which is really just like a spreadsheet, a grid of numbers, looking at the inputs as an array and doing some type of math that basically takes.', 'start': 823.237, 'duration': 10.71}, {'end': 844.657, 'text': 'take that array of inputs, multiply it by that matrix of weights and generate the outputs of this hidden layer.', 'start': 833.947, 'duration': 10.71}], 'summary': 'Classic approach: storing weighted connections in a matrix, like a spreadsheet, to generate outputs of the hidden layer.', 'duration': 34.876, 'max_score': 809.781, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w809781.jpg'}, {'end': 972.321, 'src': 'heatmap', 'start': 931.714, 'weight': 3, 'content': [{'end': 941.183, 'text': 'The weights can be expressed like this, 1, 1, 1, 2, 2, 1, 2, 2.', 'start': 931.714, 'duration': 9.469}, {'end': 945.826, 'text': 'So this is a way of expressing the weights.', 'start': 941.184, 'duration': 4.642}, {'end': 953.069, 'text': 'And a way of expressing the inputs, I could write it like this, x1, x2.', 'start': 946.926, 'duration': 6.143}, {'end': 961.012, 'text': "So I'm making the case that I have two inputs and I have four weights.", 'start': 955.23, 'duration': 5.782}, {'end': 966.735, 'text': 'And I could write it out like a matrix of numbers, a 2 by 2 matrix.', 'start': 961.653, 'duration': 5.082}, {'end': 972.321, 'text': 'And this is essentially a 2 by 1 matrix.', 'start': 968.38, 'duration': 3.941}], 'summary': 'The weights are expressed as 1, 1, 1, 2, 2, 1, 2, 2. there are 2 inputs and 4 weights.', 'duration': 31.137, 'max_score': 931.714, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w931714.jpg'}, {'end': 1167.04, 'src': 'embed', 'start': 1144.134, 'weight': 0, 'content': [{'end': 1153.536, 'text': "writing a little simple JavaScript matrix library And ultimately, once we've done that, we can come back here and see how,", 'start': 1144.134, 'duration': 9.402}, {'end': 1161.479, 'text': 'if we have that library written, we can then use it to do the math between the inputs and the hidden and the hidden to the output.', 'start': 1153.536, 'duration': 7.943}, {'end': 1167.04, 'text': "And ultimately later, we're also going to go backwards through the network to tweak values and train it.", 'start': 1161.719, 'duration': 5.321}], 'summary': 'Creating a javascript matrix library for neural network calculations.', 'duration': 22.906, 'max_score': 1144.134, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w1144134.jpg'}, {'end': 1217.988, 'src': 'embed', 'start': 1188.967, 'weight': 4, 'content': [{'end': 1190.928, 'text': 'And there are JavaScript matrix libraries.', 'start': 1188.967, 'duration': 1.961}, {'end': 1196.19, 'text': "But I'm going to kind of unpack some of this and write a lot of the code from scratch just to have a sense of how it's working.", 'start': 1191.508, 'duration': 4.682}, {'end': 1200.295, 'text': 'So hopefully, as you were watching the video, you saw a little annotation.', 'start': 1196.912, 'duration': 3.383}, {'end': 1202.276, 'text': 'This is actually incorrect.', 'start': 1201.175, 'duration': 1.101}, {'end': 1204.278, 'text': 'I mean, everything about this math is correct.', 'start': 1202.316, 'duration': 1.962}, {'end': 1214.566, 'text': 'This matches this, right? The weighted sum is x1 times weight 1 from 1 to 1, x2 times weight from 2 to 1.', 'start': 1204.638, 'duration': 9.928}, {'end': 1217.988, 'text': 'But actually the notation, the way I wrote this matrix.', 'start': 1214.566, 'duration': 3.422}], 'summary': 'The speaker discusses javascript matrix libraries and plans to write code from scratch to understand its working.', 'duration': 29.021, 'max_score': 1188.967, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w1188967.jpg'}, {'end': 1275.775, 'src': 'embed', 'start': 1263.61, 'weight': 1, 'content': [{'end': 1272.954, 'text': "the next video I'm going to look at the matrix math again and then write a generic library that does that math and then come back and put it back into the neural network itself.", 'start': 1263.61, 'duration': 9.344}, {'end': 1275.495, 'text': 'Okay, so see you in the next video.', 'start': 1273.274, 'duration': 2.221}, {'end': 1275.775, 'text': 'Thanks.', 'start': 1275.515, 'duration': 0.26}], 'summary': 'Creating a generic library for matrix math in the next video.', 'duration': 12.165, 'max_score': 1263.61, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w1263610.jpg'}], 'start': 767.805, 'title': 'Neural network weighted connections', 'summary': 'Discusses the concept of weighted connections and matrices in neural networks, emphasizing the mathematical operations involved and the relevance of linear algebra, with a focus on implementing a javascript matrix library for neural network operations.', 'chapters': [{'end': 1275.775, 'start': 767.805, 'title': 'Neural network weighted connections', 'summary': 'Discusses the concept of weighted connections and matrices in neural networks, emphasizing the mathematical operations involved and the relevance of linear algebra, with a focus on implementing a javascript matrix library for neural network operations.', 'duration': 507.97, 'highlights': ['The classic approach in neural networks involves storing weighted connections in a matrix, which is essentially a grid of numbers, where the inputs are represented as an array and multiplied by the matrix of weights to generate outputs for the hidden layer.', 'The weights in a neural network can be expressed as a matrix, with the inputs as a 2x1 matrix and the weights as a 2x2 matrix, demonstrating the fundamental application of linear algebra in neural network operations.', 'The chapter emphasizes the importance of understanding linear algebra in neural network operations, with a plan to delve into matrix notation and mathematical operations, including the creation of a JavaScript matrix library for neural network computations.', 'The upcoming videos will cover detailed explanations of matrix math and the creation of a generic library for performing neural network operations, followed by the integration of the library into the neural network implementation.', 'The author acknowledges the availability of JavaScript matrix libraries like NumPy but opts to build the code from scratch to gain a comprehensive understanding of the underlying operations and mathematics involved.']}], 'duration': 507.97, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IlmNhFxre0w/pics/IlmNhFxre0w767805.jpg', 'highlights': ['The chapter emphasizes the importance of understanding linear algebra in neural network operations, with a plan to delve into matrix notation and mathematical operations, including the creation of a JavaScript matrix library for neural network computations.', 'The upcoming videos will cover detailed explanations of matrix math and the creation of a generic library for performing neural network operations, followed by the integration of the library into the neural network implementation.', 'The classic approach in neural networks involves storing weighted connections in a matrix, which is essentially a grid of numbers, where the inputs are represented as an array and multiplied by the matrix of weights to generate outputs for the hidden layer.', 'The weights in a neural network can be expressed as a matrix, with the inputs as a 2x1 matrix and the weights as a 2x2 matrix, demonstrating the fundamental application of linear algebra in neural network operations.', 'The author acknowledges the availability of JavaScript matrix libraries like NumPy but opts to build the code from scratch to gain a comprehensive understanding of the underlying operations and mathematics involved.']}], 'highlights': ['The neural network system is structured to have an input layer, a hidden layer, and an output layer, with the ability to create a new neural network by specifying the number of input neurons, hidden neurons, and output neurons.', 'The neural network consists of 3 input neurons, 4 hidden neurons, and 2 output neurons, forming a multi-layered perceptron.', 'The training process involves tweaking the weights of the connections to achieve optimal results in a neural network-based learning system.', 'The classic approach in neural networks involves storing weighted connections in a matrix, which is essentially a grid of numbers, where the inputs are represented as an array and multiplied by the matrix of weights to generate outputs for the hidden layer.', 'The chapter discusses creating a neural network with a specific architecture of 3 input nodes, a variable number of hidden nodes, and 1 output node, emphasizing the importance of determining the shape of the data, the desired output, and the number of hidden neurons.']}