title
Coding Challenge #100: Neuroevolution Flappy Bird - Part 1
description
In this challenge, I use my JavaScript neural network library and a genetic algorithm to train an agent to play Flappy Bird. Code: https://thecodingtrain.com/challenges/100-neuroevolution-flappy-bird
p5.js Web Editor Sketches:
đšī¸ Neuroevolution Flappy Bird: https://editor.p5js.org/codingtrain/sketches/wb-QHVXBq
đšī¸ Neuroevolution Flappy Bird -- Part 5: https://editor.p5js.org/codingtrain/sketches/iyPZnViI9
đšī¸ Flappy Bird - Improved code: https://editor.p5js.org/codingtrain/sketches/T_cecG2MD
Other Parts of this Challenge:
đē Neuroevolution Flappy Bird - Part 2: https://youtu.be/YtRA6tqgJBc
đē Neuroevolution Flappy Bird - Part 3: https://youtu.be/3lvj9jvERvs
đē Neuroevolution Flappy Bird - Part 4: https://youtu.be/HrvNpbnjEG8
đē Neuroevolution Flappy Bird - Part 5: https://youtu.be/U9wiMM3BqLU
đĨ Previous video: https://youtu.be/KtPpoMThKUs?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
đĨ Next video: https://youtu.be/fUkF-YLLXeg?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
đĨ All videos: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
References:
đģ TensorFlow.js: https://js.tensorflow.org/
đģ ml5.js: https://github.com/ml5js
đģ JS Neural Network: https://github.com/CodingTrain/Toy-Neural-Network-JS/
đģ Flappy Bird Clone: https://github.com/CodingTrain/Flappy-Bird-Clone
đģ Neuroevolution documentation: https://github.com/shiffman/NOC-S18/tree/master/week10
Videos:
đ Coding Challenge #31: Flappy Bird: https://youtu.be/cXgA1d_E-jY
đ My Genetic Algorithm playlist: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6bJM3VgzjNV5YxVxUwzALHV
đ My Neural Networks playlist: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6aCibgK1PTWWu9by6XFdCfh
Live Stream Archive:
đ´ Coding Train Live 132: https://youtu.be/ipMekhLS2M0?t=2934s
Related Coding Challenges:
đ #29 Smart Rockets in p5.js: https://youtu.be/bGz7mv2vD6g
đ #31 Flappy Bird: https://youtu.be/cXgA1d_E-jY
đ #41 Clappy Bird: https://youtu.be/aKiyCeIuwn4
Timestamps:
0:00 Introducing today's topic: Flappy Birds
1:44 Pieces of the Code
4:11 Reinforcement Learning
6:45 Make multiple birds that make random decisions and use a genetic alorithm
7:40 Each Bird object will have its own neural network "brain"
9:40 The architecture for the neural network
14:03 Write a think function
16:04 Code the inputs
19:00 Find the closest pipe
22:22 Summary and next steps
Editing by Mathieu Blanchette
Animations by Jason Heglund
Music from Epidemic Sound
đ Website: http://thecodingtrain.com/
đž Share Your Creation! https://thecodingtrain.com/guides/passenger-showcase-guide
đŠ Suggest Topics: https://github.com/CodingTrain/Suggestion-Box
đĄ GitHub: https://github.com/CodingTrain
đŦ Discord: https://thecodingtrain.com/discord
đ Membership: http://youtube.com/thecodingtrain/join
đ Store: https://standard.tv/codingtrain
đī¸ Twitter: https://twitter.com/thecodingtrain
đ¸ Instagram: https://www.instagram.com/the.coding.train/
đĨ Coding Challenges: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH
đĨ Intro to Programming: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6Zy51Q-x9tMWIv9cueOFTFA
đ p5.js: https://p5js.org
đ p5.js Web Editor: https://editor.p5js.org/
đ Processing: https://processing.org
đ Code of Conduct: https://github.com/CodingTrain/Code-of-Conduct
This description was auto-generated. If you see a problem, please open an issue: https://github.com/CodingTrain/thecodingtrain.com/issues/new
#neuralnetwork #reinforcementlearning #geneticalgorithm #evolution #flappybird #javascript #p5js
detail
{'title': 'Coding Challenge #100: Neuroevolution Flappy Bird - Part 1', 'heatmap': [{'end': 492.262, 'start': 444.772, 'weight': 0.826}, {'end': 856.45, 'start': 816.526, 'weight': 0.912}, {'end': 932.044, 'start': 914.747, 'weight': 1}], 'summary': 'Series delves into implementing neuroevolution in a coding challenge, using genetic algorithms and neural networks to train agents in a flappy bird clone, while exploring reinforcement learning, neural network structure, optimization, and debugging code for the game.', 'chapters': [{'end': 230.2, 'segs': [{'end': 29.616, 'src': 'embed', 'start': 1.42, 'weight': 0, 'content': [{'end': 3.761, 'text': "It's time for coding challenge number 100.", 'start': 1.42, 'duration': 2.341}, {'end': 6.843, 'text': 'Here we go.', 'start': 3.761, 'duration': 3.082}, {'end': 14.968, 'text': "In this coding challenge, I am going to do something that I have been wanting to do on this channel for, I don't know, about 100 coding challenges.", 'start': 7.263, 'duration': 7.705}, {'end': 20.07, 'text': "I'm going to make a project that involves neuroevolution.", 'start': 15.548, 'duration': 4.522}, {'end': 28.335, 'text': 'I am going to combine both neural networks and genetic algorithms into an agent, into a simulation of a simple game.', 'start': 20.411, 'duration': 7.924}, {'end': 29.616, 'text': "And I'm going to train.", 'start': 28.435, 'duration': 1.181}], 'summary': 'Creating a neuroevolution project combining neural networks and genetic algorithms for a simple game simulation.', 'duration': 28.196, 'max_score': 1.42, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw1420.jpg'}, {'end': 75.325, 'src': 'embed', 'start': 49.838, 'weight': 1, 'content': [{'end': 58.78, 'text': 'This is using the Flappy Bird clone GitHub repository that various folks have contributed design ideas to and is actually running Neuro Evolution right now.', 'start': 49.838, 'duration': 8.942}, {'end': 64.501, 'text': 'And you can see these trains are kind of getting better and better at making their way through these little gaps.', 'start': 58.8, 'duration': 5.701}, {'end': 67.202, 'text': "So I'm going to program this myself.", 'start': 64.782, 'duration': 2.42}, {'end': 70.403, 'text': "Now I'm going to make a version of this in this coding challenge.", 'start': 67.222, 'duration': 3.181}, {'end': 75.325, 'text': "The difference, however, is I'm going to want to work in a simpler place.", 'start': 71.443, 'duration': 3.882}], 'summary': 'Neuro evolution is being used to improve flappy bird clone trains, and the speaker plans to program a simpler version in a coding challenge.', 'duration': 25.487, 'max_score': 49.838, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw49838.jpg'}, {'end': 230.2, 'src': 'embed', 'start': 208.388, 'weight': 2, 'content': [{'end': 218.313, 'text': 'And those are a function called copy, which allows me to make a complete copy of a neural network in its current state, as well as mutate,', 'start': 208.388, 'duration': 9.925}, {'end': 224.877, 'text': 'which is a function that allows me to make some minor random changes to the state of that neural network.', 'start': 219.033, 'duration': 5.844}, {'end': 226.798, 'text': 'And when I say state, I mean all of the weights.', 'start': 224.897, 'duration': 1.901}, {'end': 230.2, 'text': "So these are some things that I'm going to definitely need to make use of.", 'start': 227.238, 'duration': 2.962}], 'summary': "Functions like 'copy' and 'mutate' modify neural network state.", 'duration': 21.812, 'max_score': 208.388, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw208388.jpg'}], 'start': 1.42, 'title': 'Implementing neuroevolution in coding challenge', 'summary': 'Discusses the use of neuroevolution to train agents in a coding challenge by combining neural networks and genetic algorithms in a flappy bird clone, using a toy neural network library.', 'chapters': [{'end': 230.2, 'start': 1.42, 'title': 'Neuroevolution in coding challenge 100', 'summary': 'Discusses the implementation of neuroevolution in a coding challenge, combining neural networks and genetic algorithms to train agents in a simple game, using a flappy bird clone and a toy neural network library.', 'duration': 228.78, 'highlights': ['The chapter discusses the implementation of neuroevolution in a coding challenge, combining neural networks and genetic algorithms to train agents in a simple game. This is the central theme of the chapter, focusing on the integration of neural networks and genetic algorithms for agent training.', 'Using a Flappy Bird clone and a toy neural network library for the project. The project involves utilizing a Flappy Bird clone and a toy neural network library, which provides the foundational elements for the implementation of neuroevolution.', "Introduction of specific functionalities in the neural network class to apply a genetic algorithm, including 'copy' and 'mutate' functions. The neural network class is enhanced with functionalities like 'copy' and 'mutate' to facilitate the application of a genetic algorithm for making copies and introducing random changes to the neural network."]}], 'duration': 228.78, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw1420.jpg', 'highlights': ['The chapter discusses the implementation of neuroevolution in a coding challenge, combining neural networks and genetic algorithms to train agents in a simple game.', 'The project involves utilizing a Flappy Bird clone and a toy neural network library, which provides the foundational elements for the implementation of neuroevolution.', "The neural network class is enhanced with functionalities like 'copy' and 'mutate' to facilitate the application of a genetic algorithm for making copies and introducing random changes to the neural network."]}, {'end': 404.418, 'segs': [{'end': 304.33, 'src': 'embed', 'start': 255.6, 'weight': 0, 'content': [{'end': 261.164, 'text': "Now, reinforcement learning is a field of research that's been around for a very long time.", 'start': 255.6, 'duration': 5.564}, {'end': 265.488, 'text': 'It predates deep learning, neural networks, all that sort of stuff.', 'start': 261.505, 'duration': 3.983}, {'end': 269.35, 'text': 'And I would love to come do lots more tutorials about reinforcement learning.', 'start': 265.528, 'duration': 3.822}, {'end': 279.258, 'text': 'The recent advances that you might have seen with things like DeepMind and its bot that learned to beat all of the Atari games.', 'start': 269.711, 'duration': 9.547}, {'end': 284.262, 'text': 'those were developed with a style of reinforcement learning called deep Q learning.', 'start': 279.258, 'duration': 5.004}, {'end': 293.641, 'text': 'which uses a neural network and trains the neural network based on this reward-based system.', 'start': 285.655, 'duration': 7.986}, {'end': 298.325, 'text': 'So you have this idea of, in reinforcement learning, that you have a world.', 'start': 294.202, 'duration': 4.123}, {'end': 301.447, 'text': 'The world is going to be, in our case, the world of Flappy Bird.', 'start': 298.665, 'duration': 2.782}, {'end': 304.33, 'text': 'The world has a given state.', 'start': 302.408, 'duration': 1.922}], 'summary': "Reinforcement learning predates deep learning, and recent advances include deepmind's bot that mastered atari games using deep q learning.", 'duration': 48.73, 'max_score': 255.6, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw255600.jpg'}, {'end': 412.49, 'src': 'embed', 'start': 381.757, 'weight': 1, 'content': [{'end': 385.782, 'text': 'The payoff for certain things that I might do in a game scenario.', 'start': 381.757, 'duration': 4.025}, {'end': 393.529, 'text': "how do I know, if I'm playing, say, the game chess, if this move is really going to be a good move for way down the line?", 'start': 385.782, 'duration': 7.747}, {'end': 396.833, 'text': 'This is something I really need to figure out.', 'start': 394.05, 'duration': 2.783}, {'end': 399.35, 'text': 'It gets very complicated very quickly.', 'start': 397.848, 'duration': 1.502}, {'end': 402.355, 'text': 'The good news is I can use a different system.', 'start': 399.511, 'duration': 2.844}, {'end': 404.418, 'text': 'I can say you know what?', 'start': 402.735, 'duration': 1.683}, {'end': 412.49, 'text': "Instead of trying to actually figure out whether it was a good or bad decision, I'm just going to make a whole lot of these birds.", 'start': 404.838, 'duration': 7.652}], 'summary': 'Assessing the value of moves in chess and using a different system to make decisions.', 'duration': 30.733, 'max_score': 381.757, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw381757.jpg'}], 'start': 230.48, 'title': 'Reinforcement learning in neural networks', 'summary': 'Explores reinforcement learning in neural networks, referencing the deepmind bot that mastered atari games and discussing the complications of evaluating decisions in a game scenario.', 'chapters': [{'end': 404.418, 'start': 230.48, 'title': 'Reinforcement learning in neural networks', 'summary': 'Explores the concept of reinforcement learning in neural networks, referencing the deepmind bot that mastered atari games and discussing the complications of evaluating decisions in a game scenario.', 'duration': 173.938, 'highlights': ['Reinforcement learning involves training a neural network based on a reward-based system, such as the deep Q learning used in the DeepMind bot, which learned to beat Atari games.', 'The chapter discusses the challenge of evaluating decisions in a game scenario, highlighting the difficulty of determining the payoff for certain actions and the need to figure out the long-term consequences.', 'The concept of reinforcement learning predates deep learning and neural networks, with the potential for future tutorials on the topic mentioned.']}], 'duration': 173.938, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw230480.jpg', 'highlights': ['Reinforcement learning involves training a neural network based on a reward-based system, such as the deep Q learning used in the DeepMind bot, which learned to beat Atari games.', 'The chapter discusses the challenge of evaluating decisions in a game scenario, highlighting the difficulty of determining the payoff for certain actions and the need to figure out the long-term consequences.', 'The concept of reinforcement learning predates deep learning and neural networks, with the potential for future tutorials on the topic mentioned.']}, {'end': 711.974, 'segs': [{'end': 443.992, 'src': 'embed', 'start': 404.838, 'weight': 0, 'content': [{'end': 412.49, 'text': "Instead of trying to actually figure out whether it was a good or bad decision, I'm just going to make a whole lot of these birds.", 'start': 404.838, 'duration': 7.652}, {'end': 414.231, 'text': 'maybe like 1, 000 of them.', 'start': 412.87, 'duration': 1.361}, {'end': 418.075, 'text': 'They will all have random neural networks, random neural networks.', 'start': 414.512, 'duration': 3.563}, {'end': 420.398, 'text': 'They will just make random decisions.', 'start': 418.556, 'duration': 1.842}, {'end': 425.583, 'text': 'And maybe some of those random decisions happen to be better than some of the other random decisions.', 'start': 420.758, 'duration': 4.825}, {'end': 430.168, 'text': 'I will then take those into a new generation and repopulate the world with 100 new birds,', 'start': 425.944, 'duration': 4.224}, {'end': 434.41, 'text': 'based off of those ones that did well over and over and over again.', 'start': 431.409, 'duration': 3.001}, {'end': 436.85, 'text': 'The same principles of a genetic algorithm.', 'start': 434.45, 'duration': 2.4}, {'end': 443.992, 'text': "So now I might refer to you, if you have not ever worked with a genetic algorithm before, I'll refer to you to my genetic algorithm tutorial series.", 'start': 436.91, 'duration': 7.082}], 'summary': 'Creating 1000 birds with random neural networks, selecting 100 best-performing for new generation using genetic algorithm.', 'duration': 39.154, 'max_score': 404.838, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw404838.jpg'}, {'end': 492.262, 'src': 'heatmap', 'start': 444.772, 'weight': 0.826, 'content': [{'end': 450.594, 'text': "Okay, so let's go and write a little bit of code and then we're going to have to come back here to the whiteboard I know in a second.", 'start': 444.772, 'duration': 5.822}, {'end': 452.654, 'text': "But let's start doing a little bit of code here.", 'start': 450.634, 'duration': 2.02}, {'end': 454.555, 'text': "So I'm going to go into the code.", 'start': 453.074, 'duration': 1.481}, {'end': 456.255, 'text': "I'm going to go into the bird class.", 'start': 454.615, 'duration': 1.64}, {'end': 458.696, 'text': "And right here I'm noticing, okay, this is.", 'start': 456.835, 'duration': 1.861}, {'end': 463.27, 'text': 'the function that causes the bird to jump up.', 'start': 460.489, 'duration': 2.781}, {'end': 470.373, 'text': 'How is that currently decided? That is currently decided in sketch.js here when I press a key.', 'start': 463.51, 'duration': 6.863}, {'end': 473.054, 'text': 'This will no longer be the case.', 'start': 471.614, 'duration': 1.44}, {'end': 476.076, 'text': 'This entire function is now going to be commented out.', 'start': 473.675, 'duration': 2.401}, {'end': 481.158, 'text': 'I want the bird to make its own decision whether it should jump up or not.', 'start': 476.636, 'duration': 4.522}, {'end': 492.262, 'text': "So now I'm going to go over here and I'm going to say This.brain equals new neural network.", 'start': 483.039, 'duration': 9.223}], 'summary': 'Modifying bird class to make its own decision using a neural network.', 'duration': 47.49, 'max_score': 444.772, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw444772.jpg'}, {'end': 515.552, 'src': 'embed', 'start': 493.294, 'weight': 2, 'content': [{'end': 501.461, 'text': 'So the idea here is that the inside the bird object, the bird object will have its own neural network.', 'start': 493.294, 'duration': 8.167}, {'end': 505.444, 'text': 'And its neural network will be the thing that makes the decision whether it should jump or not.', 'start': 501.561, 'duration': 3.883}, {'end': 506.685, 'text': "Ah, I'm remembering.", 'start': 505.684, 'duration': 1.001}, {'end': 510.448, 'text': 'This is not an original idea I have, of course.', 'start': 506.705, 'duration': 3.743}, {'end': 515.552, 'text': "I will include some links in this video's description to other references and papers on neuroevolution.", 'start': 510.808, 'duration': 4.744}], 'summary': 'Bird object has its own neural network to make jumping decisions.', 'duration': 22.258, 'max_score': 493.294, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw493294.jpg'}, {'end': 711.974, 'src': 'embed', 'start': 679.461, 'weight': 3, 'content': [{'end': 685.587, 'text': "I would love to try that as an experiment as a counterpoint to the way I'm going to do it in this video.", 'start': 679.461, 'duration': 6.126}, {'end': 695.111, 'text': "I'm instead going to take Somewhat of a higher level approach and I'm going to do some feature extraction manually with my brain.", 'start': 686.107, 'duration': 9.004}, {'end': 711.974, 'text': "I'm going to say that I think I'm going to decide That I think that the important Properties, the inputs, should be the Y location of the bird,", 'start': 695.111, 'duration': 16.863}], 'summary': 'Experimenting with manual feature extraction for bird location.', 'duration': 32.513, 'max_score': 679.461, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw679461.jpg'}], 'start': 404.838, 'title': 'Genetic algorithm and neural network for decision making in flappy bird', 'summary': 'Explores using genetic algorithms to create a population of birds with random neural networks, evaluating their decision-making abilities, and implementing a neural network for a flappy bird game, with consideration of inputs such as the y location of the bird or the pixels of the game.', 'chapters': [{'end': 443.992, 'start': 404.838, 'title': 'Genetic algorithm for decision making', 'summary': 'Explores the concept of using genetic algorithms to create a population of birds with random neural networks, evaluating their decision-making abilities, and repopulating the world with the most successful birds, based on the principles of genetic algorithms.', 'duration': 39.154, 'highlights': ['Creating a population of 1000 birds with random neural networks.', 'Evaluating the random decisions made by the birds to identify successful ones for repopulation.', 'Using the principles of genetic algorithms to repopulate the world with 100 new birds based on successful predecessors.']}, {'end': 711.974, 'start': 444.772, 'title': 'Neural network for flappy bird', 'summary': 'Discusses implementing a neural network for a flappy bird game, including the decision-making process of the bird, the architecture of the neural network with one output, and the consideration of inputs as the y location of the bird or the pixels of the game.', 'duration': 267.202, 'highlights': ["The bird's decision to jump will be determined by its own neural network, replacing the current function in sketch.js, enhancing autonomy and decision-making within the game.", 'The neural network architecture includes one output, simplifying the decision-making process to jumping or not, with the output value determining the action.', 'Consideration is given to using the Y location of the bird or the pixels of the game as inputs for the neural network, providing insights into potential input data for decision-making in the game.']}], 'duration': 307.136, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw404838.jpg', 'highlights': ['Creating a population of 1000 birds with random neural networks.', 'Using the principles of genetic algorithms to repopulate the world with 100 new birds based on successful predecessors.', "The bird's decision to jump will be determined by its own neural network, enhancing autonomy and decision-making within the game.", 'Consideration is given to using the Y location of the bird or the pixels of the game as inputs for the neural network.']}, {'end': 926.119, 'segs': [{'end': 856.45, 'src': 'heatmap', 'start': 751.631, 'weight': 0, 'content': [{'end': 753.612, 'text': 'This might work with just one hidden node.', 'start': 751.631, 'duration': 1.981}, {'end': 755.454, 'text': 'It might work.', 'start': 754.933, 'duration': 0.521}, {'end': 758.282, 'text': 'with no hidden layer at all, even, in fact.', 'start': 756.281, 'duration': 2.001}, {'end': 761.883, 'text': 'It might work with just two of these inputs and be able to figure it out.', 'start': 758.702, 'duration': 3.181}, {'end': 768.746, 'text': "But let's just decide that we're also going to use four, just for the sake of having some hidden nodes.", 'start': 762.284, 'duration': 6.462}, {'end': 769.727, 'text': 'We can experiment.', 'start': 769.026, 'duration': 0.701}, {'end': 776.049, 'text': 'Does it work better if I make 64 hidden, or four, or eight? Trial and error is your friend in machine learning systems.', 'start': 769.747, 'duration': 6.302}, {'end': 782.152, 'text': 'OK, so now, this is now the diagram of the neural network.', 'start': 776.65, 'duration': 5.502}, {'end': 789.76, 'text': 'I must sit here and connect all of the various lines.', 'start': 784.515, 'duration': 5.245}, {'end': 795.345, 'text': 'So the inputs come in here, and the output comes in here.', 'start': 791.942, 'duration': 3.403}, {'end': 799.529, 'text': 'Four inputs, four hidden, one output.', 'start': 796.246, 'duration': 3.283}, {'end': 806.022, 'text': 'If you want to learn more about how a neural network works, all of the internals that are going on here,', 'start': 801.46, 'duration': 4.562}, {'end': 810.144, 'text': "what all these connections are in the weights, I'll refer you back to my neural network tutorial series.", 'start': 806.022, 'duration': 4.122}, {'end': 812.745, 'text': 'But this is pretty similar also to what I did in the color predictor.', 'start': 810.344, 'duration': 2.401}, {'end': 815.566, 'text': 'There were three inputs and two outputs.', 'start': 812.925, 'duration': 2.641}, {'end': 816.066, 'text': "I can't remember.", 'start': 815.606, 'duration': 0.46}, {'end': 826.03, 'text': "All right, so now we've decided that I want to have four inputs, four hidden nodes, and one output.", 'start': 816.526, 'duration': 9.504}, {'end': 828.791, 'text': 'So we now have the brain of our bird.', 'start': 826.45, 'duration': 2.341}, {'end': 840.679, 'text': 'And then in update, the bird is going to decide, oh, this is more complicated than I thought.', 'start': 829.963, 'duration': 10.716}, {'end': 842.362, 'text': "Let's write a new function.", 'start': 841.24, 'duration': 1.122}, {'end': 844.305, 'text': "Let's write a function called think.", 'start': 842.863, 'duration': 1.442}, {'end': 856.45, 'text': 'So in think, basically what I want to do is I want to say, let the output equal this.brain predict, and I want to send in the inputs.', 'start': 845.307, 'duration': 11.143}], 'summary': 'Experimenting with different numbers of hidden nodes and inputs in a neural network for machine learning.', 'duration': 92.674, 'max_score': 751.631, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw751631.jpg'}, {'end': 906.258, 'src': 'embed', 'start': 880.244, 'weight': 5, 'content': [{'end': 886.188, 'text': "So this is the idea here, that the neural network library that I'm using has a function called predict.", 'start': 880.244, 'duration': 5.944}, {'end': 890.632, 'text': 'When I call that function, I send in the inputs.', 'start': 887.389, 'duration': 3.243}, {'end': 895.407, 'text': 'The inputs go feed through times.', 'start': 891.843, 'duration': 3.564}, {'end': 898.49, 'text': 'the weights summed activation function times.', 'start': 895.407, 'duration': 3.083}, {'end': 900.432, 'text': 'the weights summed activation function.', 'start': 898.49, 'duration': 1.942}, {'end': 901.313, 'text': 'I get the output.', 'start': 900.492, 'duration': 0.821}, {'end': 906.258, 'text': "And based on that output, if it's greater than 0.5, the bird jumps.", 'start': 902.174, 'duration': 4.084}], 'summary': "Neural network library uses predict function to determine bird's jump based on output > 0.5.", 'duration': 26.014, 'max_score': 880.244, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw880244.jpg'}], 'start': 711.974, 'title': 'Neural network structure and function', 'summary': 'Covers determining neural network structure, including inputs, outputs, and hidden nodes, with practical examples such as a bird movement prediction model using four inputs, four hidden nodes, and one output, and introduces decision-making functions.', 'chapters': [{'end': 816.066, 'start': 711.974, 'title': 'Neural network structure decision', 'summary': 'Discusses determining the structure of a neural network, including the number of inputs, outputs, and hidden nodes, with the possibility of using trial and error in deciding the number of hidden nodes and references to a neural network tutorial series and a color predictor.', 'duration': 104.092, 'highlights': ['Determining the number of inputs, outputs, and hidden nodes in a neural network The chapter discusses deciding on the number of inputs (4), outputs (1), and hidden nodes, with the possibility of using trial and error in determining the number of hidden nodes.', 'Consideration of using trial and error in determining the number of hidden nodes The chapter mentions using trial and error to determine the optimal number of hidden nodes, with the suggestion that experimentation can be conducted to find the most effective configuration.', 'Reference to a neural network tutorial series and a color predictor The speaker refers to a neural network tutorial series for learning about the internal workings of neural networks and connections, and also mentions a color predictor project with three inputs and two outputs.']}, {'end': 926.119, 'start': 816.526, 'title': 'Neural network function for bird movement', 'summary': "Explains the process of implementing a neural network with four inputs, four hidden nodes, and one output to determine the bird's movement based on the output value, with a threshold of 0.5, and introduces the 'think' function for decision-making.", 'duration': 109.593, 'highlights': ['The neural network has four inputs, four hidden nodes, and one output.', "The 'think' function determines the bird's movement based on the output value with a threshold of 0.5.", "The 'predict' function of the neural network library is used to calculate the output based on the input array of four numbers."]}], 'duration': 214.145, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw711974.jpg', 'highlights': ['Determining the number of inputs, outputs, and hidden nodes in a neural network (4 inputs, 1 output)', 'Consideration of using trial and error in determining the number of hidden nodes for optimal configuration', 'Reference to a neural network tutorial series and a color predictor project with 3 inputs and 2 outputs', 'The neural network has 4 inputs, 4 hidden nodes, and 1 output', "The 'think' function determines the bird's movement based on the output value with a threshold of 0.5", "The 'predict' function of the neural network library is used to calculate the output based on the input array of 4 numbers"]}, {'end': 1092.159, 'segs': [{'end': 1065.11, 'src': 'embed', 'start': 1021.391, 'weight': 0, 'content': [{'end': 1029.494, 'text': 'Inputs 3 equal pipes index 0 X right So this is the idea here.', 'start': 1021.391, 'duration': 8.103}, {'end': 1033.657, 'text': 'I want all of those things to be the inputs to the neural network.', 'start': 1029.515, 'duration': 4.142}, {'end': 1035.898, 'text': "I don't need this anymore, right?", 'start': 1033.657, 'duration': 2.241}, {'end': 1039.201, 'text': 'because I am getting them from the properties that sing.', 'start': 1035.898, 'duration': 3.303}, {'end': 1042.002, 'text': 'and now let me run this and see what happens.', 'start': 1039.201, 'duration': 2.801}, {'end': 1046.707, 'text': "Yeah, it's kind of jumping all the time, All right, so this didn't really seem to improve.", 'start': 1042.002, 'duration': 4.705}, {'end': 1048.067, 'text': "Ooh, look, ooh, it's doing something.", 'start': 1046.727, 'duration': 1.34}, {'end': 1049.608, 'text': 'That one actually kind of almost did something.', 'start': 1048.188, 'duration': 1.42}, {'end': 1051.648, 'text': "It's making some of its own decisions.", 'start': 1049.648, 'duration': 2}, {'end': 1052.968, 'text': 'That was very exciting.', 'start': 1051.968, 'duration': 1}, {'end': 1055.429, 'text': "All right, well, I've got some problems here.", 'start': 1053.488, 'duration': 1.941}, {'end': 1062.39, 'text': 'Number one is when I sort of made my pretend version of it, I purposely made numbers between 0 and 1.', 'start': 1055.649, 'duration': 6.741}, {'end': 1065.11, 'text': 'So one thing I should really do is normalize all these values.', 'start': 1062.39, 'duration': 2.72}], 'summary': 'Neural network using 3 equal pipe inputs, making own decisions, needs values normalization.', 'duration': 43.719, 'max_score': 1021.391, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw1021391.jpg'}], 'start': 927.16, 'title': 'Optimizing neural network inputs', 'summary': "Explores the process of optimizing inputs for a neural network, discussing the utilization of bird and pipe positions, and the need for data normalization, aiming to enhance decision-making capabilities and observations of the network's performance.", 'chapters': [{'end': 1092.159, 'start': 927.16, 'title': 'Neural network input optimization', 'summary': "Explores the process of optimizing inputs for a neural network, discussing the utilization of bird and pipe positions, and the need for data normalization, in an attempt to enhance decision-making capabilities and observations of the network's performance.", 'duration': 164.999, 'highlights': ["The chapter discusses the process of optimizing inputs for a neural network by utilizing the bird and pipe positions, and the need for data normalization. The author focuses on the utilization of the bird's y position, the pipe's top and bottom positions, and the pipe's x position as inputs for the neural network to enhance its decision-making capabilities.", "The author emphasizes the need for data normalization, intending to scale the values between 0 and 1 to improve the network's performance. The author plans to normalize the values of the bird's y position, the pipe's top and bottom positions, and the pipe's x position by dividing them by the height and width, aiming to enhance the network's decision-making process."]}], 'duration': 164.999, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw927160.jpg', 'highlights': ['The chapter explores optimizing inputs for a neural network using bird and pipe positions.', 'Emphasizes the need for data normalization to scale values between 0 and 1 for improved performance.']}, {'end': 1422.434, 'segs': [{'end': 1141.179, 'src': 'embed', 'start': 1092.74, 'weight': 3, 'content': [{'end': 1093.42, 'text': "Oh, it's doing something.", 'start': 1092.74, 'duration': 0.68}, {'end': 1094.601, 'text': "Oh my god, it's making something.", 'start': 1093.66, 'duration': 0.941}, {'end': 1097.123, 'text': 'Oh, that was a good one, right? That was a pretty good one.', 'start': 1094.861, 'duration': 2.262}, {'end': 1099.785, 'text': "All right, so you can see that it's doing something.", 'start': 1097.964, 'duration': 1.821}, {'end': 1100.946, 'text': "It's trying to think.", 'start': 1099.825, 'duration': 1.121}, {'end': 1102.927, 'text': "But I've got another problem.", 'start': 1101.706, 'duration': 1.221}, {'end': 1103.848, 'text': "I don't know.", 'start': 1103.408, 'duration': 0.44}, {'end': 1111.493, 'text': 'Is pipes0 the right pipe? I think I probably need an algorithm to check which pipe is actually the one closest.', 'start': 1103.868, 'duration': 7.625}, {'end': 1112.394, 'text': 'Let me remember.', 'start': 1111.753, 'duration': 0.641}, {'end': 1113.795, 'text': 'Let me look at the main sketch.', 'start': 1112.414, 'duration': 1.381}, {'end': 1117.637, 'text': 'How do I add the pipes? Yeah, so first of all, the pipes are getting added to the end.', 'start': 1114.115, 'duration': 3.522}, {'end': 1126.543, 'text': 'And if the pipe goes off screen, it gets deleted from its spot, which would typically be the earliest one.', 'start': 1118.758, 'duration': 7.785}, {'end': 1130.826, 'text': 'So it might work.', 'start': 1127.524, 'duration': 3.302}, {'end': 1134.192, 'text': "But let's be smart about this.", 'start': 1132.571, 'duration': 1.621}, {'end': 1141.179, 'text': "Let's just say, this is probably overdoing it, but let's actually find the closest pipe.", 'start': 1134.393, 'duration': 6.786}], 'summary': 'Developing code to identify and manage pipes in a program, exploring the closest pipe algorithm.', 'duration': 48.439, 'max_score': 1092.74, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw1092740.jpg'}, {'end': 1399.146, 'src': 'embed', 'start': 1353.849, 'weight': 0, 'content': [{'end': 1362.896, 'text': 'I have developed a neural network architecture for the bird to read in the state, sort of evaluate the state of the game,', 'start': 1353.849, 'duration': 9.047}, {'end': 1366.178, 'text': 'make a guess as to what it should do, should it jump up, should it not?', 'start': 1362.896, 'duration': 3.282}, {'end': 1368.74, 'text': 'based on that, and then I have put that into motion.', 'start': 1366.178, 'duration': 2.562}, {'end': 1373.896, 'text': 'That neural network is part of the bird object.', 'start': 1369.161, 'duration': 4.735}, {'end': 1383.502, 'text': 'now it has a brain 441, and then it has a think function which sends the data about the game into the neural network and decides to go up or not.', 'start': 1373.896, 'duration': 9.606}, {'end': 1391.524, 'text': 'Now what I need to do is actually implement the neuroevolution part.', 'start': 1384.863, 'duration': 6.661}, {'end': 1393.984, 'text': "So I've implemented the neuro part.", 'start': 1391.904, 'duration': 2.08}, {'end': 1399.146, 'text': "I designed a way for the bird to make a decision based on how it's evaluating the scene.", 'start': 1394.124, 'duration': 5.022}], 'summary': 'Developed a neural network architecture for bird with 441 neurons to evaluate game state and make decisions.', 'duration': 45.297, 'max_score': 1353.849, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw1353849.jpg'}], 'start': 1092.74, 'title': 'Debugging code for pipe algorithm and flappy bird neural network', 'summary': 'Involves debugging a code for a pipe algorithm, ensuring the functionality of adding and deleting pipes off screen, and implementing a quick algorithm using a neural network architecture in a flappy bird game. it also discusses the plan to implement the neuroevolution part with a genetic algorithm.', 'chapters': [{'end': 1141.179, 'start': 1092.74, 'title': 'Debugging code for pipe algorithm', 'summary': 'Involves debugging a code for a pipe algorithm, trying to find the closest pipe and ensuring the functionality of adding and deleting pipes off screen.', 'duration': 48.439, 'highlights': ['The main concern is identifying the right pipe, possibly requiring an algorithm to check which pipe is closest.', 'The process of adding pipes to the end and deleting off-screen pipes is discussed, aiming to find the closest pipe for efficiency.']}, {'end': 1422.434, 'start': 1144.822, 'title': 'Flappy bird neural network', 'summary': 'Discusses implementing a quick algorithm to find the closest pipe in a flappy bird game using a neural network architecture, and the plan to implement the neuroevolution part with a genetic algorithm.', 'duration': 277.612, 'highlights': ['Implemented a system to find the closest pipe in a Flappy Bird game using a quick algorithm based on distance and position, with the use of a neural network architecture for decision-making. The speaker explains the process of finding the closest pipe in the game using an algorithm based on distance and position, and the integration of a neural network architecture for decision-making.', 'Developed a neural network architecture for the bird to evaluate the game state and make decisions, and created a brain and think function within the bird object for decision-making. The speaker details the development of a neural network architecture for the bird to evaluate the game state and make decisions, including the creation of a brain and think function within the bird object.', 'Plan to implement the neuroevolution part with a genetic algorithm to create a population of birds and evolve them over time for decision-making. The speaker discusses the plan to implement the neuroevolution part with a genetic algorithm to create a population of birds and evolve them over time for decision-making.']}], 'duration': 329.694, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/c6y21FkaUqw/pics/c6y21FkaUqw1092740.jpg', 'highlights': ['Implemented a system to find the closest pipe in a Flappy Bird game using a quick algorithm based on distance and position, with the use of a neural network architecture for decision-making.', 'Plan to implement the neuroevolution part with a genetic algorithm to create a population of birds and evolve them over time for decision-making.', 'Developed a neural network architecture for the bird to evaluate the game state and make decisions, and created a brain and think function within the bird object for decision-making.', 'The process of adding pipes to the end and deleting off-screen pipes is discussed, aiming to find the closest pipe for efficiency.', 'The main concern is identifying the right pipe, possibly requiring an algorithm to check which pipe is closest.']}], 'highlights': ['The project involves utilizing a Flappy Bird clone and a toy neural network library, which provides the foundational elements for the implementation of neuroevolution.', 'The chapter discusses the implementation of neuroevolution in a coding challenge, combining neural networks and genetic algorithms to train agents in a simple game.', "The neural network class is enhanced with functionalities like 'copy' and 'mutate' to facilitate the application of a genetic algorithm for making copies and introducing random changes to the neural network.", 'Reinforcement learning involves training a neural network based on a reward-based system, such as the deep Q learning used in the DeepMind bot, which learned to beat Atari games.', 'The concept of reinforcement learning predates deep learning and neural networks, with the potential for future tutorials on the topic mentioned.', 'Creating a population of 1000 birds with random neural networks.', 'Using the principles of genetic algorithms to repopulate the world with 100 new birds based on successful predecessors.', "The bird's decision to jump will be determined by its own neural network, enhancing autonomy and decision-making within the game.", 'The chapter explores optimizing inputs for a neural network using bird and pipe positions.', 'Emphasizes the need for data normalization to scale values between 0 and 1 for improved performance.', 'Implemented a system to find the closest pipe in a Flappy Bird game using a quick algorithm based on distance and position, with the use of a neural network architecture for decision-making.', 'Plan to implement the neuroevolution part with a genetic algorithm to create a population of birds and evolve them over time for decision-making.', 'Developed a neural network architecture for the bird to evaluate the game state and make decisions, and created a brain and think function within the bird object for decision-making.']}