title
A more interesting self-driving neural network model - Python plays GTA p.13

description
Text tutorials and sample code: https://pythonprogramming.net/game-frames-open-cv-python-plays-gta-v/ Training data and trained model: https://pythonprogramming.net/more-interesting-self-driving-python-plays-gta-v/ Project Github: https://github.com/sentdex/pygta5 https://twitter.com/sentdex https://www.facebook.com/pythonprogramming.net/ https://plus.google.com/+sentdex

detail
{'title': 'A more interesting self-driving neural network model - Python plays GTA p.13', 'heatmap': [{'end': 841.089, 'start': 809.362, 'weight': 0.86}, {'end': 891, 'start': 878.727, 'weight': 0.773}], 'summary': 'The self-driving car project has made progress with 300,000 training samples for the ai model, showing improved performance and loss reduction. the frame processing achieved 20 frames per second, enhancing decision-making capabilities, and the car racing commentary highlighted the intensity and unpredictability of the race.', 'chapters': [{'end': 123.559, 'segs': [{'end': 101.532, 'src': 'embed', 'start': 8.525, 'weight': 0, 'content': [{'end': 40.964, 'text': 'Hey! All right, welcome to part 13 of our Grand Theft Auto self-driving car Python tutorial series.', 'start': 8.525, 'duration': 32.439}, {'end': 51.927, 'text': 'So what we have at the moment is the latest version of the self-driving car slash motorcycle slash whatever.', 'start': 41.584, 'duration': 10.343}, {'end': 59.693, 'text': 'so what I want to do is just kind of show what I got to get up to this point.', 'start': 53.468, 'duration': 6.225}, {'end': 64.775, 'text': 'obviously, this is not the best AI in the world, but you know it works.', 'start': 59.693, 'duration': 5.082}, {'end': 68.599, 'text': 'so first of all, let me go ahead and put myself somewhere useful again.', 'start': 64.775, 'duration': 3.824}, {'end': 76.085, 'text': 'let me grab another motorcycle.', 'start': 68.599, 'duration': 7.486}, {'end': 87.605, 'text': 'Hopefully nobody hits me and we need to put on our helmet for safety.', 'start': 82.142, 'duration': 5.463}, {'end': 95.329, 'text': "Okay So let me pause this for now and let's talk about, uh, what we got up to this point.", 'start': 87.625, 'duration': 7.704}, {'end': 97.15, 'text': 'So here is the tensor board for this.', 'start': 95.409, 'duration': 1.741}, {'end': 100.171, 'text': 'Um, basically I got 300,000 training samples.', 'start': 97.97, 'duration': 2.201}, {'end': 101.532, 'text': "That's this purple line here.", 'start': 100.191, 'duration': 1.341}], 'summary': 'The self-driving car project has 300,000 training samples and a working ai model.', 'duration': 93.007, 'max_score': 8.525, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz88525.jpg'}], 'start': 8.525, 'title': 'Self-driving car python tutorial', 'summary': 'Showcases the progress made in the self-driving car project, including 300,000 training samples for the ai model and the presenter expressing the need for improvement.', 'chapters': [{'end': 123.559, 'start': 8.525, 'title': 'Self-driving car python tutorial', 'summary': 'Showcases the progress made in the self-driving car project, including 300,000 training samples for the ai model, with the presenter expressing the need for improvement.', 'duration': 115.034, 'highlights': ['The presenter discusses the tensor board and reveals the utilization of 300,000 training samples for the self-driving car AI model.', 'The presenter highlights the imperfection of the AI and expresses the need for improvement in its capabilities.', 'The presenter demonstrates the process of setting up the environment and acquiring a motorcycle for the project.']}], 'duration': 115.034, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz88525.jpg', 'highlights': ['The presenter discusses the utilization of 300,000 training samples for the self-driving car AI model.', "The presenter expresses the need for improvement in the AI's capabilities.", 'The presenter demonstrates the process of setting up the environment and acquiring a motorcycle for the project.']}, {'end': 706.839, 'segs': [{'end': 192.037, 'src': 'embed', 'start': 127.123, 'weight': 0, 'content': [{'end': 130.644, 'text': 'On TensorBoard now, the purple line is the main one.', 'start': 127.123, 'duration': 3.521}, {'end': 136.247, 'text': 'The blue line is the original just kind of eight epochs of training that just kind of shows you how well it did.', 'start': 130.684, 'duration': 5.563}, {'end': 139.848, 'text': 'Again, just like usual, the end of it seems to do better.', 'start': 136.847, 'duration': 3.001}, {'end': 147.831, 'text': 'But this one also has 300,000 training samples, whereas this one, this blue line, if I recall right, had about 100,000 training samples.', 'start': 139.908, 'duration': 7.923}, {'end': 153.592, 'text': "But then I decided, nah, let's go ahead and shoot for the moon and do about 300,000.", 'start': 148.651, 'duration': 4.941}, {'end': 161.184, 'text': 'so, even as we take away smoothing, we can see, initially it was so volatile, but towards the end it really started to smooth out.', 'start': 153.593, 'duration': 7.591}, {'end': 167.532, 'text': 'so anyway, and then same thing, loss was pretty volatile even all the way up until the very, very end, but smooth it out.', 'start': 161.184, 'duration': 6.348}, {'end': 168.093, 'text': 'it did pretty good.', 'start': 167.532, 'duration': 0.561}, {'end': 170.776, 'text': 'so um, so yeah.', 'start': 169.054, 'duration': 1.722}, {'end': 171.897, 'text': "so that's the training data.", 'start': 170.776, 'duration': 1.121}, {'end': 175.441, 'text': 'i will put a link in the description for my training data.', 'start': 171.897, 'duration': 3.544}, {'end': 177.062, 'text': 'i might also just host the model.', 'start': 175.441, 'duration': 1.621}, {'end': 179.505, 'text': 'the model took a really long time to train.', 'start': 177.062, 'duration': 2.443}, {'end': 186.532, 'text': "it took about, i'm trying to think, maybe, uh, 15 hours, or i guess that's not really a long time, to be honest,", 'start': 179.505, 'duration': 7.027}, {'end': 190.335, 'text': "but it took about 15 hours of just running this and that's on the titan x.", 'start': 186.532, 'duration': 3.803}, {'end': 192.037, 'text': 'so um, so yeah.', 'start': 190.335, 'duration': 1.702}], 'summary': 'Training model with 300,000 samples, 15 hours on titan x, showed improved performance.', 'duration': 64.914, 'max_score': 127.123, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8127123.jpg'}, {'end': 345.687, 'src': 'embed', 'start': 309.699, 'weight': 3, 'content': [{'end': 312.581, 'text': "And kind of what I was hoping to, I still don't have it yet, but..", 'start': 309.699, 'duration': 2.882}, {'end': 314.783, 'text': 'If we could have PID control.', 'start': 313.442, 'duration': 1.341}, {'end': 316.725, 'text': "That's what we need, man.", 'start': 315.744, 'duration': 0.981}, {'end': 318.006, 'text': "That's what we need, everybody.", 'start': 316.865, 'duration': 1.141}, {'end': 319.047, 'text': 'Put your minds to it.', 'start': 318.146, 'duration': 0.901}, {'end': 319.667, 'text': "Let's do it.", 'start': 319.067, 'duration': 0.6}, {'end': 322.55, 'text': "That's the next thing I'm going to personally be working on.", 'start': 320.568, 'duration': 1.982}, {'end': 328.655, 'text': 'And so my thought is that we can take these outputs and actually have these outputs set the PID.', 'start': 322.59, 'duration': 6.065}, {'end': 331.037, 'text': "So in this case, we'd be almost 100% turn..", 'start': 328.735, 'duration': 2.302}, {'end': 335.981, 'text': "Turn left, for example, but in some of these, like you, wouldn't be.", 'start': 333.639, 'duration': 2.342}, {'end': 345.687, 'text': "you'd be like here 73% turn left, 20% forward and then so much turn right, which would probably maybe subtract this from the turn or Just ignore this.", 'start': 335.981, 'duration': 9.706}], 'summary': 'Working on implementing pid control for setting outputs, aiming for precise control.', 'duration': 35.988, 'max_score': 309.699, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8309699.jpg'}, {'end': 396.483, 'src': 'embed', 'start': 371.144, 'weight': 4, 'content': [{'end': 380.311, 'text': "Basically what I've got coded here is just because I was finding that it was maybe sometimes not pressing it long enough and then other times it was pressing it way too much.", 'start': 371.144, 'duration': 9.167}, {'end': 383.233, 'text': "So I've forced a sort of sleep here.", 'start': 380.851, 'duration': 2.382}, {'end': 390.057, 'text': 'So basically, it presses, and it will sleep for that time and hold that for 0.5.', 'start': 383.313, 'duration': 6.744}, {'end': 392.579, 'text': 'zero nine seconds again.', 'start': 390.057, 'duration': 2.522}, {'end': 394.841, 'text': 'i feel like if we had pig control.', 'start': 392.579, 'duration': 2.262}, {'end': 396.483, 'text': 'um, this would handle for the.', 'start': 394.841, 'duration': 1.642}], 'summary': 'Code implemented to address inconsistent button pressing, now includes a 0.09-second sleep to ensure proper control.', 'duration': 25.339, 'max_score': 371.144, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8371144.jpg'}], 'start': 127.123, 'title': 'Training data and model development', 'summary': 'Compares the performance of models trained with 100k and 300k samples, showing better performance and loss reduction with 300k samples. it also covers training data, 15-hour model training time, ai imperfections, pid control implementation, and code adjustments for better performance.', 'chapters': [{'end': 170.776, 'start': 127.123, 'title': 'Training comparison: 100k vs 300k samples', 'summary': 'Compares the performance of a model trained with 100,000 samples against one trained with 300,000 samples, showing improved volatility and smoothing in the latter, leading to better performance and loss reduction.', 'duration': 43.653, 'highlights': ['The model trained with 300,000 samples showed improved smoothing and reduced volatility compared to the model trained with 100,000 samples, leading to better performance and loss reduction. 300,000 samples vs 100,000 samples', 'The blue line representing the model trained with 100,000 samples showed higher volatility, while the purple line of the 300,000 samples model demonstrated better performance. Volatility comparison between 100k and 300k samples models']}, {'end': 706.839, 'start': 170.776, 'title': 'Training data and model development', 'summary': "Discusses the training data, model training time of 15 hours, the ai's imperfections, planned pid control implementation, and the adjustments made to the code for better performance.", 'duration': 536.063, 'highlights': ['The model took approximately 15 hours to train on the Titan X GPU, with the alternative of taking a year on a CPU and RAM.', "The AI's imperfections are observed, including the need for PID control to improve performance.", 'The adjustments made to the code include forcing a sleep time to account for the human limitation in pressing keys, and the need to clear memory to prevent program failures.']}], 'duration': 579.716, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8127123.jpg', 'highlights': ['The model trained with 300,000 samples showed improved smoothing and reduced volatility compared to the model trained with 100,000 samples, leading to better performance and loss reduction. 300,000 samples vs 100,000 samples', 'The blue line representing the model trained with 100,000 samples showed higher volatility, while the purple line of the 300,000 samples model demonstrated better performance. Volatility comparison between 100k and 300k samples models', 'The model took approximately 15 hours to train on the Titan X GPU, with the alternative of taking a year on a CPU and RAM.', "The AI's imperfections are observed, including the need for PID control to improve performance.", 'The adjustments made to the code include forcing a sleep time to account for the human limitation in pressing keys, and the need to clear memory to prevent program failures.']}, {'end': 951.177, 'segs': [{'end': 739.783, 'src': 'embed', 'start': 708.797, 'weight': 0, 'content': [{'end': 710.139, 'text': 'Okay, and we are back.', 'start': 708.797, 'duration': 1.342}, {'end': 716.085, 'text': "Okay, now with the new grab screen function, it's 0.06 about-ish.", 'start': 710.639, 'duration': 5.446}, {'end': 718.328, 'text': 'Pressing T again.', 'start': 717.547, 'duration': 0.781}, {'end': 725.716, 'text': "This isn't actually the one I want.", 'start': 724.435, 'duration': 1.281}, {'end': 727.779, 'text': 'I like the one that sleeps better, but anyway.', 'start': 725.757, 'duration': 2.022}, {'end': 736.619, 'text': 'Okay, so with the repaired grab screen, we can process frames at more like 20 frames a second or so.', 'start': 729.051, 'duration': 7.568}, {'end': 739.783, 'text': "So yeah, so that's that.", 'start': 736.719, 'duration': 3.064}], 'summary': 'Improved grab screen function achieves processing 20 frames/second.', 'duration': 30.986, 'max_score': 708.797, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8708797.jpg'}, {'end': 841.089, 'src': 'heatmap', 'start': 763.682, 'weight': 1, 'content': [{'end': 766.563, 'text': "If the forward threshold is above 70, we're just going to go forward.", 'start': 763.682, 'duration': 2.881}, {'end': 774.945, 'text': 'Otherwise, as long as the turn threshold suggestion is basically above or the turn threshold is beyond 0.75, so like 75%,', 'start': 766.903, 'duration': 8.042}, {'end': 783.484, 'text': "certain that it wants to take a turn, Then we'll take a turn, but we don't want to take a turn if we're only like 60% certain or something like that.", 'start': 774.945, 'duration': 8.539}, {'end': 786.267, 'text': 'So let me bring back the sleep time.', 'start': 784.325, 'duration': 1.942}, {'end': 788.77, 'text': 'Save that.', 'start': 788.149, 'duration': 0.621}, {'end': 793.755, 'text': "And probably this, I don't even know what we'll have to do to get this to come back.", 'start': 789.991, 'duration': 3.764}, {'end': 798.721, 'text': 'But anyway, I just wanted to show that it is at least 20 frames a second.', 'start': 793.835, 'duration': 4.886}, {'end': 805.398, 'text': "okay, and now for your viewing pleasure, let's just let this thing run for a little bit.", 'start': 799.693, 'duration': 5.705}, {'end': 806.819, 'text': "it's not the best.", 'start': 805.398, 'duration': 1.421}, {'end': 809.362, 'text': "uh, let's see how it does so.", 'start': 806.819, 'duration': 2.543}, {'end': 815.287, 'text': "t so this doesn't end.", 'start': 809.362, 'duration': 5.925}, {'end': 817.069, 'text': "well, he's got to get out of this lane.", 'start': 815.287, 'duration': 1.782}, {'end': 830.702, 'text': 'there we go, oh man, Oh gosh.', 'start': 817.069, 'duration': 13.633}, {'end': 834.665, 'text': 'Oh, my gosh.', 'start': 831.583, 'duration': 3.082}, {'end': 835.966, 'text': 'I think he got just lucky there.', 'start': 834.685, 'duration': 1.281}, {'end': 841.089, 'text': 'But, yeah, really wants to eat that meaty in there.', 'start': 836.626, 'duration': 4.463}], 'summary': 'Decision-making thresholds for forward and turn based on quantifiable data, with 20 frames per second performance.', 'duration': 67.02, 'max_score': 763.682, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8763682.jpg'}, {'end': 909.006, 'src': 'heatmap', 'start': 878.727, 'weight': 0.773, 'content': [{'end': 880.548, 'text': "And I'm sure that was on purpose.", 'start': 878.727, 'duration': 1.821}, {'end': 891, 'text': 'Miss him.', 'start': 890.56, 'duration': 0.44}, {'end': 892.081, 'text': 'Thank.. Oh, okay.', 'start': 891.2, 'duration': 0.881}, {'end': 898.443, 'text': "Oh, what? I think he's trying to hit every vehicle on the road.", 'start': 895.762, 'duration': 2.681}, {'end': 903.844, 'text': 'Sometimes I think he just..', 'start': 902.544, 'duration': 1.3}, {'end': 904.765, 'text': 'Oh, that was a mistake.', 'start': 903.844, 'duration': 0.921}, {'end': 906.045, 'text': "That's not good.", 'start': 905.185, 'duration': 0.86}, {'end': 909.006, 'text': 'Oh Whew.', 'start': 906.065, 'duration': 2.941}], 'summary': 'Conversation about driving, including potential accidents and mistakes.', 'duration': 30.279, 'max_score': 878.727, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8878727.jpg'}, {'end': 951.177, 'src': 'embed', 'start': 939, 'weight': 4, 'content': [{'end': 943.603, 'text': 'Why are you in the oncoming traffic? Get out of the oncoming.', 'start': 939, 'duration': 4.603}, {'end': 951.177, 'text': 'Oh! Oh! Questions, comments, leave them below.', 'start': 944.303, 'duration': 6.874}], 'summary': 'Urgent warning to get out of oncoming traffic.', 'duration': 12.177, 'max_score': 939, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8939000.jpg'}], 'start': 708.797, 'title': 'Improved frame processing', 'summary': 'Discusses achieving 20 frames per second in frame processing, using repaired grab screen function, with specific turn thresholds set at 70% for forward movement and 75% for taking a turn, enhancing decision-making capabilities and also highlights a car racing commentary with around 20 frames per second, featuring commentary on the race, including near misses and potential accidents, emphasizing the intensity and unpredictability of the race.', 'chapters': [{'end': 788.77, 'start': 708.797, 'title': 'Improving frame processing speed', 'summary': 'Discusses the improved frame processing speed of 20 frames per second using the repaired grab screen function, with specific turn thresholds set at 70% for forward movement and 75% for taking a turn, enhancing decision-making capabilities.', 'duration': 79.973, 'highlights': ['The repaired grab screen function allows processing frames at a speed of 20 frames per second, increasing efficiency.', 'The turn threshold for taking a turn is set at 75%, ensuring a high level of certainty before making a turn decision.', 'The forward threshold is set at 70%, indicating the decision to move forward when this threshold is exceeded.']}, {'end': 951.177, 'start': 789.991, 'title': 'Car racing commentary', 'summary': 'Highlights a car racing commentary with around 20 frames per second, featuring commentary on the race, including near misses and potential accidents, emphasizing the intensity and unpredictability of the race.', 'duration': 161.186, 'highlights': ['The commentary showcases a car race with at least 20 frames per second, emphasizing the speed and intensity of the race.', 'The commentator provides real-time reactions to the race, expressing surprise and concern over near misses and potential accidents on the track.', 'The commentary includes moments of suspense and unpredictability, creating an immersive experience for viewers as they witness the race unfold.']}], 'duration': 242.38, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/nWJZ4w0HKz8/pics/nWJZ4w0HKz8708797.jpg', 'highlights': ['The repaired grab screen function allows processing frames at a speed of 20 frames per second, increasing efficiency.', 'The commentary showcases a car race with at least 20 frames per second, emphasizing the speed and intensity of the race.', 'The turn threshold for taking a turn is set at 75%, ensuring a high level of certainty before making a turn decision.', 'The forward threshold is set at 70%, indicating the decision to move forward when this threshold is exceeded.', 'The commentator provides real-time reactions to the race, expressing surprise and concern over near misses and potential accidents on the track.', 'The commentary includes moments of suspense and unpredictability, creating an immersive experience for viewers as they witness the race unfold.']}], 'highlights': ['The model trained with 300,000 samples showed improved smoothing and reduced volatility compared to the model trained with 100,000 samples, leading to better performance and loss reduction. 300,000 samples vs 100,000 samples', 'The repaired grab screen function allows processing frames at a speed of 20 frames per second, increasing efficiency.', 'The frame processing achieved 20 frames per second, enhancing decision-making capabilities, and the car racing commentary highlighted the intensity and unpredictability of the race.', 'The presenter discusses the utilization of 300,000 training samples for the self-driving car AI model.', 'The presenter demonstrates the process of setting up the environment and acquiring a motorcycle for the project.', 'The blue line representing the model trained with 100,000 samples showed higher volatility, while the purple line of the 300,000 samples model demonstrated better performance. Volatility comparison between 100k and 300k samples models', 'The turn threshold for taking a turn is set at 75%, ensuring a high level of certainty before making a turn decision.', 'The forward threshold is set at 70%, indicating the decision to move forward when this threshold is exceeded.', 'The commentary includes moments of suspense and unpredictability, creating an immersive experience for viewers as they witness the race unfold.']}