title
Coding Challenge #158: Shape Classifier Neural Network with ml5.js

description
In this challenge, I demonstrate the entire process of training and deploying a machine learning classification model in JavaScript -- data collection, model training, and prediction! 💻 Code: https://thecodingtrain.com/challenges/158-shape-classifier 🎥 ml5.js: DoodleNet: https://youtu.be/ABN_DWnM5GQ Links discussed in this video: 🔗 Processing: https://processing.org/ 🔗 p5.js Web Editor: https://editor.p5js.org/ 🔗 ml5.js Loading a pre trained model: https://learn.ml5js.org/#/reference/neural-network?id=loading-a-pre-trained-model 🔗 p5.js copy() reference: https://p5js.org/reference/#/p5.Image/copy 🔗 DoodleNet (GitHub): https://github.com/yining1023/doodleNet Other videos mentioned in this video: 🎥 Workflow Series: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6Zu_uqEA6NqhLzKLACwU74X 🎥 ml5.js: Training a CNN for Image Classification: https://youtu.be/hWurN0XhzLY 🎥 ml5.js: Train a Neural Network with Pixels as Input: https://youtu.be/UaKab6h9Z0I 🎥 ml5.js: Train Your Own Neural Network: https://youtu.be/8HEgeAbYphA 🎥 ml5.js: What is a Convolutional Neural Network?: https://youtu.be/qPKsVAI_W6M 🎥 ml5.js: Save Neural Network Trained Model: https://youtu.be/wUrg9Hjkhg0 🎥 Quick, Draw! - Coding Challenge #122: https://youtu.be/yLuk0twx8Hc Timestamps: 0:00 Introduction 1:33 Generating the dataset in Processing 5:50 Loading images in p5.js 8:34 Create a ml5 neural network 10:54 Adding the data 12:39 About training the model 13:53 Test training 15:21 Training with full dataset 16:44 Improving the dataset 18:20 Saved model 19:01 Separate sketch for prediction 19:40 Loading the model 21:13 Testing the model with shapes drawn in p5.js 24:41 Mouse drawn shapes 31:01 Using a webcam and a notebook 33:49 Wrap up 34:33 What's next? 🚂 Website: http://thecodingtrain.com/ 👾 Share Your Creation! https://thecodingtrain.com/Guides/community-contribution-guide.html 🚩 Suggest Topics: https://github.com/CodingTrain/Rainbow-Topics 💡 GitHub: https://github.com/CodingTrain 💬 Discord: https://discord.gg/hPuGy2g 💖 Membership: http://youtube.com/thecodingtrain/join 🛒 Store: https://standard.tv/codingtrain 📚 Books: https://www.amazon.com/shop/thecodingtrain 🖋️ Twitter: https://twitter.com/thecodingtrain 📸 Instagram: https://www.instagram.com/the.coding.train/ 🎥 Coding Challenges: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZiZxtDDRCi6uhfTH4FilpH 🎥 Intro to Programming: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6Zy51Q-x9tMWIv9cueOFTFA 🔗 p5.js: https://p5js.org 🔗 p5.js Web Editor: https://editor.p5js.org/ 🔗 Processing: https://processing.org 📄 Code of Conduct: https://github.com/CodingTrain/Code-of-Conduct

detail
{'title': 'Coding Challenge #158: Shape Classifier Neural Network with ml5.js', 'heatmap': [{'end': 108.833, 'start': 85.598, 'weight': 1}, {'end': 174.586, 'start': 146.928, 'weight': 0.724}, {'end': 372.957, 'start': 345.829, 'weight': 0.821}, {'end': 521.893, 'start': 453.981, 'weight': 0.897}, {'end': 1191.458, 'start': 1167.469, 'weight': 0.722}, {'end': 1645.297, 'start': 1621.138, 'weight': 0.882}, {'end': 1800.01, 'start': 1772.95, 'weight': 0.747}], 'summary': 'Using ml5.js, the video showcases a p5.js sketch employing a convolutional neural network model to recognize shapes, achieving 75% confidence in identifying circles, 76% for triangles, and accurate identification of squares, while addressing model flaws. it covers the process of generating a data set for training using code and tools such as processing and p5.js, involving a total of 300 low-resolution images and a neural network with dimensions of 128x128x4. the chapter also covers configuring a convolutional neural network for image classification using ml5, addressing normalization of data, training code, setting training options, and debugging issues. the exploration includes training a convolutional neural network over 50 epochs, encountering slow running time and fluctuating loss, leading to a decision to lower image resolution for faster training, and details deploying a pre-trained model to a web application using ml5, achieving 99% accuracy for identifying a circle and 98% accuracy for identifying a square. it also covers resizing images in p5, drawing and classifying images with user inputs, and displaying machine learning results on a web page, using techniques such as resizing, drawing with interactions, and formatting confidence percentages. additionally, it discusses optimizing javascript graphics for improved performance and user interaction, and demonstrates the process of training a neural network to recognize hand-drawn shapes with 97% accuracy for squares and 91% for triangles, introducing the possibility of recognizing more complex drawings using a pre-trained model.', 'chapters': [{'end': 64.768, 'segs': [{'end': 64.768, 'src': 'embed', 'start': 19.336, 'weight': 0, 'content': [{'end': 28.281, 'text': 'I have a p5.js sketch that is loading a neural network, a convolutional neural network model that I trained myself in this video with my own data.', 'start': 19.336, 'duration': 8.945}, {'end': 30.282, 'text': 'to recognize shapes.', 'start': 28.601, 'duration': 1.681}, {'end': 33.125, 'text': 'It only knows about circles, squares, and triangles.', 'start': 30.563, 'duration': 2.562}, {'end': 36.327, 'text': 'But I have this document camera here, live and running.', 'start': 33.445, 'duration': 2.882}, {'end': 37.288, 'text': 'You can see my hand there.', 'start': 36.347, 'duration': 0.941}, {'end': 38.769, 'text': 'I just drew that circle.', 'start': 37.588, 'duration': 1.181}, {'end': 42.432, 'text': 'And it is guessing with a 75% confidence that it is a circle.', 'start': 38.869, 'duration': 3.563}, {'end': 44.614, 'text': "Let's now see some other shapes.", 'start': 42.773, 'duration': 1.841}, {'end': 48.497, 'text': 'Here is a triangle, 76%.', 'start': 44.754, 'duration': 3.743}, {'end': 49.859, 'text': 'And I can switch this over.', 'start': 48.497, 'duration': 1.362}, {'end': 51.42, 'text': 'And I can look at this square.', 'start': 50.299, 'duration': 1.121}, {'end': 53.481, 'text': "And boy, is it confident that that's a square.", 'start': 51.52, 'duration': 1.961}, {'end': 56.184, 'text': 'I can even kind of turn it sideways.', 'start': 53.602, 'duration': 2.582}, {'end': 58.145, 'text': "Ah, it thinks it's a circle now.", 'start': 56.944, 'duration': 1.201}, {'end': 59.465, 'text': 'So there are some flaws.', 'start': 58.205, 'duration': 1.26}, {'end': 64.768, 'text': 'And you will see why those flaws might be there as you watch this video.', 'start': 59.726, 'duration': 5.042}], 'summary': 'A p5.js sketch uses a self-trained convolutional neural network to recognize shapes, achieving 75-76% accuracy.', 'duration': 45.432, 'max_score': 19.336, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E19336.jpg'}], 'start': 1.465, 'title': '2020 train whistle coding challenge', 'summary': "Showcases a p5.js sketch employing a convolutional neural network model to recognize shapes, achieving 75% confidence in identifying circles, 76% for triangles, and accurate identification of squares, while highlighting the model's flaws.", 'chapters': [{'end': 64.768, 'start': 1.465, 'title': '2020 train whistle coding challenge', 'summary': 'Showcases a p5.js sketch using a convolutional neural network model to recognize shapes, achieving 75% confidence in identifying circles, 76% for triangles, and accurate identification of squares, while also highlighting the flaws in the model.', 'duration': 63.303, 'highlights': ['The p5.js sketch utilizes a convolutional neural network model to recognize shapes, achieving 75% confidence in identifying circles, 76% for triangles, and accurate identification of squares.', "The neural network model was trained with the presenter's own data to recognize circles, squares, and triangles.", "The model's identification of shapes exhibits flaws, as demonstrated by the instance where turning a square sideways led to a mistaken identification as a circle."]}], 'duration': 63.303, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1465.jpg', 'highlights': ['The p5.js sketch employs a convolutional neural network model to recognize shapes with 75% confidence for circles and 76% for triangles.', "The neural network model was trained with the presenter's own data to recognize circles, squares, and triangles.", "The model's identification of shapes exhibits flaws, as demonstrated by the instance where turning a square sideways led to a mistaken identification as a circle."]}, {'end': 605.229, 'segs': [{'end': 108.833, 'src': 'heatmap', 'start': 65.007, 'weight': 2, 'content': [{'end': 69.65, 'text': 'So what do you need before you get started? I use Processing.', 'start': 65.007, 'duration': 4.643}, {'end': 72.292, 'text': 'You can download that at processing.org.', 'start': 69.89, 'duration': 2.402}, {'end': 74.133, 'text': 'I use p5.js.', 'start': 72.732, 'duration': 1.401}, {'end': 78.194, 'text': 'You can get the web editor at editor.p5js.org.', 'start': 74.773, 'duration': 3.421}, {'end': 81.576, 'text': "And I'm also working with p5.js locally on my computer,", 'start': 78.495, 'duration': 3.081}, {'end': 85.118, 'text': "which is something you might want to check out my workflow series if that's unfamiliar to you.", 'start': 81.576, 'duration': 3.542}, {'end': 86.739, 'text': 'See you at the end of this video.', 'start': 85.598, 'duration': 1.141}, {'end': 87.94, 'text': "No, I won't see you at the end.", 'start': 87.019, 'duration': 0.921}, {'end': 89.361, 'text': "I mean, I won't see you anyway.", 'start': 88.08, 'duration': 1.281}, {'end': 90.281, 'text': "I don't know what's going on.", 'start': 89.561, 'duration': 0.72}, {'end': 92.763, 'text': 'Enjoy this coding challenge.', 'start': 90.501, 'duration': 2.262}, {'end': 95.785, 'text': 'The first step of this project is collecting a data set.', 'start': 93.143, 'duration': 2.642}, {'end': 99.147, 'text': 'I could certainly draw shapes myself and take pictures of them,', 'start': 95.945, 'duration': 3.202}, {'end': 103.65, 'text': 'but a quick way I can generate a data set for training is just generating it with code.', 'start': 99.147, 'duration': 4.503}, {'end': 108.833, 'text': "So in this case, I'm going to use processing, a Java-based programming environment that I use in a lot of my videos.", 'start': 104.05, 'duration': 4.783}], 'summary': 'Use processing and p5.js for coding challenges and data generation. check out workflow series for local setup.', 'duration': 38.643, 'max_score': 65.007, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E65007.jpg'}, {'end': 174.586, 'src': 'heatmap', 'start': 146.928, 'weight': 0.724, 'content': [{'end': 151.632, 'text': "One is I want to make sure the circle doesn't bleed off of the edges of the canvas itself.", 'start': 146.928, 'duration': 4.704}, {'end': 154.514, 'text': 'So I can adjust its xy position based on its size.', 'start': 152.012, 'duration': 2.502}, {'end': 158.457, 'text': 'And I want them to be darker just to make them sort of higher contrast for my training data set.', 'start': 154.754, 'duration': 3.703}, {'end': 161.98, 'text': 'Great Generating lots of circles.', 'start': 160.539, 'duration': 1.441}, {'end': 164.362, 'text': 'Then I just need to save them to the hard drive.', 'start': 162.18, 'duration': 2.182}, {'end': 171.725, 'text': "The save frame function will save me a bunch of PNGs to a folder that I'm calling data.", 'start': 167.523, 'duration': 4.202}, {'end': 174.586, 'text': 'And then I can use these pound symbols.', 'start': 172.105, 'duration': 2.481}], 'summary': "Adjust circle position, create darker circles, generate and save pngs to 'data' folder.", 'duration': 27.658, 'max_score': 146.928, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E146928.jpg'}, {'end': 374.96, 'src': 'heatmap', 'start': 343.488, 'weight': 1, 'content': [{'end': 345.569, 'text': 'And here are a bunch of triangles.', 'start': 343.488, 'duration': 2.081}, {'end': 347.189, 'text': "Awesome I've collected my data.", 'start': 345.829, 'duration': 1.36}, {'end': 349.05, 'text': 'How long did that take? Not too long.', 'start': 347.229, 'duration': 1.821}, {'end': 349.57, 'text': 'Not too long.', 'start': 349.11, 'duration': 0.46}, {'end': 351.211, 'text': "I'm ready to start training the model.", 'start': 349.61, 'duration': 1.601}, {'end': 353.852, 'text': 'Now I have two projects.', 'start': 351.491, 'duration': 2.361}, {'end': 358.513, 'text': 'I have this processing sketch that generated a lot of shapes and put them in a data folder.', 'start': 354.012, 'duration': 4.501}, {'end': 363.235, 'text': "I now have a blank, empty p5.js sketch that's just creating a canvas.", 'start': 358.893, 'duration': 4.342}, {'end': 372.957, 'text': 'And a really important thing about that sketch is that index.html is importing also the ml5 library, at least version 0.6.0.', 'start': 364.115, 'duration': 8.842}, {'end': 374.96, 'text': "I'll note here that I'm working in Visual Studio Code.", 'start': 372.958, 'duration': 2.002}], 'summary': 'Collected data for training model, using p5.js and ml5 library in visual studio code.', 'duration': 31.472, 'max_score': 343.488, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E343488.jpg'}, {'end': 521.893, 'src': 'heatmap', 'start': 453.981, 'weight': 0.897, 'content': [{'end': 458.224, 'text': 'Take i, always render it with four digits, no decimal places.', 'start': 453.981, 'duration': 4.243}, {'end': 460.046, 'text': "And then I'll call that index.", 'start': 458.805, 'duration': 1.241}, {'end': 464.035, 'text': "Oh, and I'm taking i plus 1.", 'start': 461.414, 'duration': 2.621}, {'end': 466.955, 'text': "And let's just draw one of those images to make sure it's working.", 'start': 464.035, 'duration': 2.92}, {'end': 473.837, 'text': 'And how am I going to look at this code in the browser? I need to run a local web server.', 'start': 469.516, 'duration': 4.321}, {'end': 479.018, 'text': "I'm using a global node module HTTP server to do that.", 'start': 473.857, 'duration': 5.161}, {'end': 481.979, 'text': 'But Visual Studio Code has live server plugins.', 'start': 479.679, 'duration': 2.3}, {'end': 483.34, 'text': 'So many different ways you could do this.', 'start': 482.039, 'duration': 1.301}, {'end': 484.68, 'text': "Let's take a look at it.", 'start': 483.84, 'duration': 0.84}, {'end': 487.044, 'text': 'And there we go.', 'start': 486.383, 'duration': 0.661}, {'end': 488.305, 'text': 'It looks a little pixelated.', 'start': 487.064, 'duration': 1.241}, {'end': 494.27, 'text': "That's because, remember, the data set I collected was very low resolution images, 128 by 128.", 'start': 488.345, 'duration': 5.925}, {'end': 502.397, 'text': 'Ultimately, to train the machine learning model in this context, I want to use low resolution images to have less data storage, faster results.', 'start': 494.27, 'duration': 8.127}, {'end': 506.941, 'text': 'And ultimately, the detail of a high resolution circle is not super meaningful here.', 'start': 502.737, 'duration': 4.204}, {'end': 508.963, 'text': 'Just taking a peek at a square also.', 'start': 507.261, 'duration': 1.702}, {'end': 511.228, 'text': 'There we go.', 'start': 510.848, 'duration': 0.38}, {'end': 512.269, 'text': "There's one of my squares.", 'start': 511.268, 'duration': 1.001}, {'end': 512.568, 'text': 'All right.', 'start': 512.349, 'duration': 0.219}, {'end': 513.609, 'text': "I think we're in pretty good shape.", 'start': 512.629, 'duration': 0.98}, {'end': 518.611, 'text': 'Next step, create a neural network, an ml5 neural network.', 'start': 514.309, 'duration': 4.302}, {'end': 521.893, 'text': "Let's call it shape classifier.", 'start': 518.852, 'duration': 3.041}], 'summary': 'Using low-res images for faster ml model training with ml5 neural network.', 'duration': 67.912, 'max_score': 453.981, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E453981.jpg'}, {'end': 518.611, 'src': 'embed', 'start': 494.27, 'weight': 0, 'content': [{'end': 502.397, 'text': 'Ultimately, to train the machine learning model in this context, I want to use low resolution images to have less data storage, faster results.', 'start': 494.27, 'duration': 8.127}, {'end': 506.941, 'text': 'And ultimately, the detail of a high resolution circle is not super meaningful here.', 'start': 502.737, 'duration': 4.204}, {'end': 508.963, 'text': 'Just taking a peek at a square also.', 'start': 507.261, 'duration': 1.702}, {'end': 511.228, 'text': 'There we go.', 'start': 510.848, 'duration': 0.38}, {'end': 512.269, 'text': "There's one of my squares.", 'start': 511.268, 'duration': 1.001}, {'end': 512.568, 'text': 'All right.', 'start': 512.349, 'duration': 0.219}, {'end': 513.609, 'text': "I think we're in pretty good shape.", 'start': 512.629, 'duration': 0.98}, {'end': 518.611, 'text': 'Next step, create a neural network, an ml5 neural network.', 'start': 514.309, 'duration': 4.302}], 'summary': 'Using low-resolution images for training ml model, prioritizing faster results over high-resolution details.', 'duration': 24.341, 'max_score': 494.27, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E494270.jpg'}, {'end': 566.358, 'src': 'embed', 'start': 539.036, 'weight': 5, 'content': [{'end': 543.421, 'text': 'Probably the most important property here is the shape of the input data.', 'start': 539.036, 'duration': 4.385}, {'end': 544.863, 'text': 'What do I mean by shape??', 'start': 543.481, 'duration': 1.382}, {'end': 553.332, 'text': 'Well, in this case, for doing an image classification problem with a convolutional neural network.', 'start': 548.707, 'duration': 4.625}, {'end': 558.475, 'text': 'The dimensions of the image are fundamental to how the neural network deals with the data.', 'start': 554.133, 'duration': 4.342}, {'end': 561.816, 'text': 'What is the width of the image? What is the height? This is easy here.', 'start': 558.875, 'duration': 2.941}, {'end': 562.477, 'text': 'I know what it is.', 'start': 561.896, 'duration': 0.581}, {'end': 563.397, 'text': 'I made this data.', 'start': 562.537, 'duration': 0.86}, {'end': 566.358, 'text': "It's 128 by 128.", 'start': 563.697, 'duration': 2.661}], 'summary': 'Input data shape is crucial for image classification. image dimensions are 128x128.', 'duration': 27.322, 'max_score': 539.036, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E539036.jpg'}], 'start': 65.007, 'title': 'Data set generation and image classifier training', 'summary': 'Covers the process of generating a data set for training using code and tools such as processing and p5.js, and creating a data set of shapes, including circles, squares, and triangles. it involves a total of 300 low-resolution images and a neural network with dimensions of 128x128x4.', 'chapters': [{'end': 103.65, 'start': 65.007, 'title': 'Data set generation with code', 'summary': 'Discusses the process of generating a data set for training using code, using tools such as processing, p5.js, and a local computer workflow series.', 'duration': 38.643, 'highlights': ['The first step of the project involves collecting a data set for training, and one efficient method is to generate it using code.', 'The tools mentioned for this purpose include Processing, p5.js, and a local computer workflow series.', 'Processing and p5.js are recommended tools for data set generation, and the web editor for p5.js can be accessed at editor.p5js.org.']}, {'end': 605.229, 'start': 104.05, 'title': 'Creating and training image classifier', 'summary': 'Details the process of creating a data set of shapes, including circles, squares, and triangles, using processing, and then training a machine learning model with ml5.js, involving a total of 300 low-resolution images and a neural network with dimensions of 128x128x4.', 'duration': 501.179, 'highlights': ["Creating a data set of shapes with Processing The speaker demonstrates using Processing to generate a data set of 100 circles, squares, and triangles, and saving them as PNG images in a 'data' folder.", 'Training a machine learning model with ml5.js The process involves creating a neural network with dimensions of 128x128x4 to classify the low-resolution images of shapes, including circles, squares, and triangles, totaling 300 images in the data set.', 'Adjusting the image dimensions for the neural network The speaker explains the importance of specifying the shape of the input data for the convolutional neural network, which includes dimensions of 128x128x4, representing the width, height, and channels of the low-resolution images.']}], 'duration': 540.222, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E65007.jpg', 'highlights': ['The process involves creating a neural network with dimensions of 128x128x4 to classify the low-resolution images of shapes, including circles, squares, and triangles, totaling 300 images in the data set.', "The speaker demonstrates using Processing to generate a data set of 100 circles, squares, and triangles, and saving them as PNG images in a 'data' folder.", 'Processing and p5.js are recommended tools for data set generation, and the web editor for p5.js can be accessed at editor.p5js.org.', 'The first step of the project involves collecting a data set for training, and one efficient method is to generate it using code.', 'The tools mentioned for this purpose include Processing, p5.js, and a local computer workflow series.', 'The speaker explains the importance of specifying the shape of the input data for the convolutional neural network, which includes dimensions of 128x128x4, representing the width, height, and channels of the low-resolution images.']}, {'end': 895.164, 'segs': [{'end': 697.325, 'src': 'embed', 'start': 627.033, 'weight': 0, 'content': [{'end': 630.374, 'text': "But I'm just going to use the default ML5 settings and see if they work OK.", 'start': 627.033, 'duration': 3.341}, {'end': 634.114, 'text': 'Finally, I want to set debug to true.', 'start': 632.213, 'duration': 1.901}, {'end': 639.076, 'text': "That will trigger an automatic visualization of the loss function as it's training.", 'start': 634.214, 'duration': 4.862}, {'end': 644.198, 'text': 'What is the loss function? I will get to that once we see it graphed on screen.', 'start': 639.176, 'duration': 5.022}, {'end': 647.62, 'text': 'I almost thought it was done, right? I configured my neural network.', 'start': 644.658, 'duration': 2.962}, {'end': 648.84, 'text': "Here's the shape of the data.", 'start': 647.72, 'duration': 1.12}, {'end': 650.141, 'text': "Here's the task I want to do.", 'start': 649.1, 'duration': 1.041}, {'end': 652.302, 'text': "And make sure you show me how it's going while you're running.", 'start': 650.181, 'duration': 2.121}, {'end': 654.564, 'text': 'But I have to add the data.', 'start': 652.842, 'duration': 1.722}, {'end': 660.449, 'text': 'I have to tell the neural network, this is the data you should be working with when you run your training process.', 'start': 655.144, 'duration': 5.305}, {'end': 664.493, 'text': "Luckily for me, there's a function in ml5 called addData.", 'start': 661.069, 'duration': 3.424}, {'end': 668.216, 'text': 'And what do I add? I add the image and its associated label.', 'start': 664.773, 'duration': 3.443}, {'end': 669.217, 'text': "Let's look at how that works.", 'start': 668.296, 'duration': 0.921}, {'end': 682.1, 'text': 'The training data set should include the input associated with its target, the image associated with its label.', 'start': 676.018, 'duration': 6.082}, {'end': 686.501, 'text': 'So I can configure an object with a image property that is this actual circle image.', 'start': 682.14, 'duration': 4.361}, {'end': 688.682, 'text': 'ml5 knows about p5 images.', 'start': 686.822, 'duration': 1.86}, {'end': 695.444, 'text': 'ml5 can work with raw pixel data, but this is super convenient here that I could just give it the p5 image I loaded.', 'start': 689.382, 'duration': 6.062}, {'end': 697.325, 'text': 'And then the label, of course, is a string.', 'start': 695.804, 'duration': 1.521}], 'summary': 'Configured neural network with training data and labels for visualization.', 'duration': 70.292, 'max_score': 627.033, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E627033.jpg'}, {'end': 755.518, 'src': 'embed', 'start': 726.227, 'weight': 2, 'content': [{'end': 730.209, 'text': 'Oh, triangle, why do you have to have a different number of characters than circles and squares?', 'start': 726.227, 'duration': 3.982}, {'end': 735.03, 'text': 'Another really important step here is to call NormalizeData.', 'start': 730.789, 'duration': 4.241}, {'end': 738.612, 'text': 'This is one of the ultimate conveniences of working with ML5 library.', 'start': 735.29, 'duration': 3.322}, {'end': 748.615, 'text': 'It will analyze your data, look at the minimums and maximum ranges of pixel values, and then normalize all the inputs to numbers between 0 and 1.', 'start': 738.872, 'duration': 9.743}, {'end': 750.156, 'text': 'Certainly, we could do this manually.', 'start': 748.615, 'duration': 1.541}, {'end': 752.177, 'text': "I've demonstrated that in other videos.", 'start': 750.216, 'duration': 1.961}, {'end': 755.518, 'text': "But it's a nice convenience that ML5 offers to you.", 'start': 752.457, 'duration': 3.061}], 'summary': 'Ml5 library offers convenient data normalization to numbers between 0 and 1, saving manual effort.', 'duration': 29.291, 'max_score': 726.227, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E726227.jpg'}, {'end': 804.367, 'src': 'embed', 'start': 776.359, 'weight': 3, 'content': [{'end': 778.36, 'text': 'But what a nice confluence of events.', 'start': 776.359, 'duration': 2.001}, {'end': 780.721, 'text': 'The coding train trains the model.', 'start': 778.86, 'duration': 1.861}, {'end': 784.882, 'text': 'The arguments that I need are any training options.', 'start': 781.581, 'duration': 3.301}, {'end': 787.642, 'text': "So there's lots of properties and parameters I can set during training.", 'start': 784.922, 'duration': 2.72}, {'end': 791.063, 'text': 'Those are covered in more detail in other ML5 videos that I have.', 'start': 787.662, 'duration': 3.401}, {'end': 794.324, 'text': 'In this case, the only one I really want to specify is epochs.', 'start': 791.323, 'duration': 3.001}, {'end': 797.601, 'text': "And I'm going to say 50.", 'start': 796.34, 'duration': 1.261}, {'end': 804.367, 'text': '50 means I want to send all of the data, all 300 images, in through the neural network 50 times.', 'start': 797.601, 'duration': 6.766}], 'summary': 'The model is trained with 300 images 50 times by setting epochs to 50.', 'duration': 28.008, 'max_score': 776.359, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E776359.jpg'}], 'start': 605.529, 'title': 'Configuring and training image classification model with ml5', 'summary': 'Covers configuring a convolutional neural network for image classification using ml5, including default settings, adding training data, and visualization of the loss function during training. it also discusses the training process for an image classification model, addressing normalization of data, training code, setting training options, and debugging issues.', 'chapters': [{'end': 722.911, 'start': 605.529, 'title': 'Configuring convolutional neural network', 'summary': 'Explores configuring a convolutional neural network for image classification using ml5, including default settings and the process of adding training data, with a focus on visualization of the loss function during training.', 'duration': 117.382, 'highlights': ['The chapter focuses on configuring a convolutional neural network for image classification using ML5, with an emphasis on default settings and the process of adding training data.', "Setting debug to true triggers an automatic visualization of the loss function during training, providing insights into the network's performance.", 'The process involves adding training data, which includes associating input images with their respective labels for effective training.', 'The training data set consists of input images associated with their targets, demonstrating the importance of providing relevant data for the training process.', 'ML5 allows convenient addition of images and their associated labels using the addData function, simplifying the process of providing training data.']}, {'end': 895.164, 'start': 726.227, 'title': 'Image classification model training with ml5', 'summary': 'Discusses the training process for an image classification model using ml5, covering the normalization of data, training code, setting training options (e.g., epochs), and addressing debugging issues.', 'duration': 168.937, 'highlights': ['ML5 library offers the convenience of NormalizeData, which analyzes and normalizes data to numbers between 0 and 1. The NormalizeData function in ML5 library analyzes the pixel values of the data and normalizes them to a range between 0 and 1, providing a convenient way to preprocess the input data for training.', 'The training code for an image classification model in ML5 requires setting training options, such as specifying the number of epochs (e.g., 50). Setting training options for an image classification model in ML5 involves specifying parameters like epochs, which determines the number of times the entire dataset is fed through the neural network during training.', 'Debugging issues were encountered with the ML5 library, involving the requirement to fix a bug related to setting the Boolean value true. Encountering a bug in ML5 library that required fixing, specifically related to providing the Boolean value true as an argument, which was identified as a debugging issue during the training process.']}], 'duration': 289.635, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E605529.jpg', 'highlights': ['The training data set consists of input images associated with their targets, demonstrating the importance of providing relevant data for the training process.', "Setting debug to true triggers an automatic visualization of the loss function during training, providing insights into the network's performance.", 'The NormalizeData function in ML5 library analyzes the pixel values of the data and normalizes them to a range between 0 and 1, providing a convenient way to preprocess the input data for training.', 'Setting training options for an image classification model in ML5 involves specifying parameters like epochs, which determines the number of times the entire dataset is fed through the neural network during training.', 'ML5 allows convenient addition of images and their associated labels using the addData function, simplifying the process of providing training data.']}, {'end': 1481.022, 'segs': [{'end': 944.012, 'src': 'embed', 'start': 913.354, 'weight': 0, 'content': [{'end': 921.958, 'text': 'How close is the model matching fitting that training data all the way down to 0? So this is a very good sign that this graph is going down.', 'start': 913.354, 'duration': 8.604}, {'end': 924.559, 'text': "I'm going to add one more line of code,", 'start': 922.378, 'duration': 2.181}, {'end': 930.883, 'text': "because what I want to do when it finishes training is I also want to save the model so that I don't have to run the training process again.", 'start': 924.559, 'duration': 6.324}, {'end': 932.745, 'text': "I'm going to freeze that model.", 'start': 930.903, 'duration': 1.842}, {'end': 944.012, 'text': "I'm going to put it back to 100 images and run that training process again and maybe take a break and do some meditative deep breathing for a little while.", 'start': 932.765, 'duration': 11.247}], 'summary': 'Model training is going well, saving it for future use, planning to continue with more training and relaxation.', 'duration': 30.658, 'max_score': 913.354, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E913354.jpg'}, {'end': 997.708, 'src': 'embed', 'start': 968.735, 'weight': 1, 'content': [{'end': 971.417, 'text': "First of all, it's running incredibly slow.", 'start': 968.735, 'duration': 2.682}, {'end': 973.578, 'text': 'And oh, now that loss is going down.', 'start': 971.917, 'duration': 1.661}, {'end': 980.521, 'text': "I'm feeling like I think I might have pushed it a little bit too much with my 128 by 128 images.", 'start': 974.458, 'duration': 6.063}, {'end': 989.666, 'text': 'And I think maybe reducing the resolution to 64 by 64 will give me the same accuracy in my ultimate results but will allow me to train the model much faster.', 'start': 980.961, 'duration': 8.705}, {'end': 994.807, 'text': "But I'm not exactly sure why it's having such difficulty learning here.", 'start': 990.606, 'duration': 4.201}, {'end': 997.708, 'text': 'And the loss is kind of fluctuating quite a bit.', 'start': 995.288, 'duration': 2.42}], 'summary': 'Model training slow, reducing image resolution to 64x64 for faster training and similar accuracy.', 'duration': 28.973, 'max_score': 968.735, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E968735.jpg'}, {'end': 1108.063, 'src': 'embed', 'start': 1073.704, 'weight': 4, 'content': [{'end': 1076.549, 'text': "I'm going to go take a little break and come back in a minute.", 'start': 1073.704, 'duration': 2.845}, {'end': 1084.264, 'text': "So I don't know really if I just got kind of unlucky with that initial data set,", 'start': 1079.54, 'duration': 4.724}, {'end': 1088.427, 'text': 'but certainly having it run faster would allow me to experiment a bit more.', 'start': 1084.264, 'duration': 4.163}, {'end': 1092.971, 'text': "And the ultimate goal of what I'm doing here is not to train the perfect shape classifier,", 'start': 1088.587, 'duration': 4.384}, {'end': 1099.356, 'text': 'but to show you the steps and tools along the way for you to experiment with different kinds of data and different ways of doing this.', 'start': 1092.971, 'duration': 6.385}, {'end': 1108.063, 'text': 'Great So my model now, because I had the save function, has saved to the downloads folder of the default downloads folder of the browser.', 'start': 1099.716, 'duration': 8.347}], 'summary': 'Increasing speed would enable more experimentation with data and tools.', 'duration': 34.359, 'max_score': 1073.704, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1073704.jpg'}, {'end': 1194.979, 'src': 'heatmap', 'start': 1167.469, 'weight': 0.722, 'content': [{'end': 1172.375, 'text': "There's ultimately no way to reverse engineer the training data from the trained neural network itself.", 'start': 1167.469, 'duration': 4.906}, {'end': 1178.962, 'text': "So that's something that I could be sort of happy about if I was worried about that data uploading to a public web application.", 'start': 1172.635, 'duration': 6.327}, {'end': 1181.165, 'text': 'So here I am in the P5 web editor.', 'start': 1179.443, 'duration': 1.722}, {'end': 1187.832, 'text': 'First thing I want to do is upload the model, create a folder called model, upload file.', 'start': 1181.265, 'duration': 6.567}, {'end': 1191.458, 'text': 'grab these three files, drag them in.', 'start': 1189.158, 'duration': 2.3}, {'end': 1194.979, 'text': "I also want to make sure that I'm importing the ml5 library itself.", 'start': 1192.159, 'duration': 2.82}], 'summary': 'Neural network training data cannot be reverse engineered. uploading model and importing ml5 library in p5 web editor.', 'duration': 27.51, 'max_score': 1167.469, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1167469.jpg'}, {'end': 1481.022, 'src': 'embed', 'start': 1448.616, 'weight': 2, 'content': [{'end': 1449.697, 'text': 'Drum roll, please.', 'start': 1448.616, 'duration': 1.081}, {'end': 1454.039, 'text': "I knew it's a circle.", 'start': 1449.717, 'duration': 4.322}, {'end': 1456.52, 'text': "99% That's super thrilling.", 'start': 1454.059, 'duration': 2.461}, {'end': 1461.543, 'text': "Let's just try really quickly a rectangle.", 'start': 1456.861, 'duration': 4.682}, {'end': 1465.946, 'text': 'Drum roll please.', 'start': 1464.851, 'duration': 1.095}, {'end': 1472.434, 'text': 'Square, 98% square.', 'start': 1470.072, 'duration': 2.362}, {'end': 1473.895, 'text': 'Oh, it is so confident.', 'start': 1472.734, 'duration': 1.161}, {'end': 1475.777, 'text': 'Thank you, neural network friend.', 'start': 1474.056, 'duration': 1.721}, {'end': 1481.022, 'text': 'So this is working with data that matches precisely the training data set.', 'start': 1476.217, 'duration': 4.805}], 'summary': 'Neural network identifies shapes with 99% accuracy, confident in its predictions.', 'duration': 32.406, 'max_score': 1448.616, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1448616.jpg'}], 'start': 895.745, 'title': 'Neural network training and model deployment', 'summary': 'Explores training a convolutional neural network over 50 epochs, encountering slow running time and fluctuating loss, leading to a decision to lower image resolution for faster training. it also details deploying a pre-trained model to a web application using ml5, achieving 99% accuracy for identifying a circle and 98% accuracy for identifying a square.', 'chapters': [{'end': 1092.971, 'start': 895.745, 'title': 'Neural network training and model optimization', 'summary': 'Explores the training process of a convolutional neural network over 50 epochs, encountering slow running time and fluctuating loss, leading to a decision to lower the image resolution to 64 by 64 for faster training and clearer image recognition.', 'duration': 197.226, 'highlights': ["The loss measure during the training process decreases to 0 over 50 epochs, indicating a good sign for the model's fitting to the training data.", 'Decision to lower the image resolution to 64 by 64 to allow faster model training and clearer image recognition.', 'The process of freezing the trained model to avoid rerunning the training process is mentioned.', 'The intention to experiment more by allowing faster model training is expressed, aiming for the ultimate goal of training the perfect shape classifier.']}, {'end': 1481.022, 'start': 1092.971, 'title': 'Model deployment and testing in ml5', 'summary': 'Details the process of deploying a pre-trained model to a web application using ml5, uploading model files, and testing the model with hand-drawn shapes, achieving 99% accuracy for identifying a circle and 98% accuracy for identifying a square.', 'duration': 388.051, 'highlights': ['The speaker deploys a pre-trained model to a web application using ml5 and uploads model files to a folder in the application. Demonstrates the process of deploying a pre-trained model to a web application using ml5 and uploading model files to a folder in the application, emphasizing the practical application of the topic.', 'The model is tested with hand-drawn shapes, achieving 99% accuracy for identifying a circle and 98% accuracy for identifying a square. Illustrates the successful testing of the model with hand-drawn shapes, achieving 99% accuracy for identifying a circle and 98% accuracy for identifying a square, showcasing the effectiveness of the model deployment.', 'The speaker discusses the architecture and weights of the neural network and mentions the ml5 series for more detailed coverage of the topic. Provides insight into the architecture and weights of the neural network, directing viewers to the ml5 series for more comprehensive information on the topic, guiding further learning and exploration.']}], 'duration': 585.277, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E895745.jpg', 'highlights': ['The loss measure decreases to 0 over 50 epochs, indicating a good sign for model fitting.', 'Decision to lower image resolution to 64 by 64 for faster training and clearer recognition.', 'Deploying a pre-trained model to a web app using ml5, achieving 99% accuracy for identifying a circle and 98% accuracy for identifying a square.', 'Freezing the trained model to avoid rerunning the training process.', 'Intention to experiment more for faster model training and the ultimate goal of training the perfect shape classifier.']}, {'end': 1755.776, 'segs': [{'end': 1530.006, 'src': 'embed', 'start': 1498.515, 'weight': 0, 'content': [{'end': 1503.719, 'text': "And when I click the mouse, I'm going to create a separate graphics object.", 'start': 1498.515, 'duration': 5.204}, {'end': 1505.821, 'text': "I'll call it input.", 'start': 1504.94, 'duration': 0.881}, {'end': 1510.465, 'text': "That's 64 by 64.", 'start': 1507.422, 'duration': 3.043}, {'end': 1516.95, 'text': "And then I believe in P5, there's a way for me to copy one image to another and reduce its resolution.", 'start': 1510.465, 'duration': 6.485}, {'end': 1522.694, 'text': 'Something like input copy canvas.', 'start': 1517.37, 'duration': 5.324}, {'end': 1524.382, 'text': 'Something like that.', 'start': 1523.922, 'duration': 0.46}, {'end': 1525.883, 'text': "Let's look at the P5 reference.", 'start': 1524.582, 'duration': 1.301}, {'end': 1530.006, 'text': "Right here under loading and displaying pixels, there's a copy function.", 'start': 1526.264, 'duration': 3.742}], 'summary': 'Creating a 64x64 graphics object and copying image with reduced resolution in p5.', 'duration': 31.491, 'max_score': 1498.515, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1498515.jpg'}, {'end': 1671.31, 'src': 'heatmap', 'start': 1621.138, 'weight': 1, 'content': [{'end': 1623.28, 'text': 'Put background of 255 in setup.', 'start': 1621.138, 'duration': 2.142}, {'end': 1624.481, 'text': 'Try this again.', 'start': 1623.54, 'duration': 0.941}, {'end': 1626.762, 'text': 'So this is me drawing.', 'start': 1625.762, 'duration': 1}, {'end': 1629.625, 'text': "Ultimately, I'm going to try to draw a square and have it recognize it.", 'start': 1626.862, 'duration': 2.763}, {'end': 1630.645, 'text': "Let's press a key.", 'start': 1629.845, 'duration': 0.8}, {'end': 1632.587, 'text': 'Oh, look at that.', 'start': 1632.086, 'duration': 0.501}, {'end': 1633.387, 'text': 'It worked.', 'start': 1633.047, 'duration': 0.34}, {'end': 1634.969, 'text': "There's my 64 by 64.", 'start': 1634.068, 'duration': 0.901}, {'end': 1635.889, 'text': 'So that works.', 'start': 1634.969, 'duration': 0.92}, {'end': 1636.79, 'text': 'Ah, so easy.', 'start': 1636.17, 'duration': 0.62}, {'end': 1638.071, 'text': 'Ah, beautiful.', 'start': 1637.09, 'duration': 0.981}, {'end': 1638.752, 'text': 'Oh, muah.', 'start': 1638.251, 'duration': 0.501}, {'end': 1639.792, 'text': 'I love this.', 'start': 1639.192, 'duration': 0.6}, {'end': 1645.297, 'text': "OK, so now let's make this stroke weight 8.", 'start': 1640.273, 'duration': 5.024}, {'end': 1646.337, 'text': 'So I make it much thicker.', 'start': 1645.297, 'duration': 1.04}, {'end': 1650.721, 'text': "And I don't need to, when I press the key, classify the image.", 'start': 1646.798, 'duration': 3.923}, {'end': 1654.726, 'text': 'I want to do this always.', 'start': 1653.666, 'duration': 1.06}, {'end': 1657.427, 'text': "So let's make this a function called classifyImage.", 'start': 1654.806, 'duration': 2.621}, {'end': 1662.568, 'text': "And I'm going to start doing that as soon as the model is loaded.", 'start': 1659.607, 'duration': 2.961}, {'end': 1671.31, 'text': "classifyImage And once I get some results, I'm going to call classifyImage again.", 'start': 1663.128, 'duration': 8.182}], 'summary': 'Drawing a square, recognizing it, and adjusting stroke weight in a programming setup.', 'duration': 55.149, 'max_score': 1621.138, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1621138.jpg'}, {'end': 1755.776, 'src': 'embed', 'start': 1698.971, 'weight': 2, 'content': [{'end': 1700.391, 'text': 'I want to see the results.', 'start': 1698.971, 'duration': 1.42}, {'end': 1704.412, 'text': "I'll make a div to show the results.", 'start': 1702.612, 'duration': 1.8}, {'end': 1708.233, 'text': "I'll start it with it saying loading model.", 'start': 1706.173, 'duration': 2.06}, {'end': 1715.998, 'text': 'And then The label is results index 0 dot label.', 'start': 1710.534, 'duration': 5.464}, {'end': 1720.843, 'text': 'And the confidence is results index 0 dot confidence.', 'start': 1717.039, 'duration': 3.804}, {'end': 1723.746, 'text': "One of these days, I'll use something called object destructuring.", 'start': 1721.203, 'duration': 2.543}, {'end': 1725.488, 'text': 'I got to make a video to show you how to do that.', 'start': 1723.766, 'duration': 1.722}, {'end': 1727.57, 'text': 'But I like to write it in this longhand way.', 'start': 1725.728, 'duration': 1.842}, {'end': 1730.273, 'text': "And then let's just put, oh, it's not labels.", 'start': 1727.97, 'duration': 2.303}, {'end': 1731.294, 'text': 'I want it to be the label.', 'start': 1730.313, 'duration': 0.981}, {'end': 1734.677, 'text': 'Now I want to see the label.', 'start': 1731.894, 'duration': 2.783}, {'end': 1738.673, 'text': 'And then also the confidence.', 'start': 1735.652, 'duration': 3.021}, {'end': 1741.293, 'text': "And I'm going to do some stuff to this confidence.", 'start': 1739.353, 'duration': 1.94}, {'end': 1752.395, 'text': "I'm going to just make sure it only shows two digits, number format it, and also multiply it by 100 so it's kind of like a percentage.", 'start': 1741.433, 'duration': 10.962}, {'end': 1754.395, 'text': "I'm going to show it like a percentage.", 'start': 1752.415, 'duration': 1.98}, {'end': 1755.776, 'text': "I think that's right.", 'start': 1754.435, 'duration': 1.341}], 'summary': 'Creating a div to display model results with label and confidence, formatting confidence as a percentage.', 'duration': 56.805, 'max_score': 1698.971, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1698971.jpg'}], 'start': 1481.422, 'title': 'Image manipulation and machine learning integration', 'summary': 'Covers resizing images in p5, drawing and classifying images with user inputs, and displaying machine learning results on a web page, using techniques such as resizing, drawing with interactions, and formatting confidence percentages.', 'chapters': [{'end': 1580.171, 'start': 1481.422, 'title': 'Resizing image in p5', 'summary': 'Demonstrates how to resize an image using p5, discussing the process of creating a separate graphics object, copying and reducing the resolution of an image, and exploring the p5 reference for relevant functions.', 'duration': 98.749, 'highlights': ["The process involves creating a separate graphics object with dimensions of 64 by 64 and using P5's copy function to reduce the resolution of the image.", 'The speaker discusses the use of the copy function in P5 to copy a region of the canvas to another region or from a source image, emphasizing the need for a source image and specifying source xy with height to the destination xy with height.']}, {'end': 1671.31, 'start': 1580.751, 'title': 'Drawing and classifying images', 'summary': 'Discusses the process of drawing and classifying images using mouse interactions and keyboard inputs, ultimately achieving the recognition of a 64 by 64 square, as well as the modification of stroke weight for drawing.', 'duration': 90.559, 'highlights': ['The process involves drawing on a canvas using mouse interactions and recognizing a 64 by 64 square by pressing a key, achieving successful recognition.', 'The stroke weight is adjusted to 8, making the drawing much thicker.', 'A function called classifyImage is created to classify the image always, regardless of keyboard inputs, and to be called again after obtaining results.']}, {'end': 1755.776, 'start': 1671.43, 'title': 'Displaying ml results on web page', 'summary': 'Demonstrates how to display machine learning results on a web page by creating a div to show the results, accessing the label and confidence of the results, and formatting the confidence to display as a percentage.', 'duration': 84.346, 'highlights': ["Creating a div to show machine learning results, with the initial text 'loading model'", 'Accessing the label and confidence of the machine learning results and displaying them on the web page', 'Formatting the confidence to display as a percentage by limiting it to two digits and multiplying it by 100']}], 'duration': 274.354, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1481422.jpg', 'highlights': ["Using P5's copy function to reduce image resolution to 64 by 64 dimensions", 'Drawing on canvas with mouse interactions and recognizing a 64 by 64 square', "Creating a div to display machine learning results with initial text 'loading model'", 'Adjusting stroke weight to 8 for thicker drawing', 'Formatting confidence to display as a percentage by limiting it to two digits and multiplying by 100', 'Creating classifyImage function to classify the image always and be called after obtaining results']}, {'end': 2152.12, 'segs': [{'end': 1800.01, 'src': 'heatmap', 'start': 1756.256, 'weight': 0, 'content': [{'end': 1756.856, 'text': "Let's run it.", 'start': 1756.256, 'duration': 0.6}, {'end': 1758.836, 'text': 'Loading model.', 'start': 1758.056, 'duration': 0.78}, {'end': 1761.817, 'text': 'Error Cannot read property HTML of undefined.', 'start': 1759.056, 'duration': 2.761}, {'end': 1762.957, 'text': 'Results div.', 'start': 1761.877, 'duration': 1.08}, {'end': 1766.765, 'text': 'Results, that should say results div.', 'start': 1765.003, 'duration': 1.762}, {'end': 1771.709, 'text': "Another thing, actually, is I don't need to make a new graphics object every single time.", 'start': 1767.886, 'duration': 3.823}, {'end': 1775.432, 'text': 'This could be a global variable.', 'start': 1772.95, 'duration': 2.482}, {'end': 1782.218, 'text': 'Just move that stuff around.', 'start': 1781.017, 'duration': 1.201}, {'end': 1782.638, 'text': 'And here we go.', 'start': 1782.278, 'duration': 0.36}, {'end': 1783.339, 'text': "Let's try this now.", 'start': 1782.698, 'duration': 0.641}, {'end': 1784.419, 'text': 'Loading well.', 'start': 1784.019, 'duration': 0.4}, {'end': 1786.461, 'text': "Square It's a square.", 'start': 1784.459, 'duration': 2.002}, {'end': 1789.684, 'text': "Let's draw a square and see if we can make it more confident.", 'start': 1787.722, 'duration': 1.962}, {'end': 1800.01, 'text': "Square Now, I don't need to see the little tiny image that's useful here, but I don't need to see that so I can take that out.", 'start': 1791.488, 'duration': 8.522}], 'summary': 'Debugging and optimizing code for model loading and graphic object creation, resulting in improved performance.', 'duration': 30.205, 'max_score': 1756.256, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1756256.jpg'}, {'end': 2000.506, 'src': 'embed', 'start': 1961.811, 'weight': 1, 'content': [{'end': 1964.494, 'text': 'Switch it to this camera and run it again.', 'start': 1961.811, 'duration': 2.683}, {'end': 1967.897, 'text': "And great, I've got my document camera.", 'start': 1964.514, 'duration': 3.383}, {'end': 1969.078, 'text': 'Here we go, everybody.', 'start': 1968.277, 'duration': 0.801}, {'end': 1970.96, 'text': 'Drum roll, please.', 'start': 1969.819, 'duration': 1.141}, {'end': 1977.095, 'text': "Square It's a circle.", 'start': 1975.674, 'duration': 1.421}, {'end': 1981.137, 'text': 'Circle Not too confident, but I did get circle.', 'start': 1978.095, 'duration': 3.042}, {'end': 1981.617, 'text': 'All right.', 'start': 1981.157, 'duration': 0.46}, {'end': 1985.119, 'text': "Let's try a triangle.", 'start': 1982.037, 'duration': 3.082}, {'end': 1989.281, 'text': 'Square Come on.', 'start': 1988.2, 'duration': 1.081}, {'end': 1993.203, 'text': 'So my suspicion here is that I might need a thicker marker.', 'start': 1989.681, 'duration': 3.522}, {'end': 1995.003, 'text': "Let's try making it a little thicker.", 'start': 1993.503, 'duration': 1.5}, {'end': 1997.885, 'text': 'There we go.', 'start': 1997.204, 'duration': 0.681}, {'end': 1998.325, 'text': 'Look at that.', 'start': 1997.905, 'duration': 0.42}, {'end': 2000.506, 'text': 'Triangle, 91%.', 'start': 1998.685, 'duration': 1.821}], 'summary': 'Using a thicker marker resulted in 91% accuracy in identifying shapes.', 'duration': 38.695, 'max_score': 1961.811, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1961811.jpg'}, {'end': 2145.818, 'src': 'embed', 'start': 2119.53, 'weight': 2, 'content': [{'end': 2124.872, 'text': "And so in the next video, which should hopefully be a very short one, I'm just going to show you basically this exact same sketch.", 'start': 2119.53, 'duration': 5.342}, {'end': 2131.273, 'text': "But I'm going to load Yining Shi's DoodleNet model and see how it recognizes my drawings.", 'start': 2125.312, 'duration': 5.961}, {'end': 2136.755, 'text': 'that are not just of simple shapes like squares, circles, and triangles.', 'start': 2132.413, 'duration': 4.342}, {'end': 2138.595, 'text': 'OK, thanks for watching this video.', 'start': 2136.815, 'duration': 1.78}, {'end': 2139.516, 'text': 'I hope you enjoyed it.', 'start': 2138.695, 'duration': 0.821}, {'end': 2143.057, 'text': 'I hope you are able to learn something and make something creative with it.', 'start': 2139.636, 'duration': 3.421}, {'end': 2145.818, 'text': "And I can't wait to see you, what happens in future videos.", 'start': 2143.097, 'duration': 2.721}], 'summary': "Next video: showcasing yining shi's doodlenet model recognizing complex drawings.", 'duration': 26.288, 'max_score': 2119.53, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E2119530.jpg'}], 'start': 1756.256, 'title': 'Javascript graphics optimization and drawing recognition with neural network', 'summary': 'Discusses optimizing javascript graphics for improved performance and user interaction, and demonstrates the process of training a neural network to recognize hand-drawn shapes with 97% accuracy for squares and 91% for triangles, introducing the possibility of recognizing more complex drawings using a pre-trained model.', 'chapters': [{'end': 1860.373, 'start': 1756.256, 'title': 'Javascript graphics optimization', 'summary': 'Discusses optimizing javascript graphics by reusing global variables, removing unnecessary elements, and improving user experience by enlarging visual elements and adding a clear button, resulting in improved performance and user interaction.', 'duration': 104.117, 'highlights': ['Reusing global variables instead of creating new graphics objects every single time can optimize the JavaScript code, improving performance.', 'Removing unnecessary elements, such as the little tiny image, and enlarging visual elements can enhance user experience by making the graphics more visible and improving interaction.', 'Adding a clear button and using an anonymous function as the callback for mouse pressed can improve user interaction and provide a better user experience.', 'Optimizing JavaScript graphics by reusing global variables and removing unnecessary elements can lead to improved performance and user experience.']}, {'end': 2152.12, 'start': 1862.498, 'title': 'Drawing recognition with neural network', 'summary': 'Demonstrates the process of training a convolutional neural network to recognize hand-drawn shapes, achieving 97% accuracy for squares and 91% for triangles, and introduces the possibility of training the model to recognize more complex drawings using a pre-trained doodlenet model.', 'duration': 289.622, 'highlights': ['The model achieved 97% accuracy in recognizing squares and 91% accuracy in recognizing triangles. The narrator trained a convolutional neural network to recognize hand-drawn shapes, achieving 97% accuracy for squares and 91% for triangles.', 'Introduction of the pre-trained DoodleNet model for recognizing more complex drawings. The chapter introduces the possibility of training the model to recognize more complex drawings using a pre-trained DoodleNet model.']}], 'duration': 395.864, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/3MqJzMvHE3E/pics/3MqJzMvHE3E1756256.jpg', 'highlights': ['Reusing global variables and removing unnecessary elements can optimize JavaScript code, improving performance and user experience.', 'The model achieved 97% accuracy in recognizing squares and 91% accuracy in recognizing triangles.', 'Introduction of the pre-trained DoodleNet model for recognizing more complex drawings.']}], 'highlights': ["The model's identification of shapes exhibits flaws, as demonstrated by the instance where turning a square sideways led to a mistaken identification as a circle.", 'The process involves creating a neural network with dimensions of 128x128x4 to classify the low-resolution images of shapes, including circles, squares, and triangles, totaling 300 images in the data set.', 'The training data set consists of input images associated with their targets, demonstrating the importance of providing relevant data for the training process.', 'The loss measure decreases to 0 over 50 epochs, indicating a good sign for model fitting.', 'Decision to lower image resolution to 64 by 64 for faster training and clearer recognition.', 'Deploying a pre-trained model to a web app using ml5, achieving 99% accuracy for identifying a circle and 98% accuracy for identifying a square.', "Using P5's copy function to reduce image resolution to 64 by 64 dimensions", 'Reusing global variables and removing unnecessary elements can optimize JavaScript code, improving performance and user experience.', 'The model achieved 97% accuracy in recognizing squares and 91% accuracy in recognizing triangles.']}