title

MIT 6.S191 (2020): Deep Learning New Frontiers

description

MIT Introduction to Deep Learning 6.S191: Lecture 6
Deep Learning Limitations and New Frontiers
Lecturer: Ava Soleimany
January 2020
For all lectures, slides, and lab materials: http://introtodeeplearning.com
Lecture Outline
0:00 - Introduction
0:58 - Course logistics
3:59 - Upcoming guest lectures
5:35 - Deep learning and expressivity of NNs
10:02 - Generalization of deep models
14:14 - Adversarial attacks
17:00 - Limitations summary
18:18 - Structure in deep learning
22:53 - Uncertainty & bayesian deep learning
28:09 - Deep evidential regression
33:08 - AutoML
36:43 - Conclusion
Subscribe to stay up to date with new deep learning lectures at MIT, or follow us @MITDeepLearning on Twitter and Instagram to stay fully-connected!!

detail

{'title': 'MIT 6.S191 (2020): Deep Learning New Frontiers', 'heatmap': [{'end': 965.989, 'start': 894.858, 'weight': 0.79}, {'end': 1123.855, 'start': 1073.484, 'weight': 0.886}, {'end': 1170.394, 'start': 1146.488, 'weight': 0.837}, {'end': 1292.254, 'start': 1232.206, 'weight': 0.709}, {'end': 1397.462, 'start': 1326.919, 'weight': 0.761}, {'end': 1582.571, 'start': 1457.068, 'weight': 0.726}, {'end': 1651.092, 'start': 1626.459, 'weight': 0.9}, {'end': 1744.512, 'start': 1667, 'weight': 0.814}, {'end': 1832.517, 'start': 1763.584, 'weight': 0.866}, {'end': 1996.26, 'start': 1959.595, 'weight': 0.744}, {'end': 2068.449, 'start': 2036.404, 'weight': 0.882}], 'summary': "Covers course logistics, new research directions, and final project instructions. it explores the impact of deep learning on various fields, discusses limitations including overfitting and vulnerability to adversarial attacks, and delves into adversarial examples, network architecture design, bayesian deep learning, uncertainty estimation, and automating the neural network pipeline with google's automl.", 'chapters': [{'end': 332.695, 'segs': [{'end': 36.018, 'src': 'embed', 'start': 2.896, 'weight': 6, 'content': [{'end': 18.822, 'text': "you OK, so as Alexander mentioned, in this final lecture, I'll be discussing some of the limitations of deep learning.", 'start': 2.896, 'duration': 15.926}, {'end': 29.731, 'text': "And, as with any technology not just deep learning or other areas of computer science it's important to be mindful of not only what that technology can enable,", 'start': 19.022, 'duration': 10.709}, {'end': 36.018, 'text': 'but also what are some of the caveats and limitations when considering such approaches.', 'start': 29.731, 'duration': 6.287}], 'summary': 'Discussing limitations of deep learning technology.', 'duration': 33.122, 'max_score': 2.896, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs2896.jpg'}, {'end': 113.754, 'src': 'embed', 'start': 63.509, 'weight': 1, 'content': [{'end': 69.433, 'text': "And so we'll be distributing them today at the end of the lecture portion of the class.", 'start': 63.509, 'duration': 5.924}, {'end': 80.087, 'text': "And at that time we'll take a little bit of time to discuss the logistics of how you can come and receive a t-shirt for your participation in the course.", 'start': 70.314, 'duration': 9.773}, {'end': 89.944, 'text': 'So to check in quickly on where we are in the course, This is going to be the last lecture given by Alexander and myself.', 'start': 81.769, 'duration': 8.175}, {'end': 96.792, 'text': 'And tomorrow and Friday, we will have a series of guest lectures from leading researchers in industry.', 'start': 90.184, 'duration': 6.608}, {'end': 102.058, 'text': "Today, we'll also have our final lab on reinforcement learning.", 'start': 96.812, 'duration': 5.246}, {'end': 107.906, 'text': 'And thank you, everyone, who has been submitting your submissions for the lab competitions.', 'start': 102.779, 'duration': 5.127}, {'end': 111.391, 'text': 'The deadline for doing so is tomorrow at 5 PM.', 'start': 108.407, 'duration': 2.984}, {'end': 113.754, 'text': "And that's for lab one, two, and three.", 'start': 111.711, 'duration': 2.043}], 'summary': 'Last lecture today, guest lectures tomorrow and friday, lab competition deadline tomorrow at 5 pm.', 'duration': 50.245, 'max_score': 63.509, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs63509.jpg'}, {'end': 287.433, 'src': 'embed', 'start': 189.944, 'weight': 0, 'content': [{'end': 191.065, 'text': "They're going to be three minutes.", 'start': 189.944, 'duration': 1.121}, {'end': 195.849, 'text': "And we're going to hold you to that three minute window as strictly as we can.", 'start': 191.525, 'duration': 4.324}, {'end': 202.957, 'text': 'And so there is a link on this slide, which you can find on the PDF version.', 'start': 196.911, 'duration': 6.046}, {'end': 208.322, 'text': "that's going to take you to a document where the instructions for the final project are laid out,", 'start': 202.957, 'duration': 5.365}, {'end': 212.326, 'text': 'including the details for group submission and slide submission.', 'start': 208.322, 'duration': 4.004}, {'end': 219.687, 'text': 'And yeah, here are additional links for the final project proposal.', 'start': 214.384, 'duration': 5.303}, {'end': 230.534, 'text': 'And the second option to fulfill the credit requirement is a short one-page review of a recent deep learning paper.', 'start': 221.829, 'duration': 8.705}, {'end': 237.419, 'text': 'And this is going to be due on the last day of class by Friday at 1 p.m. via email to us.', 'start': 230.554, 'duration': 6.865}, {'end': 242.383, 'text': "OK, so tomorrow we're going to have two guest speakers.", 'start': 239.52, 'duration': 2.863}, {'end': 252.112, 'text': "We're going to have David Cox from IBM, who is actually the director of the MIT IBM Watson AI Lab come and speak.", 'start': 243.103, 'duration': 9.009}, {'end': 258.938, 'text': "And we're also going to have Animesh Garg, who's a professor at U Toronto and a research scientist at NVIDIA.", 'start': 252.773, 'duration': 6.165}, {'end': 263.603, 'text': "And he's going to speak about robotics and robot learning.", 'start': 259.539, 'duration': 4.064}, {'end': 273.834, 'text': "And the lab portion of tomorrow's class will be dedicated to just open office hours where you can work with your project partners on the final project.", 'start': 264.384, 'duration': 9.45}, {'end': 275.916, 'text': 'You can continue work on the labs.', 'start': 274.154, 'duration': 1.762}, {'end': 280.661, 'text': 'You can come and ask us and the TAs any further questions.', 'start': 275.956, 'duration': 4.705}, {'end': 285.993, 'text': "And on Friday, we're going to have two additional guest speakers.", 'start': 282.372, 'duration': 3.621}, {'end': 287.433, 'text': 'So Chuan Li,', 'start': 286.513, 'duration': 0.92}], 'summary': 'Upcoming class schedule: 3-minute presentations, final project details, guest speakers from ibm and nvidia, and open office hours for project work.', 'duration': 97.489, 'max_score': 189.944, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs189944.jpg'}], 'start': 2.896, 'title': 'Course logistics and final project instructions', 'summary': 'Discusses the limitations of deep learning, introduces new research directions, provides logistical and course-related announcements including class t-shirt distribution and guest lectures, outlines final project instructions for group submission and slide submission, and announces guest speakers from ibm and u toronto for upcoming lectures.', 'chapters': [{'end': 161.34, 'start': 2.896, 'title': 'Limitations of deep learning and course logistics', 'summary': 'Discusses the limitations of deep learning, introduces new research directions to address these limitations, and provides important logistical and course-related announcements, including the distribution of class t-shirts, upcoming guest lectures, and project presentation options.', 'duration': 158.444, 'highlights': ['The chapter emphasizes the importance of being mindful of the limitations of deep learning and other technologies, while also discussing new research directions to address these limitations.', 'The class t-shirts have arrived and will be distributed at the end of the lecture portion of the class, with a discussion on the logistics of receiving them for participation in the course.', 'The upcoming schedule includes guest lectures from leading researchers in industry, the final lab on reinforcement learning, and the deadline for lab competition submissions, with options for fulfilling credit requirements through project proposal competitions.']}, {'end': 332.695, 'start': 162.338, 'title': 'Final project instructions and guest speakers', 'summary': 'Outlines the final project instructions, including the details for group submission, slide submission, and the credit requirements, and announces the guest speakers, including david cox from ibm and animesh garg from u toronto, for the upcoming lectures.', 'duration': 170.357, 'highlights': ['The final project instructions including group submission, slide submission, and credit requirements are outlined, with a three-minute presentation time limit and project proposal and awards ceremony scheduled for Friday.', 'Guest speakers David Cox from IBM and Animesh Garg from U Toronto are scheduled for the upcoming lectures, with other guest speakers from Lambda Labs and Google Brain team also presenting on Friday.', 'The credit requirement options include a final project and a one-page review of a recent deep learning paper, due on the last day of class by Friday at 1 p.m. via email.', 'The lab portion of the upcoming class will be dedicated to open office hours for working on the final project, labs, and asking questions to the instructors and TAs.']}], 'duration': 329.799, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs2896.jpg', 'highlights': ['The final project instructions including group submission, slide submission, and credit requirements are outlined, with a three-minute presentation time limit and project proposal and awards ceremony scheduled for Friday.', 'The upcoming schedule includes guest lectures from leading researchers in industry, the final lab on reinforcement learning, and the deadline for lab competition submissions, with options for fulfilling credit requirements through project proposal competitions.', 'The class t-shirts have arrived and will be distributed at the end of the lecture portion of the class, with a discussion on the logistics of receiving them for participation in the course.', 'The credit requirement options include a final project and a one-page review of a recent deep learning paper, due on the last day of class by Friday at 1 p.m. via email.', 'Guest speakers David Cox from IBM and Animesh Garg from U Toronto are scheduled for the upcoming lectures, with other guest speakers from Lambda Labs and Google Brain team also presenting on Friday.', 'The lab portion of the upcoming class will be dedicated to open office hours for working on the final project, labs, and asking questions to the instructors and TAs.', 'The chapter emphasizes the importance of being mindful of the limitations of deep learning and other technologies, while also discussing new research directions to address these limitations.']}, {'end': 605.431, 'segs': [{'end': 416.156, 'src': 'embed', 'start': 362.703, 'weight': 0, 'content': [{'end': 371.01, 'text': 'robotics and a whole host of other applications and areas of impact like natural language processing, finance and security.', 'start': 362.703, 'duration': 8.307}, {'end': 378.636, 'text': "And along with this, hopefully you've also established a more concrete understanding of how these neural networks actually work.", 'start': 371.03, 'duration': 7.606}, {'end': 390.807, 'text': "And largely we've been dealing with algorithms that take as input data in some form you know, as signals, as images or other sensory data,", 'start': 380.478, 'duration': 10.329}, {'end': 394.33, 'text': 'to directly produce a decision at the output or a prediction.', 'start': 390.807, 'duration': 3.523}, {'end': 403.037, 'text': "And we've also seen ways in which these algorithms can be used in the opposite direction to generatively sample from them,", 'start': 395.351, 'duration': 7.686}, {'end': 407.121, 'text': 'to create brand new instances and data examples.', 'start': 403.037, 'duration': 4.084}, {'end': 416.156, 'text': "But really what we've been talking about is algorithms that are very well optimized to perform at a single task,", 'start': 409.192, 'duration': 6.964}], 'summary': 'Neural networks impact robotics, finance, security; optimized for single task.', 'duration': 53.453, 'max_score': 362.703, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs362703.jpg'}, {'end': 468.224, 'src': 'embed', 'start': 444.773, 'weight': 4, 'content': [{'end': 454.355, 'text': 'And what it states is that a feedforward neural net with a single hidden layer could be sufficient to approximate any function.', 'start': 444.773, 'duration': 9.582}, {'end': 466.462, 'text': "And we've seen with deep learning models that use multiple hidden layers, and this theorem is actually completely ignoring that and saying oh,", 'start': 456.092, 'duration': 10.37}, {'end': 468.224, 'text': 'you just need one hidden layer.', 'start': 466.462, 'duration': 1.762}], 'summary': 'A single hidden layer neural net can approximate any function, disregarding the use of multiple hidden layers in deep learning models.', 'duration': 23.451, 'max_score': 444.773, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs444773.jpg'}, {'end': 514.423, 'src': 'embed', 'start': 491.67, 'weight': 5, 'content': [{'end': 501.494, 'text': 'First, this theorem is making no guarantees about the number of hidden units or the size of the hidden layer that would be required to make this approximation.', 'start': 491.67, 'duration': 9.824}, {'end': 510.058, 'text': "And it's also leaving open the question of how you actually go about finding those weights and optimizing the network for that task.", 'start': 502.455, 'duration': 7.603}, {'end': 514.423, 'text': 'It just proves that one theoretically does exist.', 'start': 511.181, 'duration': 3.242}], 'summary': 'The theorem proves the existence of a network for approximation, without specifying hidden units or layer size.', 'duration': 22.753, 'max_score': 491.67, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs491670.jpg'}, {'end': 589.026, 'src': 'embed', 'start': 563.416, 'weight': 3, 'content': [{'end': 570.564, 'text': 'it also in some senses provided some degree of false hope to the AI community that neural nets could be used to solve any problem.', 'start': 563.416, 'duration': 7.148}, {'end': 573.477, 'text': 'And this hype can be very dangerous.', 'start': 571.676, 'duration': 1.801}, {'end': 579.681, 'text': 'And when you look back at the history of AI and sort of the peaks and the falls of the literature,', 'start': 573.517, 'duration': 6.164}, {'end': 589.026, 'text': 'there have been these two AI winters where research in AI and neural networks specifically came to a halt and experienced a decline.', 'start': 579.681, 'duration': 9.345}], 'summary': 'False hope from neural nets led to dangerous hype in ai, causing two ai winters.', 'duration': 25.61, 'max_score': 563.416, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs563416.jpg'}], 'start': 334.574, 'title': 'Deep learning impact and limitations', 'summary': 'Explores the impact of deep learning on fields like autonomous vehicles, medicine, reinforcement learning, robotics, natural language processing, finance, and security. it also discusses the limitations of neural networks, focusing on challenges of generalization, optimization, and overhype in ai.', 'chapters': [{'end': 416.156, 'start': 334.574, 'title': 'Deep learning impact and function', 'summary': 'Explores the impact of deep learning on various fields, such as autonomous vehicles, medicine, reinforcement learning, robotics, natural language processing, finance, and security, while also delving into the technical understanding of neural networks and their applications in decision-making and generative sampling.', 'duration': 81.582, 'highlights': ["Neural networks' impact on fields like autonomous vehicles, medicine, reinforcement learning, robotics, natural language processing, finance, and security.", 'The technical understanding of neural networks and their application in decision-making and generative sampling.', 'The optimization of algorithms for performing single tasks with input data in the form of signals, images, or sensory data.']}, {'end': 605.431, 'start': 416.156, 'title': 'Limitations of neural networks', 'summary': 'Discusses the limitations of neural networks, focusing on the universal approximation theorem and its implications for deep learning, emphasizing the challenges of generalization, optimization, and overhype in ai.', 'duration': 189.275, 'highlights': ['The universal approximation theorem states that a feedforward neural net with a single hidden layer could be sufficient to approximate any function, disregarding the use of multiple hidden layers in deep learning models.', 'The theorem does not guarantee the number of hidden units or the size of the hidden layer required for the approximation, leaving questions about weight optimization and network generalization unanswered.', 'The overhype of deep learning and the universal approximation theorem has led to false hope and potentially dangerous misconceptions about the capabilities of neural networks, contributing to historical peaks and declines in AI research.']}], 'duration': 270.857, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs334574.jpg', 'highlights': ["Neural networks' impact on fields like autonomous vehicles, medicine, reinforcement learning, robotics, natural language processing, finance, and security.", 'The technical understanding of neural networks and their application in decision-making and generative sampling.', 'The optimization of algorithms for performing single tasks with input data in the form of signals, images, or sensory data.', 'The overhype of deep learning and the universal approximation theorem has led to false hope and potentially dangerous misconceptions about the capabilities of neural networks, contributing to historical peaks and declines in AI research.', 'The universal approximation theorem states that a feedforward neural net with a single hidden layer could be sufficient to approximate any function, disregarding the use of multiple hidden layers in deep learning models.', 'The theorem does not guarantee the number of hidden units or the size of the hidden layer required for the approximation, leaving questions about weight optimization and network generalization unanswered.']}, {'end': 930.579, 'segs': [{'end': 632.577, 'src': 'embed', 'start': 606.661, 'weight': 3, 'content': [{'end': 614.526, 'text': 'One of my favorite examples of a potential danger of deep neural nets comes from this paper from a couple years ago,', 'start': 606.661, 'duration': 7.865}, {'end': 619.449, 'text': 'named Understanding Deep Neural Networks Requires Rethinking Generalization.', 'start': 614.526, 'duration': 4.923}, {'end': 621.39, 'text': 'And this was a paper from Google.', 'start': 620.029, 'duration': 1.361}, {'end': 624.372, 'text': 'And really what they did was quite simple.', 'start': 622.19, 'duration': 2.182}, {'end': 628.814, 'text': 'They took images from this huge image data set called ImageNet.', 'start': 624.732, 'duration': 4.082}, {'end': 632.577, 'text': 'And each of these images is annotated with a label right?', 'start': 629.835, 'duration': 2.742}], 'summary': "Google's paper highlights dangers of deep neural nets using imagenet data.", 'duration': 25.916, 'max_score': 606.661, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs606661.jpg'}, {'end': 759.391, 'src': 'embed', 'start': 715.919, 'weight': 0, 'content': [{'end': 720.764, 'text': 'But what was really interesting was what happened when they looked at the accuracy on the training set.', 'start': 715.919, 'duration': 4.845}, {'end': 722.907, 'text': 'And this is what they found.', 'start': 721.905, 'duration': 1.002}, {'end': 732.917, 'text': 'They found that, no matter how much they randomized the labels, the model was able to get 100% accuracy, or close to 100% accuracy,', 'start': 723.747, 'duration': 9.17}, {'end': 737.943, 'text': "on the training set, meaning that it's basically fitting to the data and their labels.", 'start': 732.917, 'duration': 5.026}, {'end': 748.002, 'text': 'And this is really a powerful example, because it shows once again, in a similar way as the universal approximation theorem,', 'start': 739.895, 'duration': 8.107}, {'end': 755.688, 'text': 'that deep neural nets are very, very good at being able to perfectly fit, or very close to perfectly fit, any function,', 'start': 748.002, 'duration': 7.686}, {'end': 759.391, 'text': 'even if that function is this random mapping from data to labels.', 'start': 755.688, 'duration': 3.703}], 'summary': 'Deep neural nets can achieve close to 100% accuracy in fitting any function, even with randomized labels.', 'duration': 43.472, 'max_score': 715.919, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs715919.jpg'}, {'end': 838.879, 'src': 'embed', 'start': 813.598, 'weight': 4, 'content': [{'end': 819.682, 'text': 'Well, there are absolutely no guarantees on what the training data looks like in these regions,', 'start': 813.598, 'duration': 6.084}, {'end': 823.125, 'text': "what the data looks like in these regions that the network hasn't seen before.", 'start': 819.682, 'duration': 3.443}, {'end': 829.95, 'text': 'And this is absolutely one of the most significant limitations that exists in modern deep learning.', 'start': 823.705, 'duration': 6.245}, {'end': 838.879, 'text': 'And this raises the questions of what happens when we look at these places where the model has insufficient or no training data?', 'start': 831.997, 'duration': 6.882}], 'summary': 'Limited training data in regions poses significant challenge in modern deep learning.', 'duration': 25.281, 'max_score': 813.598, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs813598.jpg'}, {'end': 910.524, 'src': 'embed', 'start': 880.343, 'weight': 5, 'content': [{'end': 886.369, 'text': 'such that when we take the result after that perturbation and now feed it back into our neural network,', 'start': 880.343, 'duration': 6.026}, {'end': 892.735, 'text': 'it generates a completely nonsensical prediction, like ostrich, about what is actually in that image.', 'start': 886.369, 'duration': 6.366}, {'end': 898.264, 'text': 'And so this is maybe a little bit shocking.', 'start': 894.858, 'duration': 3.406}, {'end': 899.266, 'text': 'Why is it doing this?', 'start': 898.364, 'duration': 0.902}, {'end': 904.275, 'text': 'And how is this perturbation being created to fool the network in such a way?', 'start': 899.306, 'duration': 4.969}, {'end': 910.524, 'text': "So remember, when we're training our networks, we use gradient descent.", 'start': 906.041, 'duration': 4.483}], 'summary': 'Perturbation in neural network leads to nonsensical prediction, raising questions about its creation and impact.', 'duration': 30.181, 'max_score': 880.343, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs880343.jpg'}], 'start': 606.661, 'title': 'Deep learning limitations', 'summary': "Discusses the dangers of deep neural nets' overfitting with an example achieving 100% accuracy on a training set, but near-zero accuracy on the test set, and the limitations of deep learning in dealing with insufficient training data, vulnerability to adversarial attacks, and the impact of perturbations on predictions.", 'chapters': [{'end': 788.329, 'start': 606.661, 'title': 'Deep neural nets generalization', 'summary': "Highlights the danger of deep neural nets' overfitting by sharing an example from a google paper where a deep neural net was able to achieve 100% accuracy on a training set even when the labels were completely randomized, while the accuracy on the test set tended to zero.", 'duration': 181.668, 'highlights': ['The deep neural net achieved 100% accuracy or close to it on the training set, even when the labels were completely randomized.', 'The accuracy of the resulting model on the test set progressively tended to zero as the labels were randomized.', 'Deep neural nets are very good at being able to perfectly fit, or very close to perfectly fit, any function, even if that function is a random mapping from data to labels.', 'The paper from Google demonstrated the danger of overfitting in deep neural nets by showing how the model could fit to the data and their labels even when the labels were completely randomized.']}, {'end': 930.579, 'start': 789.893, 'title': 'Limitations of deep learning', 'summary': 'Discusses the limitations of deep learning, particularly in dealing with data regions with insufficient training and the vulnerability to adversarial attacks, highlighting the challenges of making predictions in uncertain scenarios and the impact of perturbations on neural network predictions.', 'duration': 140.686, 'highlights': ['The limitations of deep learning in dealing with data regions with insufficient training and the vulnerability to adversarial attacks', 'The impact of perturbations on neural network predictions']}], 'duration': 323.918, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs606661.jpg', 'highlights': ['Deep neural net achieved 100% accuracy on training set with randomized labels', "Model's accuracy on test set tended to zero as labels were randomized", 'Deep neural nets can perfectly fit any function, even random mappings', "Google's paper demonstrated danger of overfitting in deep neural nets", 'Deep learning limitations: insufficient training data, vulnerability to adversarial attacks', 'Impact of perturbations on neural network predictions']}, {'end': 1477.593, 'segs': [{'end': 1036.402, 'src': 'embed', 'start': 960.387, 'weight': 0, 'content': [{'end': 965.989, 'text': 'And an extension of this was recently done by a group of students here at MIT,', 'start': 960.387, 'duration': 5.602}, {'end': 977.092, 'text': 'where they devised an algorithm for synthesizing adversarial examples that were robust to different transformations, like changing the shape,', 'start': 965.989, 'duration': 11.103}, {'end': 978.853, 'text': 'scaling color changes, et cetera.', 'start': 977.092, 'duration': 1.761}, {'end': 985.142, 'text': 'And what was really cool is they moved from beyond the 2D setting to the 3D setting,', 'start': 979.753, 'duration': 5.389}, {'end': 991.853, 'text': 'where they actually 3D printed physical objects that were designed to fool a neural network.', 'start': 985.142, 'duration': 6.711}, {'end': 1000.723, 'text': 'And this was the first demonstration of actual adversarial examples that existed in the physical world, in the 3D world.', 'start': 993.277, 'duration': 7.446}, {'end': 1005.106, 'text': 'And so here, they 3D printed a bunch of these adversarial turtles.', 'start': 1001.403, 'duration': 3.703}, {'end': 1012.652, 'text': 'And when they fed images of these turtles to a neural network trained to classify these images,', 'start': 1005.607, 'duration': 7.045}, {'end': 1018.557, 'text': 'the network incorrectly classified these adversarial examples as rifles rather than turtles.', 'start': 1012.652, 'duration': 5.905}, {'end': 1028.435, 'text': 'And so this just gives you a taste of some of the limitations that exist for neural networks and deep learning.', 'start': 1020.97, 'duration': 7.465}, {'end': 1036.402, 'text': 'And other examples are listed here, including the fact that they can be subject to algorithmic bias,', 'start': 1029.036, 'duration': 7.366}], 'summary': 'Mit students created 3d printed adversarial examples that fooled a neural network, showcasing limitations of deep learning.', 'duration': 76.015, 'max_score': 960.387, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs960387.jpg'}, {'end': 1123.855, 'src': 'heatmap', 'start': 1073.484, 'weight': 0.886, 'content': [{'end': 1082.488, 'text': "We'll talk about how we can represent uncertainty and understand when our model is uncertain or not confident in its predictions.", 'start': 1073.484, 'duration': 9.004}, {'end': 1086.57, 'text': 'And finally, how we can move past deep learning,', 'start': 1083.349, 'duration': 3.221}, {'end': 1097.235, 'text': 'where models are built to solve a single problem and potentially move towards building models that are capable to address many different tasks.', 'start': 1086.57, 'duration': 10.665}, {'end': 1105.51, 'text': "Okay, so first we'll talk about how we can encode structure and domain knowledge into designing deep neural nets.", 'start': 1098.786, 'duration': 6.724}, {'end': 1115.436, 'text': "And we've already seen an example of this in the case of convolutional neural networks that are very well equipped to deal with spatial data and spatial information.", 'start': 1106.19, 'duration': 9.246}, {'end': 1123.855, 'text': "And if you consider a fully connected network as sort of the baseline, there's no sense of structure there.", 'start': 1116.892, 'duration': 6.963}], 'summary': 'Discussing uncertainty representation, model confidence, and expanding beyond single-task deep learning.', 'duration': 50.371, 'max_score': 1073.484, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1073484.jpg'}, {'end': 1170.394, 'src': 'heatmap', 'start': 1146.488, 'weight': 0.837, 'content': [{'end': 1158.483, 'text': "And recently, researchers have moved on to develop neural networks that are very well suited to handle another class of data, and that's of graphs.", 'start': 1146.488, 'duration': 11.995}, {'end': 1165.59, 'text': 'And graphs are an irregular data structure that encode this very, very rich structural information.', 'start': 1158.723, 'duration': 6.867}, {'end': 1170.394, 'text': 'And that structural information is very important to the problem that can be considered.', 'start': 1165.791, 'duration': 4.603}], 'summary': 'Researchers have developed neural networks for handling graph data structures, which encode rich structural information important for problem-solving.', 'duration': 23.906, 'max_score': 1146.488, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1146488.jpg'}, {'end': 1220.579, 'src': 'embed', 'start': 1195.159, 'weight': 2, 'content': [{'end': 1199.864, 'text': 'And there are a whole class of problems that can be represented this way.', 'start': 1195.159, 'duration': 4.705}, {'end': 1210.454, 'text': "And an idea that arises is how can we extend neural networks to learn and process the data that's present in these graph structures?", 'start': 1200.524, 'duration': 9.93}, {'end': 1217.637, 'text': 'And this falls very nicely from an extension of convolutional neural nets.', 'start': 1212.313, 'duration': 5.324}, {'end': 1220.579, 'text': 'And with convolutional neural networks.', 'start': 1218.517, 'duration': 2.062}], 'summary': 'Extending neural networks to process graph structures using convolutional neural nets.', 'duration': 25.42, 'max_score': 1195.159, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1195159.jpg'}, {'end': 1292.254, 'src': 'heatmap', 'start': 1232.206, 'weight': 0.709, 'content': [{'end': 1237.745, 'text': 'And as we go across the entirety of the image,', 'start': 1232.206, 'duration': 5.539}, {'end': 1244.308, 'text': 'the idea is we can apply this set of weights to extract particular local features that are present in the image.', 'start': 1237.745, 'duration': 6.563}, {'end': 1247.529, 'text': 'And different sets of weights extract different features.', 'start': 1245.088, 'duration': 2.441}, {'end': 1251.311, 'text': 'In graph convolutional networks.', 'start': 1249.47, 'duration': 1.841}, {'end': 1259.394, 'text': "the idea is very similar, where now, rather than processing a 2D matrix that represents an image, we're processing a graph.", 'start': 1251.311, 'duration': 8.083}, {'end': 1265.309, 'text': 'And what graph convolutional networks use is a kernel of weights, a set of weights.', 'start': 1260.345, 'duration': 4.964}, {'end': 1273.997, 'text': 'And rather than sliding this set of weights across a 2D matrix, the weights are applied to each of the different nodes present in the graph.', 'start': 1266.11, 'duration': 7.887}, {'end': 1282.305, 'text': 'And so the network is looking at a node and the neighbors of that node, and it goes across the entirety of the graph in this manner.', 'start': 1274.838, 'duration': 7.467}, {'end': 1292.254, 'text': 'and aggregates information about a node and its neighbors and encodes that into a high level representation.', 'start': 1283.703, 'duration': 8.551}], 'summary': 'Graph convolutional networks use weights to extract features from nodes and neighbors in a graph.', 'duration': 60.048, 'max_score': 1232.206, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1232206.jpg'}, {'end': 1354.07, 'src': 'embed', 'start': 1326.919, 'weight': 3, 'content': [{'end': 1338.608, 'text': "And it's basically just this unordered cloud of points where there is some spatial dependence and it represents sort of the depth and our perception of the 3D world.", 'start': 1326.919, 'duration': 11.689}, {'end': 1346.086, 'text': 'And just like images, you can perform classification or segmentation on this 3D point data.', 'start': 1339.983, 'duration': 6.103}, {'end': 1354.07, 'text': 'And it turns out that graph convolutional networks can also be extended to handle and analyze this point cloud data.', 'start': 1347.287, 'duration': 6.783}], 'summary': 'Graph convolutional networks analyze spatial 3d point cloud data for classification and segmentation.', 'duration': 27.151, 'max_score': 1326.919, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1326919.jpg'}, {'end': 1409.892, 'src': 'heatmap', 'start': 1326.919, 'weight': 4, 'content': [{'end': 1338.608, 'text': "And it's basically just this unordered cloud of points where there is some spatial dependence and it represents sort of the depth and our perception of the 3D world.", 'start': 1326.919, 'duration': 11.689}, {'end': 1346.086, 'text': 'And just like images, you can perform classification or segmentation on this 3D point data.', 'start': 1339.983, 'duration': 6.103}, {'end': 1354.07, 'text': 'And it turns out that graph convolutional networks can also be extended to handle and analyze this point cloud data.', 'start': 1347.287, 'duration': 6.783}, {'end': 1370.345, 'text': 'And the way this is done is by dynamically computing a graph based on these point clouds that essentially creates a mesh that preserves the local depth and the spatial structure present in the point cloud.', 'start': 1354.931, 'duration': 15.414}, {'end': 1384.414, 'text': 'Okay, so that gives you a taste of how different types of data and different network structures can be used to encode prior knowledge into our network.', 'start': 1372.687, 'duration': 11.727}, {'end': 1393.099, 'text': 'Another area that has garnered a lot of interest in recent years is this question of uncertainty,', 'start': 1385.415, 'duration': 7.684}, {'end': 1397.462, 'text': 'and how do we know how confident a model is in its predictions?', 'start': 1393.099, 'duration': 4.363}, {'end': 1403.29, 'text': "So let's consider a really simple example, a classification example.", 'start': 1398.789, 'duration': 4.501}, {'end': 1409.892, 'text': "And what we've learned so far is that we can use a network to output a classification probability.", 'start': 1404.171, 'duration': 5.721}], 'summary': 'Graph convolutional networks can analyze unordered 3d point cloud data, encoding prior knowledge and addressing uncertainty in predictions.', 'duration': 24.477, 'max_score': 1326.919, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1326919.jpg'}], 'start': 932.25, 'title': 'Adversarial examples and network architecture design', 'summary': 'Delves into the impact of adversarial examples, including a mit study on 3d printed objects fooling a neural network, and extends on designing network architecture with prior knowledge, processing graph and point cloud data, and understanding model uncertainty.', 'chapters': [{'end': 1063.899, 'start': 932.25, 'title': 'Adversarial examples in neural networks', 'summary': 'Explores the creation and impact of adversarial examples, including a recent mit study that synthesized 3d printed physical objects to fool a neural network, demonstrating the limitations and susceptibility of neural networks to adversarial attacks and algorithmic bias.', 'duration': 131.649, 'highlights': ['A group of students at MIT devised an algorithm for synthesizing adversarial examples robust to different transformations, including 3D printed physical objects that were designed to fool a neural network.', 'The neural network incorrectly classified the 3D printed adversarial turtles as rifles rather than turtles, showcasing the susceptibility of neural networks to adversarial attacks.', 'Neural networks have limitations, including susceptibility to adversarial attacks and algorithmic bias, and being extremely data hungry.']}, {'end': 1477.593, 'start': 1064.7, 'title': 'Designing network architecture with prior knowledge', 'summary': 'Discusses encoding structure and prior domain knowledge into network architecture, extending neural networks for processing graph and point cloud data, and understanding model uncertainty and confidence in predictions.', 'duration': 412.893, 'highlights': ['Graph convolutional networks extending neural networks for processing graph data', 'Extending graph convolutional networks to analyze 3D point cloud data', 'Importance of understanding model uncertainty and confidence in predictions']}], 'duration': 545.343, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs932250.jpg', 'highlights': ['MIT students devised algorithm for synthesizing adversarial examples robust to different transformations', 'Neural network misclassified 3D printed adversarial turtles as rifles, showcasing susceptibility to attacks', 'Graph convolutional networks extend neural networks for processing graph data', 'Extending graph convolutional networks to analyze 3D point cloud data', 'Importance of understanding model uncertainty and confidence in predictions', 'Neural networks have limitations, including susceptibility to adversarial attacks and algorithmic bias']}, {'end': 1986.295, 'segs': [{'end': 1560.799, 'src': 'embed', 'start': 1508.444, 'weight': 0, 'content': [{'end': 1512.127, 'text': 'what is done is, rather than directly learning the weights,', 'start': 1508.444, 'duration': 3.683}, {'end': 1523.274, 'text': 'the neural network actually approximates a posterior probability distribution over the weights given the data X and the labels Y.', 'start': 1512.127, 'duration': 11.147}, {'end': 1531.759, 'text': "And Bayesian neural networks are considered Bayesian because we can rewrite this posterior P of W given X and Y using Bayes' rule.", 'start': 1523.274, 'duration': 8.485}, {'end': 1540.712, 'text': 'But computationally, it turns out that actually computing this posterior distribution is infeasible and intractable.', 'start': 1532.81, 'duration': 7.902}, {'end': 1551.956, 'text': 'So what has been done is there have been different approaches and different ways to try to approximate this distribution using sampling operations.', 'start': 1541.613, 'duration': 10.343}, {'end': 1560.799, 'text': 'And one example of how you can use sampling to approximate this posterior is by using dropout,', 'start': 1554.355, 'duration': 6.444}], 'summary': 'Neural network approximates posterior distribution over weights using sampling, such as dropout.', 'duration': 52.355, 'max_score': 1508.444, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1508444.jpg'}, {'end': 1651.092, 'src': 'heatmap', 'start': 1626.459, 'weight': 0.9, 'content': [{'end': 1633.986, 'text': "And if we do this many times, say t times, we're going to obtain different predictions from the model every time.", 'start': 1626.459, 'duration': 7.527}, {'end': 1643.807, 'text': 'And by looking at the expected value of those predictions and the variance in those predictions, we can get a sense of how uncertain the model is.', 'start': 1635, 'duration': 8.807}, {'end': 1651.092, 'text': 'And one application of this is in the context of depth estimation.', 'start': 1645.968, 'duration': 5.124}], 'summary': 'Using t iterations gives varied predictions, helping assess model uncertainty. applicable in depth estimation.', 'duration': 24.633, 'max_score': 1626.459, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1626459.jpg'}, {'end': 1744.512, 'src': 'heatmap', 'start': 1667, 'weight': 0.814, 'content': [{'end': 1673.226, 'text': "And what you can see here in the image on the right is that there's this particular band, a hotspot of uncertainty.", 'start': 1667, 'duration': 6.226}, {'end': 1680.852, 'text': 'And that corresponds to that portion of the image where the two cars are overlapping, which kind of makes sense, right?', 'start': 1673.866, 'duration': 6.986}, {'end': 1687.138, 'text': 'You may not have as clear of a sense of the depth in that region in particular.', 'start': 1681.112, 'duration': 6.026}, {'end': 1692.29, 'text': 'And so to conceptualize this a bit further.', 'start': 1689.008, 'duration': 3.282}, {'end': 1701.914, 'text': 'this is a general example of how you can ensemble different instances of models together to obtain estimates of uncertainty.', 'start': 1692.29, 'duration': 9.624}, {'end': 1706.176, 'text': "So let's say we're working in the context of self-driving cars, right?", 'start': 1702.475, 'duration': 3.701}, {'end': 1713.18, 'text': 'And our task is given an input image to predict a steering wheel angle that will be used to control the car.', 'start': 1706.637, 'duration': 6.543}, {'end': 1715.724, 'text': "And that's mu, the mean.", 'start': 1714.383, 'duration': 1.341}, {'end': 1723.913, 'text': 'And in order to estimate the uncertainty, we can take an ensemble of many different instances of a model like this.', 'start': 1717.306, 'duration': 6.607}, {'end': 1730.579, 'text': 'And in the case of dropout sampling, each model will have different sets of weights that are being dropped out.', 'start': 1724.774, 'duration': 5.805}, {'end': 1736.925, 'text': "And from each model, we're going to get a different estimate of the predicted steering wheel angle.", 'start': 1731.36, 'duration': 5.565}, {'end': 1744.512, 'text': "And we can aggregate many of these different estimates together, and they're going to lie along some distribution.", 'start': 1738.126, 'duration': 6.386}], 'summary': 'Ensemble models for self-driving cars predict uncertainty with dropout sampling.', 'duration': 77.512, 'max_score': 1667, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1667000.jpg'}, {'end': 1723.913, 'src': 'embed', 'start': 1673.866, 'weight': 3, 'content': [{'end': 1680.852, 'text': 'And that corresponds to that portion of the image where the two cars are overlapping, which kind of makes sense, right?', 'start': 1673.866, 'duration': 6.986}, {'end': 1687.138, 'text': 'You may not have as clear of a sense of the depth in that region in particular.', 'start': 1681.112, 'duration': 6.026}, {'end': 1692.29, 'text': 'And so to conceptualize this a bit further.', 'start': 1689.008, 'duration': 3.282}, {'end': 1701.914, 'text': 'this is a general example of how you can ensemble different instances of models together to obtain estimates of uncertainty.', 'start': 1692.29, 'duration': 9.624}, {'end': 1706.176, 'text': "So let's say we're working in the context of self-driving cars, right?", 'start': 1702.475, 'duration': 3.701}, {'end': 1713.18, 'text': 'And our task is given an input image to predict a steering wheel angle that will be used to control the car.', 'start': 1706.637, 'duration': 6.543}, {'end': 1715.724, 'text': "And that's mu, the mean.", 'start': 1714.383, 'duration': 1.341}, {'end': 1723.913, 'text': 'And in order to estimate the uncertainty, we can take an ensemble of many different instances of a model like this.', 'start': 1717.306, 'duration': 6.607}], 'summary': 'Ensemble models to estimate uncertainty in self-driving cars.', 'duration': 50.047, 'max_score': 1673.866, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1673866.jpg'}, {'end': 1832.517, 'src': 'heatmap', 'start': 1763.584, 'weight': 0.866, 'content': [{'end': 1769.408, 'text': "But if they're clustered very closely together, the model is more certain, more confident in its prediction.", 'start': 1763.584, 'duration': 5.824}, {'end': 1776.509, 'text': 'And these estimates are actually being drawn from an underlying distribution.', 'start': 1771.425, 'duration': 5.084}, {'end': 1782.673, 'text': 'And what Ensembleng is trying to do is to sample from this underlying distribution.', 'start': 1776.709, 'duration': 5.964}, {'end': 1790.459, 'text': 'But it turns out that we can approximate and model this distribution directly using a neural network.', 'start': 1783.654, 'duration': 6.805}, {'end': 1795.358, 'text': "And this means that we're learning what is called an evidential distribution.", 'start': 1791.397, 'duration': 3.961}, {'end': 1803.66, 'text': 'And effectively, the evidential distribution captures how much evidence the model has in support of a prediction.', 'start': 1796.058, 'duration': 7.602}, {'end': 1824.912, 'text': 'And the way that we can train these evidential networks is by first trying to maximize the fit of the inferred distribution to the data and also minimizing the evidence that the model has for cases when the model makes errors.', 'start': 1805.421, 'duration': 19.491}, {'end': 1832.517, 'text': 'And if you train a network using this approach, you can generate calibrated,', 'start': 1826.673, 'duration': 5.844}], 'summary': 'Ensembleng uses neural network to model evidential distribution for confident predictions.', 'duration': 68.933, 'max_score': 1763.584, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1763584.jpg'}, {'end': 1924.04, 'src': 'embed', 'start': 1874.203, 'weight': 1, 'content': [{'end': 1879.145, 'text': "I'm uncertain about my prediction in this region because I haven't seen the data before.", 'start': 1874.203, 'duration': 4.942}, {'end': 1891.826, 'text': 'And by using an evidential distribution, our network actually generates these predictions of uncertainty that scale,', 'start': 1880.006, 'duration': 11.82}, {'end': 1896.088, 'text': 'as the model has less and less data or less and less evidence.', 'start': 1891.826, 'duration': 4.262}, {'end': 1908.796, 'text': 'And so these uncertainties are also robust to adversarial perturbations and adversarial changes, like similar to those that we saw previously.', 'start': 1897.813, 'duration': 10.983}, {'end': 1918.658, 'text': "And in fact, if an input suppose an image is adversarially perturbed and it's increasingly adversarially perturbed,", 'start': 1909.516, 'duration': 9.142}, {'end': 1924.04, 'text': 'the estimates of uncertainty are also going to increase as the degree of perturbation increases.', 'start': 1918.658, 'duration': 5.382}], 'summary': 'Predictions of uncertainty scale with less evidence, robust to adversarial changes.', 'duration': 49.837, 'max_score': 1874.203, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1874203.jpg'}, {'end': 1986.295, 'src': 'embed', 'start': 1956.673, 'weight': 6, 'content': [{'end': 1959.595, 'text': 'And this was work done a couple of years ago,', 'start': 1956.673, 'duration': 2.922}, {'end': 1969.667, 'text': 'where they actually used estimates of uncertainty to improve the quality of the segmentations and depth estimations that they made.', 'start': 1959.595, 'duration': 10.072}, {'end': 1977.651, 'text': 'And what they showed was compared to a baseline model without any estimates of uncertainty,', 'start': 1969.687, 'duration': 7.964}, {'end': 1986.295, 'text': 'they could actually use these metrics to improve the performance of their model at segmentation and depth estimation.', 'start': 1977.651, 'duration': 8.644}], 'summary': 'Using uncertainty estimates improved segmentation and depth estimation performance compared to baseline model.', 'duration': 29.622, 'max_score': 1956.673, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1956673.jpg'}], 'start': 1480.461, 'title': 'Bayesian deep learning and uncertainty estimation', 'summary': 'Covers bayesian deep learning and its approach to approximating posterior probability distribution over weights in neural networks, including the use of dropout. it also discusses uncertainty estimation in neural networks through dropout sampling and evidential distribution, emphasizing its applications in depth estimation and steering angle prediction for self-driving cars, as well as its robustness to adversarial perturbations.', 'chapters': [{'end': 1560.799, 'start': 1480.461, 'title': 'Bayesian deep learning overview', 'summary': 'Discusses bayesian deep learning and its approach to approximating the posterior probability distribution over weights in neural networks using sampling operations, with a focus on the infeasibility of computing the distribution and the use of dropout as an example of such approximation.', 'duration': 80.338, 'highlights': ['Bayesian neural nets approximate a posterior probability distribution over the weights given the data X and the labels Y.', 'Computing the posterior distribution is infeasible and intractable, leading to the use of different approaches and sampling operations for approximation.', 'An example of approximation is using dropout to approximate the posterior distribution over weights in neural networks.']}, {'end': 1986.295, 'start': 1560.799, 'title': 'Uncertainty estimation in neural networks', 'summary': 'Discusses the concept of uncertainty estimation in neural networks, particularly through dropout sampling and evidential distribution, highlighting its application in depth estimation and steering angle prediction for self-driving cars, and its robustness to adversarial perturbations.', 'duration': 425.496, 'highlights': ['Ensemble different instances of models to obtain estimates of uncertainty.', 'Using evidential distribution to generate calibrated, accurate estimates of uncertainty for every prediction.', 'Robustness of uncertainties to adversarial perturbations.', 'Application in depth estimation and steering angle prediction for self-driving cars.', 'Improvement in segmentation and depth estimation using estimates of uncertainty.']}], 'duration': 505.834, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1480461.jpg', 'highlights': ['Bayesian neural nets approximate a posterior probability distribution over the weights given the data X and the labels Y.', 'Using evidential distribution to generate calibrated, accurate estimates of uncertainty for every prediction.', 'An example of approximation is using dropout to approximate the posterior distribution over weights in neural networks.', 'Ensemble different instances of models to obtain estimates of uncertainty.', 'Robustness of uncertainties to adversarial perturbations.', 'Application in depth estimation and steering angle prediction for self-driving cars.', 'Improvement in segmentation and depth estimation using estimates of uncertainty.', 'Computing the posterior distribution is infeasible and intractable, leading to the use of different approaches and sampling operations for approximation.']}, {'end': 2288.19, 'segs': [{'end': 2068.449, 'src': 'heatmap', 'start': 2020.014, 'weight': 0, 'content': [{'end': 2029.92, 'text': "And as models get more and more complex, they require some degree of expert knowledge, some of which you've hopefully learned through this course,", 'start': 2020.014, 'duration': 9.906}, {'end': 2036.404, 'text': "to select the particular architecture of the network that's being used, to,", 'start': 2029.92, 'duration': 6.484}, {'end': 2042.708, 'text': 'selecting and tuning hyperparameters and adjusting the network to perform as best as it possibly can.', 'start': 2036.404, 'duration': 6.304}, {'end': 2052.574, 'text': 'What Google did was they built a learning algorithm that can be used to automatically learn a machine learning model to solve a given problem.', 'start': 2043.428, 'duration': 9.146}, {'end': 2055.676, 'text': 'And this is called AutoML, or automated machine learning.', 'start': 2052.614, 'duration': 3.062}, {'end': 2060.98, 'text': 'And the way it works is it uses a reinforcement learning framework.', 'start': 2056.657, 'duration': 4.323}, {'end': 2068.449, 'text': 'In this framework, there is a controller neural network, which is sort of the agent.', 'start': 2063.587, 'duration': 4.862}], 'summary': 'Automl uses a reinforcement learning framework to automatically learn machine learning models, as demonstrated by google.', 'duration': 48.435, 'max_score': 2020.014, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs2020014.jpg'}, {'end': 2198.736, 'src': 'embed', 'start': 2173.427, 'weight': 1, 'content': [{'end': 2183.21, 'text': "And what Google has done is that they've actually generated a pipeline for this and put this service on the cloud so that you, as a user,", 'start': 2173.427, 'duration': 9.783}, {'end': 2189.613, 'text': 'can provide it with a data set and a set of metrics that you want to optimize over.', 'start': 2183.21, 'duration': 6.403}, {'end': 2198.736, 'text': 'And this AutoML framework will spit out candidate child networks that can be deployed for your task of interest.', 'start': 2190.253, 'duration': 8.483}], 'summary': 'Google has developed an automl framework on the cloud for optimizing over user-provided data and metrics.', 'duration': 25.309, 'max_score': 2173.427, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs2173427.jpg'}, {'end': 2270.403, 'src': 'embed', 'start': 2248.335, 'weight': 4, 'content': [{'end': 2260.68, 'text': 'are able to learn tasks and use the analytical process that goes into that to generalize to other examples in our life and other problems that we may encounter,', 'start': 2248.335, 'duration': 12.345}, {'end': 2270.403, 'text': 'whereas neural networks and AI right now are still very much constrained and optimized to perform well at particular individual problems.', 'start': 2260.68, 'duration': 9.723}], 'summary': 'Humans can generalize tasks, while ai is optimized for specific problems.', 'duration': 22.068, 'max_score': 2248.335, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs2248335.jpg'}], 'start': 1988.096, 'title': "Automating neural network pipeline and google's automl", 'summary': "Discusses the need for automating the neural network pipeline and google's automl, highlighting the complexity of model tuning, the impact of automl in reducing the burden on engineers, and the distinction between human intelligence and ai capabilities.", 'chapters': [{'end': 2042.708, 'start': 1988.096, 'title': 'Automating neural network pipeline', 'summary': 'Discusses the need for automating the neural network pipeline due to the complexity of tuning and optimizing models, requiring expert knowledge to select architectures and tuning hyperparameters.', 'duration': 54.612, 'highlights': ['Neural networks need to be finely tuned and optimized for the task of interest, requiring expert knowledge to select the particular architecture and tune hyperparameters.', 'As models get more complex, they require some degree of expert knowledge to perform as best as possible.']}, {'end': 2288.19, 'start': 2043.428, 'title': "Google's automl and the future of ai", 'summary': "Discusses google's automl, a reinforcement learning framework using a controller neural network to automatically generate and improve machine learning models, significantly reducing the burden on engineers for hyperparameter optimization and architecture selection, and highlights the distinction between human intelligence and ai capabilities.", 'duration': 244.762, 'highlights': ["Google's AutoML uses a reinforcement learning framework with a controller neural network to automatically generate and improve machine learning models.", 'The controller agent iteratively produces new architectures, tests them, and uses feedback to improve the child network over thousands of iterations.', "AutoML's service on the cloud allows users to provide a dataset and a set of metrics to optimize over, producing candidate child networks deployable for specific tasks.", "AI's current limitations are highlighted in contrast to human intelligence, emphasizing the need to bridge the gap to enable AI to generalize to other problems."]}], 'duration': 300.094, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tfM_DdbGTLs/pics/tfM_DdbGTLs1988096.jpg', 'highlights': ["Google's AutoML uses a reinforcement learning framework with a controller neural network to automatically generate and improve machine learning models.", "AutoML's service on the cloud allows users to provide a dataset and a set of metrics to optimize over, producing candidate child networks deployable for specific tasks.", 'Neural networks need to be finely tuned and optimized for the task of interest, requiring expert knowledge to select the particular architecture and tune hyperparameters.', 'As models get more complex, they require some degree of expert knowledge to perform as best as possible.', "AI's current limitations are highlighted in contrast to human intelligence, emphasizing the need to bridge the gap to enable AI to generalize to other problems."]}], 'highlights': ['Deep neural net achieved 100% accuracy on training set with randomized labels', 'Neural network misclassified 3D printed adversarial turtles as rifles, showcasing susceptibility to attacks', "Google's AutoML uses a reinforcement learning framework with a controller neural network to automatically generate and improve machine learning models", 'Bayesian neural nets approximate a posterior probability distribution over the weights given the data X and the labels Y', 'The final project instructions including group submission, slide submission, and credit requirements are outlined, with a three-minute presentation time limit and project proposal and awards ceremony scheduled for Friday', 'The upcoming schedule includes guest lectures from leading researchers in industry, the final lab on reinforcement learning, and the deadline for lab competition submissions, with options for fulfilling credit requirements through project proposal competitions', 'The class t-shirts have arrived and will be distributed at the end of the lecture portion of the class, with a discussion on the logistics of receiving them for participation in the course', 'Covers course logistics, new research directions, and final project instructions', "Neural networks' impact on fields like autonomous vehicles, medicine, reinforcement learning, robotics, natural language processing, finance, and security", 'The optimization of algorithms for performing single tasks with input data in the form of signals, images, or sensory data']}