title
Sacha Arnoud, Director of Engineering, Waymo - MIT Self-Driving Cars

description
This is a talk by Sacha Arnoud for course 6.S094: Deep Learning for Self-Driving Cars (2018 version). Sacha is the Director of Engineering at Waymo and his talk is titled "The rise of machine learning in self-driving cars." This class is free and open to everyone. It is an introduction to the practice of deep learning through the applied theme of building a self-driving car. INFO: Course website: https://selfdrivingcars.mit.edu Contact: deepcars@mit.edu CONNECT: - If you enjoyed this video, please subscribe to this channel. - AI Podcast: https://lexfridman.com/ai/ - Show your support: https://www.patreon.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Twitter: https://twitter.com/lexfridman - Facebook: https://www.facebook.com/lexfridman - Instagram: https://www.instagram.com/lexfridman - Slack: https://deep-mit-slack.herokuapp.com LINKS: Playlist: https://goo.gl/SLCb1y Lecture 1: Deep Learning - https://youtu.be/-6INDaLcuJY Lecture 2: Self-Driving Cars - https://youtu.be/_OCjqIgxwHw Lecture 3: Deep Reinforcement Learning - https://youtu.be/MQ6pP65o7OM Lecture 4: Computer Vision - https://youtu.be/CLOAswsxudo Lecture 5: Deep Learning for Human Sensing - https://youtu.be/Z2GfE8pLyxc Guest talk: Sacha Arnoud, Waymo - https://youtu.be/LSX3qdy0dFg Guest talk: Emilio Frazolli, nuTonomy - https://youtu.be/dWSbItd0HEA Guest talk: Sterling Anderson, Aurora - https://youtu.be/HKBhP9JISF0 2017: Guest talk: Sertac Karaman, MIT - https://youtu.be/0fLSf3NO0-s Guest talk: Chris Gerdes, Stanford - https://youtu.be/LDprUza7yT4

detail
{'title': 'Sacha Arnoud, Director of Engineering, Waymo - MIT Self-Driving Cars', 'heatmap': [{'end': 221.091, 'start': 175.873, 'weight': 1}, {'end': 3837.565, 'start': 3781.318, 'weight': 0.776}], 'summary': "Sacha arnoud, director of engineering at waymo, discusses the company's milestones, including driving over four million miles autonomously, the potential impact of self-driving technology on safety, accessibility, and efficiency, and the technical and industrial challenges. waymo's aggressive testing of 1000-mile loops in diverse environments, completion of 10 loops in san francisco in 2010, and achievement of confidence in operating driverless cars in phoenix. the impact of deep learning on address mapping and challenges of implementing real-time processing in autonomous cars. importance of sensor data, challenges in understanding and predicting complex behaviors, impact of deep learning on driving safety, challenges in sensor data processing, and understanding vehicle semantics. waymo's achievement of reaching 4 million miles in six months, ability to simulate and augment miles for testing, and challenges in snow identification and semantic understanding in self-driving car development.", 'chapters': [{'end': 530.538, 'segs': [{'end': 151.552, 'src': 'embed', 'start': 126.195, 'weight': 2, 'content': [{'end': 132.601, 'text': "and I'll try to put that in a current hole so that you can see how those pieces fit together to build the system we have today.", 'start': 126.195, 'duration': 6.406}, {'end': 138.462, 'text': 'And last but not least, I think, as Lex mentioned,', 'start': 133.919, 'duration': 4.543}, {'end': 145.127, 'text': 'it takes a lot more actually than algorithms to build a sophisticated system such as our self-driving cars.', 'start': 138.462, 'duration': 6.665}, {'end': 151.552, 'text': 'And fundamentally, it takes a full industrial project to make that happen.', 'start': 146.308, 'duration': 5.244}], 'summary': "Building self-driving cars requires more than just algorithms, it's a full industrial project.", 'duration': 25.357, 'max_score': 126.195, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg126195.jpg'}, {'end': 221.091, 'src': 'heatmap', 'start': 175.873, 'weight': 1, 'content': [{'end': 176.874, 'text': "It's a very hot topic.", 'start': 175.873, 'duration': 1.001}, {'end': 179.155, 'text': 'And for very good reasons.', 'start': 178.275, 'duration': 0.88}, {'end': 184.732, 'text': 'I can tell you for sure that 2017 It has been a great year for Waymo.', 'start': 180.055, 'duration': 4.677}, {'end': 190.208, 'text': 'Actually, only a year ago, in January 2017, Waymo became its own company.', 'start': 185.053, 'duration': 5.155}, {'end': 198.88, 'text': 'So that was a major milestone and a testimony to the robustness of the solution so that we could move to a productization phase.', 'start': 191.816, 'duration': 7.064}, {'end': 207.384, 'text': 'So what you see on the picture here is our latest generation self-driving vehicle.', 'start': 199.92, 'duration': 7.464}, {'end': 211.206, 'text': 'So it is based on the Chrysler Pacifica.', 'start': 208.464, 'duration': 2.742}, {'end': 213.687, 'text': 'You can already see a bunch of sensors.', 'start': 211.526, 'duration': 2.161}, {'end': 217.749, 'text': "I'll come back to that and give you more insights on what they do and how they operate.", 'start': 214.227, 'duration': 3.522}, {'end': 221.091, 'text': "But that's the latest and greatest.", 'start': 219.21, 'duration': 1.881}], 'summary': 'Waymo became its own company in january 2017 and had a successful year, evident in their latest self-driving vehicle based on the chrysler pacifica.', 'duration': 45.218, 'max_score': 175.873, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg175873.jpg'}, {'end': 211.206, 'src': 'embed', 'start': 180.055, 'weight': 0, 'content': [{'end': 184.732, 'text': 'I can tell you for sure that 2017 It has been a great year for Waymo.', 'start': 180.055, 'duration': 4.677}, {'end': 190.208, 'text': 'Actually, only a year ago, in January 2017, Waymo became its own company.', 'start': 185.053, 'duration': 5.155}, {'end': 198.88, 'text': 'So that was a major milestone and a testimony to the robustness of the solution so that we could move to a productization phase.', 'start': 191.816, 'duration': 7.064}, {'end': 207.384, 'text': 'So what you see on the picture here is our latest generation self-driving vehicle.', 'start': 199.92, 'duration': 7.464}, {'end': 211.206, 'text': 'So it is based on the Chrysler Pacifica.', 'start': 208.464, 'duration': 2.742}], 'summary': 'Waymo had a successful 2017, becoming its own company in january and introducing the latest self-driving vehicle based on the chrysler pacifica.', 'duration': 31.151, 'max_score': 180.055, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg180055.jpg'}, {'end': 258.755, 'src': 'embed', 'start': 229.729, 'weight': 1, 'content': [{'end': 242.62, 'text': 'I personally believe and I think you will agree with me that self-driving really has the potential to deeply change the way we look about mobility and the way we move people and things around.', 'start': 229.729, 'duration': 12.891}, {'end': 254.073, 'text': "So only to cover a few aspects here, obviously I don't want to go into too many details, Safety is one of the main motivations.", 'start': 243.641, 'duration': 10.432}, {'end': 258.755, 'text': '94% of US crashes today involve human errors.', 'start': 254.093, 'duration': 4.662}], 'summary': 'Self-driving has the potential to transform mobility; 94% of us crashes involve human errors.', 'duration': 29.026, 'max_score': 229.729, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg229729.jpg'}, {'end': 368.823, 'src': 'embed', 'start': 310.111, 'weight': 3, 'content': [{'end': 321.722, 'text': 'obviously, self-driving technology has the potential to deeply change the way we think about traffic parking spots, urban environments, city design.', 'start': 310.111, 'duration': 11.611}, {'end': 324.845, 'text': "So that's why it's a very exciting topic.", 'start': 323.263, 'duration': 1.582}, {'end': 335.199, 'text': "So that's why we made it our mission at Waymo, is fundamentally to make it safe and easy to move people and things around.", 'start': 327.13, 'duration': 8.069}, {'end': 342.227, 'text': "So that's a nice mission, and we've been on it for a very long time.", 'start': 336.541, 'duration': 5.686}, {'end': 348.887, 'text': 'So actually, The whole adventure started close to 10 years ago in 2009.', 'start': 343.449, 'duration': 5.438}, {'end': 358.512, 'text': 'And at the time, that started under the umbrella of a Google project that you may have heard of called Chauffeur.', 'start': 348.887, 'duration': 9.625}, {'end': 365.215, 'text': 'And back in those days, so remember, we were before the deep learning days, at least in the industry.', 'start': 358.532, 'duration': 6.683}, {'end': 368.823, 'text': 'And so really back in those days.', 'start': 366.641, 'duration': 2.182}], 'summary': "Waymo's mission is to make it safe and easy to move people and things around, starting in 2009 under the chauffeur project.", 'duration': 58.712, 'max_score': 310.111, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg310111.jpg'}, {'end': 423.74, 'src': 'embed', 'start': 393.755, 'weight': 5, 'content': [{'end': 399.603, 'text': 'So the genesis for that work was to come up with a pretty aggressive objective.', 'start': 393.755, 'duration': 5.848}, {'end': 406.352, 'text': 'So the team, the first milestone for the team was to essentially assemble 10 100 mile loops.', 'start': 400.825, 'duration': 5.527}, {'end': 415.154, 'text': 'in Northern California, around Mountain View, and try and figure out so for a total of 1, 000 miles,', 'start': 408.669, 'duration': 6.485}, {'end': 423.74, 'text': 'and try and see if they could build a first system that would be able to go and drive those loops autonomously.', 'start': 415.154, 'duration': 8.586}], 'summary': 'Team aimed to assemble 10 100-mile loops in northern california, around mountain view, totaling 1,000 miles, to test an autonomous driving system.', 'duration': 29.985, 'max_score': 393.755, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg393755.jpg'}, {'end': 509.339, 'src': 'embed', 'start': 479.311, 'weight': 6, 'content': [{'end': 482.534, 'text': 'Some of them were even going through a dense urban area.', 'start': 479.311, 'duration': 3.223}, {'end': 486.038, 'text': 'So you can see San Francisco being driven.', 'start': 482.634, 'duration': 3.404}, {'end': 490.382, 'text': 'You can see Monterey, some of the Monterey centers being driven.', 'start': 486.078, 'duration': 4.304}, {'end': 496.688, 'text': "And as you'll see on the video, those truly bring dense urban area challenges.", 'start': 490.942, 'duration': 5.746}, {'end': 504.774, 'text': "So since I promised it, so here you're going to see some pictures of the driving.", 'start': 499.448, 'duration': 5.326}, {'end': 506.235, 'text': "It's kind of working.", 'start': 505.314, 'duration': 0.921}, {'end': 509.339, 'text': 'So here with better quality.', 'start': 507.697, 'duration': 1.642}], 'summary': 'Driving through dense urban areas in san francisco and monterey, showcasing challenges.', 'duration': 30.028, 'max_score': 479.311, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg479311.jpg'}], 'start': 0.089, 'title': "Waymo's self-driving journey and milestones", 'summary': "Details waymo's milestones, including driving over four million miles autonomously, the potential impact of self-driving technology on safety, accessibility, and efficiency, and the technical and industrial challenges. it also highlights waymo's mission to make safe and easy transportation, its 10-year journey from the chauffeur project to autonomous driving, and the aggressive testing of 1000-mile loops in diverse environments, including dense urban areas and challenging terrains.", 'chapters': [{'end': 310.111, 'start': 0.089, 'title': "Self-driving cars: waymo's milestones", 'summary': "Details waymo's recent milestones, including driving over four million miles autonomously, the potential impact of self-driving technology on safety, accessibility, and efficiency, and the technical and industrial challenges of building self-driving cars.", 'duration': 310.022, 'highlights': ['The potential impact of self-driving technology on safety, accessibility, and efficiency is discussed, with 94% of US crashes involving human errors and the technology having the potential to make mobility more available and cheaper for people (quantifiable data: 94% of US crashes involve human errors).', 'Waymo has recently driven over four million miles autonomously, marking a significant milestone in the development of self-driving technology (quantifiable data: Waymo has driven over four million miles autonomously).', 'The chapter details the technical and industrial challenges of building self-driving cars, including the need for advanced techniques, models, architectures, algorithms, and the full industrial project required to make it happen.', 'Waymo became its own company in January 2017, leading to a productization phase and the development of the latest generation self-driving vehicle based on the Chrysler Pacifica (quantifiable data: Waymo became its own company in January 2017).']}, {'end': 530.538, 'start': 310.111, 'title': "Waymo's self-driving journey", 'summary': "Highlights waymo's mission to make safe and easy transportation, its 10-year journey from the chauffeur project to autonomous driving, and the aggressive testing of 1000-mile loops in diverse environments, including dense urban areas and challenging terrains.", 'duration': 220.427, 'highlights': ["Waymo's mission is to make it safe and easy to move people and things around, driving its exciting focus on self-driving technology.", 'The 10-year journey from the Chauffeur project to autonomous driving, starting in 2009, marked the initial attempt to assess the possibility of self-driving by assembling a prototype vehicle with off-the-shelf sensors.', "The team's aggressive objective of assembling 10 100-mile loops in Northern California, totaling 1000 miles, to test the autonomous driving system's capabilities in various challenging environments, including small roads, two-way traffic, cliffs, busy highways, different weather conditions, bridges, and dense urban areas.", 'The challenging test routes included driving through Santa Cruz Mountains, with small roads, two-way traffic, and cliffs, as well as dense urban areas like San Francisco and Monterey, showcasing diverse urban area challenges.', "The aggressive testing involved driving through diverse terrains, such as dense urban areas, highways, Lake Tahoe, and bridges, to assess the self-driving system's adaptability to different road conditions and weather."]}], 'duration': 530.449, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg89.jpg', 'highlights': ['Waymo has recently driven over four million miles autonomously, marking a significant milestone in the development of self-driving technology', 'The potential impact of self-driving technology on safety, accessibility, and efficiency is discussed, with 94% of US crashes involving human errors and the technology having the potential to make mobility more available and cheaper for people', 'The chapter details the technical and industrial challenges of building self-driving cars, including the need for advanced techniques, models, architectures, algorithms, and the full industrial project required to make it happen', "Waymo's mission is to make it safe and easy to move people and things around, driving its exciting focus on self-driving technology", 'The 10-year journey from the Chauffeur project to autonomous driving, starting in 2009, marked the initial attempt to assess the possibility of self-driving by assembling a prototype vehicle with off-the-shelf sensors', "The team's aggressive objective of assembling 10 100-mile loops in Northern California, totaling 1000 miles, to test the autonomous driving system's capabilities in various challenging environments, including small roads, two-way traffic, cliffs, busy highways, different weather conditions, bridges, and dense urban areas", 'The challenging test routes included driving through Santa Cruz Mountains, with small roads, two-way traffic, and cliffs, as well as dense urban areas like San Francisco and Monterey, showcasing diverse urban area challenges', 'Waymo became its own company in January 2017, leading to a productization phase and the development of the latest generation self-driving vehicle based on the Chrysler Pacifica']}, {'end': 1062.885, 'segs': [{'end': 590.924, 'src': 'embed', 'start': 561.185, 'weight': 0, 'content': [{'end': 571.369, 'text': 'and Google decided that self-driving was worth pursuing and moved forward with the development of the technology and testing.', 'start': 561.185, 'duration': 10.184}, {'end': 576.892, 'text': "So we've been at it for all those years and have been working very hard on it.", 'start': 573.11, 'duration': 3.782}, {'end': 587.821, 'text': 'Historically, Waymo and I think all the other companies out there have been relying on what we call safety drivers to still sit behind the wheels.', 'start': 577.993, 'duration': 9.828}, {'end': 590.924, 'text': 'Even if the car is driving autonomously,', 'start': 587.921, 'duration': 3.003}], 'summary': "Google's waymo has been pursuing self-driving technology for years, relying on safety drivers during testing.", 'duration': 29.739, 'max_score': 561.185, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg561185.jpg'}, {'end': 717.401, 'src': 'embed', 'start': 690.585, 'weight': 2, 'content': [{'end': 698.127, 'text': 'So we started with a fairly constrained geographical area in Chandler, close to Phoenix, Arizona.', 'start': 690.585, 'duration': 7.542}, {'end': 706.91, 'text': 'And we are hardworking to expand the testing and the scope of our operating area since then.', 'start': 699.768, 'duration': 7.142}, {'end': 714.84, 'text': 'So that goes well beyond a single car a single day.', 'start': 712.298, 'duration': 2.542}, {'end': 717.401, 'text': 'Not only we do that continuously,', 'start': 715.88, 'duration': 1.521}], 'summary': 'Expanding testing beyond one car per day in chandler, close to phoenix, arizona.', 'duration': 26.816, 'max_score': 690.585, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg690585.jpg'}, {'end': 802.868, 'src': 'embed', 'start': 772.659, 'weight': 4, 'content': [{'end': 774.721, 'text': 'So 90% of the technology takes only 10% of the time.', 'start': 772.659, 'duration': 2.062}, {'end': 780.128, 'text': 'In other words, you need to 10x.', 'start': 778.447, 'duration': 1.681}, {'end': 785.653, 'text': 'You need to 10x the capabilities of your technology.', 'start': 781.149, 'duration': 4.504}, {'end': 791.298, 'text': 'You need to 10x your team size and find ways for more engineers and more researchers to collaborate together.', 'start': 785.974, 'duration': 5.324}, {'end': 794.981, 'text': 'You need to 10x the capabilities of your sensors.', 'start': 791.318, 'duration': 3.663}, {'end': 802.868, 'text': "You need to 10x fundamentally the overall quality of the system and your testing practices, as we'll see in a lot of the aspects of the program.", 'start': 795.001, 'duration': 7.867}], 'summary': 'To succeed, 10x technology, team size, sensors, and system quality.', 'duration': 30.209, 'max_score': 772.659, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg772659.jpg'}, {'end': 928.087, 'src': 'embed', 'start': 900.27, 'weight': 3, 'content': [{'end': 902.951, 'text': 'And all those directions are things that you can see in Google products today.', 'start': 900.27, 'duration': 2.681}, {'end': 910.793, 'text': "So, whether you're talking Google Assistant or Google Photos, speech recognition or even Google Maps,", 'start': 903.471, 'duration': 7.322}, {'end': 914.094, 'text': 'you can see the impact of deep learning in all those areas.', 'start': 910.793, 'duration': 3.301}, {'end': 920.56, 'text': 'And actually, many years ago, I myself was part of the Street View team.', 'start': 915.475, 'duration': 5.085}, {'end': 928.087, 'text': 'And I was leading an internal program, an internal project that we called Street Smart.', 'start': 922.141, 'duration': 5.946}], 'summary': 'Google products today reflect deep learning impact, including google assistant, google photos, speech recognition, and google maps.', 'duration': 27.817, 'max_score': 900.27, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg900270.jpg'}], 'start': 532.62, 'title': "Waymo's self-driving progress", 'summary': "Discusses waymo's completion of 10 loops in san francisco in 2010, and their recent milestone in november, achieving confidence in their system to operate driverless cars in phoenix, including a video demonstration of passengers using a self-driving car-hailing service with no safety driver. it also covers the expansion of self-driving car testing and deployment in chandler, arizona, emphasizing the significant effort and time required to develop safe technology for public roads and the impact of google's deep learning technology in various google products, including google maps.", 'chapters': [{'end': 689.324, 'start': 532.62, 'title': "Waymo's progress in self-driving technology", 'summary': "Discusses waymo's successful completion of 10 loops in san francisco in 2010, and the recent milestone in november where they achieved confidence in their system to operate driverless cars in the phoenix area, presenting a video demonstration of passengers using a self-driving car-hailing service with no safety driver.", 'duration': 156.704, 'highlights': ['Waymo completed 10 loops in San Francisco autonomously back in 2010, showcasing early success and experience in self-driving technology.', 'In November, Waymo achieved a level of confidence in their system that allowed them to operate driverless cars in the Phoenix area, marking a significant milestone in their development.', 'Waymo presented a video demonstration of a self-driving car-hailing service in which passengers used the car with no safety driver present, highlighting the progress towards fully autonomous operations.']}, {'end': 1062.885, 'start': 690.585, 'title': 'Chandler expansion and deep learning breakthroughs', 'summary': "Discusses the expansion of testing and deployment of self-driving cars in chandler, arizona, emphasizing the significant effort and time required to develop safe technology for public roads and the rise of deep learning, particularly the impact of google's deep learning technology in various google products, including google maps.", 'duration': 372.3, 'highlights': ['The significant effort and time required to develop safe technology for public roads, with the realization that when a technology is 90% done, it still requires 90% more work, emphasizing the need to 10x the capabilities of technology, team size, sensors, overall system quality, and testing practices.', "The rise of deep learning and the impact of Google's deep learning technology in various Google products, including Google Assistant, Google Photos, speech recognition, Google Maps, and its role in analyzing street view imagery to improve mapping strategies, address lookups, and ETA predictions.", 'The expansion of testing and deployment of self-driving cars in Chandler, Arizona, going beyond a single car a day to a growing fleet of self-driving cars, with the intention of a product launch soon, highlighting the ongoing effort to expand the operating area and achieve depth of understanding and perfection in the technology.', 'The insider view of the rise of deep learning, including the breakthroughs in algorithm development and the role of Google Brain Team in leading research and developing machine learning tools and infrastructure, pushing the field in various directions such as computer vision, speech understanding, and NLP.']}], 'duration': 530.265, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg532620.jpg', 'highlights': ['Waymo achieved confidence in their system to operate driverless cars in Phoenix, marking a significant milestone in their development.', 'Waymo presented a video demonstration of a self-driving car-hailing service with no safety driver, highlighting progress towards fully autonomous operations.', 'The expansion of testing and deployment of self-driving cars in Chandler, Arizona, going beyond a single car a day to a growing fleet, with the intention of a product launch soon.', 'The rise of deep learning and its impact on various Google products, including Google Maps, Google Assistant, and Google Photos.', 'The significant effort and time required to develop safe technology for public roads, emphasizing the need to 10x the capabilities of technology, team size, sensors, overall system quality, and testing practices.']}, {'end': 1478.76, 'segs': [{'end': 1096.463, 'src': 'embed', 'start': 1062.885, 'weight': 0, 'content': [{'end': 1072.99, 'text': "because there's no point having pixels if you cannot understand the number that's on the facade all the way to properly geolocalizing it so that you can then put it on Google Maps.", 'start': 1062.885, 'duration': 10.105}, {'end': 1082.432, 'text': "The first deep learning application that succeeded in production, and that's all the way back to 2012,", 'start': 1076.388, 'duration': 6.044}, {'end': 1096.463, 'text': 'that we had the first system in production was really the first breakthrough that we had across Alphabet on our ability to properly understand real scene situations.', 'start': 1082.432, 'duration': 14.031}], 'summary': 'First deep learning application succeeded in production in 2012, marking a breakthrough for alphabet in understanding real scene situations.', 'duration': 33.578, 'max_score': 1062.885, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1062885.jpg'}, {'end': 1211.788, 'src': 'embed', 'start': 1130.223, 'weight': 1, 'content': [{'end': 1139.949, 'text': "obviously that's in Paris where we have even more imagery, so more views of those physical numbers, that if you're able to triangulate,", 'start': 1130.223, 'duration': 9.726}, {'end': 1143.331, 'text': "you're able to localize them very accurately and have very accurate maps.", 'start': 1139.949, 'duration': 3.382}, {'end': 1149.575, 'text': "So the last example I'm going to show is in Cape Town in South Africa.", 'start': 1144.852, 'duration': 4.723}, {'end': 1155.958, 'text': 'where again, the impact of that deep learning work has been huge in terms of quality.', 'start': 1150.791, 'duration': 5.167}, {'end': 1163.167, 'text': 'So many countries today actually have up to more than 95% of addresses mapped that way.', 'start': 1156.558, 'duration': 6.609}, {'end': 1167.335, 'text': 'So doing similar things.', 'start': 1166.414, 'duration': 0.921}, {'end': 1175.683, 'text': 'So obviously, you can see a lot of parallelism between that work on 3D imagery and doing the same on the real scene on the car.', 'start': 1167.355, 'duration': 8.328}, {'end': 1179.406, 'text': 'But obviously, doing that on the car is even harder.', 'start': 1176.663, 'duration': 2.743}, {'end': 1186.753, 'text': "It's even harder because you need to do that real time and very quickly with low latency.", 'start': 1179.426, 'duration': 7.327}, {'end': 1191.137, 'text': 'And you also need to do that in an embedded system.', 'start': 1188.195, 'duration': 2.942}, {'end': 1196.119, 'text': 'So the cars have to be entirely autonomous.', 'start': 1191.737, 'duration': 4.382}, {'end': 1200.142, 'text': 'You cannot rely on a connection to a Google data center.', 'start': 1197.1, 'duration': 3.042}, {'end': 1204.424, 'text': "And first, you don't have the time in terms of latency to bring data back and forth.", 'start': 1200.622, 'duration': 3.802}, {'end': 1209.006, 'text': 'But also, you cannot rely on a connection for the safe operation of your system.', 'start': 1205.765, 'duration': 3.241}, {'end': 1211.788, 'text': 'So you need to do the processing within the car.', 'start': 1209.387, 'duration': 2.401}], 'summary': "Google's deep learning work has mapped over 95% of addresses in many countries, impacting quality and autonomy in car operations.", 'duration': 81.565, 'max_score': 1130.223, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1130223.jpg'}], 'start': 1062.885, 'title': 'Impact of deep learning on address mapping', 'summary': 'Discusses the impact of deep learning on address mapping, showcasing examples from sao paulo, paris, and cape town, with more than 95% of addresses mapped in many countries, and emphasizes the challenges of implementing real-time processing in autonomous cars.', 'chapters': [{'end': 1108.38, 'start': 1062.885, 'title': 'Deep learning breakthrough in scene understanding', 'summary': "Discusses the first successful deep learning application in production from 2012, which significantly improved alphabet's ability to understand real scene situations, leading to the proper geolocalization and visualization of pixels on google maps.", 'duration': 45.495, 'highlights': ['The first successful deep learning application in production was in 2012, marking a significant breakthrough for Alphabet in understanding real scene situations and geolocalization for Google Maps.']}, {'end': 1478.76, 'start': 1108.38, 'title': 'Impact of deep learning on address mapping', 'summary': 'Discusses the impact of deep learning on address mapping, showcasing examples from sao paulo, paris, and cape town, with more than 95% of addresses mapped in many countries, and emphasizes the challenges of implementing real-time processing in autonomous cars.', 'duration': 370.38, 'highlights': ['The impact of deep learning on address mapping, with more than 95% of addresses mapped in many countries. The deep learning work has had a huge impact, with many countries today having up to more than 95% of addresses mapped that way.', 'Challenges of implementing real-time processing in autonomous cars. Implementing real-time processing in autonomous cars is challenging due to the need for low latency, processing within the car, and the absence of reliance on a connection to a data center.', 'Examples of the impact of deep learning on address mapping in Sao Paulo, Paris, and Cape Town. The examples from Sao Paulo, Paris, and Cape Town showcase the consistent view of the addressing scheme, accurate localization of physical numbers, and huge impact in terms of quality in South Africa.']}], 'duration': 415.875, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1062885.jpg', 'highlights': ['The first successful deep learning application in production was in 2012, marking a significant breakthrough for Alphabet in understanding real scene situations and geolocalization for Google Maps.', 'The impact of deep learning on address mapping, with more than 95% of addresses mapped in many countries. The deep learning work has had a huge impact, with many countries today having up to more than 95% of addresses mapped that way.', 'Examples of the impact of deep learning on address mapping in Sao Paulo, Paris, and Cape Town. The examples from Sao Paulo, Paris, and Cape Town showcase the consistent view of the addressing scheme, accurate localization of physical numbers, and huge impact in terms of quality in South Africa.', 'Challenges of implementing real-time processing in autonomous cars. Implementing real-time processing in autonomous cars is challenging due to the need for low latency, processing within the car, and the absence of reliance on a connection to a data center.']}, {'end': 1834.253, 'segs': [{'end': 1510.199, 'src': 'embed', 'start': 1480.281, 'weight': 2, 'content': [{'end': 1486.125, 'text': 'The other big input, obviously, is what sensors are gonna give you once you get on the spot.', 'start': 1480.281, 'duration': 5.844}, {'end': 1495.094, 'text': "So sensor data is the signal that's gonna tell you what is not like what you mapped and the things.", 'start': 1487.751, 'duration': 7.343}, {'end': 1496.514, 'text': 'is the traffic light red or green??', 'start': 1495.094, 'duration': 1.42}, {'end': 1498.515, 'text': 'Where are the pedestrians??', 'start': 1497.074, 'duration': 1.441}, {'end': 1499.095, 'text': 'Where are the cars?', 'start': 1498.535, 'duration': 0.56}, {'end': 1499.635, 'text': 'What are they doing?', 'start': 1499.135, 'duration': 0.5}, {'end': 1510.199, 'text': 'So, as we saw on the initial picture, we have quite a set of sensors on our self-driving cars.', 'start': 1502.056, 'duration': 8.143}], 'summary': 'Sensor data informs self-driving cars of surroundings and traffic conditions.', 'duration': 29.918, 'max_score': 1480.281, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1480281.jpg'}, {'end': 1656.26, 'src': 'embed', 'start': 1569.886, 'weight': 0, 'content': [{'end': 1577.211, 'text': "Or it's much harder and computationally expensive to get depth information out of camera systems.", 'start': 1569.886, 'duration': 7.325}, {'end': 1585.317, 'text': 'So systems like LiDAR, for instance, when you hit objects, will give you a very good depth estimation.', 'start': 1577.912, 'duration': 7.405}, {'end': 1589.881, 'text': "But obviously, they're going to lack a lot of the semantic information that you will find on camera systems.", 'start': 1585.337, 'duration': 4.544}, {'end': 1595.328, 'text': 'So all those sensors are designed to be complementary in terms of their capabilities.', 'start': 1590.521, 'duration': 4.807}, {'end': 1604.06, 'text': 'It goes without saying that the better your sensors are, the better your perception system is going to be.', 'start': 1597.23, 'duration': 6.83}, {'end': 1618.107, 'text': "That's why, at Waymo, we took the path of designing our own sensors in-house and enhancing what's available off the shelf today,", 'start': 1606.055, 'duration': 12.052}, {'end': 1625.215, 'text': 'because it seemed important for us to go all the way to be able to build a self-driving system that we could believe in.', 'start': 1618.107, 'duration': 7.108}, {'end': 1631.844, 'text': "And so that's what perception does.", 'start': 1629.082, 'duration': 2.762}, {'end': 1636.287, 'text': 'Takes those two inputs and build a representation of the scene.', 'start': 1632.204, 'duration': 4.083}, {'end': 1646.875, 'text': 'So, at the end of the day, you have to realize that, in nature, that work of perception is really what differentiates deeply.', 'start': 1637.248, 'duration': 9.627}, {'end': 1656.26, 'text': 'differentiates what you need to do in a self-driving system as opposed to a lower-level driving assistance system.', 'start': 1646.875, 'duration': 9.385}], 'summary': 'Lidar provides good depth estimation, but lacks semantic information. waymo designs own sensors for better perception.', 'duration': 86.374, 'max_score': 1569.886, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1569886.jpg'}, {'end': 1723.625, 'src': 'embed', 'start': 1696.472, 'weight': 3, 'content': [{'end': 1704.134, 'text': "you need to understand that there's a chance that that car is going to want to avoid that bicyclist is going to swerve and you need to anticipate.", 'start': 1696.472, 'duration': 7.662}, {'end': 1712.338, 'text': 'that behavior so that you can properly decide whether you want to slow down, give space for the car, or speed up and have the car go behind you.', 'start': 1704.454, 'duration': 7.884}, {'end': 1723.625, 'text': "Those are the kinds of behaviors that go well beyond not bumping into things and that require a much deeper understanding of the world that's going on around you.", 'start': 1712.959, 'duration': 10.666}], 'summary': 'Anticipate and respond to car and bicyclist behavior to navigate safely.', 'duration': 27.153, 'max_score': 1696.472, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1696472.jpg'}], 'start': 1480.281, 'title': 'Self-driving car sensors and autonomous vehicle behavior prediction', 'summary': 'Discusses the importance of sensor data in self-driving cars, emphasizing the complementary nature of different sensor types and the impact of high-quality sensors on perception systems at waymo. it also covers the challenges of autonomous vehicles in understanding and predicting complex behaviors, such as avoiding collisions with moving objects and anticipating actions of other vehicles and pedestrians, requiring a deeper understanding of the surrounding environment and the ability to predict behaviors based on various cues and factors.', 'chapters': [{'end': 1656.26, 'start': 1480.281, 'title': 'Self-driving car sensors', 'summary': 'Discusses the importance of sensor data in self-driving cars, emphasizing the complementary nature of different sensor types and the impact of high-quality sensors on perception systems at waymo.', 'duration': 175.979, 'highlights': ['At Waymo, designing their own sensors in-house was crucial for building a reliable self-driving system, highlighting the significance of high-quality sensors in enhancing perception systems.', 'Different types of sensors such as vision systems, radar, and LiDAR are designed to be complementary, providing a dense representation, depth estimation, and semantic information, thereby improving the perception system.', 'Sensor data, including information about traffic lights, pedestrians, and cars, is essential for accurately mapping the surroundings and making real-time decisions in self-driving cars.']}, {'end': 1834.253, 'start': 1656.88, 'title': 'Autonomous vehicle behavior prediction', 'summary': 'Discusses the challenges of autonomous vehicles in understanding and predicting complex behaviors, such as avoiding collisions with moving objects and anticipating actions of other vehicles and pedestrians, requiring a deeper understanding of the surrounding environment and the ability to predict behaviors based on various cues and factors.', 'duration': 177.373, 'highlights': ['The challenge of autonomous vehicles goes beyond avoiding collisions and involves understanding and predicting complex behaviors, such as anticipating a car swerving to avoid a bicyclist, and requires a deeper understanding of the environment (e.g., identifying parked cars, flashing lights, and cones) to make informed decisions (e.g., slowing down, giving space, or speeding up) to navigate through complex scenarios.', 'Advanced understanding of the environment and semantics, such as identifying flashing lights on a police car indicating an emergency vehicle and recognizing cones as obstacles, enables autonomous vehicles to make informed decisions and navigate through complex scenarios.', 'The ability to anticipate behaviors, such as predicting someone getting out of a parked car or a car swerving to avoid an obstacle, is crucial for autonomous vehicles to adjust their trajectory and make timely decisions to ensure safe navigation through complex scenarios.']}], 'duration': 353.972, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1480281.jpg', 'highlights': ['Different types of sensors at Waymo, including vision systems, radar, and LiDAR, are designed to be complementary, providing a dense representation, depth estimation, and semantic information, thereby improving the perception system.', 'At Waymo, designing their own sensors in-house was crucial for building a reliable self-driving system, highlighting the significance of high-quality sensors in enhancing perception systems.', 'Sensor data, including information about traffic lights, pedestrians, and cars, is essential for accurately mapping the surroundings and making real-time decisions in self-driving cars.', 'The challenge of autonomous vehicles involves understanding and predicting complex behaviors, such as anticipating a car swerving to avoid a bicyclist, and requires a deeper understanding of the environment to make informed decisions to navigate through complex scenarios.', 'Advanced understanding of the environment and semantics enables autonomous vehicles to make informed decisions and navigate through complex scenarios.', 'The ability to anticipate behaviors, such as predicting someone getting out of a parked car or a car swerving to avoid an obstacle, is crucial for autonomous vehicles to adjust their trajectory and make timely decisions to ensure safe navigation through complex scenarios.']}, {'end': 2446.601, 'segs': [{'end': 1862.103, 'src': 'embed', 'start': 1834.253, 'weight': 0, 'content': [{'end': 1837.996, 'text': 'is something you need to understand in order to properly and safely drive.', 'start': 1834.253, 'duration': 3.743}, {'end': 1843.688, 'text': 'And only then, only when you have that depth of understanding,', 'start': 1840.225, 'duration': 3.463}, {'end': 1851.735, 'text': 'you can start to come up with realistic behavior predictions and trajectory predictions for all those agents on the scene,', 'start': 1843.688, 'duration': 8.047}, {'end': 1855.378, 'text': 'so that you can come up with a proper strategy for your planning control.', 'start': 1851.735, 'duration': 3.643}, {'end': 1862.103, 'text': 'So how is deep learning playing into that whole space?', 'start': 1858.14, 'duration': 3.963}], 'summary': 'Deep learning is crucial for understanding driving and making behavior predictions for agents on the road.', 'duration': 27.85, 'max_score': 1834.253, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1834253.jpg'}, {'end': 1989.546, 'src': 'embed', 'start': 1938.351, 'weight': 2, 'content': [{'end': 1946.759, 'text': "the whole bunch of data coming off your sensors is a very important task, because that reduces the computation you're gonna have to do,", 'start': 1938.351, 'duration': 8.408}, {'end': 1949.782, 'text': 'but also key to operate safely.', 'start': 1946.759, 'duration': 3.023}, {'end': 1954.986, 'text': 'A more subtle one, but important one, are around reflections.', 'start': 1951.184, 'duration': 3.802}, {'end': 1958.189, 'text': 'So we are driving a scene.', 'start': 1956.107, 'duration': 2.082}, {'end': 1960.23, 'text': "There's a car here.", 'start': 1958.789, 'duration': 1.441}, {'end': 1963.253, 'text': 'On the camera picture, the car is reflected in a bus.', 'start': 1960.651, 'duration': 2.602}, {'end': 1972.86, 'text': 'And if you just do a naive detection, especially if the bus moves along with you, which is very typical and everything moves,', 'start': 1964.033, 'duration': 8.827}, {'end': 1974.662, 'text': "then all of a sudden you're going to have two cars.", 'start': 1972.86, 'duration': 1.802}, {'end': 1975.822, 'text': 'on the scene.', 'start': 1975.402, 'duration': 0.42}, {'end': 1981.944, 'text': "And if you take that car too seriously, all the way to impacting your behavior, obviously, you're going to make mistakes.", 'start': 1976.362, 'duration': 5.582}, {'end': 1989.546, 'text': 'So here, I showed you an example of reflections on the visual range.', 'start': 1983.584, 'duration': 5.962}], 'summary': 'Managing sensor data is crucial for reducing computation and ensuring safety, while handling reflections is vital for accurate scene interpretation and avoiding erroneous behavior.', 'duration': 51.195, 'max_score': 1938.351, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1938351.jpg'}, {'end': 2111.075, 'src': 'embed', 'start': 2082.754, 'weight': 4, 'content': [{'end': 2086.297, 'text': "So, if you're not familiar with convolution layers,", 'start': 2082.754, 'duration': 3.543}, {'end': 2102.012, 'text': "that's a very popular way to do computer vision because it relies on connecting neurons with kernels that are going to learn layer after layer features of the imagery.", 'start': 2086.297, 'duration': 15.715}, {'end': 2111.075, 'text': "So those kernels typically work locally on the sub-region of the image and they're going to understand lines.", 'start': 2102.312, 'duration': 8.763}], 'summary': 'Convolution layers are a popular method in computer vision for learning features of imagery through connected neurons and kernels.', 'duration': 28.321, 'max_score': 2082.754, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg2082754.jpg'}, {'end': 2424.087, 'src': 'embed', 'start': 2400.897, 'weight': 1, 'content': [{'end': 2412.21, 'text': 'So here you would train a deep net that would directly take the whole projection of your sensor data and output boxes that encode the priors you have.', 'start': 2400.897, 'duration': 11.313}, {'end': 2417.285, 'text': 'So here, for instance, I can show you how such a thing would work for cone detection.', 'start': 2413.223, 'duration': 4.062}, {'end': 2424.087, 'text': "So you can see that we don't have all the fidelity of the per pixel cone detection, but we don't really care about that.", 'start': 2417.885, 'duration': 6.202}], 'summary': 'Train deep net for sensor data, output encoding boxes for cone detection.', 'duration': 23.19, 'max_score': 2400.897, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg2400897.jpg'}], 'start': 1834.253, 'title': 'Deep learning and sensor data challenges in driving safety', 'summary': 'Covers the impact of deep learning on driving safety, emphasizing realistic behavior predictions. it also discusses challenges in sensor data processing, focusing on efficient object detection and the need for filtering and handling reflections.', 'chapters': [{'end': 1889.453, 'start': 1834.253, 'title': 'Deep learning in driving safety', 'summary': 'Discusses the need for deep understanding in driving to make realistic behavior and trajectory predictions for agents on the scene, emphasizing the impact of deep learning in solving driving safety problems.', 'duration': 55.2, 'highlights': ['Deep understanding is essential for making realistic behavior and trajectory predictions for driving safety.', 'Deep learning plays a significant role in solving driving safety problems.', 'Having sensors in real life for driving is emphasized as a big piece of the puzzle.']}, {'end': 2446.601, 'start': 1890.174, 'title': 'Challenges in sensor data processing', 'summary': 'Discusses the challenges in processing sensor data for autonomous driving, including the need for filtering, handling reflections, and applying convolution layers, emphasizing the importance of efficient and accurate object detection.', 'duration': 556.427, 'highlights': ['Efficient Object Detection: Using convolution layers and priors for efficient object detection, such as detecting snow patches and cones, contributes to better scene understanding and processing.', 'Handling Reflections: Dealing with reflections, such as those caused by cars being reflected in buses, is crucial to prevent misinterpretation and potential driving mistakes.', 'Filtering Sensor Data: The importance of filtering sensor data to reduce computation and ensure safe operation, such as ignoring irrelevant LIDAR laser points triggered by exhaust smoke, is emphasized.', 'Challenges of Sparse Data: Discussing the challenges of processing sparse sensor data and the need to project it into 2D planes for effective processing and analysis.', 'Utilizing Convolution Layers: The use of convolution layers for computer vision, which understand lines, contours, and higher-level features, is highlighted as a common and efficient technique in processing imagery data.']}], 'duration': 612.348, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg1834253.jpg', 'highlights': ['Deep learning is crucial for realistic behavior and trajectory predictions for driving safety.', 'Efficient object detection using convolution layers and priors contributes to better scene understanding.', 'Handling reflections is crucial to prevent misinterpretation and potential driving mistakes.', 'Filtering sensor data is important to reduce computation and ensure safe operation.', 'Utilizing convolution layers for computer vision is a common and efficient technique in processing imagery data.']}, {'end': 3293.125, 'segs': [{'end': 2501.801, 'src': 'embed', 'start': 2477.813, 'weight': 0, 'content': [{'end': 2485.845, 'text': 'So school buses are not really emergency vehicles, but obviously, whether the bus has lights on or the bus has the stop sign open on the side,', 'start': 2477.813, 'duration': 8.032}, {'end': 2487.848, 'text': 'carry heavy semantics that you need to understand.', 'start': 2485.845, 'duration': 2.003}, {'end': 2492.074, 'text': 'So how do you deal with that? Back to the deep learning techniques.', 'start': 2489.13, 'duration': 2.944}, {'end': 2501.801, 'text': 'One thing you could do is take that patch, build a new convolution tower, and build a classifier on top of that.', 'start': 2493.16, 'duration': 8.641}], 'summary': 'School buses require understanding of signals for safety, tackled using deep learning techniques.', 'duration': 23.988, 'max_score': 2477.813, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg2477813.jpg'}, {'end': 2555.168, 'src': 'embed', 'start': 2523.843, 'weight': 3, 'content': [{'end': 2528.006, 'text': 'So one better thing to do is to use embeddings.', 'start': 2523.843, 'duration': 4.163}, {'end': 2537.132, 'text': "So, if you're not familiar with it, embeddings essentially are vector representations of objects that you can learn with deep nets.", 'start': 2528.246, 'duration': 8.886}, {'end': 2541.155, 'text': 'that will carry some semantic meaning of those objects.', 'start': 2537.132, 'duration': 4.023}, {'end': 2545.919, 'text': 'So for instance, given a vehicle, you can build a vector.', 'start': 2541.195, 'duration': 4.724}, {'end': 2555.168, 'text': "that's gonna carry the information that that vehicle is a school bus, whether the lights are on, whether the stop sign is open,", 'start': 2547.065, 'duration': 8.103}], 'summary': 'Use embeddings to create vector representations of objects with semantic meaning.', 'duration': 31.325, 'max_score': 2523.843, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg2523843.jpg'}, {'end': 2699.932, 'src': 'embed', 'start': 2673.446, 'weight': 2, 'content': [{'end': 2679.33, 'text': 'And the recall you need to have on pedestrian is very high and pedestrians show up in many different situations.', 'start': 2673.446, 'duration': 5.884}, {'end': 2681.811, 'text': 'So, for instance, here you have occluded pedestrians.', 'start': 2679.35, 'duration': 2.461}, {'end': 2689.837, 'text': "that you need to see because that's a good chance when you do your behavior prediction that that person here is gonna jump out of the car and you need to be ready for that.", 'start': 2681.811, 'duration': 8.026}, {'end': 2699.932, 'text': 'So last but not least, Predicting the behavior of pedestrians is really hard, because they move in any direction.', 'start': 2691.718, 'duration': 8.214}], 'summary': 'Predicting pedestrian behavior is challenging due to their high recall and unpredictable movement.', 'duration': 26.486, 'max_score': 2673.446, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg2673446.jpg'}, {'end': 2856.242, 'src': 'embed', 'start': 2828.221, 'weight': 4, 'content': [{'end': 2833.384, 'text': 'So if you have vector representations of those objects, you can start and track them over time.', 'start': 2828.221, 'duration': 5.163}, {'end': 2847.873, 'text': 'So a common technique that you can use to get there is to use recurrent neural networks that essentially are networks that will build a state that gets better and better as it gets more observations sequential observations of a real pattern.', 'start': 2834.485, 'duration': 13.388}, {'end': 2856.242, 'text': 'So for instance, coming back to the words example I gave earlier, You have one word, you see its vector representation.', 'start': 2848.233, 'duration': 8.009}], 'summary': 'Using recurrent neural networks to track vector representations of objects over time.', 'duration': 28.021, 'max_score': 2828.221, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg2828221.jpg'}], 'start': 2450.305, 'title': 'Understanding vehicle semantics, embeddings, and deep learning', 'summary': 'Addresses challenges in understanding vehicle semantics, emphasizing the identification of emergency vehicles and school buses, and proposes deep learning techniques. it also discusses the use of embeddings to represent objects and understand their semantics, challenges in understanding pedestrian behavior in self-driving tasks, and the importance of recurrent neural networks and large-scale labeling efforts.', 'chapters': [{'end': 2522.385, 'start': 2450.305, 'title': 'Understanding vehicle semantics', 'summary': 'Discussed the challenges of understanding the semantics of different types of vehicles, emphasizing the need to identify emergency vehicles and school buses and proposing deep learning techniques to address these challenges.', 'duration': 72.08, 'highlights': ['The need to understand the semantics of emergency vehicles, such as whether they are active or not, and the significance of identifying school buses based on specific attributes like lights and stop signs.', 'Proposing the use of deep learning techniques, such as building specific classifiers for identifying school buses with lights on and stop signs open, to address the challenges of understanding vehicle semantics.', 'Highlighting the potential computational cost and complexity associated with implementing deep learning techniques for understanding vehicle semantics, particularly in the context of running it on cars.']}, {'end': 3293.125, 'start': 2523.843, 'title': 'Embeddings and deep learning for understanding objects and behavior', 'summary': 'Discusses the use of embeddings to represent objects and understand their semantics, such as the example of word embeddings in nlp, and the challenges in understanding pedestrian behavior in self-driving tasks, requiring shape priors, high recall, and prediction, and the importance of recurrent neural networks and large-scale labeling efforts.', 'duration': 769.282, 'highlights': ['The use of embeddings to represent objects and understand their semantics, such as the example of word embeddings in NLP Embeddings are vector representations of objects that carry semantic meaning, as seen in word embeddings in NLP, which can help understand the semantics of sentences.', 'Challenges in understanding pedestrian behavior in self-driving tasks, requiring shape priors, high recall, and prediction Understanding pedestrian behavior presents challenges due to their deformable nature, varied poses, and unpredictable movements, requiring high recall and prediction, as well as a fine understanding of semantics.', 'Importance of recurrent neural networks and large-scale labeling efforts for understanding behavior over time and training deep learning models Recurrent neural networks combined with vector representations can track objects over time, providing a better understanding of the scene, and large-scale labeling efforts are crucial for training deep learning models in supervised tasks.']}], 'duration': 842.82, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg2450305.jpg', 'highlights': ['Proposing the use of deep learning techniques for identifying school buses with specific attributes.', 'The need to understand the semantics of emergency vehicles and school buses.', 'Challenges in understanding pedestrian behavior in self-driving tasks, requiring shape priors and high recall.', 'The use of embeddings to represent objects and understand their semantics.', 'Importance of recurrent neural networks and large-scale labeling efforts for understanding behavior over time.']}, {'end': 4009.563, 'segs': [{'end': 3369.714, 'src': 'embed', 'start': 3326.311, 'weight': 0, 'content': [{'end': 3331.253, 'text': 'One is around real world driving, another one is around simulation, and the last one is around the structure testing.', 'start': 3326.311, 'duration': 4.942}, {'end': 3332.254, 'text': "So I'll come back to that.", 'start': 3331.634, 'duration': 0.62}, {'end': 3337.291, 'text': 'In terms of real-world driving, obviously, there is no way around it.', 'start': 3333.669, 'duration': 3.622}, {'end': 3343.435, 'text': 'If you want to encounter situations and see and understand how you behave, you need to drive.', 'start': 3338.572, 'duration': 4.863}, {'end': 3349.458, 'text': 'So as you can see, the driving at Waymo has been accelerating over time, still is accelerating.', 'start': 3344.475, 'duration': 4.983}, {'end': 3357.403, 'text': 'So we crossed three million miles driven back in May 2017, and only six months later, back in November.', 'start': 3349.478, 'duration': 7.925}, {'end': 3361.53, 'text': "We reached 4 million, so that's an accelerating pace.", 'start': 3358.888, 'duration': 2.642}, {'end': 3365.072, 'text': 'Obviously, not every mile is equal.', 'start': 3363.371, 'duration': 1.701}, {'end': 3369.714, 'text': 'And what you care about are the miles that carry new situations and important situations.', 'start': 3365.352, 'duration': 4.362}], 'summary': 'Waymo has accelerated real-world driving, reaching 4 million miles in november 2017.', 'duration': 43.403, 'max_score': 3326.311, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg3326311.jpg'}, {'end': 3527.406, 'src': 'embed', 'start': 3498.114, 'weight': 2, 'content': [{'end': 3506.576, 'text': "So using Google's infrastructure, we have the ability to run a virtual fleet of 25, 000 cars 24-7 in data centers.", 'start': 3498.114, 'duration': 8.462}, {'end': 3517.538, 'text': "So those are software stacks that emulate the driving across either road miles that we've driven or modified miles that help us understand the behavior of the software.", 'start': 3506.676, 'duration': 10.862}, {'end': 3527.406, 'text': 'So, to give you another magnitude, last year alone, we drove 2.5 billion of those miles in data centers, right?', 'start': 3518.959, 'duration': 8.447}], 'summary': "Google's infrastructure can run a virtual fleet of 25,000 cars 24-7, driving 2.5 billion miles in data centers.", 'duration': 29.292, 'max_score': 3498.114, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg3498114.jpg'}, {'end': 3582.038, 'src': 'embed', 'start': 3557.918, 'weight': 3, 'content': [{'end': 3565.024, 'text': 'So we do that in a 90-acre testing facility on a former Air Force base in central California.', 'start': 3557.918, 'duration': 7.106}, {'end': 3574.932, 'text': 'that we set up with traffic lights, railroad crossings, truly trying to reproduce a real-life situation,', 'start': 3566.205, 'duration': 8.727}, {'end': 3582.038, 'text': "and where we set up very specific scenarios that we haven't necessarily encountered during regular driving, but that we want to test.", 'start': 3574.932, 'duration': 7.106}], 'summary': 'A 90-acre testing facility in central california replicates real-life situations for specific scenarios.', 'duration': 24.12, 'max_score': 3557.918, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg3557918.jpg'}, {'end': 3684.247, 'src': 'embed', 'start': 3659.65, 'weight': 4, 'content': [{'end': 3668.112, 'text': 'The first one is around growing what we call ODD, so Operating Design Domain.', 'start': 3659.65, 'duration': 8.462}, {'end': 3681.023, 'text': 'So, extending our fleet of self-driving cars, not only geographically, um so, geographically, meaning, uh, going into, uh, deploying into urban cores,', 'start': 3669.393, 'duration': 11.63}, {'end': 3684.247, 'text': 'uh, deploying into different weather conditions.', 'start': 3681.023, 'duration': 3.224}], 'summary': 'Extending the fleet of self-driving cars to new geographies and weather conditions to grow the operating design domain (odd).', 'duration': 24.597, 'max_score': 3659.65, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg3659650.jpg'}, {'end': 3837.565, 'src': 'heatmap', 'start': 3781.318, 'weight': 0.776, 'content': [{'end': 3786.301, 'text': 'and how deep the roots are all the way back in time.', 'start': 3781.318, 'duration': 4.983}, {'end': 3798.99, 'text': 'My second objective was to give you to tie in some of the technical algorithmic solutions that you may have talked about during that class into the practical cases we need to solve in the production system.', 'start': 3787.322, 'duration': 11.668}, {'end': 3801.25, 'text': 'And last but not least,', 'start': 3799.93, 'duration': 1.32}, {'end': 3813.153, 'text': 'really emphasize the scale and the engineering infrastructure work that needs to happen to really take such a project into attrition in a production system.', 'start': 3801.25, 'duration': 11.903}, {'end': 3815.733, 'text': 'Last tweet.', 'start': 3815.193, 'duration': 0.54}, {'end': 3823.095, 'text': "That's a scene with kids jumping on bags as Frogger across the scene.", 'start': 3817.554, 'duration': 5.541}, {'end': 3826.015, 'text': 'And I think we have time for a few questions.', 'start': 3824.075, 'duration': 1.94}, {'end': 3828.616, 'text': 'So maybe a hand of thank you.', 'start': 3826.535, 'duration': 2.081}, {'end': 3837.565, 'text': 'Yeah, I was wondering, you showed your car craft simulation a little bit.', 'start': 3834.684, 'duration': 2.881}], 'summary': 'The talk covers technical solutions, scale, and engineering work for production systems.', 'duration': 56.247, 'max_score': 3781.318, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg3781318.jpg'}], 'start': 3294.468, 'title': "Machine learning testing methods and waymo's accelerating driving efforts", 'summary': "Discusses the importance of real-world driving, simulation, and structured testing in ensuring machine learning readiness, and highlights waymo's achievement of reaching 4 million miles in six months, with the ability to simulate and augment miles for efficient testing, and expanding their testing domains and semantic understanding.", 'chapters': [{'end': 3343.435, 'start': 3294.468, 'title': 'Machine learning testing methods', 'summary': 'Discusses the three legs used to ensure machine learning readiness for production: real-world driving, simulation, and structured testing, with an emphasis on the importance of real-world driving for encountering and understanding various situations.', 'duration': 48.967, 'highlights': ['Real-world driving is crucial for encountering situations and understanding behavior, necessary for testing machine learning at scale and with a high safety bar.', 'The three legs used to ensure machine learning readiness for production are real-world driving, simulation, and structured testing.', 'Machine learning at scale and with a high safety bar requires extensive testing, with real-world driving being a key component.', 'Understanding how the self-driving system behaves in various situations is essential, necessitating real-world driving for comprehensive testing.']}, {'end': 4009.563, 'start': 3344.475, 'title': "Waymo's accelerating driving and simulation efforts", 'summary': "Discusses how waymo's driving has accelerated over time, reaching 4 million miles in six months, with the ability to simulate and augment miles for efficient testing, run a virtual fleet of 25,000 cars 24/7, and set up a testing facility on a former air force base in central california, while focusing on expanding the testing domains and semantic understanding.", 'duration': 665.088, 'highlights': ["Waymo's driving has accelerated over time, reaching 4 million miles in six months, with the ability to simulate and augment miles for efficient testing. Waymo crossed three million miles driven back in May 2017, and only six months later, back in November, reached 4 million miles, demonstrating an accelerating pace. The ability to simulate and augment driven miles allows for efficient testing of self-driving systems.", "Waymo has the ability to run a virtual fleet of 25,000 cars 24/7 in data centers, driving 2.5 billion miles in data centers last year. Using Google's infrastructure, Waymo can run a virtual fleet of 25,000 cars 24/7 in data centers, driving 2.5 billion miles in data centers last year, expanding the understanding of the self-driving system's behavior by three orders of magnitude.", 'Waymo has set up a 90-acre testing facility on a former Air Force base in central California to reproduce real-life driving situations and test specific scenarios not encountered during regular driving. Waymo has established a 90-acre testing facility on a former Air Force base in central California, equipped with traffic lights, railroad crossings, and specific scenarios for testing, aiming to reproduce real-life driving situations and further augment the simulation for testing self-driving systems.', 'Waymo emphasizes expanding testing domains and semantic understanding, such as growing the Operating Design Domain and focusing on semantic understanding of driving scenes. Waymo is extending its fleet geographically, deploying into different urban cores and weather conditions, and focusing on semantic understanding of driving scenes to ensure self-driving cars can navigate complex environments and scenarios safely.']}], 'duration': 715.095, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg3294468.jpg', 'highlights': ['Waymo reached 4 million miles in six months, accelerating pace, and ability to simulate and augment miles for efficient testing.', 'Real-world driving is crucial for encountering situations and understanding behavior, necessary for testing machine learning at scale and with a high safety bar.', "Waymo can run a virtual fleet of 25,000 cars 24/7 in data centers, driving 2.5 billion miles last year, expanding the understanding of the self-driving system's behavior.", 'Waymo established a 90-acre testing facility on a former Air Force base in central California, equipped with traffic lights, railroad crossings, and specific scenarios for testing.', 'Waymo emphasizes expanding testing domains and semantic understanding, such as growing the Operating Design Domain and focusing on semantic understanding of driving scenes.']}, {'end': 4399.329, 'segs': [{'end': 4035.8, 'src': 'embed', 'start': 4009.563, 'weight': 0, 'content': [{'end': 4013.786, 'text': 'grow the number of people who will be able to productively participate in your engineering project.', 'start': 4009.563, 'duration': 4.223}, {'end': 4020.031, 'text': "And that's where the robustness we need to bring into our development environment.", 'start': 4014.887, 'duration': 5.144}, {'end': 4027.296, 'text': 'our testing is really key to be able to grow that team at the biggest scale.', 'start': 4020.031, 'duration': 7.265}, {'end': 4030.934, 'text': 'and essentially explore all those paths and come up with the best one.', 'start': 4028.351, 'duration': 2.583}, {'end': 4035.8, 'text': 'And at the end of the day, the robustness of testing is the judge.', 'start': 4031.355, 'duration': 4.445}], 'summary': 'Enhance engineering team by prioritizing robust testing for scalability.', 'duration': 26.237, 'max_score': 4009.563, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg4009563.jpg'}, {'end': 4132.281, 'src': 'embed', 'start': 4104.937, 'weight': 1, 'content': [{'end': 4108.779, 'text': "But the sixth one that happened, if you don't generalize, actually is going to fall through.", 'start': 4104.937, 'duration': 3.842}, {'end': 4119.371, 'text': 'So really the complexity of what you need to do is extract the core principles that make you safely drive.', 'start': 4110.261, 'duration': 9.11}, {'end': 4126.298, 'text': 'And have the algorithms learn those principles rather than the specifics of any situation.', 'start': 4120.993, 'duration': 5.305}, {'end': 4132.281, 'text': 'Because as you said, the parameter space of a real scene is infinite.', 'start': 4127.059, 'duration': 5.222}], 'summary': 'To safely drive, extract core principles for algorithms to learn, as real scene parameter space is infinite.', 'duration': 27.344, 'max_score': 4104.937, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg4104937.jpg'}, {'end': 4226.604, 'src': 'embed', 'start': 4194.842, 'weight': 4, 'content': [{'end': 4202.587, 'text': "I mean if you're, many different types of snow could actually have pretty different impacts on driving,", 'start': 4194.842, 'duration': 7.745}, {'end': 4207.91, 'text': 'whether it be just like a flurry or if it were to be the kind of like a really heavy blizzard like we just had.', 'start': 4202.587, 'duration': 5.323}, {'end': 4218.477, 'text': 'Yeah, I think if you look at it from an algorithmic point of view, that may make sense.', 'start': 4207.93, 'duration': 10.547}, {'end': 4226.604, 'text': "Maybe something I'd like to emphasize a little more is the very hard line to walk.", 'start': 4220.799, 'duration': 5.805}], 'summary': 'Different snow types have varying impacts on driving, from flurries to heavy blizzards. emphasizing the difficulty of walking the line.', 'duration': 31.762, 'max_score': 4194.842, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg4194842.jpg'}, {'end': 4284.919, 'src': 'embed', 'start': 4257.722, 'weight': 3, 'content': [{'end': 4261.504, 'text': 'Even more importantly, having, for instance,', 'start': 4257.722, 'duration': 3.782}, {'end': 4267.268, 'text': "it wouldn't make sense to have a behavior prediction on every snowflake of the things you see on the side of the road right?", 'start': 4261.504, 'duration': 5.764}, {'end': 4268.869, 'text': 'And you need to group.', 'start': 4267.548, 'duration': 1.321}, {'end': 4270.21, 'text': "that's the whole point of segmentation.", 'start': 4268.869, 'duration': 1.341}, {'end': 4280.816, 'text': 'You need to group what you see into semantic objects that are likely to exhibit a behavior as a whole and reason at that level of abstraction,', 'start': 4270.25, 'duration': 10.566}, {'end': 4284.919, 'text': 'to have a meaningful semantic understanding that you need to drive essentially right?', 'start': 4280.816, 'duration': 4.103}], 'summary': 'Segmentation groups objects for behavior prediction to drive with meaningful semantic understanding.', 'duration': 27.197, 'max_score': 4257.722, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg4257722.jpg'}, {'end': 4393.447, 'src': 'embed', 'start': 4348.929, 'weight': 5, 'content': [{'end': 4354.451, 'text': 'But really, different sensors or different systems are not going to make the same mistakes.', 'start': 4348.929, 'duration': 5.522}, {'end': 4356.372, 'text': "And so they're going to complement each other.", 'start': 4355.151, 'duration': 1.221}, {'end': 4360.253, 'text': "And that's a very important piece of redundancy that we build into the system.", 'start': 4356.392, 'duration': 3.861}, {'end': 4368.597, 'text': 'The other one is also, even in the reflection case, is in the understanding.', 'start': 4360.954, 'duration': 7.643}, {'end': 4376.86, 'text': "So the way you as a human wouldn't be fooled is because you understand and you know it's not a thing that can happen.", 'start': 4368.697, 'duration': 8.163}, {'end': 4384.283, 'text': "The same way you know that car reflecting in the bus, there's no way you can see through the bus of a real car behind it.", 'start': 4376.9, 'duration': 7.383}, {'end': 4393.447, 'text': 'So that level of semantic understanding is what is going to tell you what is true and what is not, or what is a mistake, an error in your stack.', 'start': 4384.864, 'duration': 8.583}], 'summary': 'Different sensors provide redundancy and complement each other to reduce errors in the system, emphasizing the importance of semantic understanding for distinguishing between true and false information.', 'duration': 44.518, 'max_score': 4348.929, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg4348929.jpg'}], 'start': 4009.563, 'title': 'Challenges in autonomous car development', 'summary': 'Emphasizes robust testing for team growth, challenges in snow identification, and the importance of semantic understanding and perception in self-driving car development.', 'chapters': [{'end': 4148.928, 'start': 4009.563, 'title': 'Robust testing for engineering projects', 'summary': 'Emphasizes the importance of robust testing in growing the team at scale, ensuring the robustness of testing as the judge of approach effectiveness, and the need for algorithms to learn core principles rather than specific situations in self-driving car development.', 'duration': 139.365, 'highlights': ['The robustness of testing is key to growing the team at the biggest scale, exploring all paths, and coming up with the best approach.', 'The robustness of testing serves as the judge of whether an approach works or not, emphasizing its importance in development.', 'Algorithms need to learn core principles for safe driving rather than specifics of any situation to handle the infinite parameter space of real scenes.', 'The goal is to bring more diversity to the learning of general principles, rather than enumerating all possibilities for self-driving car development.']}, {'end': 4280.816, 'start': 4149.648, 'title': 'Challenges in snow identification for autonomous cars', 'summary': 'Discusses the challenges of identifying various types of snow for autonomous cars, and the trade-off between algorithmic capabilities and computational feasibility in processing snow-related data, emphasizing the importance of grouping semantic objects for behavior prediction.', 'duration': 131.168, 'highlights': ['The trade-off between algorithmic capabilities and computational feasibility in processing snow-related data, emphasizing the importance of grouping semantic objects for behavior prediction (e.g., not processing every snowflake individually, but grouping them into semantic objects) to reason at a higher level of abstraction.', 'The difficulty of identifying snow due to its various shapes and the potential impact of different types of snow on driving, such as flurries versus heavy blizzards, and the suggestion of creating a wider array of object embeddings for different types of snow to address this challenge.', "The consideration of the computational feasibility in implementing a wide array of object embeddings for various types of snow, highlighting the challenge of balancing algorithmic possibilities with the computational resources available for the car's processing power."]}, {'end': 4399.329, 'start': 4280.816, 'title': 'Semantic understanding and perception', 'summary': 'Discusses the importance of semantic understanding and the potential challenges in perception-based systems, emphasizing the need for complementary sensors and semantic understanding to mitigate errors and adversarial attacks.', 'duration': 118.513, 'highlights': ['Errors can occur in every model, such as the example of non-adversarial errors like the reflection case, where real-life situations can lead to confusion without the need for adversarial attacks.', 'Complementary sensors and systems can mitigate errors by not making the same mistakes, providing an essential redundancy in the system.', 'Semantic understanding plays a crucial role in differentiating true information from errors, similar to how humans rely on understanding to discern between real and mistaken perceptions.']}], 'duration': 389.766, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/LSX3qdy0dFg/pics/LSX3qdy0dFg4009563.jpg', 'highlights': ['Robust testing is crucial for team growth and development, exploring all paths.', 'Algorithms need to learn core principles for safe driving to handle real scenes.', 'Diversity in learning general principles is essential for self-driving car development.', 'Grouping semantic objects for behavior prediction is crucial for processing snow-related data.', 'Balancing algorithmic possibilities with computational resources is a challenge in snow identification.', 'Complementary sensors and systems provide essential redundancy to mitigate errors.', 'Semantic understanding is crucial for discerning true information from errors.']}], 'highlights': ['Waymo has recently driven over four million miles autonomously, marking a significant milestone in the development of self-driving technology', 'The potential impact of self-driving technology on safety, accessibility, and efficiency is discussed, with 94% of US crashes involving human errors and the technology having the potential to make mobility more available and cheaper for people', 'Waymo achieved confidence in their system to operate driverless cars in Phoenix, marking a significant milestone in their development.', 'The expansion of testing and deployment of self-driving cars in Chandler, Arizona, going beyond a single car a day to a growing fleet, with the intention of a product launch soon.', 'The first successful deep learning application in production was in 2012, marking a significant breakthrough for Alphabet in understanding real scene situations and geolocalization for Google Maps.', 'Different types of sensors at Waymo, including vision systems, radar, and LiDAR, are designed to be complementary, providing a dense representation, depth estimation, and semantic information, thereby improving the perception system.', 'Deep learning is crucial for realistic behavior and trajectory predictions for driving safety.', 'Waymo reached 4 million miles in six months, accelerating pace, and ability to simulate and augment miles for efficient testing.', 'Real-world driving is crucial for encountering situations and understanding behavior, necessary for testing machine learning at scale and with a high safety bar.', 'Robust testing is crucial for team growth and development, exploring all paths.', 'Algorithms need to learn core principles for safe driving to handle real scenes.', 'Diversity in learning general principles is essential for self-driving car development.', 'Grouping semantic objects for behavior prediction is crucial for processing snow-related data.', 'Complementary sensors and systems provide essential redundancy to mitigate errors.', 'Semantic understanding is crucial for discerning true information from errors.']}