title
Self-Driving Cars: State of the Art (2019)

description
Introductory lecture of the MIT Self-Driving Cars series (6.S094) with an overview of the autonomous vehicle industry in 2018 and looking forward to 2019, including Waymo, Tesla, Cruise, Ford, GM, and out-of-the-box ideas of boring tunnels, flying cars, connected vehicles, and more. This covers the state of the art in terms of industry developments and not the perception and planning algorithm development. The latter will be covered in detail in future lectures. For more lecture videos on deep learning, reinforcement learning (RL), artificial intelligence (AI & AGI), and podcast conversations, visit our website or follow TensorFlow code tutorials on our GitHub repo. INFO: Website: https://deeplearning.mit.edu GitHub: https://github.com/lexfridman/mit-deep-learning Slides: http://bit.ly/2HLcQ4s Playlist: http://bit.ly/2S1MVdy OUTLINE: 0:00 - Introduction 1:53 - 2018 in review 4:49 - Fatalities 8:29 - Taxi services 10:54 - Predictions 16:55 - Human-centered autonomy 19:42 - Levels of autonomy and proliferation strategies 24:48 - Out-of-the-box ideas 27:28 - Who will be first? 29:26 - Historical context 31:05 - Underlying beliefs of the industry and public 32:32 - Driving is hard 35:32 - Humans are amazing 37:10 - Humans and automation don't mix well? 41:55 - Two approaches: Lidar vs Vision 49:54 - In the meantime… data 52:49 - The road ahead CONNECT: - If you enjoyed this video, please subscribe to this channel. - Twitter: https://twitter.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/lexfridman - Instagram: https://www.instagram.com/lexfridman

detail
{'title': 'Self-Driving Cars: State of the Art (2019)', 'heatmap': [{'end': 2099.212, 'start': 2062.724, 'weight': 1}], 'summary': 'Discusses the mission of autonomous vehicles to improve mobility, efficiency, and safety, highlighting sobering statistics of car crash fatalities. it also covers milestones achieved in 2018, deployment predictions, industry innovation, human-robot interaction, sensor comparison, and perception challenges in autonomous driving.', 'chapters': [{'end': 120.621, 'segs': [{'end': 98.556, 'src': 'embed', 'start': 25.589, 'weight': 0, 'content': [{'end': 30.55, 'text': "So the problem, the mission, the dream, the thing that we're trying to solve.", 'start': 25.589, 'duration': 4.961}, {'end': 39.673, 'text': "for many it may be about entrepreneurial possibilities of making money and so on, but really it's about improving access to mobility,", 'start': 30.55, 'duration': 9.123}, {'end': 47.015, 'text': "moving people around the world that don't have that ability, whether it has to do with age or purely access of where you live.", 'start': 39.673, 'duration': 7.342}, {'end': 58.147, 'text': 'We want to increase the efficiency of how people move about, the ability to be productive in the time we spend in traffic and transportation.', 'start': 48.254, 'duration': 9.893}, {'end': 68.757, 'text': 'One of the most hated things in terms of stress and motion The thing in our lives that if we could just with a snap of a finger remove is traffic.', 'start': 59.009, 'duration': 9.748}, {'end': 75.76, 'text': 'So the ability to convert that into efficiency, into a productive aspect, into a positive aspect of life.', 'start': 69.217, 'duration': 6.543}, {'end': 82.763, 'text': 'And really the most important thing, at least for me and for many of us working in this space, is to save lives.', 'start': 76.34, 'duration': 6.423}, {'end': 87.285, 'text': 'Prevent crashes that lead to injuries, prevent crashes that lead to fatalities.', 'start': 83.243, 'duration': 4.042}, {'end': 92.647, 'text': "Here's a counter, every 23 seconds somebody in the world dies in a car auto crash.", 'start': 87.725, 'duration': 4.922}, {'end': 98.556, 'text': 'It should be a sobering, it is for me, thing that I think about every single day.', 'start': 93.615, 'duration': 4.941}], 'summary': 'Mission: improve access to mobility, increase efficiency, and save lives by preventing car crashes.', 'duration': 72.967, 'max_score': 25.589, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ25589.jpg'}], 'start': 0.049, 'title': 'Autonomous vehicles: state of the art', 'summary': 'Discusses the mission of autonomous vehicles, emphasizing the goal to improve mobility, increase efficiency, and save lives, with a focus on preventing crashes that lead to injuries and fatalities, with a sobering statistic of a death in a car crash every 23 seconds worldwide.', 'chapters': [{'end': 120.621, 'start': 0.049, 'title': 'Autonomous vehicles: state of the art', 'summary': 'Discusses the mission of autonomous vehicles, emphasizing the goal to improve mobility, increase efficiency, and save lives, with a focus on preventing crashes that lead to injuries and fatalities, with a sobering statistic of a death in a car crash every 23 seconds worldwide.', 'duration': 120.572, 'highlights': ['The main goal of autonomous vehicles is to save lives by preventing crashes that lead to injuries and fatalities, with a staggering statistic of a death in a car crash every 23 seconds worldwide.', 'The focus is on improving access to mobility and increasing the efficiency of how people move about, aiming to convert the stress of traffic into a productive aspect of life.', 'The mission of autonomous vehicles extends beyond entrepreneurial possibilities and aims to address the challenges of moving people who lack access due to age or location.']}], 'duration': 120.572, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ49.jpg', 'highlights': ['The main goal of autonomous vehicles is to save lives by preventing crashes that lead to injuries and fatalities, with a staggering statistic of a death in a car crash every 23 seconds worldwide.', 'The focus is on improving access to mobility and increasing the efficiency of how people move about, aiming to convert the stress of traffic into a productive aspect of life.', 'The mission of autonomous vehicles extends beyond entrepreneurial possibilities and aims to address the challenges of moving people who lack access due to age or location.']}, {'end': 755.337, 'segs': [{'end': 176.231, 'src': 'embed', 'start': 120.621, 'weight': 0, 'content': [{'end': 124.904, 'text': 'I would like to review what happened in 2018..', 'start': 120.621, 'duration': 4.283}, {'end': 135.571, 'text': 'So first, Waymo has done incredible work in deploying and testing their vehicles in various domains and have, in October,', 'start': 124.904, 'duration': 10.667}, {'end': 139.393, 'text': 'reached the mark of 10 million miles driven autonomously.', 'start': 135.571, 'duration': 3.822}, {'end': 140.994, 'text': 'This is an incredible accomplishment.', 'start': 139.413, 'duration': 1.581}, {'end': 147.799, 'text': "It's truly a big step for fully autonomous vehicles in terms of deployment.", 'start': 141.294, 'duration': 6.505}, {'end': 151.786, 'text': "And obviously it's growing and growing by day.", 'start': 148.843, 'duration': 2.943}, {'end': 156.392, 'text': "And we'll have Drago here from Waymo to talk about their work there.", 'start': 152.547, 'duration': 3.845}, {'end': 163.36, 'text': "Then on the L2, on the semi-autonomous side, that's the pair, that's the mirror side of this equation.", 'start': 156.932, 'duration': 6.428}, {'end': 176.231, 'text': "The other incredible number that's perhaps less talked about is the 1 billion mile mark reached by Tesla in the semi-autonomous driving of autopilot.", 'start': 164.5, 'duration': 11.731}], 'summary': 'In 2018, waymo drove 10 million miles autonomously, while tesla reached 1 billion miles in semi-autonomous driving.', 'duration': 55.61, 'max_score': 120.621, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ120621.jpg'}, {'end': 247.089, 'src': 'embed', 'start': 217.787, 'weight': 2, 'content': [{'end': 218.747, 'text': "So it's computer vision.", 'start': 217.787, 'duration': 0.96}, {'end': 221.729, 'text': 'And how does computer vision work in the modern day?', 'start': 219.448, 'duration': 2.281}, {'end': 226.253, 'text': 'Especially with the second iteration of autopilot hardware?', 'start': 223.171, 'duration': 3.082}, {'end': 227.174, 'text': "there's a neural network.", 'start': 226.253, 'duration': 0.921}, {'end': 228.695, 'text': "there's a set of neural networks behind it.", 'start': 227.174, 'duration': 1.521}, {'end': 230.176, 'text': "That's super exciting.", 'start': 229.135, 'duration': 1.041}, {'end': 247.089, 'text': "That is probably the largest deployment of neural networks in the world that has a direct impact on a human life that's able to decide,", 'start': 230.416, 'duration': 16.673}], 'summary': 'Computer vision with neural networks is the largest deployment impacting human life.', 'duration': 29.302, 'max_score': 217.787, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ217787.jpg'}, {'end': 355.951, 'src': 'embed', 'start': 329.11, 'weight': 5, 'content': [{'end': 334.034, 'text': 'Now the two aspects here that are sobering and really important to think about.', 'start': 329.11, 'duration': 4.924}, {'end': 342.6, 'text': 'as we talk about the progression of autonomous vehicles proliferation in our world is our response as a public,', 'start': 334.034, 'duration': 8.566}, {'end': 348.465, 'text': 'from the general public to the engineers, to the media and so on, how we think about these fatalities.', 'start': 342.6, 'duration': 5.865}, {'end': 353.249, 'text': "And obviously there's a disproportionate amount of attention given to these fatalities.", 'start': 348.805, 'duration': 4.444}, {'end': 355.951, 'text': "And that's something as engineers you have to also think about.", 'start': 353.609, 'duration': 2.342}], 'summary': 'Public response to autonomous vehicle fatalities needs consideration by engineers and media.', 'duration': 26.841, 'max_score': 329.11, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ329110.jpg'}, {'end': 487.216, 'src': 'embed', 'start': 449.414, 'weight': 6, 'content': [{'end': 454.015, 'text': "Tesla is a modern vehicle that's much safer than the general population of manually driven vehicles.", 'start': 449.414, 'duration': 4.601}, {'end': 459.757, 'text': 'Autopilot is driven on only particular kinds of roads on the highway, primarily most of the miles.', 'start': 454.515, 'duration': 5.242}, {'end': 465.518, 'text': 'The kinds of people that drive autopilot, all these kinds of factors need to be considered when you compare the two.', 'start': 460.757, 'duration': 4.761}, {'end': 470, 'text': 'But when you just look at the numbers, Tesla autopilot is three times safer.', 'start': 466.239, 'duration': 3.761}, {'end': 474.364, 'text': "than manually driven vehicles, but that's not the right way to look at it.", 'start': 470.62, 'duration': 3.744}, {'end': 487.216, 'text': "And for anyone that's ever taken a statistics class, three fatalities is not a large number by which to make any significant conclusions.", 'start': 474.944, 'duration': 12.272}], 'summary': 'Tesla autopilot is three times safer than manually driven vehicles, but needs to be considered with other factors.', 'duration': 37.802, 'max_score': 449.414, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ449414.jpg'}, {'end': 691.368, 'src': 'embed', 'start': 659.46, 'weight': 7, 'content': [{'end': 663.824, 'text': 'And when you yourself think about what it means, when will they be here?', 'start': 659.46, 'duration': 4.364}, {'end': 671.33, 'text': 'When will autonomous vehicles arrive, such that the Uber that you call will be autonomous and not populated by a driver?', 'start': 665.105, 'duration': 6.225}, {'end': 680.402, 'text': 'So the thing we have to think about is what we think about, how we define autonomous, what that experience looks like.', 'start': 671.39, 'duration': 9.012}, {'end': 684.944, 'text': 'And most importantly in these discussions, we have to think about scale.', 'start': 681.142, 'duration': 3.802}, {'end': 691.368, 'text': 'So we here at MIT, our group, MIT Human-Centered Autonomous Vehicle.', 'start': 686.645, 'duration': 4.723}], 'summary': "Autonomous vehicles' arrival and scale are key considerations for mit's human-centered autonomous vehicle group.", 'duration': 31.908, 'max_score': 659.46, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ659460.jpg'}], 'start': 120.621, 'title': 'Autonomous vehicle milestones and progress', 'summary': "Highlights the milestones achieved in 2018, such as waymo's 10 million autonomously driven miles and tesla's 1 billion miles with autopilot, and discusses the progress, public perception, safety comparison, and deployment challenges of autonomous vehicles.", 'chapters': [{'end': 267.339, 'start': 120.621, 'title': 'Autonomous vehicle milestones in 2018', 'summary': "Highlights the milestones achieved in 2018, including waymo's 10 million autonomously driven miles and tesla's 1 billion miles with autopilot, emphasizing the impact of neural networks in modern computer vision for autonomous driving.", 'duration': 146.718, 'highlights': ['Waymo reached 10 million miles driven autonomously, a significant achievement for fully autonomous vehicles.', 'Tesla reached the 1 billion mile mark in semi-autonomous driving with autopilot, showcasing its advanced capabilities for highway driving.', 'The use of neural networks in modern computer vision for autonomous driving is highlighted as the largest deployment impacting human life, making life-critical decisions many times a second.', 'The autopilot system relies on computer vision with a primary sensor being a camera, emphasizing the vast amount of data collected and processed for autonomous driving.', "The chapter emphasizes the significance of Waymo and Tesla's milestones in advancing autonomous driving technology in 2018."]}, {'end': 755.337, 'start': 267.339, 'title': 'Autonomous vehicle progress and public perception', 'summary': 'Discusses the progress of autonomous vehicles, including the significant accomplishment of one billion miles driven and the sobering incidents of fatalities in autonomous and semi-autonomous vehicles, while highlighting the public response, comparison of safety, and the challenges of achieving meaningful deployment.', 'duration': 487.998, 'highlights': ['The significant accomplishment of one billion miles driven by autonomous vehicles One billion miles driven by autonomous vehicles is highlighted as a significant accomplishment.', 'The sobering incidents of fatalities in autonomous and semi-autonomous vehicles The sobering incidents of fatalities in autonomous and semi-autonomous vehicles are discussed, emphasizing the importance of public response and the challenges faced by engineers.', "Comparison of safety between autonomous vehicles and manually driven vehicles The comparison of safety between autonomous vehicles and manually driven vehicles is discussed, highlighting the threefold safety advantage of Tesla's Autopilot over manually driven vehicles and the challenges in comparing the two.", 'Challenges of achieving meaningful deployment of autonomous vehicles The challenges of achieving meaningful deployment of autonomous vehicles are emphasized, with a focus on the concept of scale and the need for widespread adoption beyond prototype demonstrations.', 'Public perception and response to autonomous vehicle fatalities The public perception and response to autonomous vehicle fatalities are highlighted, with a focus on the disproportionate attention given to these incidents and the impact on the progress of autonomous vehicles.']}], 'duration': 634.716, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ120621.jpg', 'highlights': ['Tesla reached the 1 billion mile mark in semi-autonomous driving with autopilot, showcasing its advanced capabilities for highway driving.', 'Waymo reached 10 million miles driven autonomously, a significant achievement for fully autonomous vehicles.', 'The use of neural networks in modern computer vision for autonomous driving is highlighted as the largest deployment impacting human life, making life-critical decisions many times a second.', 'The autopilot system relies on computer vision with a primary sensor being a camera, emphasizing the vast amount of data collected and processed for autonomous driving.', "The chapter emphasizes the significance of Waymo and Tesla's milestones in advancing autonomous driving technology in 2018.", 'The sobering incidents of fatalities in autonomous and semi-autonomous vehicles are discussed, emphasizing the importance of public response and the challenges faced by engineers.', "Comparison of safety between autonomous vehicles and manually driven vehicles is discussed, highlighting the threefold safety advantage of Tesla's Autopilot over manually driven vehicles and the challenges in comparing the two.", 'Challenges of achieving meaningful deployment of autonomous vehicles are emphasized, with a focus on the concept of scale and the need for widespread adoption beyond prototype demonstrations.', 'Public perception and response to autonomous vehicle fatalities are highlighted, with a focus on the disproportionate attention given to these incidents and the impact on the progress of autonomous vehicles.']}, {'end': 1749.419, 'segs': [{'end': 813.045, 'src': 'embed', 'start': 756.318, 'weight': 0, 'content': [{'end': 762.822, 'text': "I'm not a marketing PR person, so I don't understand why everybody has to make a prediction, but they all seem to.", 'start': 756.318, 'duration': 6.504}, {'end': 769.966, 'text': 'All the major automakers have made a prediction of when they will be able to deploy autonomous vehicles.', 'start': 763.742, 'duration': 6.224}, {'end': 780.345, 'text': 'Tesla has made, in early 2017, A prediction that they will have autonomous vehicles in 2018.', 'start': 771.546, 'duration': 8.799}, {'end': 784.33, 'text': "In 2018, they've now adjusted the prediction to 2019.", 'start': 780.345, 'duration': 3.985}, {'end': 793.3, 'text': 'Nissan, Honda, Toyota have made prediction for 2020 under certain constraints in highway, urban, Hyundai and Volvo.', 'start': 784.33, 'duration': 8.97}, {'end': 806.904, 'text': 'has in 2021 with BMW and Ford, Ford saying at scale, so large scale deployment in 2021, and Chrysler in 21, and Daimler saying in the early 20s.', 'start': 795.361, 'duration': 11.543}, {'end': 813.045, 'text': 'So there is the predictions that are extremely optimistic,', 'start': 807.764, 'duration': 5.281}], 'summary': 'Major automakers have made predictions for autonomous vehicle deployment, ranging from 2018 to early 20s.', 'duration': 56.727, 'max_score': 756.318, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ756318.jpg'}, {'end': 966.107, 'src': 'embed', 'start': 937.229, 'weight': 10, 'content': [{'end': 943.192, 'text': 'For Rodney, the vehicles are really fully autonomous or beyond 2050.', 'start': 937.229, 'duration': 5.963}, {'end': 958.501, 'text': 'But he believes in the 30s there will be a significant a major city will be able to allocate a significant region of that city where manual driving is fully banned,', 'start': 943.192, 'duration': 15.309}, {'end': 966.107, 'text': 'which is the way he believes those autonomous vehicles can really proliferate when you ban manually driven vehicles in certain parts.', 'start': 958.501, 'duration': 7.606}], 'summary': 'Rodney predicts fully autonomous vehicles by 2050, with manual driving banned in major cities by the 2030s to enable the proliferation of autonomous vehicles.', 'duration': 28.878, 'max_score': 937.229, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ937229.jpg'}, {'end': 1018.577, 'src': 'embed', 'start': 996.082, 'weight': 5, 'content': [{'end': 1004.007, 'text': "so even when you have every car that's built that's fully autonomous, it's still going to take time for that dissipation of vehicles to happen.", 'start': 996.082, 'duration': 7.925}, {'end': 1013.393, 'text': 'And so my own view, beyond predictions, to take a little pause into the ridiculous and the fun to explain the view.', 'start': 1004.447, 'duration': 8.946}, {'end': 1018.577, 'text': 'Yes, that is me playing guitar in our autonomous vehicle.', 'start': 1016.155, 'duration': 2.422}], 'summary': 'Transition to fully autonomous cars will take time, even with all cars being autonomous.', 'duration': 22.495, 'max_score': 996.082, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ996082.jpg'}, {'end': 1097.106, 'src': 'embed', 'start': 1066.888, 'weight': 4, 'content': [{'end': 1073.452, 'text': "They're cautious and therefore slower and therefore more annoying in the way we think about actually how we navigate this world.", 'start': 1066.888, 'duration': 6.564}, {'end': 1077.495, 'text': 'We take risk, we drive assertively, we speed over the speed limit all the time.', 'start': 1073.712, 'duration': 3.783}, {'end': 1080.257, 'text': 'That is not how autonomous vehicles today operate.', 'start': 1077.795, 'duration': 2.462}, {'end': 1082.058, 'text': "So they're not going to get us there faster.", 'start': 1080.497, 'duration': 1.561}, {'end': 1086.361, 'text': "And for every promise, every hope that they're going to be cheaper.", 'start': 1082.458, 'duration': 3.903}, {'end': 1097.106, 'text': "Really, there's still significant investment going into them and there's not good economics in the near term of how to make them obviously significantly cheaper.", 'start': 1086.821, 'duration': 10.285}], 'summary': 'Autonomous vehicles are slower, not faster or cheaper as expected.', 'duration': 30.218, 'max_score': 1066.888, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1066888.jpg'}, {'end': 1206.783, 'src': 'embed', 'start': 1181.383, 'weight': 8, 'content': [{'end': 1189.269, 'text': "So, in the language that we're talking about here, there's several levels of autonomy that are defined from level 0 to level 4, level 0 no automation.", 'start': 1181.383, 'duration': 7.886}, {'end': 1193.392, 'text': '4 and 5, level 3, 4 and 5 increasing automation.', 'start': 1189.269, 'duration': 4.123}, {'end': 1196.215, 'text': 'So level 2 is when the driver is still responsible.', 'start': 1193.453, 'duration': 2.762}, {'end': 1206.783, 'text': "Level 3, 4, 5 is when there's less and less responsibility, but really in 3, 4, 5 there's parts of the driving where the liability is on the car.", 'start': 1196.915, 'duration': 9.868}], 'summary': 'Autonomy levels range from 0 to 5, where level 3, 4, and 5 entail increasing automation and less driver responsibility.', 'duration': 25.4, 'max_score': 1181.383, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1181383.jpg'}, {'end': 1308.458, 'src': 'embed', 'start': 1279.665, 'weight': 7, 'content': [{'end': 1282.506, 'text': 'On the fully autonomous side.', 'start': 1279.665, 'duration': 2.841}, {'end': 1289.431, 'text': "the way to think about it, with the predictions that we're talking about here,", 'start': 1282.506, 'duration': 6.925}, {'end': 1292.593, 'text': "is there's several different possibilities of how to deploy these vehicles.", 'start': 1289.431, 'duration': 3.162}, {'end': 1301.951, 'text': 'One is last mile delivery of goods and services, like the groceries.', 'start': 1295.786, 'duration': 6.165}, {'end': 1308.458, 'text': 'These are zero occupancy vehicles delivering groceries or delivering human beings at the last mile.', 'start': 1302.773, 'duration': 5.685}], 'summary': 'Fully autonomous vehicles can be deployed for last mile delivery of goods and services, such as groceries, with zero occupancy vehicles.', 'duration': 28.793, 'max_score': 1279.665, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1279665.jpg'}, {'end': 1680.344, 'src': 'embed', 'start': 1649.69, 'weight': 9, 'content': [{'end': 1651.671, 'text': 'Back in my day, we used to drive in the street.', 'start': 1649.69, 'duration': 1.981}, {'end': 1655.212, 'text': 'OK, so 10, 000 vehicles.', 'start': 1652.671, 'duration': 2.541}, {'end': 1658.927, 'text': "if that's the bar.", 'start': 1658.026, 'duration': 0.901}, {'end': 1663.392, 'text': 'I sort of out of curiosity, asked did a little public poll?', 'start': 1659.487, 'duration': 3.905}, {'end': 1675.226, 'text': '3, 000 people responded asked who will be first to deploy 10, 000 fully autonomous cars operating on public roads without a safety driver?', 'start': 1663.392, 'duration': 11.834}, {'end': 1680.344, 'text': 'And several options percolated, with Tesla getting 50%,', 'start': 1676.96, 'duration': 3.384}], 'summary': '3,000 people polled on deployment of 10,000 autonomous cars, with tesla leading at 50%.', 'duration': 30.654, 'max_score': 1649.69, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1649690.jpg'}], 'start': 756.318, 'title': 'Autonomous vehicle deployment and innovation', 'summary': "Covers automakers' deployment predictions with tesla aiming for 2019 and others targeting 2020-2021. it discusses engineers' cautious approach, safety challenges, levels of autonomy, potential deployment scenarios, and disruptive innovation in the industry, including a public poll on the deployment of 10,000 fully autonomous cars.", 'chapters': [{'end': 813.045, 'start': 756.318, 'title': 'Autonomous vehicle deployment predictions', 'summary': "Discusses various automakers' predictions for the deployment of autonomous vehicles, with tesla aiming for 2019, followed by nissan, honda, toyota for 2020, and hyundai, volvo, bmw, ford, chrysler, and daimler targeting 2021 and beyond.", 'duration': 56.727, 'highlights': ['Tesla predicted autonomous vehicles by 2018, now adjusted to 2019.', 'Nissan, Honda, Toyota aiming for 2020 deployment.', 'Hyundai, Volvo, BMW, Ford, Chrysler, and Daimler targeting 2021 and beyond.', 'Various automakers have made optimistic predictions for autonomous vehicle deployment.']}, {'end': 1331.71, 'start': 813.045, 'title': 'Autonomous vehicle deployment', 'summary': 'Discusses the cautious approach of leading engineers towards autonomous vehicles, the spectrum of optimism and pessimism around their adoption, the challenges of safety, speed, and economics, and the importance of creating a better human experience. it also outlines the levels of autonomy and the potential deployment scenarios for fully autonomous vehicles.', 'duration': 518.665, 'highlights': ['The cautious approach of leading engineers towards autonomous vehicles Leading engineers like Carl, Yama, and Gil Pratt advocate for a cautious approach, emphasizing the difficulty of removing the human from the loop of automation.', 'The spectrum of optimism and pessimism around autonomous vehicle adoption The chapter presents the differing viewpoints of Elon Musk, who predicts fully autonomous vehicles by 2019, and Rodney, who believes it will not happen before 2050, with a significant city potentially banning manual driving in the 30s and majority of US cities banning manual driving in the 40s.', 'Challenges of safety, speed, and economics in autonomous vehicle adoption The text discusses the slower nature of autonomous vehicles, their cautious driving style, the lack of significant cost reduction, and the importance of creating a better human experience for their adoption.', 'Levels of autonomy and deployment scenarios for fully autonomous vehicles The chapter outlines the levels of autonomy from 0 to 5, emphasizes the distinction between human-centered autonomy and full autonomy, and discusses potential deployment scenarios including last mile delivery and trucking on the highway.']}, {'end': 1749.419, 'start': 1332.391, 'title': 'Autonomous vehicle innovation', 'summary': 'Discusses the various levels of autonomy in autonomous vehicles, including highway driving, urban routes, zero occupancy ride sharing delivery, semi-autonomous driving, and futuristic ideas such as connected vehicles, tunnels, and flying cars. it also highlights a public poll on the deployment of 10,000 fully autonomous cars and the wide-open opportunities for disruptive innovation in the autonomous vehicle industry.', 'duration': 417.028, 'highlights': ['The chapter discusses the various levels of autonomy in autonomous vehicles, including highway driving, urban routes, zero occupancy ride sharing delivery, semi-autonomous driving, and futuristic ideas such as connected vehicles, tunnels, and flying cars. It covers the automatability of highway driving, defining urban routes for taxi services and personalized public transport, closed communities with defined constraints and customer base, zero occupancy ride-sharing delivery, and the semi-autonomous approach with teleoperation and high autonomy on highways.', 'It also highlights a public poll on the deployment of 10,000 fully autonomous cars. The poll revealed that 50% of the vote went to Tesla, 21% to Waymo, and 14% to other options, indicating the current leaders in the autonomous vehicle space and the wide-open opportunities for disruptive innovation.', 'The wide-open opportunities for disruptive innovation in the autonomous vehicle industry. The transcript emphasizes the presence of companies taking advantage of the hype and investment in the space, alongside companies genuinely aiming to solve the autonomous vehicle problem and become the next disruptive force in the industry.']}], 'duration': 993.101, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ756318.jpg', 'highlights': ['Tesla predicted autonomous vehicles by 2018, now adjusted to 2019.', 'Nissan, Honda, Toyota aiming for 2020 deployment.', 'Hyundai, Volvo, BMW, Ford, Chrysler, and Daimler targeting 2021 and beyond.', 'Various automakers have made optimistic predictions for autonomous vehicle deployment.', 'The cautious approach of leading engineers towards autonomous vehicles.', 'The spectrum of optimism and pessimism around autonomous vehicle adoption.', 'Challenges of safety, speed, and economics in autonomous vehicle adoption.', 'Levels of autonomy and deployment scenarios for fully autonomous vehicles.', 'The chapter discusses the various levels of autonomy in autonomous vehicles.', 'It also highlights a public poll on the deployment of 10,000 fully autonomous cars.', 'The wide-open opportunities for disruptive innovation in the autonomous vehicle industry.']}, {'end': 2114.08, 'segs': [{'end': 1799.429, 'src': 'embed', 'start': 1749.419, 'weight': 0, 'content': [{'end': 1759.483, 'text': 'but currently Tesla, with the human, with the semi-autonomous vehicle approach, working towards trying to become fully autonomous, and Waymo,', 'start': 1749.419, 'duration': 10.064}, {'end': 1765.946, 'text': 'starting with the fully autonomous, working towards achieving scale at the fully autonomous, are the leaders in the space.', 'start': 1759.483, 'duration': 6.463}, {'end': 1775.608, 'text': "given that ranking in 2019, let's take a quick step back to 2005 with the DARPA challenge when the story began.", 'start': 1765.946, 'duration': 9.662}, {'end': 1781.293, 'text': 'The race to the desert, when Stanley from Stanford won a race to the desert.', 'start': 1775.628, 'duration': 5.665}, {'end': 1785.398, 'text': "That really captivated people's imagination about what's possible.", 'start': 1782.234, 'duration': 3.164}, {'end': 1789.262, 'text': 'And a lot of people have said that the autonomous vehicle problem is solved in 2005.', 'start': 1785.718, 'duration': 3.544}, {'end': 1796.567, 'text': 'They really said, The idea was, especially because in 2004, nobody finished that race.', 'start': 1789.262, 'duration': 7.305}, {'end': 1797.768, 'text': '2005, four cars finished the race.', 'start': 1796.587, 'duration': 1.181}, {'end': 1799.429, 'text': 'It was like, well, we cracked it.', 'start': 1798.128, 'duration': 1.301}], 'summary': 'Tesla and waymo are leaders in autonomous vehicles. in 2005, the darpa challenge showcased the progress, with four cars finishing the race.', 'duration': 50.01, 'max_score': 1749.419, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1749419.jpg'}, {'end': 1900.853, 'src': 'embed', 'start': 1856.43, 'weight': 4, 'content': [{'end': 1863.884, 'text': 'And the rest is just, for industry to figure out some of the details of how to make an app and make a business out of it.', 'start': 1856.43, 'duration': 7.454}, {'end': 1871.807, 'text': "So that could be true, and the underlying beliefs there is that driving is an easy task, that it's solvable.", 'start': 1865.046, 'duration': 6.761}, {'end': 1879.349, 'text': "The thing that we do as human beings that it's pretty formalizable, it's pretty easy to solve with autonomy.", 'start': 1872.127, 'duration': 7.222}, {'end': 1882.589, 'text': 'The other idea is that humans are bad at driving.', 'start': 1880.489, 'duration': 2.1}, {'end': 1884.009, 'text': 'This is a common belief.', 'start': 1882.969, 'duration': 1.04}, {'end': 1887.33, 'text': 'Not me, not you, but everybody else.', 'start': 1884.61, 'duration': 2.72}, {'end': 1889.971, 'text': 'Nobody in this room, but everybody else is a terrible driver.', 'start': 1887.79, 'duration': 2.181}, {'end': 1896.452, 'text': 'The kind of intuition that we have about our experience of traffic leads us to believe that humans are just really bad at driving.', 'start': 1890.511, 'duration': 5.941}, {'end': 1900.853, 'text': 'And from the human factor, psychology side.', 'start': 1897.172, 'duration': 3.681}], 'summary': 'Industry to figure out app details for easy, solvable autonomous driving; common belief in human bad driving', 'duration': 44.423, 'max_score': 1856.43, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1856430.jpg'}, {'end': 1955.588, 'src': 'embed', 'start': 1931.876, 'weight': 3, 'content': [{'end': 1938.52, 'text': 'People are, they tune out, they over trust the system, they misinterpret the system and they lose vigilance.', 'start': 1931.876, 'duration': 6.644}, {'end': 1940.552, 'text': 'Those are the three underlying beliefs.', 'start': 1939.07, 'duration': 1.482}, {'end': 1947.519, 'text': 'It very well could be true, but what if it is not? So we have to consider that it is not.', 'start': 1941.132, 'duration': 6.387}, {'end': 1954.406, 'text': 'The driving task is easy, because if you think the driving task is easy and formalizable and solved by autonomous vehicles,', 'start': 1948.64, 'duration': 5.766}, {'end': 1955.588, 'text': 'you have to solve this problem.', 'start': 1954.406, 'duration': 1.182}], 'summary': 'People over trust the system, misinterpret it, and lose vigilance, posing challenges for autonomous vehicles.', 'duration': 23.712, 'max_score': 1931.876, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1931876.jpg'}, {'end': 2023.912, 'src': 'embed', 'start': 1984.885, 'weight': 6, 'content': [{'end': 1992.81, 'text': 'so, in snowy, icy weather, rain, limited visibility conditions, you have 100, 200 milliseconds to make a decision.', 'start': 1984.885, 'duration': 7.925}, {'end': 1996.032, 'text': 'Your algorithm, based on the perception, has to make a control decision.', 'start': 1993.55, 'duration': 2.482}, {'end': 2001.636, 'text': 'Then you have to deal with the non-verbal communication with pedestrians.', 'start': 1999.114, 'duration': 2.522}, {'end': 2015.207, 'text': "These unreasonable, irrational creatures, us human beings, you have to not only understand the intent of the movement that's anticipated,", 'start': 2002.421, 'duration': 12.786}, {'end': 2019.77, 'text': 'so anticipating the trajectory of the pedestrian, you also have to assert yourself in a game-theoretic way.', 'start': 2015.207, 'duration': 4.563}, {'end': 2023.912, 'text': 'As crazy as it might sound, you have to threaten yourself, you have to take a risk.', 'start': 2020.51, 'duration': 3.402}], 'summary': 'In adverse weather conditions, autonomous vehicles have 100-200 milliseconds to make control decisions and navigate non-verbal communication with pedestrians, while also asserting themselves in a game-theoretic way.', 'duration': 39.027, 'max_score': 1984.885, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1984885.jpg'}, {'end': 2099.212, 'src': 'heatmap', 'start': 2062.724, 'weight': 1, 'content': [{'end': 2064.525, 'text': 'If driving is easy we have to solve that too.', 'start': 2062.724, 'duration': 1.801}, {'end': 2068.569, 'text': 'And of course the thing I showed yesterday with the coast runners and the boat going around.', 'start': 2064.665, 'duration': 3.904}, {'end': 2081.143, 'text': 'and all the ethical dilemmas, from the moral machine to the more serious engineering aspects,', 'start': 2071.058, 'duration': 10.085}, {'end': 2092.527, 'text': 'from the unintended consequences that arise from having to formalize the objective function under which a planning algorithm operates.', 'start': 2081.143, 'duration': 11.384}, {'end': 2099.212, 'text': "If there's any learning that as I showed yesterday, a boat on the left driven by a human wants to finish the race.", 'start': 2092.808, 'duration': 6.404}], 'summary': 'Addressing ethical dilemmas in driving algorithms and unintended consequences of formalizing objective functions.', 'duration': 36.488, 'max_score': 2062.724, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2062724.jpg'}, {'end': 2099.212, 'src': 'embed', 'start': 2071.058, 'weight': 8, 'content': [{'end': 2081.143, 'text': 'and all the ethical dilemmas, from the moral machine to the more serious engineering aspects,', 'start': 2071.058, 'duration': 10.085}, {'end': 2092.527, 'text': 'from the unintended consequences that arise from having to formalize the objective function under which a planning algorithm operates.', 'start': 2081.143, 'duration': 11.384}, {'end': 2099.212, 'text': "If there's any learning that as I showed yesterday, a boat on the left driven by a human wants to finish the race.", 'start': 2092.808, 'duration': 6.404}], 'summary': 'Discussion on ethical dilemmas and unintended consequences in planning algorithms.', 'duration': 28.154, 'max_score': 2071.058, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2071058.jpg'}], 'start': 1749.419, 'title': 'Advancements in autonomous vehicles', 'summary': 'Delves into the early advancements in autonomous vehicles, covering the darpa challenges in 2004 and 2007, and the rise of tesla and waymo. it also addresses the challenges of autonomous driving, including human driving assumptions and the difficulties in maintaining vigilance and trust in autonomous systems. furthermore, it explores the challenges of vehicle-to-vehicle and vehicle-to-pedestrian nonverbal communication in adverse weather conditions in boston, highlighting ethical dilemmas in algorithmic decision-making.', 'chapters': [{'end': 1855.91, 'start': 1749.419, 'title': 'Autonomous vehicle race', 'summary': 'Discusses the early advancements in autonomous vehicles, including the darpa challenges in 2004 and 2007, and the emergence of tesla and waymo as leaders in the space, with waymo focusing on fully autonomous vehicles and tesla working towards full autonomy with a semi-autonomous approach.', 'duration': 106.491, 'highlights': ['The emergence of Tesla and Waymo as leaders in autonomous vehicles Tesla and Waymo are currently the leaders in the autonomous vehicle space, with Tesla working towards full autonomy with a semi-autonomous approach and Waymo focused on achieving scale with fully autonomous vehicles.', 'The significance of the DARPA challenges in 2004 and 2007 The DARPA challenges in 2004 and 2007 marked significant advancements in autonomous vehicles, with the race to the desert sparking imagination and the Urban Grand Challenge demonstrating progress in urban driving.', 'Initial perception of solving the autonomy problem in 2005 In 2005, the completion of the DARPA challenge led to the perception that the autonomous vehicle problem was solved; however, critics pointed out that the challenges of urban driving were not comparable to desert driving.']}, {'end': 1955.588, 'start': 1856.43, 'title': 'Challenges of autonomous driving', 'summary': 'Discusses the underlying beliefs about driving, including the assumptions that driving is an easy and solvable task, humans are bad at driving, and the challenges of maintaining vigilance and trust in autonomous systems, backed by over 70 years of research.', 'duration': 99.158, 'highlights': ["Humans' tendency to tune out, over trust, misinterpret, and lose vigilance when monitoring systems, as evidenced by over 70 years of research.", 'The common belief that humans are bad at driving, supported by the intuition derived from the experience of traffic.', 'The underlying belief that driving is an easy and formalizable task, which autonomous vehicles must solve.']}, {'end': 2114.08, 'start': 1956.488, 'title': 'Challenges in vehicle-to-vehicle nonverbal communication', 'summary': 'Discusses the challenges of vehicle-to-vehicle and vehicle-to-pedestrian nonverbal communication in boston, emphasizing the need for quick decision-making in adverse weather conditions and the ethical dilemmas in algorithmic decision-making.', 'duration': 157.592, 'highlights': ['The need for quick decision-making in adverse weather conditions, such as snowy, icy weather, rain, and limited visibility, where drivers have 100-200 milliseconds to make control decisions. Emphasizes the time constraints and environmental factors affecting decision-making, highlighting the complexities of algorithmic control.', 'The challenges of understanding and anticipating pedestrian movement, as well as the need to assert oneself in a game-theoretic way when interacting with pedestrians. Highlights the complexities of interpreting pedestrian behavior and the strategic decision-making required by drivers in such interactions.', 'The ethical dilemmas and unintended consequences arising from formalizing the objective function for planning algorithms, as demonstrated by the boat race scenario. Illustrates the ethical and engineering challenges in optimizing rewards while considering potential unintended outcomes and the impact of objective functions.']}], 'duration': 364.661, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ1749419.jpg', 'highlights': ['Tesla and Waymo are leaders in autonomous vehicles, with different approaches.', 'DARPA challenges in 2004 and 2007 marked significant advancements in autonomous vehicles.', 'Initial perception of solving the autonomy problem in 2005 was challenged by critics.', 'Humans tend to tune out, over trust, misinterpret, and lose vigilance when monitoring systems.', 'Common belief that humans are bad at driving, supported by traffic experience.', 'Driving is perceived as an easy and formalizable task, which autonomous vehicles must solve.', 'Drivers have 100-200 milliseconds to make control decisions in adverse weather conditions.', 'Challenges of understanding and anticipating pedestrian movement and strategic decision-making.', 'Ethical dilemmas and unintended consequences arising from formalizing the objective function for planning algorithms.']}, {'end': 2665.873, 'segs': [{'end': 2196.598, 'src': 'embed', 'start': 2136.325, 'weight': 0, 'content': [{'end': 2147.373, 'text': "The fact that we have 540 million years worth of data on our visual perception system means we don't understand how damn impressive it is to be able to perceive and understand the scene in a split second,", 'start': 2136.325, 'duration': 11.048}, {'end': 2157.8, 'text': 'maintain context, maintain an understanding of performing all the visual localization tasks, about anticipating the physics of the scene, and so on.', 'start': 2147.373, 'duration': 10.427}, {'end': 2160.281, 'text': "And then there's a control side.", 'start': 2159.039, 'duration': 1.242}, {'end': 2163.966, 'text': "The humans don't give enough credit to ourselves.", 'start': 2160.301, 'duration': 3.665}, {'end': 2165.108, 'text': "We're incredible.", 'start': 2164.146, 'duration': 0.962}, {'end': 2173.36, 'text': 'A state of the art soccer player on the left and a state of the art robot on the right.', 'start': 2165.608, 'duration': 7.752}, {'end': 2190.032, 'text': "I think there's like four or five times he scores.", 'start': 2185.414, 'duration': 4.618}, {'end': 2191.177, 'text': 'All right.', 'start': 2190.052, 'duration': 1.125}, {'end': 2196.598, 'text': "And that's all the movement and so on involved with that.", 'start': 2194.235, 'duration': 2.363}], 'summary': 'Our visual perception system is impressive, with 540 million years of data, enabling us to perform tasks and anticipate physics in a split second, surpassing state-of-the-art robots in a soccer game.', 'duration': 60.273, 'max_score': 2136.325, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2136325.jpg'}, {'end': 2251.48, 'src': 'embed', 'start': 2221.121, 'weight': 3, 'content': [{'end': 2223.103, 'text': "There's just noisy, broken communication.", 'start': 2221.121, 'duration': 1.982}, {'end': 2229.647, 'text': 'So humans are incredible in terms of our ability to understand the world and in terms of our ability to act in that world.', 'start': 2223.563, 'duration': 6.084}, {'end': 2240.335, 'text': "And the fact that humans, the idea, the view, the popular view grounded in the psychology that humans and automations don't mix well.", 'start': 2230.528, 'duration': 9.807}, {'end': 2244.378, 'text': 'Overtrust, misunderstanding, loss of vigilance, decrement, and so on.', 'start': 2240.915, 'duration': 3.463}, {'end': 2247.036, 'text': "That's not an obvious fact.", 'start': 2245.575, 'duration': 1.461}, {'end': 2249.038, 'text': 'It happens a lot in the lab.', 'start': 2247.697, 'duration': 1.341}, {'end': 2251.48, 'text': 'Most of the experiments are actually in the lab.', 'start': 2249.118, 'duration': 2.362}], 'summary': 'Humans have strong understanding and action abilities, but face challenges when mixing with automation, leading to overtrust, misunderstanding, and loss of vigilance.', 'duration': 30.359, 'max_score': 2221.121, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2221121.jpg'}, {'end': 2344.04, 'src': 'embed', 'start': 2314.985, 'weight': 1, 'content': [{'end': 2321.93, 'text': '000 moments of transfer of control they are always vigilant.', 'start': 2314.985, 'duration': 6.945}, {'end': 2327.033, 'text': "There's not a moment once in this data set where they respond too late.", 'start': 2322.25, 'duration': 4.783}, {'end': 2332.011, 'text': 'to a critical situation, to a challenging world situation.', 'start': 2329.649, 'duration': 2.362}, {'end': 2342.498, 'text': "Now, the data set, 22 vehicles, that's 0.1% or less than the full Tesla fleet that has autopilot.", 'start': 2332.651, 'duration': 9.847}, {'end': 2344.04, 'text': "But it's still an inkling.", 'start': 2342.999, 'duration': 1.041}], 'summary': '22 out of 22 tesla vehicles with autopilot responded promptly to critical situations, demonstrating high vigilance.', 'duration': 29.055, 'max_score': 2314.985, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2314985.jpg'}, {'end': 2428.196, 'src': 'embed', 'start': 2404.497, 'weight': 4, 'content': [{'end': 2412.279, 'text': "And if you approach the shared autonomy side, the semi-autonomous, where you're only responsible for a large percentage, but not 100% of the driving,", 'start': 2404.497, 'duration': 7.782}, {'end': 2419.161, 'text': 'then you have to solve the human side, the human interaction, the sensing, what the driver is doing, the collaborating,', 'start': 2412.279, 'duration': 6.882}, {'end': 2423.242, 'text': 'communicating with the driver and the personalization aspect that learns with the driver.', 'start': 2419.161, 'duration': 4.081}, {'end': 2428.196, 'text': 'As I said, you can go online.', 'start': 2426.955, 'duration': 1.241}], 'summary': 'Developing semi-autonomous systems entails addressing human interaction, sensing, collaboration, and personalization.', 'duration': 23.699, 'max_score': 2404.497, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2404497.jpg'}, {'end': 2529.174, 'src': 'embed', 'start': 2483.994, 'weight': 5, 'content': [{'end': 2491.316, 'text': "you're sort of balancing the time when it's a critical time, to pay attention when it's not, and communicating effectively, learning with the driver.", 'start': 2483.994, 'duration': 7.322}, {'end': 2494.917, 'text': 'That problem is a fundamental machine learning problem.', 'start': 2491.616, 'duration': 3.301}, {'end': 2500.178, 'text': "There's a lot of data, visible light, everything about the driver, and it's a psychology problem.", 'start': 2495.317, 'duration': 4.861}, {'end': 2508.4, 'text': "So we have data, we have complicated human beings, and it's a human-robot interaction problem that deserves solving.", 'start': 2500.698, 'duration': 7.702}, {'end': 2518.131, 'text': "But as you'll hear on the Beyond, the human side, looking out into the world, people that are trying to solve the fully autonomous vehicle,", 'start': 2509.58, 'duration': 8.551}, {'end': 2520.312, 'text': "it's really a two approach consideration.", 'start': 2518.131, 'duration': 2.181}, {'end': 2527.594, 'text': 'One approach is vision, cameras, and deep learning.', 'start': 2520.972, 'duration': 6.622}, {'end': 2529.174, 'text': 'Collect a huge amount of data.', 'start': 2527.694, 'duration': 1.48}], 'summary': 'Balancing critical time, communication, and learning with the driver is a fundamental machine learning problem with a focus on human-robot interaction and vision-based approaches.', 'duration': 45.18, 'max_score': 2483.994, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2483994.jpg'}], 'start': 2114.461, 'title': 'Human-robot interaction and capabilities', 'summary': 'Explores human visual perception and control, emphasizing human-automation interaction and highlighting the superior performance of humans in certain tasks. it also discusses observations from 22 teslas over a two-year period, revealing that drivers used autopilot for 34% of the miles and had 0 late responses in 26,000 moments of control transfer, while comparing vision-based cameras with deep learning and lidar for mapping and perception in autonomous driving.', 'chapters': [{'end': 2271.575, 'start': 2114.461, 'title': 'Human vs automation: understanding and capabilities', 'summary': 'Discusses the impressive capabilities of human visual perception and control, highlighting the misconceptions about human-automation interaction and emphasizing the superior performance of humans in certain tasks compared to robots.', 'duration': 157.114, 'highlights': ['Humans possess an incredible ability to perceive and understand the world, demonstrated through the extensive 540 million years worth of data on visual perception systems.', 'The chapter emphasizes the superior performance of humans in tasks such as soccer playing compared to state-of-the-art robots, showcasing the remarkable control and intuition possessed by humans.', "The discussion challenges the popular view grounded in psychology that humans and automation don't mix well, highlighting the misconceptions and overtrust in automation that are not obvious facts."]}, {'end': 2665.873, 'start': 2271.655, 'title': 'Human-robot interaction in autonomous driving', 'summary': 'Discusses the findings from observing 22 teslas over a two-year period, revealing that drivers used autopilot for 34% of the miles and had 0 late responses in 26,000 moments of control transfer. it delves into the challenges of building systems for shared autonomy and fully autonomous driving, comparing the use of vision-based cameras with deep learning and lidar for mapping and perception.', 'duration': 394.218, 'highlights': ['The data from observing 22 Teslas over two years showed that drivers used autopilot for 34% of the miles with no late responses in 26,000 moments of control transfer, demonstrating high vigilance (quantifiable: 34% of miles, 0 late responses).', 'The chapter discusses the challenges and trade-offs in building systems for shared autonomy and fully autonomous driving, emphasizing the need to solve various aspects such as mapping, localization, scene perception, control, planning, and external communication, as well as the human interaction and collaboration (quantifiable: trade-offs, various aspects to be solved).', 'It highlights the two approaches for perception in autonomous driving: vision-based cameras with deep learning, which require a huge amount of data and are feasible and cost-effective due to the world being designed for human eyes, and LIDAR-based mapping, which offers consistency, reliability, and explainability, especially in the context of accurate sensors (quantifiable: feasibility and cost-effectiveness of vision-based cameras, consistency and reliability of LIDAR).', 'The transcript addresses the fundamental machine learning problem of effectively managing driver attention, especially in critical situations, and emphasizes the need for communication and learning with the driver, considering the increasing smartphone use in cars (quantifiable: managing driver attention, increasing smartphone use).', 'It raises the importance of human-robot interaction in autonomous driving, highlighting that it is a complex problem deserving attention and emphasizing the need to understand the psychology and behavior of human drivers (quantifiable: importance of human-robot interaction, understanding human psychology).']}], 'duration': 551.412, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2114461.jpg', 'highlights': ['Humans possess an incredible ability to perceive and understand the world, demonstrated through extensive 540 million years worth of data on visual perception systems.', 'The data from observing 22 Teslas over two years showed that drivers used autopilot for 34% of the miles with no late responses in 26,000 moments of control transfer, demonstrating high vigilance (quantifiable: 34% of miles, 0 late responses).', 'The chapter emphasizes the superior performance of humans in tasks such as soccer playing compared to state-of-the-art robots, showcasing the remarkable control and intuition possessed by humans.', "The discussion challenges the popular view grounded in psychology that humans and automation don't mix well, highlighting the misconceptions and overtrust in automation that are not obvious facts.", 'The chapter discusses the challenges and trade-offs in building systems for shared autonomy and fully autonomous driving, emphasizing the need to solve various aspects such as mapping, localization, scene perception, control, planning, and external communication, as well as the human interaction and collaboration (quantifiable: trade-offs, various aspects to be solved).', 'It highlights the two approaches for perception in autonomous driving: vision-based cameras with deep learning, which require a huge amount of data and are feasible and cost-effective due to the world being designed for human eyes, and LIDAR-based mapping, which offers consistency, reliability, and explainability, especially in the context of accurate sensors (quantifiable: feasibility and cost-effectiveness of vision-based cameras, consistency and reliability of LIDAR).', 'The transcript addresses the fundamental machine learning problem of effectively managing driver attention, especially in critical situations, and emphasizes the need for communication and learning with the driver, considering the increasing smartphone use in cars (quantifiable: managing driver attention, increasing smartphone use).', 'It raises the importance of human-robot interaction in autonomous driving, highlighting that it is a complex problem deserving attention and emphasizing the need to understand the psychology and behavior of human drivers (quantifiable: importance of human-robot interaction, understanding human psychology).']}, {'end': 3249.868, 'segs': [{'end': 2905.971, 'src': 'embed', 'start': 2880.862, 'weight': 0, 'content': [{'end': 2890.288, 'text': "in extreme conditions and it's just susceptible to all these kinds of problems and doesn't detect speed unless you do some tricky structure from motion kind of things.", 'start': 2880.862, 'duration': 9.426}, {'end': 2897.53, 'text': "So here's where sensor fusion steps in and everybody works together to build an entire picture.", 'start': 2890.648, 'duration': 6.882}, {'end': 2899.21, 'text': "That's how this plot works.", 'start': 2897.57, 'duration': 1.64}, {'end': 2900.59, 'text': 'You can stack it on top of each other.', 'start': 2899.23, 'duration': 1.36}, {'end': 2905.971, 'text': 'So if you look at a suite that, for example, Tesla is using, which is ultrasonic radar and camera,', 'start': 2901.03, 'duration': 4.941}], 'summary': 'Sensor fusion enables building a comprehensive picture using ultrasonic radar, camera, and other sensors.', 'duration': 25.109, 'max_score': 2880.862, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2880862.jpg'}, {'end': 2979.471, 'src': 'embed', 'start': 2952.957, 'weight': 3, 'content': [{'end': 2956.618, 'text': "When you say, oh crap, I don't know what to do, the human catches.", 'start': 2952.957, 'duration': 3.661}, {'end': 2965.441, 'text': 'In the fully autonomous mode, is what Waymo is working on, and others, the fail-safe is LIDAR.', 'start': 2957.778, 'duration': 7.663}, {'end': 2967.102, 'text': 'The fail-safe is maps.', 'start': 2966.101, 'duration': 1.001}, {'end': 2970.545, 'text': "that you can't rely on the human.", 'start': 2968.103, 'duration': 2.442}, {'end': 2979.471, 'text': "but you know this road so well that if the cameras freak out, if there's any of the sensors freak out that you're able to, you have such good maps.", 'start': 2970.545, 'duration': 8.926}], 'summary': 'In fully autonomous mode, waymo relies on fail-safe lidar and maps to navigate roads, minimizing dependence on humans.', 'duration': 26.514, 'max_score': 2952.957, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2952957.jpg'}, {'end': 3046.301, 'src': 'embed', 'start': 3003.221, 'weight': 1, 'content': [{'end': 3006.202, 'text': "Manually driven cars are still happening, so there's a lot of data happening.", 'start': 3003.221, 'duration': 2.981}, {'end': 3014.404, 'text': "Semi-autonomous cars, every company is now releasing more and more semi-autonomous technology, so that's all data.", 'start': 3006.242, 'duration': 8.162}, {'end': 3019.438, 'text': "And what that boils down to is the two paths they're walking towards.", 'start': 3015.455, 'duration': 3.983}, {'end': 3023.081, 'text': "It's Vision versus LiDAR, L2 versus L4.", 'start': 3020.119, 'duration': 2.962}, {'end': 3025.363, 'text': 'Semi-autonomous versus fully autonomous.', 'start': 3023.542, 'duration': 1.821}, {'end': 3028.926, 'text': 'Tesla on the semi-autonomous front has reached one billion miles.', 'start': 3025.864, 'duration': 3.062}, {'end': 3032.53, 'text': 'Waymo the leader on the autonomous front has reached 10 million miles.', 'start': 3029.367, 'duration': 3.163}, {'end': 3034.972, 'text': "The pros and cons, I've outlined them.", 'start': 3033.13, 'duration': 1.842}, {'end': 3042.958, 'text': "One, the vision one, the one I'm obviously very excited about, and machine learning research is excited about,", 'start': 3035.672, 'duration': 7.286}, {'end': 3046.301, 'text': 'which fundamentally relies on huge data and deep learning.', 'start': 3042.958, 'duration': 3.343}], 'summary': 'Semi-autonomous vs fully autonomous: tesla 1b miles, waymo 10m miles.', 'duration': 43.08, 'max_score': 3003.221, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ3003221.jpg'}, {'end': 3230.639, 'src': 'embed', 'start': 3207.544, 'weight': 4, 'content': [{'end': 3214.668, 'text': "That's actually the defining problem for artificial intelligence in the 21st century is that touches people in a real way.", 'start': 3207.544, 'duration': 7.124}, {'end': 3219.191, 'text': 'And I think cars, autonomous vehicles, is one of the big ways that that happens.', 'start': 3214.948, 'duration': 4.243}, {'end': 3227.516, 'text': 'We get to deal with the psychology, the philosophy, the sociology aspects of it, how we socially think about it, to the robotics problem,', 'start': 3219.411, 'duration': 8.105}, {'end': 3228.737, 'text': 'to the perception problem.', 'start': 3227.516, 'duration': 1.221}, {'end': 3230.639, 'text': "It's a fascinating space to explore.", 'start': 3228.817, 'duration': 1.822}], 'summary': "Ai's challenge in 21st century: autonomous vehicles impact society, psychology, philosophy, and robotics.", 'duration': 23.095, 'max_score': 3207.544, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ3207544.jpg'}], 'start': 2666.313, 'title': 'Sensor comparison and perception challenges', 'summary': 'Compares lidar, radar, and cameras for autonomous vehicles, emphasizing sensor fusion, discusses vision vs lidar, l2 vs l4, and milestones achieved by tesla and waymo, and explores challenges in reliable sensor suites, ai impact, and pedestrian detection.', 'chapters': [{'end': 2900.59, 'start': 2666.313, 'title': 'Sensor comparison for autonomous vehicles', 'summary': "Discusses the pros and cons of lidar, radar, and cameras for autonomous vehicles, highlighting their cost, resolution, range, and performance in various weather conditions, and emphasizes the importance of sensor fusion for an autonomous vehicle's perception system.", 'duration': 234.277, 'highlights': ['LiDAR offers high resolution and works in dark and variable lighting conditions, but is expensive and lacks rich textural contrast information. LiDAR provides good resolution and range, but is also expensive and not suitable for extreme weather conditions. It lacks rich textural contrast information and is sensitive to fog and rain, making it less reliable in adverse weather conditions.', 'Cameras are cheap, provide rich information, and have the best range, but are not effective in dark or extreme conditions and struggle with speed detection. Cameras are cost-effective with high resolution and a wide range. However, they are not suitable for dark or extreme conditions and struggle with speed detection. They provide rich information, but their performance is limited in adverse conditions.', 'Radar is cheap, has good range, and performs well in extreme weather, but has low resolution and does not provide rich information. Radar is cost-effective with good range and performance in extreme weather conditions. However, it has low resolution and does not provide rich information, limiting its capabilities in certain situations.', 'Sensor fusion is essential for creating a comprehensive perception system for autonomous vehicles. Sensor fusion is crucial for integrating the strengths of different sensors and compensating for their individual limitations, ultimately creating a comprehensive perception system for autonomous vehicles.']}, {'end': 3090.283, 'start': 2901.03, 'title': 'Vision vs lidar in autonomous vehicles', 'summary': 'Discusses the comparison between vision-based and lidar-based approaches in autonomous vehicles, highlighting the emergence of semi-autonomous and fully autonomous technologies, the debate between vision vs lidar and l2 vs l4, and the milestones achieved by tesla and waymo in terms of miles driven.', 'duration': 189.253, 'highlights': ['Tesla has reached one billion miles on semi-autonomous driving, while Waymo has reached 10 million miles on fully autonomous driving. The comparison of milestones achieved by Tesla in semi-autonomous driving (1 billion miles) and Waymo in fully autonomous driving (10 million miles) showcases the advancements in autonomous technologies.', 'The debate between Vision vs LiDAR and L2 vs L4 in the development of autonomous vehicles is outlined, emphasizing the reliance on huge data and deep learning for the vision-based approach. The discussion on the debate between Vision vs LiDAR and L2 vs L4 emphasizes the reliance on huge data and deep learning for the vision-based approach, reflecting the ongoing advancements in autonomous vehicle technologies.', 'The comparison of different fail-safe methods in semi-autonomous and fully autonomous driving, where human intervention serves as the fail-safe in the former, while LiDAR and maps act as fail-safes in the latter. The comparison of fail-safe methods in semi-autonomous and fully autonomous driving highlights the role of human intervention as a fail-safe in the former, and the reliance on LiDAR and maps as fail-safes in the latter, demonstrating the differing safety measures in autonomous driving technologies.']}, {'end': 3249.868, 'start': 3090.283, 'title': 'Challenges in autonomous vehicle perception', 'summary': 'Discusses the challenges of reliable sensor suites, the capabilities of lidar and cameras, and the potential impact of artificial intelligence in autonomous vehicles, highlighting the complexities in detecting pedestrians and the potential of ai to impact and save lives.', 'duration': 159.585, 'highlights': ["The challenge of reliable sensor suites and the need for accurate perception in autonomous vehicles, with the discussion of LiDAR's ability to detect solid body objects and cameras' effectiveness in detecting fog or smoke.", "The complexities in detecting pedestrians and making decisions in scenarios involving distracted or impaired pedestrians, highlighting the difficulty in addressing the 'four Ds' - drunk, drugged, drowsy, or distracted individuals - and the challenges this poses for autonomous vehicles.", 'The potential impact of artificial intelligence in autonomous vehicles, emphasizing the opportunity to use AI algorithms to directly impact and save lives, and the significance of this as a defining problem for artificial intelligence in the 21st century.', 'The exploration of the interdisciplinary aspects of autonomous vehicles, from psychology, philosophy, and sociology to the robotics and perception problems, showcasing the multifaceted nature of the challenges and the excitement in exploring this space.']}], 'duration': 583.555, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sRxaMDDMWQQ/pics/sRxaMDDMWQQ2666313.jpg', 'highlights': ['Sensor fusion is crucial for integrating the strengths of different sensors and compensating for their individual limitations, ultimately creating a comprehensive perception system for autonomous vehicles.', 'The comparison of milestones achieved by Tesla in semi-autonomous driving (1 billion miles) and Waymo in fully autonomous driving (10 million miles) showcases the advancements in autonomous technologies.', 'The debate between Vision vs LiDAR and L2 vs L4 emphasizes the reliance on huge data and deep learning for the vision-based approach, reflecting the ongoing advancements in autonomous vehicle technologies.', 'The comparison of different fail-safe methods in semi-autonomous and fully autonomous driving highlights the role of human intervention as a fail-safe in the former, and the reliance on LiDAR and maps as fail-safes in the latter, demonstrating the differing safety measures in autonomous driving technologies.', 'The potential impact of artificial intelligence in autonomous vehicles, emphasizing the opportunity to use AI algorithms to directly impact and save lives, and the significance of this as a defining problem for artificial intelligence in the 21st century.']}], 'highlights': ['The main goal of autonomous vehicles is to save lives by preventing crashes that lead to injuries and fatalities, with a staggering statistic of a death in a car crash every 23 seconds worldwide.', 'Tesla reached the 1 billion mile mark in semi-autonomous driving with autopilot, showcasing its advanced capabilities for highway driving.', 'Humans possess an incredible ability to perceive and understand the world, demonstrated through extensive 540 million years worth of data on visual perception systems.', 'Sensor fusion is crucial for integrating the strengths of different sensors and compensating for their individual limitations, ultimately creating a comprehensive perception system for autonomous vehicles.']}