title
Stanford Webinar - Building Safe and Reliable Autonomous Systems

description
AI is changing fast. Building safe, trustworthy AI systems and establishing confidence in their behavior and robustness is crucial for their success and adoption in society. In this conversation with Dr. Anthony Corso, he discusses techniques for building safe and reliable autonomous systems using state of the art machine learning techniques for high-stakes applications such as healthcare, transportation, and critical infrastructure. View Anthony's course: https://online.stanford.edu/courses/xaa101-designing-reliable-and-robust-ai-systems About the speaker: Anthony is the executive director of the Stanford Center for AI Safety and the associate director of research for the SAIL-Toyota Center. His current research is split between developing verifiably robust autonomy and the using AI algorithms to tackle climate change. Learn more about Anthony: https://anthonylcorso.com/ Chapters 0:00 Introduction 01:38 Dr. Corso intro to reliable AI 03:24 Risks with Autonomous Systems 04:43 How AI Systems Fail 06:13 Can AI be more safe than humans? 07:44 Challenges & Scalability 08:58 Generalizability 11:19 Existential Risks of AI 13:17 AI Ethics 14:39 Applications of AI Systems 15:29 How to build safe AI Systems 21:00 Testing for Rare Events 22:05 Testing & Formal Verification 26:40 What is Robustness? 29:20 Uncertainty Quantification & Fallback Strategies

detail
{'title': 'Stanford Webinar - Building Safe and Reliable Autonomous Systems', 'heatmap': [], 'summary': "The webinar covers dr. corso's role at stanford, ai in transportation and safety, risks of ai and adversarial attacks, equitable deployment of ai systems, safety challenges in self-driving cars, a course on ai in safety-critical domains, and the future of autonomous systems, highlighting the need for robust and resilient ai in safety-critical applications.", 'chapters': [{'end': 87.569, 'segs': [{'end': 53.336, 'src': 'embed', 'start': 28.468, 'weight': 0, 'content': [{'end': 36.431, 'text': "He is also the executive director of the Stanford Center for AI Safety, which is broadly the topic we're going to be talking about today.", 'start': 28.468, 'duration': 7.963}, {'end': 44.514, 'text': 'His research is focused on the use of algorithmic decision-making for safety-critical applications, emphasizing the creation of robust,', 'start': 37.331, 'duration': 7.183}, {'end': 45.974, 'text': 'reliable autonomous systems.', 'start': 44.514, 'duration': 1.46}, {'end': 48.375, 'text': 'And this is a really important topic.', 'start': 46.494, 'duration': 1.881}, {'end': 53.336, 'text': "And obviously, it's been important for a long time, and that's something that Anthony and I will talk about a little bit.", 'start': 48.395, 'duration': 4.941}], 'summary': 'Research focuses on algorithmic decision-making for safety-critical applications and emphasizes the creation of robust, reliable autonomous systems.', 'duration': 24.868, 'max_score': 28.468, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M28468.jpg'}], 'start': 10.356, 'title': 'Dr. corso', 'summary': 'Introduces dr. corso, a post-doctoral researcher at stanford university, discussing his role in the stanford intelligent systems laboratory and the stanford center for ai safety, emphasizing the importance of algorithmic decision-making for safety-critical applications and the upcoming seminar on the topic in the last week of july.', 'chapters': [{'end': 87.569, 'start': 10.356, 'title': 'Introduction to dr. corso', 'summary': 'Introduces dr. corso, a post-doctoral researcher at stanford university, discussing his role in the stanford intelligent systems laboratory and the stanford center for ai safety, emphasizing the importance of algorithmic decision-making for safety-critical applications and the upcoming seminar on the topic in the last week of july.', 'duration': 77.213, 'highlights': ["Dr. Corso's role in the Stanford Intelligent Systems Laboratory and the Stanford Center for AI Safety, focusing on algorithmic decision-making for safety-critical applications.", 'Discussion of the upcoming seminar on the topic of algorithmic decision-making for safety-critical applications in the last week of July.', "Mention of Dr. Corso's position as the executive director of the Stanford Center for AI Safety."]}], 'duration': 77.213, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M10356.jpg', 'highlights': ["Dr. Corso's role in the Stanford Intelligent Systems Laboratory and the Stanford Center for AI Safety, focusing on algorithmic decision-making for safety-critical applications.", "Mention of Dr. Corso's position as the executive director of the Stanford Center for AI Safety.", 'Discussion of the upcoming seminar on the topic of algorithmic decision-making for safety-critical applications in the last week of July.']}, {'end': 372.901, 'segs': [{'end': 183.589, 'src': 'embed', 'start': 115.307, 'weight': 0, 'content': [{'end': 116.287, 'text': 'here at Stanford.', 'start': 115.307, 'duration': 0.98}, {'end': 127.431, 'text': 'And a lot of the research that I did was thinking about the ways in which we can use AI or algorithmic decision making to improve both the safety and efficiency of transportation systems.', 'start': 116.528, 'duration': 10.903}, {'end': 132.193, 'text': 'So, you know, something like a million people every year die in car accidents.', 'start': 128.191, 'duration': 4.002}, {'end': 138.175, 'text': 'And so, if we can improve the safety of things like driving cars, then we can save a lot of lives.', 'start': 132.453, 'duration': 5.722}, {'end': 145.998, 'text': 'And so the topic that I mostly studied was around the use of AI systems to perform these tasks safely.', 'start': 138.975, 'duration': 7.023}, {'end': 152.241, 'text': "But then the question naturally arises how do you check whether or not you've built a very safe system?", 'start': 146.479, 'duration': 5.762}, {'end': 162.846, 'text': 'So I did my dissertation work on developing algorithms that allow you to stress, test and ensure the safety of these really high stakes systems.', 'start': 152.861, 'duration': 9.985}, {'end': 172.482, 'text': "And since then I've carried on as a postdoc and have continued to think about the areas in which we use AI systems that influence and affect a lot of people.", 'start': 163.326, 'duration': 9.156}, {'end': 176.549, 'text': 'And we really need to make sure that they are robust and reliable before deploying them.', 'start': 172.542, 'duration': 4.007}, {'end': 179.245, 'text': "That's great.", 'start': 178.905, 'duration': 0.34}, {'end': 181.447, 'text': 'So Anthony, I think maybe we can take a step back.', 'start': 179.405, 'duration': 2.042}, {'end': 183.589, 'text': 'So there are a couple things there.', 'start': 181.467, 'duration': 2.122}], 'summary': 'Research focuses on using ai to improve transportation safety, aiming to reduce car accident fatalities, and developing algorithms to ensure the safety of ai systems.', 'duration': 68.282, 'max_score': 115.307, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M115307.jpg'}, {'end': 292.559, 'src': 'embed', 'start': 207.966, 'weight': 3, 'content': [{'end': 211.347, 'text': 'what that broadly covers and what the risks are involved with those?', 'start': 207.966, 'duration': 3.381}, {'end': 215.689, 'text': "You've already touched on it in terms of car accidents, but can you build on that a little bit??", 'start': 211.367, 'duration': 4.322}, {'end': 217.309, 'text': 'Yeah, absolutely.', 'start': 216.569, 'duration': 0.74}, {'end': 224.232, 'text': "So I think of autonomous systems as really any system that's going to perform a task that historically has been done by a human.", 'start': 217.349, 'duration': 6.883}, {'end': 227.713, 'text': 'So this can be driving cars is a prime example.', 'start': 224.412, 'duration': 3.301}, {'end': 230.474, 'text': 'And autonomous systems.', 'start': 228.553, 'duration': 1.921}, {'end': 238.159, 'text': "although we've had autonomous systems for a very long time, People are becoming much more aware of them with the advent of AI and machine learning,", 'start': 230.474, 'duration': 7.685}, {'end': 250.808, 'text': "which are techniques that allow computer systems to learn from data and basically enable us to process information in real time and enable this higher level of autonomy that we've been seeing over the past couple of years.", 'start': 238.159, 'duration': 12.649}, {'end': 258.411, 'text': 'The problem is, however, that a lot of these technologies, although they are absolutely amazing and enable us to do incredible things,', 'start': 251.568, 'duration': 6.843}, {'end': 261.413, 'text': 'often fail in kind of unexpected ways.', 'start': 258.411, 'duration': 3.002}, {'end': 264.495, 'text': 'They tend to be a little bit more brittle than, say, humans are.', 'start': 261.713, 'duration': 2.782}, {'end': 272.679, 'text': 'So they may work in a very controlled, say, laboratory setting, but as soon as you deploy these systems and the environment changes ever so slightly,', 'start': 265.135, 'duration': 7.544}, {'end': 273.94, 'text': 'you can begin running into problems.', 'start': 272.679, 'duration': 1.261}, {'end': 280.363, 'text': "And this is, of course, very dangerous if you're deploying these systems in a domain like driving where people's lives are at risk.", 'start': 273.96, 'duration': 6.403}, {'end': 287.775, 'text': 'Could you you know I actually have something I want to circle back onto about safety,', 'start': 283.271, 'duration': 4.504}, {'end': 291.218, 'text': 'but could we elaborate on the brittleness of some of these systems?', 'start': 287.775, 'duration': 3.443}, {'end': 292.559, 'text': 'Can you build on what that means?', 'start': 291.258, 'duration': 1.301}], 'summary': 'Autonomous systems, including ai and machine learning, have potential but may be brittle and prone to unexpected failures.', 'duration': 84.593, 'max_score': 207.966, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M207966.jpg'}, {'end': 364.072, 'src': 'embed', 'start': 337.544, 'weight': 5, 'content': [{'end': 343.566, 'text': 'The problem is if you then try to deploy that model in, say, a different geographic domain, where things are slightly different,', 'start': 337.544, 'duration': 6.022}, {'end': 349.388, 'text': 'so maybe the crosswalk looks a little bit different, or people tend to wear different clothes or something like that,', 'start': 343.566, 'duration': 5.822}, {'end': 355.029, 'text': "and all of a sudden what we see is that these machine learning models' performances can drop pretty dramatically,", 'start': 349.388, 'duration': 5.641}, {'end': 358.73, 'text': 'even though the differences in kind of the environment are quite subtle.', 'start': 355.029, 'duration': 3.701}, {'end': 360.751, 'text': "And humans don't really have this problem.", 'start': 359.07, 'duration': 1.681}, {'end': 364.072, 'text': "We're able to generalize fairly well across a variety of factors.", 'start': 360.831, 'duration': 3.241}], 'summary': "Machine learning models' performance can drop dramatically in different geographic domains, despite subtle environmental differences, unlike humans.", 'duration': 26.528, 'max_score': 337.544, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M337544.jpg'}], 'start': 87.689, 'title': 'Ai in transportation and autonomous systems', 'summary': 'Discusses leveraging ai to improve transportation safety and efficiency, emphasizing potential lives saved, and the concept of autonomous systems in transportation, highlighting tasks historically done by humans and the role of ai. it also covers the brittleness of technology in real-world applications, focusing on the failure of machine learning models across different geographic domains and contrasting with human generalization abilities.', 'chapters': [{'end': 183.589, 'start': 87.689, 'title': 'Ai in transportation safety', 'summary': "Discusses anthony's research on using ai to improve safety and efficiency in transportation systems, emphasizing the potential to save lives by leveraging ai in car accident prevention and the development of algorithms to stress-test and ensure the safety of high-stakes systems.", 'duration': 95.9, 'highlights': ["Anthony's research focused on using AI to improve safety and efficiency in transportation systems, aiming to save lives as approximately a million people die in car accidents annually. Approximately a million people die in car accidents annually, motivating the need to improve safety through AI.", "He developed algorithms to stress-test and ensure the safety of high-stakes systems, addressing the challenge of verifying the safety of AI-driven systems. Anthony's dissertation work involved developing algorithms to stress-test and ensure the safety of high-stakes systems, addressing the challenge of verifying the safety of AI-driven systems.", 'Anthony emphasized the importance of ensuring robustness and reliability in AI systems that influence and affect a large number of people before deploying them. Anthony stressed the need to ensure robustness and reliability in AI systems that influence and affect a large number of people before deploying them.']}, {'end': 250.808, 'start': 183.609, 'title': 'Understanding autonomous systems', 'summary': 'Discusses the concept of autonomous systems, particularly in transportation, covering the tasks historically done by humans, the role of ai and machine learning in enabling higher autonomy, and the increasing awareness of autonomous systems due to these technologies.', 'duration': 67.199, 'highlights': ['Autonomous systems encompass tasks historically performed by humans, such as driving cars.', 'AI and machine learning enable computer systems to process information in real time, leading to higher autonomy in systems.', 'The increasing awareness of autonomous systems is attributed to the advancements in AI and machine learning.']}, {'end': 372.901, 'start': 251.568, 'title': 'Brittleness of technology in real-world applications', 'summary': "Discusses the brittleness of technology in real-world applications, highlighting how machine learning models can fail in different geographic domains, affecting their performance significantly and contrasting this with humans' ability to generalize across various factors.", 'duration': 121.333, 'highlights': ["Machine learning models' performance can drop dramatically when deployed in different geographic domains, even with subtle environmental differences, contrasting with humans' ability to generalize across various factors.", "The brittleness of technologies can pose significant dangers when deployed in domains like driving, where people's lives are at risk.", 'Technologies, while enabling incredible feats, tend to be more brittle than humans and can fail unexpectedly in real-world settings.']}], 'duration': 285.212, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M87689.jpg', 'highlights': ["Anthony's research aims to save lives as approximately a million people die in car accidents annually.", 'He developed algorithms to stress-test and ensure the safety of high-stakes systems.', 'Emphasized the importance of ensuring robustness and reliability in AI systems before deploying them.', 'Autonomous systems encompass tasks historically performed by humans, such as driving cars.', 'AI and machine learning enable computer systems to process information in real time, leading to higher autonomy.', "Machine learning models' performance can drop dramatically when deployed in different geographic domains.", 'The brittleness of technologies can pose significant dangers when deployed in domains like driving.', 'Technologies, while enabling incredible feats, tend to be more brittle than humans and can fail unexpectedly.']}, {'end': 814.661, 'segs': [{'end': 441.531, 'src': 'embed', 'start': 412.812, 'weight': 0, 'content': [{'end': 415.815, 'text': 'much better than a human at a variety of tasks you really care about.', 'start': 412.812, 'duration': 3.003}, {'end': 418.397, 'text': 'So I think a great example of this would be healthcare.', 'start': 415.855, 'duration': 2.542}, {'end': 424.442, 'text': "So, if you could have an AI system that was maybe even just as good as the world's best doctor,", 'start': 418.737, 'duration': 5.705}, {'end': 429.566, 'text': "you could deploy that system very widely across the world, providing healthcare access to people who wouldn't otherwise get it.", 'start': 424.442, 'duration': 5.124}, {'end': 433.988, 'text': 'or likewise flying aircraft or flying or driving vehicles.', 'start': 430.346, 'duration': 3.642}, {'end': 441.531, 'text': 'These are things that although humans do fairly well, we can really improve the safety of and that can be really beneficial for everyone.', 'start': 434.528, 'duration': 7.003}], 'summary': 'Ai can provide better healthcare access and improve safety in aviation and driving.', 'duration': 28.719, 'max_score': 412.812, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M412812.jpg'}, {'end': 470.419, 'src': 'embed', 'start': 445.652, 'weight': 2, 'content': [{'end': 453.837, 'text': "because what we're talking about with AI systems is it allows it to, allows us to scale these systems out to a ton of people.", 'start': 445.652, 'duration': 8.185}, {'end': 458.862, 'text': "And so then any risk that's inherent in the system is now a risk affecting many, many, many more people.", 'start': 454.138, 'duration': 4.724}, {'end': 463.486, 'text': 'And so we have to be extremely careful before actually deploying these systems out into the world.', 'start': 458.902, 'duration': 4.584}, {'end': 465.354, 'text': "That's really interesting.", 'start': 464.653, 'duration': 0.701}, {'end': 470.419, 'text': 'So part of it, part of what the particular challenge of these systems is how scalable they are, right?', 'start': 465.394, 'duration': 5.025}], 'summary': 'Ai systems enable scalable deployment, with potential risks affecting large populations. caution is crucial before widespread implementation.', 'duration': 24.767, 'max_score': 445.652, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M445652.jpg'}, {'end': 511.821, 'src': 'embed', 'start': 490.399, 'weight': 1, 'content': [{'end': 501.111, 'text': 'that another sort of problem that AI systems tend to face is that they can be fairly susceptible to what we call adversarial attack or basically like intentional manipulation by people.', 'start': 490.399, 'duration': 10.712}, {'end': 503.053, 'text': 'And so you know you can do.', 'start': 501.291, 'duration': 1.762}, {'end': 507.057, 'text': "there's been some research where people can, for example, will print out a sticker.", 'start': 503.053, 'duration': 4.004}, {'end': 511.821, 'text': 'that just kind of looks like maybe a sticker you would see out in the world posted on a stop sign.', 'start': 507.057, 'duration': 4.764}], 'summary': 'Ai systems are susceptible to adversarial attacks, such as stickers resembling stop signs.', 'duration': 21.422, 'max_score': 490.399, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M490399.jpg'}], 'start': 373.442, 'title': "Ai's role and safety concerns", 'summary': 'Discusses how ai systems can enhance safety in various domains, but also highlights the risks and susceptibility to adversarial attacks, emphasizing the need for careful deployment, testing, and verification to mitigate dangers and ethical implications.', 'chapters': [{'end': 490.399, 'start': 373.442, 'title': "Ai systems' role in safety", 'summary': 'Discusses how ai systems can significantly improve safety in various domains, such as healthcare, aircraft, and vehicles, by outperforming humans, potentially providing healthcare access to more people and enhancing overall safety. however, the scalability of ai systems poses significant risks that need to be carefully addressed before deployment.', 'duration': 116.957, 'highlights': ["AI systems can potentially perform much better than humans at tasks like healthcare, aircraft operation, and driving, significantly improving safety and providing broader access to healthcare. (e.g., AI system as good as the world's best doctor could provide healthcare access to more people)", 'The scalability of AI systems increases the risk as any inherent risk in the system could potentially affect a large number of people, necessitating extreme caution before deploying these systems. (e.g., deploying a system across all of the United States or the world)', 'The discussion emphasizes the need to be extremely careful due to the scalability of AI systems, which can pose risks affecting a vast number of people, unlike individual human errors with local consequences. (e.g., a mistake made as a single driver has local consequences, while a mistake in an AI system deployed across the world affects many people)', "The potential of AI systems to outperform humans in various domains, such as healthcare, flying aircraft, and driving vehicles, is highlighted as a reason for excitement about the capabilities of AI. (e.g., AI system performing as well as the world's best doctor)", "The chapter explores the dual aspect of AI systems' role in safety, not just ensuring safety but also improving safety by potentially performing tasks more safely than humans in certain domains. (e.g., AI systems potentially doing things more safely than humans, enhancing safety in healthcare, aircraft operation, and driving)"]}, {'end': 814.661, 'start': 490.399, 'title': 'Ai safety and adversarial attacks', 'summary': 'Discusses the susceptibility of ai systems to adversarial attacks, demonstrating how intentional manipulation can lead to dangerous consequences for autonomous vehicles, and emphasizes the importance of testing and verification to mitigate risks. it also highlights the need to consider the domain and consequences of failure when deploying ai systems, as well as the potential ethical implications of ai development.', 'duration': 324.262, 'highlights': ['Adversarial Attacks on AI Systems AI systems can be manipulated through adversarial attacks, as demonstrated by the ability to deceive machine learning systems into misinterpreting a stop sign as a different traffic sign, posing a significant threat to the safety of autonomous vehicles.', 'Consequences of Adversarial Attacks If all vehicles on the road use the same algorithm, they can all be influenced by a single adversarial attack, highlighting the widespread impact and potential danger of such manipulations.', 'Risk Considerations in AI Deployment The likelihood of failure event occurrence multiplied by the magnitude of its consequences determines the overall risk, with higher consequence domains like aircraft requiring extremely low probabilities of failure, contrasting with lower risk domains such as serving ads to users.', 'Safety in Different Operating Domains The context in which AI systems operate, such as downtown city streets or open pit mines, significantly impacts safety considerations, emphasizing the importance of domain-specific risk assessment.', 'Existential Risk and Autonomous Systems The development of highly competent autonomous systems poses the risk of misalignment in their objectives, potentially leading to existential risks, although the discussion primarily focuses on current autonomous systems in operation.', 'Ethical Implications of AI Development The consideration of ethics in AI development and deployment is brought into question, reflecting the need to address ethical concerns and implications in the advancement of AI technology.']}], 'duration': 441.219, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M373442.jpg', 'highlights': ["AI systems can potentially perform much better than humans at tasks like healthcare, aircraft operation, and driving, significantly improving safety and providing broader access to healthcare. (e.g., AI system as good as the world's best doctor could provide healthcare access to more people)", 'Adversarial Attacks on AI Systems AI systems can be manipulated through adversarial attacks, as demonstrated by the ability to deceive machine learning systems into misinterpreting a stop sign as a different traffic sign, posing a significant threat to the safety of autonomous vehicles.', 'The scalability of AI systems increases the risk as any inherent risk in the system could potentially affect a large number of people, necessitating extreme caution before deploying these systems. (e.g., deploying a system across all of the United States or the world)', 'The discussion emphasizes the need to be extremely careful due to the scalability of AI systems, which can pose risks affecting a vast number of people, unlike individual human errors with local consequences. (e.g., a mistake made as a single driver has local consequences, while a mistake in an AI system deployed across the world affects many people)', "The potential of AI systems to outperform humans in various domains, such as healthcare, flying aircraft, and driving vehicles, is highlighted as a reason for excitement about the capabilities of AI. (e.g., AI system performing as well as the world's best doctor)"]}, {'end': 1128.444, 'segs': [{'end': 860.951, 'src': 'embed', 'start': 814.761, 'weight': 0, 'content': [{'end': 819.646, 'text': "Or is that maybe at a level up from these very concrete sort of technical problems that you're trying to address?", 'start': 814.761, 'duration': 4.885}, {'end': 822.167, 'text': "Yeah, so that's a fantastic question.", 'start': 820.546, 'duration': 1.621}, {'end': 826.831, 'text': "And it's an extremely important piece of the whole deployment of AI systems,", 'start': 822.227, 'duration': 4.604}, {'end': 831.375, 'text': "ensuring that these systems are deployed in a way that's equitable and fair to all groups.", 'start': 826.831, 'duration': 4.544}, {'end': 836.059, 'text': 'And in fact you can even think about the risk that, say,', 'start': 832.256, 'duration': 3.803}, {'end': 842.164, 'text': 'an AI system will behave differently at different subgroups of people based on gender or race, or something like that,', 'start': 836.059, 'duration': 6.105}, {'end': 847.328, 'text': 'as essentially a lack of robustness to various subgroups.', 'start': 842.164, 'duration': 5.164}, {'end': 852.09, 'text': 'So this is a problem that we absolutely need to mitigate.', 'start': 848.929, 'duration': 3.161}, {'end': 855.85, 'text': "It's a problem that comes from the data sets that we as humans have generated.", 'start': 852.35, 'duration': 3.5}, {'end': 860.951, 'text': 'We have bias in our society in that to show up in the data sets that we create.', 'start': 856.47, 'duration': 4.481}], 'summary': 'Addressing bias in ai deployment is crucial for equitable and fair systems.', 'duration': 46.19, 'max_score': 814.761, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M814761.jpg'}, {'end': 975.61, 'src': 'embed', 'start': 945.412, 'weight': 7, 'content': [{'end': 951.656, 'text': "I do think that a lot of what we're about to talk about here basically applies across all sorts of application domains,", 'start': 945.412, 'duration': 6.244}, {'end': 954.898, 'text': 'because a lot of the core underlying technology is very, very similar.', 'start': 951.656, 'duration': 3.242}, {'end': 965.784, 'text': 'So yeah, as Pax mentioned, you have two phases, I would say, very broadly speaking, when designing an autonomous system for a high-stakes application.', 'start': 956.118, 'duration': 9.666}, {'end': 967.845, 'text': 'The first is just building that system.', 'start': 966.084, 'duration': 1.761}, {'end': 975.61, 'text': "And so when we build that system, we have to kind of do a variety of things to ensure that we're building kind of the best possible system.", 'start': 968.626, 'duration': 6.984}], 'summary': 'Designing autonomous systems involves two phases: building the system and ensuring its quality.', 'duration': 30.198, 'max_score': 945.412, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M945412.jpg'}, {'end': 1085.664, 'src': 'embed', 'start': 1052.804, 'weight': 3, 'content': [{'end': 1059.967, 'text': 'But it took an additional eight years of verification and validation to build enough trust in that system to get the FAA stamp.', 'start': 1052.804, 'duration': 7.163}, {'end': 1062.609, 'text': 'And this is already when the system was safe enough.', 'start': 1060.608, 'duration': 2.001}, {'end': 1063.769, 'text': 'They just had to ensure it.', 'start': 1062.629, 'duration': 1.14}, {'end': 1070.033, 'text': 'you know, they had to build the trust in it in order to make sure everyone was confident that what they were deploying was going to actually be safe.', 'start': 1063.769, 'duration': 6.264}, {'end': 1079.319, 'text': 'And so both of those aspects, the designing of the system and the test and evaluation are both crucial pieces of eventually deploying a safe system.', 'start': 1070.333, 'duration': 8.986}, {'end': 1082.561, 'text': "And so, and we're going to get into this when we talk through the class.", 'start': 1080.4, 'duration': 2.161}, {'end': 1085.664, 'text': 'but can you talk a little bit more about the verification piece?', 'start': 1082.561, 'duration': 3.103}], 'summary': '8 years to verify and validate system for faa stamp, ensuring safety and trust in deployment.', 'duration': 32.86, 'max_score': 1052.804, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1052804.jpg'}], 'start': 814.761, 'title': 'Ai system deployment and safety', 'summary': 'Emphasizes equitable deployment of ai systems to mitigate biases and discusses the challenges and steps involved in designing safe ai systems, illustrated by the example of aircraft collision avoidance system development.', 'chapters': [{'end': 860.951, 'start': 814.761, 'title': 'Equitable deployment of ai systems', 'summary': 'Discusses the importance of ensuring the equitable deployment of ai systems to mitigate biases that may cause ai systems to behave differently based on gender or race, which stems from biases in the data sets created by humans.', 'duration': 46.19, 'highlights': ['The risk that an AI system will behave differently at different subgroups based on gender or race is a lack of robustness to various subgroups.', 'Mitigating biases in data sets is crucial for ensuring the equitable deployment of AI systems.', 'Biases in society show up in the data sets created by humans, contributing to the problem of AI systems behaving differently based on gender or race.']}, {'end': 1128.444, 'start': 861.871, 'title': 'Designing safe ai systems', 'summary': 'Discusses the challenges and steps involved in designing safe ai systems, emphasizing the importance of both building and verification phases, illustrated by the example of aircraft collision avoidance system development and validation.', 'duration': 266.573, 'highlights': ['The chapter emphasizes the importance of both building and verification phases in designing safe AI systems, illustrated by the example of aircraft collision avoidance system development and validation. Emphasis on both building and verification phases, example of aircraft collision avoidance system development, validation, and verification', 'The speaker explains the two broad phases involved in designing an autonomous system for high-stakes applications - building and verification. Explanation of two broad phases in designing an autonomous system', 'The example of aircraft collision avoidance system development demonstrates the extensive verification and validation process, requiring eight years to build trust and obtain FAA approval. Example of aircraft collision avoidance system development, eight years verification and validation process', 'The speaker discusses the need to ensure the safety and reliability of AI systems through data collection, use of advanced engineering methods, and understanding potential risks before deployment. Emphasis on safety and reliability, importance of data collection, use of advanced engineering methods, understanding potential risks', 'The importance of verification is highlighted, including formal verification and robust training, to ensure safety and demonstrate it to potential users. Emphasis on importance of verification, focus on formal verification and robust training']}], 'duration': 313.683, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M814761.jpg', 'highlights': ['Mitigating biases in data sets is crucial for ensuring the equitable deployment of AI systems.', 'The risk that an AI system will behave differently at different subgroups based on gender or race is a lack of robustness to various subgroups.', 'Biases in society show up in the data sets created by humans, contributing to the problem of AI systems behaving differently based on gender or race.', 'The example of aircraft collision avoidance system development demonstrates the extensive verification and validation process, requiring eight years to build trust and obtain FAA approval.', 'The speaker discusses the need to ensure the safety and reliability of AI systems through data collection, use of advanced engineering methods, and understanding potential risks before deployment.', 'The importance of verification is highlighted, including formal verification and robust training, to ensure safety and demonstrate it to potential users.', 'The chapter emphasizes the importance of both building and verification phases in designing safe AI systems, illustrated by the example of aircraft collision avoidance system development and validation.', 'The speaker explains the two broad phases involved in designing an autonomous system for high-stakes applications - building and verification.']}, {'end': 1439.379, 'segs': [{'end': 1154.1, 'src': 'embed', 'start': 1128.544, 'weight': 0, 'content': [{'end': 1136.093, 'text': "So for example, if we're designing a self-driving car, we want to make sure that that self-driving car is much, much safer than a human.", 'start': 1128.544, 'duration': 7.549}, {'end': 1140.655, 'text': 'So, for example, maybe we want it to be ten times safer than the average human driver.', 'start': 1136.893, 'duration': 3.762}, {'end': 1148.478, 'text': 'And what that means is that this will have basically a failure rate of something less than one in ten to the seven.', 'start': 1141.135, 'duration': 7.343}, {'end': 1154.1, 'text': 'So one in ten million chance of, say, causing a collision for every hour on the road.', 'start': 1148.518, 'duration': 5.582}], 'summary': 'Designing self-driving cars to be 10 times safer than human drivers, with a failure rate of less than 1 in 10 million per hour on the road.', 'duration': 25.556, 'max_score': 1128.544, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1128544.jpg'}, {'end': 1272.076, 'src': 'embed', 'start': 1232.807, 'weight': 1, 'content': [{'end': 1238.031, 'text': 'We think of AI as doing what it can to, say, avoid collisions or something like that.', 'start': 1232.807, 'duration': 5.224}, {'end': 1242.875, 'text': 'But this was using AI to actually cover the weaknesses of our automated vehicle.', 'start': 1238.051, 'duration': 4.824}, {'end': 1246.838, 'text': 'And when we do that, we can uncover a whole bunch of possible failure modes.', 'start': 1242.955, 'duration': 3.883}, {'end': 1251.602, 'text': 'And when we do, it basically allows us to understand what are the weaknesses of our system.', 'start': 1247.379, 'duration': 4.223}, {'end': 1254.424, 'text': 'Are we OK with the failure modes that we found?', 'start': 1251.842, 'duration': 2.582}, {'end': 1258.788, 'text': 'Or do we need to go back to the drawing board and fix some of these issues before we deploy it?', 'start': 1254.745, 'duration': 4.043}, {'end': 1261.49, 'text': "And that's great.", 'start': 1260.649, 'duration': 0.841}, {'end': 1264.611, 'text': "And so there's this kind of a question.", 'start': 1261.59, 'duration': 3.021}, {'end': 1265.452, 'text': "You've got rare events.", 'start': 1264.631, 'duration': 0.821}, {'end': 1272.076, 'text': "You're going to be testing something that's so safe that, in theory, the kinds of events that cause a failure are really, really rare.", 'start': 1265.552, 'duration': 6.524}], 'summary': 'Using ai to uncover weaknesses in automated vehicles and address failure modes before deployment.', 'duration': 39.269, 'max_score': 1232.807, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1232807.jpg'}, {'end': 1352.533, 'src': 'embed', 'start': 1326.916, 'weight': 4, 'content': [{'end': 1334.06, 'text': "So again I'm going to call it and this is not technical, because I'm not technical the kind of traditional systems you can reason, as you said,", 'start': 1326.916, 'duration': 7.144}, {'end': 1334.861, 'text': 'from first principles.', 'start': 1334.06, 'duration': 0.801}, {'end': 1339.384, 'text': 'We understand the physics well enough of this mechanical system that we can predict the failure points.', 'start': 1334.901, 'duration': 4.483}, {'end': 1346.849, 'text': "With an AI system, we can't reason about it based on first principles, so we need to test it in simulations.", 'start': 1339.804, 'duration': 7.045}, {'end': 1352.533, 'text': "Could we build on that? Because I know you've also, or Michael's lab has also done work around formal verification.", 'start': 1347.369, 'duration': 5.164}], 'summary': 'Understanding failure points in mechanical systems through physics, while ai systems require testing in simulations.', 'duration': 25.617, 'max_score': 1326.916, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1326916.jpg'}, {'end': 1391.245, 'src': 'embed', 'start': 1367.325, 'weight': 3, 'content': [{'end': 1375.733, 'text': "So what we're doing in formal verification is trying to construct a mathematical proof that the system has the safety behavior that we want it to.", 'start': 1367.325, 'duration': 8.408}, {'end': 1380.598, 'text': 'So for example, we might try to construct a proof that our vehicle will never collide with another vehicle.', 'start': 1375.773, 'duration': 4.825}, {'end': 1391.245, 'text': "Now, this may sound challenging, and it is, and oftentimes what's required to do these formal verification proofs is to make a bunch of assumptions.", 'start': 1381.238, 'duration': 10.007}], 'summary': "Formal verification aims to prove safety behaviors, like ensuring vehicles won't collide.", 'duration': 23.92, 'max_score': 1367.325, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1367325.jpg'}], 'start': 1128.544, 'title': 'Ensuring safety of self-driving cars', 'summary': 'Discusses the challenge of making self-driving cars ten times safer than humans, with a failure rate of less than one in ten million, and the use of ai to identify failure modes. it also covers the challenges in testing ai systems and the application of formal verification for safety behavior of autonomous vehicles.', 'chapters': [{'end': 1272.076, 'start': 1128.544, 'title': 'Enhancing safety of self-driving cars', 'summary': 'Discusses the challenge of ensuring a self-driving car is ten times safer than a human, with a failure rate of less than one in ten million, and the approach of using ai to uncover failure modes and weaknesses of the system.', 'duration': 143.532, 'highlights': ['Ensuring self-driving car safety The chapter discusses the challenge of ensuring a self-driving car is ten times safer than a human, with a failure rate of less than one in ten million.', 'Using AI to uncover failure modes The approach involves using AI to uncover failure modes and weaknesses of the system by manipulating the environment to cause the autonomous vehicle to fail.', 'Testing for rare events The chapter addresses the challenge of testing something so safe that the events causing a failure are extremely rare, requiring extensive testing and uncovering possible failure modes.']}, {'end': 1439.379, 'start': 1272.116, 'title': 'Testing ai systems for safety', 'summary': 'Discusses the challenges in testing ai systems, the need for alternative testing methods due to complex machine learning components, and the application of formal verification to construct mathematical proofs for safety behavior of autonomous vehicles.', 'duration': 167.263, 'highlights': ['Formal verification is used to construct mathematical proofs for safety behavior of systems, such as proving that a vehicle will never collide with another vehicle. Formal verification involves constructing mathematical proofs to ensure safety behavior, for example, proving that a vehicle will never collide with another vehicle.', 'Traditional systems can be reasoned about from first principles, unlike AI systems with complex machine learning components, requiring alternative testing methods. Traditional systems can be reasoned about from first principles, while AI systems with complex machine learning components require alternative testing methods due to the difficulty in reasoning about safety.', 'Testing AI systems using simulations is necessary as reasoning about them based on first principles is not feasible. Testing AI systems using simulations is necessary as reasoning about them based on first principles is not feasible due to the complex machine learning components.']}], 'duration': 310.835, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1128544.jpg', 'highlights': ['Ensuring self-driving car safety The chapter discusses the challenge of ensuring a self-driving car is ten times safer than a human, with a failure rate of less than one in ten million.', 'Using AI to uncover failure modes The approach involves using AI to uncover failure modes and weaknesses of the system by manipulating the environment to cause the autonomous vehicle to fail.', 'Testing for rare events The chapter addresses the challenge of testing something so safe that the events causing a failure are extremely rare, requiring extensive testing and uncovering possible failure modes.', 'Formal verification is used to construct mathematical proofs for safety behavior of systems, such as proving that a vehicle will never collide with another vehicle. Formal verification involves constructing mathematical proofs to ensure safety behavior, for example, proving that a vehicle will never collide with another vehicle.', 'Traditional systems can be reasoned about from first principles, unlike AI systems with complex machine learning components, requiring alternative testing methods. Traditional systems can be reasoned about from first principles, while AI systems with complex machine learning components require alternative testing methods due to the difficulty in reasoning about safety.', 'Testing AI systems using simulations is necessary as reasoning about them based on first principles is not feasible. Testing AI systems using simulations is necessary as reasoning about them based on first principles is not feasible due to the complex machine learning components.']}, {'end': 1851.287, 'segs': [{'end': 1577.773, 'src': 'embed', 'start': 1556.973, 'weight': 0, 'content': [{'end': 1566.657, 'text': "And I wanted to put those learnings into kind of one place where someone who's maybe taken some machine learning courses can go to get up to speed on kind of the latest kind of advancements,", 'start': 1556.973, 'duration': 9.684}, {'end': 1569.718, 'text': 'but also current issues around AI and machine learning.', 'start': 1566.657, 'duration': 3.061}, {'end': 1575.152, 'text': "And so to that end, we've outlined basically a five-day course.", 'start': 1570.509, 'duration': 4.643}, {'end': 1577.773, 'text': 'Each day will consist of about two hours of lectures.', 'start': 1575.212, 'duration': 2.561}], 'summary': 'Five-day course with 2-hour lectures to cover latest advancements and current issues in ai and machine learning.', 'duration': 20.8, 'max_score': 1556.973, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1556973.jpg'}, {'end': 1657.282, 'src': 'embed', 'start': 1612.294, 'weight': 1, 'content': [{'end': 1618.059, 'text': "will have similar performance even when the environment in which it's operating in changes ever so slightly.", 'start': 1612.294, 'duration': 5.765}, {'end': 1625.025, 'text': 'So, like I said, a robust model, say, for autonomous driving will do just as well in the morning as it does in the afternoon,', 'start': 1618.799, 'duration': 6.226}, {'end': 1628.347, 'text': "when the lighting conditions are different and people's behavior are slightly different.", 'start': 1625.025, 'duration': 3.322}, {'end': 1630.309, 'text': "That's what we mean by robustness.", 'start': 1628.387, 'duration': 1.922}, {'end': 1634.101, 'text': 'um great.', 'start': 1632.74, 'duration': 1.361}, {'end': 1639.766, 'text': 'so then, and the the next uh two days will be around model evaluation and verification.', 'start': 1634.101, 'duration': 5.665}, {'end': 1646.933, 'text': "so on that on the next day we'll be talking about um techniques for explaining, uh, these machine learning models.", 'start': 1639.766, 'duration': 7.167}, {'end': 1654.48, 'text': 'so, as i said, you know, often these are these very complex uh neural networks that, have you know, billions, potentially,', 'start': 1646.933, 'duration': 7.547}, {'end': 1657.282, 'text': 'of parameters that are used to process images.', 'start': 1654.48, 'duration': 2.802}], 'summary': 'Robust models maintain consistent performance despite changing environments, with a focus on model evaluation and explanation techniques for complex neural networks.', 'duration': 44.988, 'max_score': 1612.294, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1612294.jpg'}, {'end': 1733.146, 'src': 'embed', 'start': 1702.506, 'weight': 3, 'content': [{'end': 1703.846, 'text': 'that would be desirable.', 'start': 1702.506, 'duration': 1.34}, {'end': 1707.488, 'text': "We'll get to play with some libraries that people have developed to do this,", 'start': 1704.506, 'duration': 2.982}, {'end': 1710.969, 'text': 'so you can actually use it in real-world projects that you might be working on.', 'start': 1707.488, 'duration': 3.481}, {'end': 1717.946, 'text': "On day four, then we're going to move into a discussion on uncertainty quantification machine learning.", 'start': 1713.041, 'duration': 4.905}, {'end': 1719.587, 'text': "So I'll say a little bit about what I mean.", 'start': 1717.986, 'duration': 1.601}, {'end': 1720.888, 'text': 'So uncertainty.', 'start': 1719.988, 'duration': 0.9}, {'end': 1727.935, 'text': "quantification is basically you know what happens when your machine learning model, say, encounters a situation it's never seen before.", 'start': 1720.888, 'duration': 7.047}, {'end': 1733.146, 'text': 'Would you want, on the one hand, for it to just decide what it thinks is best and go forward full steam?', 'start': 1728.462, 'duration': 4.684}], 'summary': 'Practical application of libraries for machine learning in real-world projects, with focus on uncertainty quantification and handling of unseen situations.', 'duration': 30.64, 'max_score': 1702.506, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1702506.jpg'}, {'end': 1773.653, 'src': 'embed', 'start': 1751.303, 'weight': 4, 'content': [{'end': 1759.707, 'text': "And so there's a bunch of techniques for building this style of reasoning about uncertainty into the machine learning components or the AI systems that you build.", 'start': 1751.303, 'duration': 8.404}, {'end': 1762.128, 'text': 'So Anthony, just to pause you there.', 'start': 1760.787, 'duration': 1.341}, {'end': 1767.93, 'text': "So in certain quantification, it's the idea you're in a new or novel situation.", 'start': 1762.148, 'duration': 5.782}, {'end': 1773.653, 'text': "It's one recognizing, I suppose, is probably a problem, just recognizing it's a new situation.", 'start': 1768.951, 'duration': 4.702}], 'summary': 'Techniques for incorporating uncertainty into ai systems are crucial for navigating novel situations.', 'duration': 22.35, 'max_score': 1751.303, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1751303.jpg'}], 'start': 1439.539, 'title': 'Ai in safety-critical domains course', 'summary': 'Outlines a five-day course on ai in safety-critical domains, addressing issues in deploying machine learning systems, emphasizing the importance of building robust models, with each day consisting of about two hours of lectures.', 'chapters': [{'end': 1634.101, 'start': 1439.539, 'title': 'Ai in safety-critical domains course', 'summary': 'Outlines a five-day course on ai in safety-critical domains, addressing issues in deploying machine learning systems and the importance of building robust models, with a focus on techniques and examples to improve performance, with each day consisting of about two hours of lectures.', 'duration': 194.562, 'highlights': ['The course is a five-day program with each day consisting of about two hours of lectures, focusing on the use of AI in safety-critical domains and techniques to build robust machine learning models.', 'The importance of building robust machine learning models is emphasized to ensure similar performance even when the operating environment changes ever so slightly, as exemplified in the context of autonomous driving.', "The course aims to address issues encountered in deploying machine learning systems and AI systems for real-world applications, based on the speaker's years of research and experience in the field."]}, {'end': 1851.287, 'start': 1634.101, 'title': 'Machine learning model evaluation', 'summary': 'Covers techniques for explaining machine learning models, including methods for failure identification, formal verification, and uncertainty quantification, with a focus on understanding complex neural networks and building safe systems.', 'duration': 217.186, 'highlights': ['The chapter covers techniques for explaining machine learning models, including methods for failure identification, formal verification, and uncertainty quantification. This includes discussing ways to peer inside complex neural networks, using adversarial techniques for stress testing systems, and methods for formal verification to ensure resilience to adversarial attacks and other desirable properties.', 'Uncertainty quantification is discussed, focusing on the identification of novel situations and the appropriate response of machine learning models. The discussion includes techniques for building reasoning about uncertainty into machine learning components or AI systems, and the challenge of identifying novel situations where machine learning models may be extremely confident despite being wrong.', 'The chapter concludes with a discussion on tying all the topics together, emphasizing the importance of understanding and implementing the discussed techniques. The final day aims to integrate the discussed techniques, providing a comprehensive understanding of the methods for explaining machine learning models, failure identification, formal verification, and uncertainty quantification.']}], 'duration': 411.748, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1439539.jpg', 'highlights': ['The course is a five-day program with each day consisting of about two hours of lectures, focusing on the use of AI in safety-critical domains and techniques to build robust machine learning models.', 'The importance of building robust machine learning models is emphasized to ensure similar performance even when the operating environment changes ever so slightly, as exemplified in the context of autonomous driving.', "The course aims to address issues encountered in deploying machine learning systems and AI systems for real-world applications, based on the speaker's years of research and experience in the field.", 'The chapter covers techniques for explaining machine learning models, including methods for failure identification, formal verification, and uncertainty quantification.', 'Uncertainty quantification is discussed, focusing on the identification of novel situations and the appropriate response of machine learning models.', 'The chapter concludes with a discussion on tying all the topics together, emphasizing the importance of understanding and implementing the discussed techniques.']}, {'end': 2608.694, 'segs': [{'end': 1911.586, 'src': 'embed', 'start': 1872.77, 'weight': 0, 'content': [{'end': 1880.992, 'text': "And then there's been some discussions here in the US and thinking about the ways in which AI might be regulated and how some of the techniques I'm describing perhaps will fit into that regulation.", 'start': 1872.77, 'duration': 8.222}, {'end': 1887.808, 'text': "Because you might be required in the future to do many of the things I'm talking about to ensure that people stay unharmed for the systems that we build.", 'start': 1881.112, 'duration': 6.696}, {'end': 1891.479, 'text': 'Thanks, Anthony.', 'start': 1890.299, 'duration': 1.18}, {'end': 1893.48, 'text': "It's a really interesting course.", 'start': 1892.02, 'duration': 1.46}, {'end': 1896.421, 'text': "And again, it's focusing on kind of these autonomous systems.", 'start': 1893.64, 'duration': 2.781}, {'end': 1906.245, 'text': "I guess one thing is, do you see, so we're talking in the primary use case we're talking about is autonomous vehicles like cars or aviation.", 'start': 1896.461, 'duration': 9.784}, {'end': 1910.246, 'text': 'We got a lot of questions about working in warehouses, maybe industrial machinery.', 'start': 1906.305, 'duration': 3.941}, {'end': 1911.586, 'text': 'Does this still apply??', 'start': 1910.686, 'duration': 0.9}], 'summary': 'Discussions on regulating ai for ensuring system safety in autonomous vehicles and industrial machinery.', 'duration': 38.816, 'max_score': 1872.77, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1872770.jpg'}, {'end': 1987.515, 'src': 'embed', 'start': 1962.984, 'weight': 1, 'content': [{'end': 1970.969, 'text': 'In a much more controlled environment, you do have the luxury of perhaps controlling the environment in which the AI system is operating in.', 'start': 1962.984, 'duration': 7.985}, {'end': 1974.31, 'text': "But it's never so straightforward as that.", 'start': 1972.529, 'duration': 1.781}, {'end': 1975.87, 'text': 'Things will always change.', 'start': 1974.47, 'duration': 1.4}, {'end': 1987.515, 'text': "And you want your AI system say that's controlling a robotic arm to be somewhat resilient to the small changes that occur on the factory line or changes to the environment.", 'start': 1976.01, 'duration': 11.505}], 'summary': 'Ai systems need to adapt to changes in the controlled environment to be effective.', 'duration': 24.531, 'max_score': 1962.984, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1962984.jpg'}, {'end': 2121.366, 'src': 'embed', 'start': 2089.978, 'weight': 5, 'content': [{'end': 2094.54, 'text': "we'll look at a bunch of examples where you have, uh, these what we call distribution shifts.", 'start': 2089.978, 'duration': 4.562}, {'end': 2101.501, 'text': 'so this is where things in the environment change ever so slightly and the machine learning models performance drops, uh, quite a bit.', 'start': 2094.54, 'duration': 6.961}, {'end': 2103.082, 'text': "and so we'll look at a bunch of examples of that.", 'start': 2101.501, 'duration': 1.581}, {'end': 2103.502, 'text': "you'll see.", 'start': 2103.082, 'duration': 0.42}, {'end': 2104.062, 'text': 'you know.', 'start': 2103.502, 'duration': 0.56}, {'end': 2108.803, 'text': 'for example, uh, changes in lighting condition or color scheme, like things like this.', 'start': 2104.062, 'duration': 4.741}, {'end': 2113.984, 'text': 'you can build a little bit of an intuition around what types of factors may influence certain types of machine learning models.', 'start': 2108.803, 'duration': 5.181}, {'end': 2121.366, 'text': 'And then the last piece of this is you absolutely need to monitor your systems as you deploy them.', 'start': 2114.904, 'duration': 6.462}], 'summary': 'Distribution shifts impact ml model performance, requiring monitoring post-deployment.', 'duration': 31.388, 'max_score': 2089.978, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M2089978.jpg'}, {'end': 2155.029, 'src': 'embed', 'start': 2130.829, 'weight': 6, 'content': [{'end': 2137.911, 'text': 'So as we talked about, making sure the system is aware when things change and can potentially alert, say, a user.', 'start': 2130.829, 'duration': 7.082}, {'end': 2143.977, 'text': 'And then also having systems that are monitoring the performance of these systems over time.', 'start': 2138.491, 'duration': 5.486}, {'end': 2148.102, 'text': 'And if you see the performance dropping, for example, then you know something has shifted.', 'start': 2143.997, 'duration': 4.105}, {'end': 2151.407, 'text': "Maybe it's time to retrain your model or intervene in some way.", 'start': 2148.122, 'duration': 3.285}, {'end': 2153.207, 'text': "That's great.", 'start': 2152.827, 'duration': 0.38}, {'end': 2155.029, 'text': 'So maybe even revisiting.', 'start': 2153.227, 'duration': 1.802}], 'summary': 'Implement system monitoring to alert users of performance changes and intervene if necessary.', 'duration': 24.2, 'max_score': 2130.829, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M2130829.jpg'}, {'end': 2217.433, 'src': 'embed', 'start': 2190.53, 'weight': 8, 'content': [{'end': 2201.986, 'text': 'But another thing you can do is just every so often, every week or something, re-evaluate your say, model on a new set of data.', 'start': 2190.53, 'duration': 11.456}, {'end': 2205.328, 'text': "So it's sort of like just a repeated validation process.", 'start': 2202.266, 'duration': 3.062}, {'end': 2213.511, 'text': 'So doing the same thing you did before you deployed it, just on an iterated base to ensure that things are staying at a high level of performance.', 'start': 2205.348, 'duration': 8.163}, {'end': 2217.433, 'text': 'So this would be, so in other words, the AI system is running.', 'start': 2214.291, 'duration': 3.142}], 'summary': 'Regularly re-evaluate model with new data to maintain high performance.', 'duration': 26.903, 'max_score': 2190.53, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M2190530.jpg'}, {'end': 2300.305, 'src': 'embed', 'start': 2272.994, 'weight': 7, 'content': [{'end': 2278.636, 'text': "And I've talked a lot about these physical changes in the environment, like changing lighting conditions and things like that.", 'start': 2272.994, 'duration': 5.642}, {'end': 2282.778, 'text': 'But distribution shift can be something that happens.', 'start': 2278.696, 'duration': 4.082}, {'end': 2291.381, 'text': "say like imagine, for example, that we have an AI system that's helping you diagnose diseases at a hospital or something.", 'start': 2282.778, 'duration': 8.603}, {'end': 2296.724, 'text': 'What would you think would happen to that system at the beginning of the COVID-19 pandemic, for example,', 'start': 2292.081, 'duration': 4.643}, {'end': 2300.305, 'text': 'where a new virus altogether has shown up on the scene?', 'start': 2296.724, 'duration': 3.581}], 'summary': 'Physical changes in environment can affect ai systems, such as in diagnosing diseases during the covid-19 pandemic.', 'duration': 27.311, 'max_score': 2272.994, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M2272994.jpg'}, {'end': 2395.575, 'src': 'embed', 'start': 2370.618, 'weight': 10, 'content': [{'end': 2377.823, 'text': "And I think something we haven't even talked about today, though I'm sure everyone has encountered it recently, is things like chat GPT.", 'start': 2370.618, 'duration': 7.205}, {'end': 2385.849, 'text': 'So these generative AI systems that are able to produce content, text or images or videos and things like that.', 'start': 2377.843, 'duration': 8.006}, {'end': 2395.575, 'text': 'And these types of systems, when deployed without much oversight, can lead to very strange and bad things happening.', 'start': 2386.649, 'duration': 8.926}], 'summary': 'Generative ai, like chat gpt, can lead to negative outcomes without oversight.', 'duration': 24.957, 'max_score': 2370.618, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M2370618.jpg'}, {'end': 2554.707, 'src': 'embed', 'start': 2528.064, 'weight': 12, 'content': [{'end': 2532.687, 'text': "but it does get a lot messier when we're talking about these really large scale generative AI systems.", 'start': 2528.064, 'duration': 4.623}, {'end': 2536.155, 'text': 'How can you know?', 'start': 2533.853, 'duration': 2.302}, {'end': 2544.58, 'text': 'do you foresee these generative AI systems being applied in domains like autonomous vehicles, or does that itself pose too great a safety risk?', 'start': 2536.155, 'duration': 8.425}, {'end': 2545.921, 'text': "I'm sorry, now we're going a little bit off.", 'start': 2544.62, 'duration': 1.301}, {'end': 2546.802, 'text': "No, it's fine.", 'start': 2546.081, 'duration': 0.721}, {'end': 2554.707, 'text': "I certainly hope not that they are not, because it's you know, I've seen so many examples of failures of reasoning,", 'start': 2547.903, 'duration': 6.804}], 'summary': 'Challenges of large-scale generative ai in safety-critical domains discussed.', 'duration': 26.643, 'max_score': 2528.064, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M2528064.jpg'}], 'start': 1853.724, 'title': 'Ai in autonomous systems', 'summary': 'Discusses ai regulation, challenges in deployment, and the future of autonomous systems, emphasizing the need for robust and resilient ai, impact of distribution shifts, and potential risks associated with generative ai.', 'chapters': [{'end': 2070.572, 'start': 1853.724, 'title': 'Ai regulation in autonomous systems', 'summary': 'Discusses the upcoming regulation around ai, focusing on its application in autonomous vehicles and industrial machinery, and emphasizes the need for robust and resilient ai systems to ensure safety and reliability in various environments.', 'duration': 216.848, 'highlights': ['The upcoming regulation around AI, particularly in the EU and the US, is discussed, emphasizing the need for techniques to ensure safety in the systems. The chapter highlights the impending AI regulation in the EU and the US, stressing the importance of techniques to ensure safety in autonomous systems.', 'The application of AI in autonomous vehicles and industrial machinery is explored, emphasizing the need for robust and resilient systems in controlled and uncontrolled environments. The discussion emphasizes the application of AI in autonomous vehicles and industrial machinery, highlighting the importance of robust systems in both controlled and uncontrolled environments.', 'Considerations for adapting AI systems to changing environmental factors are discussed, including resilience to small changes and the potential impact of lighting conditions and seasonal changes. The chapter addresses the need for AI systems to adapt to changing environmental factors, such as resilience to small changes and the impact of lighting conditions and seasonal changes.', 'The need for robust AI systems in various applications is emphasized, highlighting the importance of resilience and robustness in both warehouse and driving domains. The discussion emphasizes the importance of robust AI systems in various applications, highlighting the need for resilience and robustness in both warehouse and driving domains.', 'The challenge of reasoning about environmental parameters in AI systems is acknowledged, emphasizing the complexity and the need for domain experts to address such challenges. The chapter acknowledges the challenge of reasoning about environmental parameters in AI systems, emphasizing the complexity and the importance of domain experts in addressing such challenges.']}, {'end': 2329.934, 'start': 2070.572, 'title': 'Challenges in ai deployment', 'summary': 'Covers the challenges of deploying ai in warehouse environments, emphasizing the impact of distribution shifts and the importance of monitoring and re-evaluating systems to maintain performance and adapt to novel situations.', 'duration': 259.362, 'highlights': ['The impact of distribution shifts on machine learning models is discussed, highlighting how even slight changes in the environment can significantly affect performance.', 'The importance of monitoring and quantifying uncertainty in deployed systems is emphasized, with the need to detect performance drops and potential shifts, triggering retraining or intervention.', "The concept of distributional shift is explored, illustrating how novel situations, such as the emergence of new diseases, can drastically impact AI systems' performance and diagnosis capabilities.", 'The need for periodic re-evaluation of AI models on new data to ensure sustained high performance is emphasized, especially in dynamic environments where changes can occur over time.', 'The significance of monitoring system performance and being alert to deviations from normal operations, such as changes in sensors or environmental factors, is highlighted as crucial in AI deployment.']}, {'end': 2608.694, 'start': 2331.056, 'title': 'Future of autonomous systems', 'summary': 'Discusses the hopeful potential of ai systems in improving efficiency at a larger scale while highlighting the risks associated with the deployment of generative ai systems, such as chat gpt, leading to toxic behavior, discrimination, and misinformation at scale.', 'duration': 277.638, 'highlights': ['Generative AI systems like chat GPT can lead to toxic behavior, discrimination, and misinformation at scale, posing significant risks when deployed without oversight.', 'The hopeful potential of AI systems lies in their ability to perform tasks more quickly, efficiently, and at a larger scale, bringing great resources to people.', 'The concerns revolve around the widespread and rapid deployment of generative AI systems like chat GPT, with uncertainties about their influence on the information ecosystem and potential societal harms.', 'Principles applicable to systems interacting with the physical environment may also apply to chat GPT, but quantifying the societal harms and testing for emotional harm is a more challenging domain.', 'There are concerns about the potential application of generative AI systems in domains like autonomous vehicles due to failures of reasoning and logic, posing safety risks that need to be addressed before their deployment.']}], 'duration': 754.97, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/IDvdyv_49-M/pics/IDvdyv_49-M1853724.jpg', 'highlights': ['The impending AI regulation in the EU and the US, stressing the importance of techniques to ensure safety in autonomous systems.', 'The application of AI in autonomous vehicles and industrial machinery, highlighting the importance of robust systems in both controlled and uncontrolled environments.', 'The need for AI systems to adapt to changing environmental factors, such as resilience to small changes and the impact of lighting conditions and seasonal changes.', 'The importance of robust AI systems in various applications, emphasizing the need for resilience and robustness in both warehouse and driving domains.', 'The challenge of reasoning about environmental parameters in AI systems, emphasizing the complexity and the importance of domain experts in addressing such challenges.', 'The impact of distribution shifts on machine learning models, highlighting how even slight changes in the environment can significantly affect performance.', 'The importance of monitoring and quantifying uncertainty in deployed systems, with the need to detect performance drops and potential shifts, triggering retraining or intervention.', "The concept of distributional shift is explored, illustrating how novel situations, such as the emergence of new diseases, can drastically impact AI systems' performance and diagnosis capabilities.", 'The need for periodic re-evaluation of AI models on new data to ensure sustained high performance, especially in dynamic environments where changes can occur over time.', 'The significance of monitoring system performance and being alert to deviations from normal operations, such as changes in sensors or environmental factors, is highlighted as crucial in AI deployment.', 'Generative AI systems like chat GPT can lead to toxic behavior, discrimination, and misinformation at scale, posing significant risks when deployed without oversight.', 'The concerns revolve around the widespread and rapid deployment of generative AI systems like chat GPT, with uncertainties about their influence on the information ecosystem and potential societal harms.', 'There are concerns about the potential application of generative AI systems in domains like autonomous vehicles due to failures of reasoning and logic, posing safety risks that need to be addressed before their deployment.']}], 'highlights': ["AI systems can potentially perform much better than humans at tasks like healthcare, aircraft operation, and driving, significantly improving safety and providing broader access to healthcare. (e.g., AI system as good as the world's best doctor could provide healthcare access to more people)", 'Mitigating biases in data sets is crucial for ensuring the equitable deployment of AI systems.', 'Ensuring self-driving car safety The chapter discusses the challenge of ensuring a self-driving car is ten times safer than a human, with a failure rate of less than one in ten million.', "The webinar covers dr. corso's role at stanford, ai in transportation and safety, risks of ai and adversarial attacks, equitable deployment of ai systems, safety challenges in self-driving cars, a course on ai in safety-critical domains, and the future of autonomous systems, highlighting the need for robust and resilient ai in safety-critical applications.", 'The impending AI regulation in the EU and the US, stressing the importance of techniques to ensure safety in autonomous systems.', 'The application of AI in autonomous vehicles and industrial machinery, highlighting the importance of robust systems in both controlled and uncontrolled environments.', 'The need for AI systems to adapt to changing environmental factors, such as resilience to small changes and the impact of lighting conditions and seasonal changes.', 'The importance of robust AI systems in various applications, emphasizing the need for resilience and robustness in both warehouse and driving domains.', 'The challenge of reasoning about environmental parameters in AI systems, emphasizing the complexity and the importance of domain experts in addressing such challenges.', 'The impact of distribution shifts on machine learning models, highlighting how even slight changes in the environment can significantly affect performance.', 'The importance of monitoring and quantifying uncertainty in deployed systems, with the need to detect performance drops and potential shifts, triggering retraining or intervention.', "The concept of distributional shift is explored, illustrating how novel situations, such as the emergence of new diseases, can drastically impact AI systems' performance and diagnosis capabilities.", 'The need for periodic re-evaluation of AI models on new data to ensure sustained high performance, especially in dynamic environments where changes can occur over time.', 'The significance of monitoring system performance and being alert to deviations from normal operations, such as changes in sensors or environmental factors, is highlighted as crucial in AI deployment.', 'Generative AI systems like chat GPT can lead to toxic behavior, discrimination, and misinformation at scale, posing significant risks when deployed without oversight.', 'The concerns revolve around the widespread and rapid deployment of generative AI systems like chat GPT, with uncertainties about their influence on the information ecosystem and potential societal harms.', 'There are concerns about the potential application of generative AI systems in domains like autonomous vehicles due to failures of reasoning and logic, posing safety risks that need to be addressed before their deployment.']}