title
Lecture 5: Conditioning Continued, Law of Total Probability | Statistics 110

description
We continue further with conditional probability, and discuss the law of total probability, the so-called prosecutor's fallacy, a disease testing example, and the crucial distinction between independence and conditional independence.

detail
{'title': 'Lecture 5: Conditioning Continued, Law of Total Probability | Statistics 110', 'heatmap': [{'end': 842.188, 'start': 717.573, 'weight': 0.764}, {'end': 2674.338, 'start': 2638.604, 'weight': 0.724}], 'summary': "Covers conditional probability, problem-solving strategies, and real-world applications in medicine and law, including the counterintuitive nature of test accuracy, the prosecutor's fallacy, and conditional independence, with examples and practical insights.", 'chapters': [{'end': 187.892, 'segs': [{'end': 91.57, 'src': 'embed', 'start': 0.669, 'weight': 0, 'content': [{'end': 12.238, 'text': "Okay, so last time, we proved a lot of theorems last time, right? Bayes' rule, at least n factorial plus 3 theorems or something like that for any n.", 'start': 0.669, 'duration': 11.569}, {'end': 19.063, 'text': 'So that was a very productive day, and I wanna continue with conditional probability, thinking conditionally.', 'start': 12.238, 'duration': 6.825}, {'end': 27.049, 'text': "We did Bayes' rule, but I wanna do some examples of conditional probability and some more stuff on conditional probability.", 'start': 19.123, 'duration': 7.926}, {'end': 34.954, 'text': "Basically, the topic for today is not just probability, but it's thinking.", 'start': 29.591, 'duration': 5.363}, {'end': 43.06, 'text': 'So probability.', 'start': 40.258, 'duration': 2.802}, {'end': 48.908, 'text': 'is how to think about uncertainty and randomness.', 'start': 44.606, 'duration': 4.302}, {'end': 50.49, 'text': "That's the topic for this entire course.", 'start': 48.949, 'duration': 1.541}, {'end': 52.411, 'text': 'So this is not just a statistics course.', 'start': 50.81, 'duration': 1.601}, {'end': 54.092, 'text': 'This is a thinking course.', 'start': 52.431, 'duration': 1.661}, {'end': 58.355, 'text': 'And the math we were doing last time, the math was extremely easy.', 'start': 55.233, 'duration': 3.122}, {'end': 62.918, 'text': "I multiplied both sides by something, and then there's our theorem.", 'start': 58.455, 'duration': 4.463}, {'end': 70.943, 'text': 'So it looked really easy, but how to actually think about it and how to apply it is not always easy.', 'start': 63.839, 'duration': 7.104}, {'end': 72.624, 'text': "In fact, it's often subtle.", 'start': 71.023, 'duration': 1.601}, {'end': 76.545, 'text': 'So I want to do some examples and a few more theorems along those lines.', 'start': 72.724, 'duration': 3.821}, {'end': 86.929, 'text': "So I like to say thinking conditionally, that's one of the biggest themes in this whole course.", 'start': 78.986, 'duration': 7.943}, {'end': 91.57, 'text': 'Using conditional probability, conditional thinking is a condition for thinking.', 'start': 87.049, 'duration': 4.521}], 'summary': 'Focus on conditional probability and thinking; aim for conditional thinking in course.', 'duration': 90.901, 'max_score': 0.669, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8669.jpg'}, {'end': 158.992, 'src': 'embed', 'start': 129.275, 'weight': 3, 'content': [{'end': 139.11, 'text': 'Now, one strategy that we already talked a little bit about is to try simple and extreme cases.', 'start': 129.275, 'duration': 9.835}, {'end': 144.058, 'text': 'That is extremely useful in a wide variety of problems.', 'start': 141.314, 'duration': 2.744}, {'end': 158.992, 'text': "So I did my undergrad at Caltech, and at Caltech everyone's hero is Richard Feynman, who was one of the greatest physicists of the 20th century.", 'start': 151.169, 'duration': 7.823}], 'summary': 'Strategy of using simple and extreme cases is useful for a wide variety of problems.', 'duration': 29.717, 'max_score': 129.275, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8129275.jpg'}], 'start': 0.669, 'title': 'Conditional probability and problem-solving strategies', 'summary': "Emphasizes conditional thinking, problem-solving strategies, and the ineffectiveness of the feynman algorithm, building on theorems and bayes' rule from the previous session.", 'chapters': [{'end': 127.394, 'start': 0.669, 'title': 'Conditional probability and thinking', 'summary': "Discusses conditional probability, emphasizing the importance of conditional thinking and problem-solving strategies, following the previous session's focus on theorems and bayes' rule.", 'duration': 126.725, 'highlights': ["The topic for today is not just probability, but it's thinking. So probability is how to think about uncertainty and randomness, emphasizing the course's focus on thinking and problem-solving strategies.", 'Thinking conditionally is one of the biggest themes in this whole course, highlighting the significance of conditional probability and its application in problem-solving.', "The math we were doing last time, the math was extremely easy. I multiplied both sides by something, and then there's our theorem, indicating the simplicity of the mathematical operations but the complexity in applying and understanding them.", "The chapter discusses conditional probability, emphasizing the importance of conditional thinking and problem-solving strategies, following the previous session's focus on theorems and Bayes' rule."]}, {'end': 187.892, 'start': 129.275, 'title': 'Problem-solving strategies', 'summary': 'Discusses the usefulness of trying simple and extreme cases in problem-solving, highlighting the feynman algorithm as an ineffective general problem-solving method.', 'duration': 58.617, 'highlights': ['Trying simple and extreme cases is extremely useful in a wide variety of problems.', 'The Feynman algorithm, involving writing down the problem, thinking hard about it, and writing down the solution, was considered ineffective for most people.']}], 'duration': 187.223, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8669.jpg', 'highlights': ["Emphasizes conditional thinking, problem-solving strategies, and the ineffectiveness of the Feynman algorithm, building on theorems and Bayes' rule from the previous session.", 'Thinking conditionally is one of the biggest themes in this whole course, highlighting the significance of conditional probability and its application in problem-solving.', "The chapter discusses conditional probability, emphasizing the importance of conditional thinking and problem-solving strategies, following the previous session's focus on theorems and Bayes' rule.", 'Trying simple and extreme cases is extremely useful in a wide variety of problems.', "The math we were doing last time, the math was extremely easy. I multiplied both sides by something, and then there's our theorem, indicating the simplicity of the mathematical operations but the complexity in applying and understanding them."]}, {'end': 504.969, 'segs': [{'end': 226.368, 'src': 'embed', 'start': 188.032, 'weight': 0, 'content': [{'end': 189.492, 'text': 'So this is one strategy.', 'start': 188.032, 'duration': 1.46}, {'end': 193.493, 'text': "And this is a strategy we'll be using over and over again in the course.", 'start': 190.392, 'duration': 3.101}, {'end': 197.414, 'text': "A second strategy we'll be using over and over again.", 'start': 193.753, 'duration': 3.661}, {'end': 202.056, 'text': "that's useful in statistics, but it's useful in computer science, it's useful in math, it's useful in econ.", 'start': 197.414, 'duration': 4.642}, {'end': 205.937, 'text': 'useful all over the place is to try to break the problem up into simpler pieces.', 'start': 202.056, 'duration': 3.881}, {'end': 211.86, 'text': 'If you have a problem that seems too difficult and complicated, try to decompose it into smaller pieces.', 'start': 206.457, 'duration': 5.403}, {'end': 219.244, 'text': "Try to solve the small, it's recursive, right? If the smaller pieces are still too difficult, then break up the smaller pieces into simpler problems.", 'start': 212.56, 'duration': 6.684}, {'end': 226.368, 'text': 'So you have more problems, but each problem is easier and hopefully eventually you reach a point where you can solve each of those problems,', 'start': 219.804, 'duration': 6.564}], 'summary': 'Using the strategy of breaking problems into simpler pieces is useful in various fields like statistics, computer science, math, and econ.', 'duration': 38.336, 'max_score': 188.032, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8188032.jpg'}, {'end': 287.769, 'src': 'embed', 'start': 256.259, 'weight': 2, 'content': [{'end': 261.404, 'text': 'And suppose we wanna find the probability of B, which is some blob, B for blob.', 'start': 256.259, 'duration': 5.145}, {'end': 271.144, 'text': 'So suppose that our problem, this is a very general strategy, but suppose that our problem is still a generic problem, but less generic than this.', 'start': 263.362, 'duration': 7.782}, {'end': 279.907, 'text': "We have this complicated looking blob B, and we want to find the probability of B, but we don't know how to do it because it's this complicated blob.", 'start': 271.804, 'duration': 8.103}, {'end': 287.769, 'text': 'So instead of giving up, what we would do is break B up into pieces, find the probability of each piece, and add them up.', 'start': 280.827, 'duration': 6.942}], 'summary': 'Strategy: break complex problem into pieces, find their probabilities, and sum them up.', 'duration': 31.51, 'max_score': 256.259, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8256259.jpg'}, {'end': 339.457, 'src': 'embed', 'start': 308.307, 'weight': 3, 'content': [{'end': 311.769, 'text': "So we're gonna let A1 through An be a partition of S.", 'start': 308.307, 'duration': 3.462}, {'end': 324.772, 'text': 'The word partition just means that these sets A, which are these rectangles, are disjoint and their union is all of S.', 'start': 316.229, 'duration': 8.543}, {'end': 328.353, 'text': 'So we are just chopping up the space into disjoint pieces.', 'start': 324.772, 'duration': 3.581}, {'end': 329.714, 'text': "They don't have to look like rectangles.", 'start': 328.393, 'duration': 1.321}, {'end': 334.535, 'text': 'Chop it up however you want, as long as those pieces are disjoint and their union is the whole space.', 'start': 329.794, 'duration': 4.741}, {'end': 339.457, 'text': "That's called a partition of S.", 'start': 334.555, 'duration': 4.902}], 'summary': 'A partition of s is a disjoint set of rectangles that cover the whole space.', 'duration': 31.15, 'max_score': 308.307, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8308307.jpg'}, {'end': 457.513, 'src': 'embed', 'start': 419.783, 'weight': 4, 'content': [{'end': 423.185, 'text': "That's why I said we had n factorial theorems, because you could do it in any order.", 'start': 419.783, 'duration': 3.402}, {'end': 428.071, 'text': 'blah, blah, blah, do that for all the pieces.', 'start': 425.93, 'duration': 2.141}, {'end': 435.596, 'text': 'B given a n, p of a n.', 'start': 428.812, 'duration': 6.784}, {'end': 441.12, 'text': 'And this equation here is called the law of total probability.', 'start': 435.596, 'duration': 5.524}, {'end': 452.132, 'text': "That's the name for it, but But I prefer to just think of it as breaking up a problem into simpler pieces.", 'start': 442.141, 'duration': 9.991}, {'end': 456.153, 'text': 'So the proof is already just written here.', 'start': 453.192, 'duration': 2.961}, {'end': 457.513, 'text': "It's immediate.", 'start': 456.813, 'duration': 0.7}], 'summary': 'The law of total probability simplifies problems by breaking them into pieces.', 'duration': 37.73, 'max_score': 419.783, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8419783.jpg'}], 'start': 188.032, 'title': 'Problem-solving strategies', 'summary': 'Introduces problem-solving strategies emphasizing breaking down complex problems into simpler pieces and solving them recursively, applicable in statistics, computer science, math, and economics, and discusses breaking down complex probability problems using examples.', 'chapters': [{'end': 226.368, 'start': 188.032, 'title': 'Problem-solving strategies', 'summary': 'Introduces problem-solving strategies, emphasizing the importance of breaking down complex problems into simpler pieces and solving them recursively, applicable in various fields such as statistics, computer science, math, and economics.', 'duration': 38.336, 'highlights': ['The importance of breaking down complex problems into simpler pieces and solving them recursively is emphasized, applicable in various fields such as statistics, computer science, math, and economics.', 'The strategy of decomposing problems into smaller pieces and solving them individually is highlighted as a useful approach in problem-solving.']}, {'end': 504.969, 'start': 226.368, 'title': 'Breaking down problems for probability', 'summary': 'Discusses the strategy of breaking a complex probability problem into simpler pieces, using the example of finding the probability of a complicated blob b by breaking it up into disjoint pieces and then adding them up.', 'duration': 278.601, 'highlights': ['The strategy of breaking a complex probability problem into simpler pieces is illustrated using the example of finding the probability of a complicated blob B by breaking it up into disjoint pieces and then adding them up.', 'The concept of a partition of S is explained, where sets A, representing the disjoint pieces, form a union that covers the entire space.', "The law of total probability is introduced as a way of breaking up a problem into simpler pieces, with the proof being immediate and dependent on the chosen partition's effectiveness.", 'The effectiveness of breaking up a problem into simpler pieces depends on the quality of the chosen partition, with statistics being described as both a science and an art that requires practice.']}], 'duration': 316.937, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8188032.jpg', 'highlights': ['The importance of breaking down complex problems into simpler pieces and solving them recursively is emphasized, applicable in various fields such as statistics, computer science, math, and economics.', 'The strategy of decomposing problems into smaller pieces and solving them individually is highlighted as a useful approach in problem-solving.', 'The strategy of breaking a complex probability problem into simpler pieces is illustrated using the example of finding the probability of a complicated blob B by breaking it up into disjoint pieces and then adding them up.', 'The concept of a partition of S is explained, where sets A, representing the disjoint pieces, form a union that covers the entire space.', "The law of total probability is introduced as a way of breaking up a problem into simpler pieces, with the proof being immediate and dependent on the chosen partition's effectiveness."]}, {'end': 1240.212, 'segs': [{'end': 594.66, 'src': 'embed', 'start': 505.349, 'weight': 0, 'content': [{'end': 507.09, 'text': 'You just need experience with that.', 'start': 505.349, 'duration': 1.741}, {'end': 515.073, 'text': "The more problems you do, then the better you'll get at guessing what would be a useful partition and what would be a useless partition.", 'start': 507.11, 'duration': 7.963}, {'end': 520.653, 'text': "So that's just the general idea of why it's conditional.", 'start': 517.506, 'duration': 3.147}, {'end': 525.403, 'text': "Basically, there's two main reasons conditional probability is very important.", 'start': 521.575, 'duration': 3.828}, {'end': 529.505, 'text': "One is that it's important in its own right right?", 'start': 526.943, 'duration': 2.562}, {'end': 535.668, 'text': 'Because, like I was talking about last time, conditional probabilities just says if we get some evidence,', 'start': 529.545, 'duration': 6.123}, {'end': 537.909, 'text': 'how do we update our probabilities based on the evidence?', 'start': 535.668, 'duration': 2.241}, {'end': 539.73, 'text': "So that's just a very general, important problem.", 'start': 537.929, 'duration': 1.801}, {'end': 545.894, 'text': "The second reason it's extremely important is that even if we wanted an unconditional probability like here,", 'start': 540.231, 'duration': 5.663}, {'end': 549.776, 'text': 'p of b is unconditional still a lot of times.', 'start': 545.894, 'duration': 3.882}, {'end': 553.178, 'text': 'we need to use conditional probability to break it up into simpler pieces.', 'start': 549.776, 'duration': 3.402}, {'end': 554.679, 'text': 'All right.', 'start': 554.419, 'duration': 0.26}, {'end': 558.044, 'text': "Let's do some examples.", 'start': 557.023, 'duration': 1.021}, {'end': 565.726, 'text': "All right, I'll start with one that seems really simple, but actually is kind of surprising, I think.", 'start': 561.385, 'duration': 4.341}, {'end': 576.81, 'text': 'So suppose we have two random cards just from a standard deck.', 'start': 570.288, 'duration': 6.522}, {'end': 583.793, 'text': 'So we get get random two card hand.', 'start': 579.031, 'duration': 4.762}, {'end': 594.66, 'text': 'So random two cards out of a 52-card deck from standard deck.', 'start': 587.018, 'duration': 7.642}], 'summary': 'Conditional probability is important for updating probabilities based on evidence and breaking up unconditional probabilities into simpler pieces.', 'duration': 89.311, 'max_score': 505.349, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8505349.jpg'}, {'end': 842.188, 'src': 'heatmap', 'start': 717.573, 'weight': 0.764, 'content': [{'end': 722.495, 'text': "So that's redundant, so I just crossed it out, okay? Divided by the probability that we have an ace.", 'start': 717.573, 'duration': 4.922}, {'end': 730.078, 'text': 'Now the probability, this is just quick review with the naive definition.', 'start': 726.517, 'duration': 3.561}, {'end': 734.18, 'text': "We can use the naive definition here cuz we're assuming all two card hands are equally likely.", 'start': 730.098, 'duration': 4.082}, {'end': 744.888, 'text': 'The probability that both cards are aces, well, you can choose whether to do this problem using order or without order.', 'start': 737.045, 'duration': 7.843}, {'end': 748.549, 'text': "But let's just do it without order, because I don't really care about the order of the hand.", 'start': 744.968, 'duration': 3.581}, {'end': 755.752, 'text': "If the card consists of two aces, there's two possibilities, right? Choose two out of the four aces.", 'start': 750.89, 'duration': 4.862}, {'end': 761.975, 'text': "And the denominator, we know, is 52 choose 2, because we're just picking two cards out of 52.", 'start': 756.112, 'duration': 5.863}, {'end': 762.755, 'text': 'Naive definition.', 'start': 761.975, 'duration': 0.78}, {'end': 768.349, 'text': "The denominator, the probability that we have an ace, there's two ways to do that.", 'start': 764.346, 'duration': 4.003}, {'end': 772.593, 'text': 'Either we could break it up into cases.', 'start': 769.45, 'duration': 3.143}, {'end': 773.714, 'text': "So there's two cases.", 'start': 772.913, 'duration': 0.801}, {'end': 776.596, 'text': 'Either we have two aces, and you can find that.', 'start': 773.854, 'duration': 2.742}, {'end': 777.737, 'text': 'We just did that.', 'start': 777.037, 'duration': 0.7}, {'end': 780.9, 'text': 'Or we have one ace and one non-ace.', 'start': 778.478, 'duration': 2.422}, {'end': 782.601, 'text': 'Those are two disjoint cases.', 'start': 781.36, 'duration': 1.241}, {'end': 783.442, 'text': 'We could add them up.', 'start': 782.661, 'duration': 0.781}, {'end': 789.067, 'text': "I think it's a little bit easier to do the complement, but you'll get the same thing either way if you do it correctly.", 'start': 784.283, 'duration': 4.784}, {'end': 790.368, 'text': 'We do the complement.', 'start': 789.427, 'duration': 0.941}, {'end': 796.336, 'text': "then what we're saying is it's 1 minus the probability that neither card is an ace.", 'start': 791.374, 'duration': 4.962}, {'end': 805.961, 'text': 'The probability that neither card is an ace, well, there are 48 non-aces in a deck, right? 52 minus 4.', 'start': 796.937, 'duration': 9.024}, {'end': 807.281, 'text': 'We can choose any two of them.', 'start': 805.961, 'duration': 1.32}, {'end': 810.863, 'text': 'Divided by 52, choose 2.', 'start': 807.642, 'duration': 3.221}, {'end': 813.704, 'text': 'And if you simplify this, you get 1 over 33.', 'start': 810.863, 'duration': 2.841}, {'end': 818.887, 'text': 'So about a 3% chance of this happening after simplification.', 'start': 813.704, 'duration': 5.183}, {'end': 823.099, 'text': "OK, now let's do this problem.", 'start': 821.138, 'duration': 1.961}, {'end': 835.225, 'text': "What's the probability that both are aces, given that we have the ace of spades? So p of both aces, given that we have the ace of spades.", 'start': 823.579, 'duration': 11.646}, {'end': 842.188, 'text': "Again, there's more than one way we can do this problem.", 'start': 840.067, 'duration': 2.121}], 'summary': 'Probability of drawing two aces is 1/33, about 3% chance.', 'duration': 124.615, 'max_score': 717.573, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8717573.jpg'}, {'end': 1107.014, 'src': 'embed', 'start': 1075.004, 'weight': 1, 'content': [{'end': 1081.187, 'text': "I mean, this kind of thing actually is important in gambling, but let's do one that's important just in daily life.", 'start': 1075.004, 'duration': 6.183}, {'end': 1088.283, 'text': "Suppose that we're testing for a disease.", 'start': 1084.921, 'duration': 3.362}, {'end': 1101.23, 'text': 'So this is a problem that comes up every day everywhere in the world in the medical context.', 'start': 1092.646, 'duration': 8.584}, {'end': 1107.014, 'text': 'So suppose that a patient comes in and is getting tested for a certain disease.', 'start': 1101.811, 'duration': 5.203}], 'summary': 'Testing for a disease is a common medical challenge worldwide.', 'duration': 32.01, 'max_score': 1075.004, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81075004.jpg'}], 'start': 505.349, 'title': 'Conditional probability importance and examples', 'summary': 'Emphasizes the significance of conditional probability in updating probabilities based on evidence, highlighting its relevance in solving general problems and providing examples of its application in scenarios like drawing cards from a deck and medical testing.', 'chapters': [{'end': 549.776, 'start': 505.349, 'title': 'Conditional probability importance', 'summary': 'Discusses the importance of conditional probability in updating probabilities based on evidence and its relevance in solving general problems, emphasizing the need for experience and practice. conditional probability is important due to its significance in updating probabilities based on evidence and its relevance in solving general problems.', 'duration': 44.427, 'highlights': ['Conditional probability is important in updating probabilities based on evidence. Emphasizes the importance of conditional probability in updating probabilities based on evidence.', 'Experience and practice are essential in improving the ability to guess useful and useless partitions. Stresses the need for experience and practice in improving the ability to guess useful and useless partitions.', 'Conditional probability is important in solving general problems. Highlights the significance of conditional probability in solving general problems.']}, {'end': 1240.212, 'start': 549.776, 'title': 'Conditional probability examples', 'summary': 'Discusses conditional probability using examples of drawing two random cards from a standard deck to compute conditional probabilities for having an ace and the ace of spades, and illustrates the importance of careful conditioning in medical testing scenarios, revealing the surprising and subtle nature of conditional probability.', 'duration': 690.436, 'highlights': ["The probability that both cards are aces given that we have the ace of spades is 3 over 51, which simplifies to 1 17th, revealing the subtle and surprising nature of conditional probability. The probability of getting both aces given the ace of spades is 3 over 51, indicating the conditional probability's surprising and subtle nature.", 'The probability that both cards are aces given that we have an ace is found to be 1 over 33, indicating the surprising complexity of conditional probability even in seemingly simple scenarios. The probability of getting both aces given an ace is 1 over 33, highlighting the unexpected complexity of conditional probability in seemingly simple scenarios.', 'The chapter emphasizes the importance of careful conditioning in medical testing scenarios, where the interpretation of test accuracy and the prevalence of the disease play crucial roles. The importance of careful conditioning in medical testing scenarios is emphasized, focusing on the interpretation of test accuracy and disease prevalence.']}], 'duration': 734.863, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo8505349.jpg', 'highlights': ['Emphasizes the significance of conditional probability in updating probabilities based on evidence.', 'Highlights the importance of careful conditioning in medical testing scenarios, focusing on the interpretation of test accuracy and disease prevalence.', 'Stresses the need for experience and practice in improving the ability to guess useful and useless partitions.', 'Reveals the surprising and subtle nature of conditional probability in specific scenarios, such as drawing cards from a deck.', 'Highlights the unexpected complexity of conditional probability in seemingly simple scenarios, like drawing cards from a deck.']}, {'end': 1541.801, 'segs': [{'end': 1297.641, 'src': 'embed', 'start': 1269.707, 'weight': 0, 'content': [{'end': 1272.692, 'text': 'Patient has disease, D for disease.', 'start': 1269.707, 'duration': 2.985}, {'end': 1282.635, 'text': "Okay, when you're defining events, try to write it out as carefully as possible.", 'start': 1278.673, 'duration': 3.962}, {'end': 1291.439, 'text': "It would be tempting here to just write D for disease, okay? But that's confusing, right? Disease is not an event.", 'start': 1283.375, 'duration': 8.064}, {'end': 1293.36, 'text': 'The event is that the patient has a disease.', 'start': 1291.499, 'duration': 1.861}, {'end': 1297.641, 'text': "Now, if it's very obvious in the context what you mean, then that's fine.", 'start': 1293.74, 'duration': 3.901}], 'summary': "Emphasize defining events carefully, use specific language. avoid confusion by specifying the event as 'patient has a disease.'", 'duration': 27.934, 'max_score': 1269.707, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81269707.jpg'}, {'end': 1370.064, 'src': 'embed', 'start': 1340.304, 'weight': 2, 'content': [{'end': 1356.618, 'text': "So let's suppose that 95% accurate means that the probability of T given D equals 0.95 equals the probability of T complement given D complement.", 'start': 1340.304, 'duration': 16.314}, {'end': 1358.199, 'text': "So that's an assumption.", 'start': 1357.418, 'duration': 0.781}, {'end': 1362.903, 'text': "What this assumption is, that's an interpretation of 95% accurate.", 'start': 1359.542, 'duration': 3.361}, {'end': 1370.064, 'text': 'And what this says is if the patient has the disease, then 95% of the time the test will correctly report test positive.', 'start': 1362.923, 'duration': 7.141}], 'summary': 'Interpreting 95% accuracy as 95% chance of correct positive test if patient has disease.', 'duration': 29.76, 'max_score': 1340.304, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81340304.jpg'}, {'end': 1427.45, 'src': 'embed', 'start': 1395.128, 'weight': 4, 'content': [{'end': 1399.748, 'text': 'The patient wants to know whether he or she has the disease.', 'start': 1395.128, 'duration': 4.62}, {'end': 1404.289, 'text': 'So what the patient cares about is not p of t given d.', 'start': 1400.689, 'duration': 3.6}, {'end': 1408.59, 'text': "It's p of d given t.", 'start': 1404.289, 'duration': 4.301}, {'end': 1409.57, 'text': "So that's our goal.", 'start': 1408.59, 'duration': 0.98}, {'end': 1414.071, 'text': 'Our goal is to find p of d given t.', 'start': 1409.79, 'duration': 4.281}, {'end': 1425.949, 'text': 'Now, so one of the most common mistakes in statistics as applied in real life is confusing p of t given d with p of d given t.', 'start': 1415.481, 'duration': 10.468}, {'end': 1427.45, 'text': 'Those are completely different concepts.', 'start': 1425.949, 'duration': 1.501}], 'summary': 'Goal: determine p of d given t, not p of t given d. common mistake in statistics.', 'duration': 32.322, 'max_score': 1395.128, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81395128.jpg'}, {'end': 1528.015, 'src': 'embed', 'start': 1457.023, 'weight': 6, 'content': [{'end': 1464.564, 'text': "And we already know P We know P That's just for the population, so that's 0.01.", 'start': 1457.023, 'duration': 7.541}, {'end': 1469.812, 'text': "The only thing left is p We don't yet know p So here's a little trick.", 'start': 1464.564, 'duration': 5.248}, {'end': 1474.606, 'text': "And sometimes if you look in books, they won't state Bayes' rule this way.", 'start': 1471.945, 'duration': 2.661}, {'end': 1477.487, 'text': "They'll do something more complicated with a sum in the denominator.", 'start': 1474.866, 'duration': 2.621}, {'end': 1480.269, 'text': "But I don't consider that Bayes' rule.", 'start': 1478.128, 'duration': 2.141}, {'end': 1484.85, 'text': "This is Bayes' rule, okay? And now often the denominator is the tricky part.", 'start': 1480.409, 'duration': 4.441}, {'end': 1489.772, 'text': "And for doing the denominator, that's when we do this law of total probability, okay?", 'start': 1485.271, 'duration': 4.501}, {'end': 1493.534, 'text': "So it's very common to use Bayes' rule and the law of total probability in tandem.", 'start': 1489.792, 'duration': 3.742}, {'end': 1511.06, 'text': 'So if we expand the denominator using the law of total probability, p of t given d, p of d plus p of t given d complement, p of d complement.', 'start': 1494.414, 'duration': 16.646}, {'end': 1513.472, 'text': 'So our partition is a very obvious one.', 'start': 1511.311, 'duration': 2.161}, {'end': 1518.573, 'text': 'The partition is just saying either the patient has the disease or does not have the disease.', 'start': 1513.932, 'duration': 4.641}, {'end': 1521.594, 'text': "So we're breaking it up into those two cases right?", 'start': 1518.853, 'duration': 2.741}, {'end': 1526.595, 'text': "So it's hard to immediately see what p of t is, but it's easy as soon as we break it into two cases.", 'start': 1521.754, 'duration': 4.841}, {'end': 1528.015, 'text': 'So we have two cases.', 'start': 1526.995, 'duration': 1.02}], 'summary': "Using bayes' rule and total probability for finding p, with a population of 0.01.", 'duration': 70.992, 'max_score': 1457.023, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81457023.jpg'}], 'start': 1240.332, 'title': 'Probability concepts in medicine', 'summary': "Discusses the importance of defining events and using clear notation to avoid ambiguity. it also covers conditional probability, accuracy in medical testing, bayes' rule, and the law of total probability with a population probability of 0.01.", 'chapters': [{'end': 1316.254, 'start': 1240.332, 'title': 'Defining events and notation', 'summary': "Discusses the importance of carefully defining events and using clear notation to avoid ambiguity, emphasizing that events should be specifically defined and not confused with representations like 'd for disease'.", 'duration': 75.922, 'highlights': ["Defining events with clear notation is crucial to avoid ambiguity, as representing 'D for disease' without specifying the event can lead to confusion and misinterpretation.", "It is essential to clearly define events, such as the event 'D' representing the patient having the disease, to ensure understanding and prevent confusion."]}, {'end': 1427.45, 'start': 1316.254, 'title': 'Conditional probability and accuracy', 'summary': 'Discusses the concept of conditional probability, emphasizing the difference between p of t given d and p of d given t, and the interpretation of 95% accuracy in medical testing.', 'duration': 111.196, 'highlights': ['The interpretation of 95% accurate is that if the patient has the disease, then 95% of the time the test will correctly report test positive. Interpreting 95% accuracy: If the patient has the disease, the test correctly reports positive 95% of the time.', "The patient wants to know whether he or she has the disease, caring about p of d given t, not p of t given d. Patient's concern: Patient cares about p of d given t, not p of t given d.", 'Confusing p of t given d with p of d given t is one of the most common mistakes in statistics as applied in real life. Common mistake: Confusing p of t given d with p of d given t is a frequent error in real-life statistics applications.']}, {'end': 1541.801, 'start': 1427.93, 'title': "Bayes' rule and law of total probability", 'summary': "Explains the application of bayes' rule and the law of total probability to calculate the probability of a patient having a disease, with a population probability of 0.01, demonstrating the use of these concepts in tandem for a clear partition and calculation of probabilities.", 'duration': 113.871, 'highlights': ["The chapter explains Bayes' rule as the relationship between probabilities, particularly P is P, and demonstrates its application in determining the probability of a patient having a disease with a known population probability of 0.01.", "It highlights the use of the law of total probability in tandem with Bayes' rule to calculate the denominator, providing a clear partition of cases where the patient either has the disease or does not have the disease.", 'The chapter emphasizes breaking down the calculation into two cases - the patient having the disease and the patient not having the disease - to simplify the determination of the probability of the patient having the disease.', "It demonstrates the straightforward application of the known probabilities and partition to calculate the final probability, showcasing the practical use of Bayes' rule and the law of total probability in a specific scenario."]}], 'duration': 301.469, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81240332.jpg', 'highlights': ["Defining events with clear notation is crucial to avoid ambiguity, as representing 'D for disease' without specifying the event can lead to confusion and misinterpretation.", "It is essential to clearly define events, such as the event 'D' representing the patient having the disease, to ensure understanding and prevent confusion.", 'The interpretation of 95% accurate is that if the patient has the disease, then 95% of the time the test will correctly report test positive.', 'Interpreting 95% accuracy: If the patient has the disease, the test correctly reports positive 95% of the time.', 'The patient wants to know whether he or she has the disease, caring about p of d given t, not p of t given d.', "Patient's concern: Patient cares about p of d given t, not p of t given d.", "The chapter explains Bayes' rule as the relationship between probabilities, particularly P is P, and demonstrates its application in determining the probability of a patient having a disease with a known population probability of 0.01.", "It highlights the use of the law of total probability in tandem with Bayes' rule to calculate the denominator, providing a clear partition of cases where the patient either has the disease or does not have the disease.", 'The chapter emphasizes breaking down the calculation into two cases - the patient having the disease and the patient not having the disease - to simplify the determination of the probability of the patient having the disease.', "It demonstrates the straightforward application of the known probabilities and partition to calculate the final probability, showcasing the practical use of Bayes' rule and the law of total probability in a specific scenario."]}, {'end': 1970, 'segs': [{'end': 1594.689, 'src': 'embed', 'start': 1570.591, 'weight': 0, 'content': [{'end': 1579.957, 'text': "So even though the test is supposedly 95% accurate in this sense, there's only a 16% chance that the patient has the disease.", 'start': 1570.591, 'duration': 9.366}, {'end': 1584.441, 'text': 'And that seems surprising both to most patients and to most doctors at first.', 'start': 1580.358, 'duration': 4.083}, {'end': 1594.689, 'text': 'And in fact, there was a study done at Harvard where they asked something like 60 Harvard doctors a question very similar to this.', 'start': 1585.001, 'duration': 9.688}], 'summary': 'Test is 95% accurate, but only 16% chance patient has disease.', 'duration': 24.098, 'max_score': 1570.591, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81570591.jpg'}, {'end': 1749.766, 'src': 'embed', 'start': 1721.238, 'weight': 1, 'content': [{'end': 1727.963, 'text': "And just speaking roughly, if you have 1, 000 patients, about 10 of them will have the disease, right? That's 1%.", 'start': 1721.238, 'duration': 6.725}, {'end': 1734.388, 'text': "Maybe not exactly 10, but just roughly, intuitively speaking, we'd imagine 1, 000 patients, 10 have the disease.", 'start': 1727.963, 'duration': 6.425}, {'end': 1738.05, 'text': "And let's suppose that for those 10, the test is correct every time.", 'start': 1734.928, 'duration': 3.122}, {'end': 1739.932, 'text': 'So all 10 of them test positive.', 'start': 1738.11, 'duration': 1.822}, {'end': 1749.766, 'text': 'OK, now what about the other 900 and whatever people? 10 people have the disease, so 990 people do not have the disease.', 'start': 1741.479, 'duration': 8.287}], 'summary': 'Roughly, out of 1000 patients, 10 have the disease, with 10 testing positive.', 'duration': 28.528, 'max_score': 1721.238, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81721238.jpg'}, {'end': 1870.785, 'src': 'embed', 'start': 1839.613, 'weight': 3, 'content': [{'end': 1842.935, 'text': "You get evidence, and you update your probabilities using Bayes' rule.", 'start': 1839.613, 'duration': 3.322}, {'end': 1851.288, 'text': "And here's one really crucial and beautiful fact about Bayes' rule, is it has a certain coherency property.", 'start': 1843.959, 'duration': 7.329}, {'end': 1859.257, 'text': 'And I kind of work out the math of this in one of the strategic practice problems, but let me just tell you the intuition right now.', 'start': 1852.789, 'duration': 6.468}, {'end': 1870.785, 'text': "Suppose that you get two pieces of evidence, not just one, okay? So suppose you're investigating a crime and there's two possibilities.", 'start': 1860.515, 'duration': 10.27}], 'summary': "Bayes' rule updates probabilities with evidence, showing coherency with multiple pieces of evidence.", 'duration': 31.172, 'max_score': 1839.613, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81839613.jpg'}, {'end': 1970, 'src': 'embed', 'start': 1939.515, 'weight': 4, 'content': [{'end': 1948.177, 'text': "So I call these biohazards, basically just common mistakes, but they're hazardous to your health if you make them, so you have to be really careful.", 'start': 1939.515, 'duration': 8.662}, {'end': 1951.078, 'text': 'These are common mistakes with conditional probability.', 'start': 1948.658, 'duration': 2.42}, {'end': 1968.96, 'text': "So one is the one we just talked about, confusing P with P, And we know that Bayes' rule is how you can connect these two,", 'start': 1952.419, 'duration': 16.541}, {'end': 1970, 'text': "but they're not the same thing.", 'start': 1968.96, 'duration': 1.04}], 'summary': "Common mistakes in conditional probability can be hazardous to your health, such as confusing p with p, despite the connection through bayes' rule.", 'duration': 30.485, 'max_score': 1939.515, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81939515.jpg'}], 'start': 1542.662, 'title': "Test accuracy and bayes' rule", 'summary': "Discusses the counterintuitive nature of test accuracy, demonstrating how a supposedly 95% accurate test yields a 16% chance of a patient actually having the disease, as shown by a harvard study. it also explains the principle of updating probabilities using bayes' rule and common mistakes with conditional probability, emphasizing flexibility and consistency in updating probabilities with evidence.", 'chapters': [{'end': 1815.779, 'start': 1542.662, 'title': 'Understanding test accuracy', 'summary': 'Discusses the counterintuitive nature of test accuracy, highlighting how a supposedly 95% accurate test only yields a 16% chance of a patient actually having the disease, as demonstrated by a harvard study and a hypothetical scenario with 1,000 patients.', 'duration': 273.117, 'highlights': ["The test's supposedly 95% accuracy results in only a 16% chance of the patient having the disease, as demonstrated by a Harvard study. 95% accuracy leads to 16% chance of patient having the disease.", 'In a hypothetical scenario with 1,000 patients, approximately 10 would have the disease, and 50 people without the disease would test positive, highlighting the counterintuitive nature of test accuracy. In a scenario with 1,000 patients, 10 have the disease, and 50 without the disease test positive.', '80% of the doctors in a Harvard study guessed extremely high numbers, like 95%, for the chance of a patient having the disease, reflecting the widespread misunderstanding of test accuracy. 80% of doctors guessed extremely high numbers for the chance of patient having the disease.']}, {'end': 1970, 'start': 1816.86, 'title': "Bayes' rule and conditional probability", 'summary': "Explains the principle of updating probabilities using bayes' rule, highlighting its coherency property and common mistakes with conditional probability, emphasizing the flexibility and consistency in updating probabilities with evidence.", 'duration': 153.14, 'highlights': ["Bayes' rule has a coherency property, allowing for consistent updating of probabilities with one or multiple pieces of evidence, in any order. Bayes' rule maintains consistency in updating probabilities, whether it's done in one step or multiple steps with one or more pieces of evidence, ensuring the final result is coherent.", 'The importance of being cautious about common mistakes with conditional probability and confusing P with P. Emphasizes the significance of avoiding common mistakes in conditional probability, particularly the confusion between two different probabilities, P and P, to ensure accurate calculations and interpretations.', "The principle of updating probabilities using Bayes' rule, where evidence leads to updating probabilities. Explains the fundamental concept of updating probabilities using Bayes' rule, demonstrating how evidence, such as symptoms in a medical context, leads to the adjustment of probabilities, illustrating the practical application of the principle."]}], 'duration': 427.338, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81542662.jpg', 'highlights': ['95% accuracy leads to 16% chance of patient having the disease.', 'In a scenario with 1,000 patients, 10 have the disease, and 50 without the disease test positive.', '80% of doctors guessed extremely high numbers for the chance of patient having the disease.', "Bayes' rule maintains consistency in updating probabilities, whether it's done in one step or multiple steps with one or more pieces of evidence, ensuring the final result is coherent.", 'Emphasizes the significance of avoiding common mistakes in conditional probability, particularly the confusion between two different probabilities, P and P, to ensure accurate calculations and interpretations.', "Explains the fundamental concept of updating probabilities using Bayes' rule, demonstrating how evidence, such as symptoms in a medical context, leads to the adjustment of probabilities, illustrating the practical application of the principle."]}, {'end': 2307.88, 'segs': [{'end': 2001.022, 'src': 'embed', 'start': 1970.34, 'weight': 2, 'content': [{'end': 1972.78, 'text': "Sometimes this is called the prosecutor's fallacy.", 'start': 1970.34, 'duration': 2.44}, {'end': 1979.082, 'text': "And if you wanna find examples of this, you can just search online for prosecutor's fallacy.", 'start': 1974.121, 'duration': 4.961}, {'end': 1984.243, 'text': "That's kind of unfair to prosecutors because defense attorneys make the same mistake.", 'start': 1979.522, 'duration': 4.721}, {'end': 1986.463, 'text': 'Doctors make the same mistake.', 'start': 1985.043, 'duration': 1.42}, {'end': 1991.184, 'text': "People make that mistake all the time in everyday life, so it's unfair to just pick on prosecutors.", 'start': 1986.503, 'duration': 4.681}, {'end': 2001.022, 'text': "The reason it's called the prosecutor's fallacy is that It's a common situation that and maybe sometimes the prosecutor does it deliberately,", 'start': 1991.784, 'duration': 9.238}], 'summary': "The prosecutor's fallacy is a common mistake made by various professionals and individuals, not just prosecutors.", 'duration': 30.682, 'max_score': 1970.34, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81970340.jpg'}, {'end': 2047.549, 'src': 'embed', 'start': 2016.565, 'weight': 0, 'content': [{'end': 2021.849, 'text': 'And the mistake would be focusing entirely on the probability of the evidence given innocence.', 'start': 2016.565, 'duration': 5.284}, {'end': 2027.633, 'text': 'And you want the probability of innocence given evidence, right? So those two things get confused.', 'start': 2022.009, 'duration': 5.624}, {'end': 2037.106, 'text': 'Let me mention one legal example, which is a very sad true story called the Sally Clark case.', 'start': 2029.804, 'duration': 7.302}, {'end': 2047.549, 'text': 'This is an extreme example, but there are many other legal cases that are similar in flavor to varying or, in varying degrees,', 'start': 2038.187, 'duration': 9.362}], 'summary': 'Focusing on evidence probability can lead to confusion. example: sally clark case.', 'duration': 30.984, 'max_score': 2016.565, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82016565.jpg'}], 'start': 1970.34, 'title': "Prosecutor's fallacy", 'summary': "Discusses the prosecutor's fallacy, exemplified by sally clark's wrongful conviction due to a misunderstanding of probability, underscoring the critical impact of probability understanding in legal cases.", 'chapters': [{'end': 2307.88, 'start': 1970.34, 'title': "Prosecutor's fallacy in legal cases", 'summary': "Discusses the prosecutor's fallacy, highlighting the extreme case of sally clark, a british woman wrongly convicted of murdering her babies due to a misunderstanding of probability, which resulted in years of imprisonment and a tragic outcome. the chapter emphasizes the importance of understanding probability in legal cases and its potential consequences.", 'duration': 337.54, 'highlights': ['Misapplication of Probability in Legal Cases The chapter illustrates the misapplication of probability in legal cases through the extreme example of Sally Clark, who was wrongly convicted of murdering her babies based on a probability calculation that ignored the prior probability of innocence, leading to years of imprisonment and a tragic outcome.', "Prosecutor's Fallacy in Legal System The discussion sheds light on the prosecutor's fallacy, emphasizing that it is not exclusive to prosecutors and occurs in various professions, highlighting the unfairness of singling out prosecutors for this mistake.", 'Impact of Misunderstanding Probability The case of Sally Clark serves as a poignant example of the profound impact of misunderstanding probability in legal cases, leading to wrongful convictions, prolonged imprisonment, and tragic consequences, emphasizing the critical need for a proper understanding of probability in legal proceedings.']}], 'duration': 337.54, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo81970340.jpg', 'highlights': ['The case of Sally Clark serves as a poignant example of the profound impact of misunderstanding probability in legal cases, leading to wrongful convictions, prolonged imprisonment, and tragic consequences, emphasizing the critical need for a proper understanding of probability in legal proceedings.', 'Misapplication of Probability in Legal Cases The chapter illustrates the misapplication of probability in legal cases through the extreme example of Sally Clark, who was wrongly convicted of murdering her babies based on a probability calculation that ignored the prior probability of innocence, leading to years of imprisonment and a tragic outcome.', "Prosecutor's Fallacy in Legal System The discussion sheds light on the prosecutor's fallacy, emphasizing that it is not exclusive to prosecutors and occurs in various professions, highlighting the unfairness of singling out prosecutors for this mistake."]}, {'end': 3000.392, 'segs': [{'end': 2340.344, 'src': 'embed', 'start': 2309.321, 'weight': 0, 'content': [{'end': 2314.005, 'text': "Okay, so that's called the prosecutor's fallacy, although it's not restricted to prosecutors.", 'start': 2309.321, 'duration': 4.684}, {'end': 2323.793, 'text': 'Now another one that I wanted to emphasize is one that I was just leading into when I mentioned this idea of prior.', 'start': 2314.865, 'duration': 8.928}, {'end': 2327.776, 'text': 'So I wanna tell you what the word prior and the word posterior mean.', 'start': 2323.813, 'duration': 3.963}, {'end': 2336.503, 'text': "So the second mistake is confusing P that's called the prior.", 'start': 2328.977, 'duration': 7.526}, {'end': 2340.344, 'text': 'Prior means before we have evidence.', 'start': 2338.203, 'duration': 2.141}], 'summary': "The prosecutor's fallacy and confusion between prior and posterior are common mistakes in evidence evaluation.", 'duration': 31.023, 'max_score': 2309.321, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82309321.jpg'}, {'end': 2430.841, 'src': 'embed', 'start': 2401.995, 'weight': 1, 'content': [{'end': 2403.876, 'text': "But it doesn't mean you just write p of a equals 1.", 'start': 2401.995, 'duration': 1.881}, {'end': 2406.037, 'text': 'p of a given a is 1.', 'start': 2403.876, 'duration': 2.161}, {'end': 2409.26, 'text': "So when you're writing down conditional probability calculations, you have to be very,", 'start': 2406.037, 'duration': 3.223}, {'end': 2417.428, 'text': 'very careful about what should you put to the left of this bar and what do you put to the right of this bar.', 'start': 2409.26, 'duration': 8.168}, {'end': 2430.841, 'text': 'okay?. And one more common mistake with probability is confusing independence with conditional independence.', 'start': 2417.428, 'duration': 13.413}], 'summary': 'Be cautious when calculating conditional probability and distinguishing independence from conditional independence.', 'duration': 28.846, 'max_score': 2401.995, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82401995.jpg'}, {'end': 2484.884, 'src': 'embed', 'start': 2453.396, 'weight': 2, 'content': [{'end': 2464.659, 'text': 'This is a more subtle mistake than these first two, but also comes up a lot in practice and can lead you to completely wrong results.', 'start': 2453.396, 'duration': 11.263}, {'end': 2471.981, 'text': 'So I wanna talk about this distinction, independence versus conditional independence.', 'start': 2465.939, 'duration': 6.042}, {'end': 2477.462, 'text': 'So first let me write the definition of conditional independence.', 'start': 2474.281, 'duration': 3.181}, {'end': 2484.884, 'text': 'So we say that, So this is a definition, but it should be kind of an intuitive definition.', 'start': 2479.803, 'duration': 5.081}], 'summary': 'Distinguishing between independence and conditional independence is crucial.', 'duration': 31.488, 'max_score': 2453.396, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82453396.jpg'}, {'end': 2609.88, 'src': 'embed', 'start': 2577.506, 'weight': 5, 'content': [{'end': 2580.148, 'text': 'And the answer is no, not in general.', 'start': 2577.506, 'duration': 2.642}, {'end': 2584.131, 'text': "So if they're conditionally independent, they may or may not be independent.", 'start': 2580.669, 'duration': 3.462}, {'end': 2589.055, 'text': "And there's an example that I worked out in detail on the strategic practice.", 'start': 2584.772, 'duration': 4.283}, {'end': 2594.019, 'text': "So I'll just talk about that example briefly, but you can read that example if you haven't already.", 'start': 2589.556, 'duration': 4.463}, {'end': 2595.941, 'text': "That's the chess player example.", 'start': 2594.52, 'duration': 1.421}, {'end': 2601.877, 'text': 'Chess opponent of unknown strength.', 'start': 2599.116, 'duration': 2.761}, {'end': 2609.88, 'text': "This is a very common situation in real life, right? It doesn't have to be chess, obviously.", 'start': 2603.858, 'duration': 6.022}], 'summary': 'Conditional independence does not guarantee general independence, as demonstrated in the chess player example.', 'duration': 32.374, 'max_score': 2577.506, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82577506.jpg'}, {'end': 2674.338, 'src': 'heatmap', 'start': 2638.604, 'weight': 0.724, 'content': [{'end': 2645.287, 'text': "Now, it's possible that after you win the first few, then that person gets demoralized and collapses and starts playing badly.", 'start': 2638.604, 'duration': 6.683}, {'end': 2649.829, 'text': "Well, it's possible that they get mad and they think harder and they want to take revenge and things like that.", 'start': 2645.647, 'duration': 4.182}, {'end': 2652.491, 'text': "Ignore all that because I'm just coming up with an example.", 'start': 2650.37, 'duration': 2.121}, {'end': 2660.514, 'text': "So let's assume that conditional on how strong of a player that person is, all the games are independent.", 'start': 2652.891, 'duration': 7.623}, {'end': 2662.635, 'text': "That's reasonable.", 'start': 2661.135, 'duration': 1.5}, {'end': 2674.338, 'text': "But that does not imply that the That doesn't imply the games are unconditionally independent if you don't condition on how strong that person is.", 'start': 2664.096, 'duration': 10.242}], 'summary': "Independent games may depend on player's strength, but not unconditionally.", 'duration': 35.734, 'max_score': 2638.604, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82638604.jpg'}, {'end': 2713.669, 'src': 'embed', 'start': 2688.329, 'weight': 3, 'content': [{'end': 2697.197, 'text': 'So even though the games are seemingly independent, the earlier games give you evidence that helps you assess the strength of your opponent.', 'start': 2688.329, 'duration': 8.868}, {'end': 2701.52, 'text': "So that gives you evidence that's relevant for predicting the later outcomes.", 'start': 2698.277, 'duration': 3.243}, {'end': 2708.986, 'text': 'Independence would mean that the earlier games that you play give you no information whatsoever that helps you predict the outcomes of the later games,', 'start': 2702.02, 'duration': 6.966}, {'end': 2713.669, 'text': 'okay?. But actually the earlier games give you a good sense of how strong that person is.', 'start': 2708.986, 'duration': 4.683}], 'summary': 'Earlier games provide evidence for predicting later outcomes, contrary to the assumption of independence.', 'duration': 25.34, 'max_score': 2688.329, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82688329.jpg'}, {'end': 2964.416, 'src': 'embed', 'start': 2935.749, 'weight': 4, 'content': [{'end': 2939.07, 'text': 'But suppose that either of these two things will cause the fire alarm to go off.', 'start': 2935.749, 'duration': 3.321}, {'end': 2944.231, 'text': "And suppose, so suppose, that's an assumption, but I'm just constructing an example so I can assume that.", 'start': 2939.61, 'duration': 4.621}, {'end': 2947.772, 'text': 'Suppose that f and c are independent, okay?', 'start': 2944.651, 'duration': 3.121}, {'end': 2964.416, 'text': "But the key observation what's the probability that there's a fire, given that the alarm goes off and no one's making popcorn??", 'start': 2948.851, 'duration': 15.565}], 'summary': 'Analyzing the probability of fire given alarm and no popcorn.', 'duration': 28.667, 'max_score': 2935.749, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82935749.jpg'}], 'start': 2309.321, 'title': 'Probability and conditional independence', 'summary': 'Discusses common probability mistakes including confusion between prior and posterior, conditional probability calculations, and independence versus conditional independence. it also explains conditional independence using the example of playing chess and discusses whether unconditional independence implies conditional independence, with a counterexample involving a fire alarm and making popcorn.', 'chapters': [{'end': 2574.003, 'start': 2309.321, 'title': 'Probability mistakes and definitions', 'summary': 'Discusses common mistakes in probability, including confusing prior and posterior, conditional probability calculations, and independence versus conditional independence.', 'duration': 264.682, 'highlights': ['The distinction between prior and posterior is emphasized, with prior representing before evidence and posterior representing after evidence, and the mistake of writing p of a equals 1 is highlighted, with the correct concept being p of a given a equals 1.', 'The importance of being careful in conditional probability calculations is emphasized, with the reminder to consider what should be put to the left and right of the bar, to avoid completely wrong arguments.', 'The discussion on the more subtle mistake of confusing independence with conditional independence is presented, with the definition of conditional independence provided, and the question of whether independence implies conditional independence and vice versa is raised.']}, {'end': 3000.392, 'start': 2577.506, 'title': 'Conditional independence in real life', 'summary': 'Explains the concept of conditional independence using the example of playing chess with an opponent of unknown strength, highlighting that game outcomes may be conditionally independent but not independent unconditionally, and also discusses the converse question of whether unconditional independence implies conditional independence, concluding with a counterexample involving a fire alarm and making popcorn.', 'duration': 422.886, 'highlights': ['The games are seemingly independent, but the earlier games give you evidence that helps you assess the strength of your opponent, highlighting that game outcomes may be conditionally independent but not independent unconditionally.', 'The chapter discusses the converse question of whether unconditional independence implies conditional independence, concluding that the answer is no, and provides a counterexample involving a fire alarm and making popcorn.', 'The example of playing chess with an opponent of unknown strength is used to illustrate that game outcomes may be conditionally independent given the strength of the opponent, but not independent unconditionally.']}], 'duration': 691.071, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/JzDvVgNDxo8/pics/JzDvVgNDxo82309321.jpg', 'highlights': ['The distinction between prior and posterior is emphasized, with prior representing before evidence and posterior representing after evidence.', 'The importance of being careful in conditional probability calculations is emphasized, with the reminder to consider what should be put to the left and right of the bar.', 'The discussion on the more subtle mistake of confusing independence with conditional independence is presented, with the definition of conditional independence provided.', 'The games are seemingly independent, but the earlier games give you evidence that helps you assess the strength of your opponent, highlighting that game outcomes may be conditionally independent but not independent unconditionally.', 'The chapter discusses the converse question of whether unconditional independence implies conditional independence, concluding that the answer is no, and provides a counterexample involving a fire alarm and making popcorn.', 'The example of playing chess with an opponent of unknown strength is used to illustrate that game outcomes may be conditionally independent given the strength of the opponent, but not independent unconditionally.']}], 'highlights': ['The chapter emphasizes the significance of avoiding common mistakes in conditional probability, particularly the confusion between two different probabilities, P and P, to ensure accurate calculations and interpretations.', "The chapter explains Bayes' rule as the relationship between probabilities, particularly P is P, and demonstrates its application in determining the probability of a patient having a disease with a known population probability of 0.01.", 'The chapter emphasizes breaking down the calculation into two cases - the patient having the disease and the patient not having the disease - to simplify the determination of the probability of the patient having the disease.', 'The strategy of breaking a complex probability problem into simpler pieces is illustrated using the example of finding the probability of a complicated blob B by breaking it up into disjoint pieces and then adding them up.', 'The importance of breaking down complex problems into simpler pieces and solving them recursively is emphasized, applicable in various fields such as statistics, computer science, math, and economics.']}