title
11.4: Introduction to Computer Vision - Processing Tutorial

description
This video covers the basic ideas behind computer vision. OpenCV for Processing (Java) and the Kinect are demonstrated. This accompanies Chapter 16 of Learning Processing: A Beginner's Guide to Programming Images, Animation, and Interaction http://learningprocessing.com/ Twitter: https://twitter.com/shiffman Source Code: https://github.com/CodingTrain/Rainbow-Code/ For a full set of tutorials about the kinect see: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6ZMlWHdcy8hAGDy6IaoxUKf Help us caption & translate this video! http://amara.org/v/Qbwq/ 📄 Code of Conduct: https://github.com/CodingTrain/Code-of-Conduct

detail
{'title': '11.4: Introduction to Computer Vision - Processing Tutorial', 'heatmap': [{'end': 192.869, 'start': 163.77, 'weight': 0.897}, {'end': 549.497, 'start': 521.14, 'weight': 0.908}, {'end': 595.596, 'start': 561.3, 'weight': 0.767}, {'end': 720.801, 'start': 682.785, 'weight': 0.926}, {'end': 909.577, 'start': 850.748, 'weight': 0.857}, {'end': 1097.938, 'start': 1083.47, 'weight': 0.757}], 'summary': "Introduces computer vision basics and applications such as user presence detection, hand gesture recognition, and clothing color identification, covering topics like finding the brightest pixel, tracking specific colors, interactive paint using cameras, motion analysis, and opencv functionalities including face detection and image processing, along with microsoft kinect's 3d scanning capabilities.", 'chapters': [{'end': 78.311, 'segs': [{'end': 53.579, 'src': 'embed', 'start': 17.118, 'weight': 1, 'content': [{'end': 23.361, 'text': "What does it mean to try to have our program, our thing we're making in processing, see the world in some capacity?", 'start': 17.118, 'duration': 6.243}, {'end': 26.182, 'text': "Maybe to determine if there's a user present.", 'start': 23.781, 'duration': 2.401}, {'end': 29.084, 'text': 'maybe to determine if the user is waving his or her hand.', 'start': 26.182, 'duration': 2.902}, {'end': 32.106, 'text': 'maybe to determine what color clothes the user is wearing.', 'start': 29.084, 'duration': 3.022}, {'end': 40.731, 'text': 'There are lots of things we might be able to figure out from a scene if we have a camera looking at that scene or an image that was taken from somewhere.', 'start': 32.326, 'duration': 8.405}, {'end': 53.579, 'text': "Okay, so this is a huge topic and somebody who actually knows what they're talking about could probably make a long series of videos going quite in depth about topics and examples in computer vision.", 'start': 40.851, 'duration': 12.728}], 'summary': 'Exploring the potential of processing in computer vision.', 'duration': 36.461, 'max_score': 17.118, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB4417118.jpg'}, {'end': 86.653, 'src': 'embed', 'start': 54.67, 'weight': 0, 'content': [{'end': 60.856, 'text': "What I'm going to do in this short 10 to 20 minutes here is simply kind of look at some of the basics.", 'start': 54.67, 'duration': 6.186}, {'end': 67.341, 'text': 'What are some of the key fundamental pieces that we need to know about or figure out in order to build our own computer vision application?', 'start': 60.896, 'duration': 6.445}, {'end': 71.525, 'text': 'And then what are some pathways to doing more stuff with that?', 'start': 67.642, 'duration': 3.883}, {'end': 76.209, 'text': "So let's kind of start with a basic, kind of classic scenario.", 'start': 71.565, 'duration': 4.644}, {'end': 78.311, 'text': "Let's say we have a scene.", 'start': 76.55, 'duration': 1.761}, {'end': 86.653, 'text': 'There is a camera from our computer pointed into a room and in that room there is a very bright light.', 'start': 79.272, 'duration': 7.381}], 'summary': 'Overview of building computer vision applications in 10-20 minutes, covering key fundamentals and pathways for further development.', 'duration': 31.983, 'max_score': 54.67, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB4454670.jpg'}], 'start': 0.289, 'title': 'Computer vision basics', 'summary': 'Introduces the concept of computer vision and its potential applications, such as user presence detection, hand gesture recognition, and clothing color identification, and outlines fundamental aspects and pathways for building a computer vision application in a 10 to 20-minute session.', 'chapters': [{'end': 78.311, 'start': 0.289, 'title': 'Computer vision basics', 'summary': 'Introduces the concept of computer vision, exploring its potential applications like user presence detection, hand gesture recognition, and clothing color identification, and outlines the fundamental aspects and pathways for building a computer vision application in a 10 to 20-minute session.', 'duration': 78.022, 'highlights': ['The chapter discusses the potential applications of computer vision, such as user presence detection, hand gesture recognition, and clothing color identification, using a camera or an image.', 'It outlines the fundamental aspects and pathways for building a computer vision application in a 10 to 20-minute session.', 'The video aims to provide a basic understanding of computer vision, focusing on key fundamental pieces and pathways for further exploration.', "Computer vision is introduced as a way of making programs 'see' the world, potentially detecting user presence, hand gestures, and clothing colors from a scene or image."]}], 'duration': 78.022, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44289.jpg', 'highlights': ['The chapter discusses potential applications of computer vision, such as user presence detection, hand gesture recognition, and clothing color identification.', 'It outlines fundamental aspects and pathways for building a computer vision application in a 10 to 20-minute session.', "Computer vision is introduced as a way of making programs 'see' the world, potentially detecting user presence, hand gestures, and clothing colors from a scene or image."]}, {'end': 639.026, 'segs': [{'end': 105.927, 'src': 'embed', 'start': 79.272, 'weight': 2, 'content': [{'end': 86.653, 'text': 'There is a camera from our computer pointed into a room and in that room there is a very bright light.', 'start': 79.272, 'duration': 7.381}, {'end': 99.536, 'text': "As this sort of weird looking light bulb thing that I drew here and maybe there is a person like I can't draw at all holding on to that light bulb and they're moving it around.", 'start': 89.594, 'duration': 9.942}, {'end': 103.297, 'text': 'Could we have a camera over here looking at this scene?', 'start': 100.156, 'duration': 3.141}, {'end': 104.377, 'text': 'track that light bulb?', 'start': 103.297, 'duration': 1.08}, {'end': 105.927, 'text': 'How would we do that?', 'start': 105.287, 'duration': 0.64}], 'summary': "A computer camera captures a bright light in a room with a person moving it. the speaker seeks to track the light bulb's movement with another camera.", 'duration': 26.655, 'max_score': 79.272, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB4479272.jpg'}, {'end': 153.95, 'src': 'embed', 'start': 125.495, 'weight': 1, 'content': [{'end': 129.097, 'text': 'and the next one has a brightness value of three, and the next one has a brightness value of four and then zero again,', 'start': 125.495, 'duration': 3.602}, {'end': 132.578, 'text': "and eventually we might get around and be like oh look, there's a pixel over here with a brightness value of 255..", 'start': 129.097, 'duration': 3.481}, {'end': 134.499, 'text': "That's really bright.", 'start': 132.578, 'duration': 1.921}, {'end': 145.305, 'text': "Let's remember where this xy location is and as we get to the end of looking at all the pixels, which one had the highest brightest value?", 'start': 135.359, 'duration': 9.946}, {'end': 151.328, 'text': 'This is searching through the pixels to find, to try to see where the brightest pixel is.', 'start': 145.525, 'duration': 5.803}, {'end': 153.95, 'text': "So let's look at a kind of variation on this for a second.", 'start': 151.528, 'duration': 2.422}], 'summary': 'Searching for the pixel with the highest brightness value, ranging from 0 to 255.', 'duration': 28.455, 'max_score': 125.495, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44125495.jpg'}, {'end': 192.869, 'src': 'heatmap', 'start': 163.77, 'weight': 0.897, 'content': [{'end': 169.534, 'text': "And you can see there I am above over here and I'm going to take this blue marker and I'm going to click on it and you can see, oh,", 'start': 163.77, 'duration': 5.764}, {'end': 172.936, 'text': "I'm kind of drawing a dot following this blue marker.", 'start': 169.534, 'duration': 3.402}, {'end': 177.32, 'text': "I'm tracking the blue color in this image that's coming from a camera.", 'start': 173.036, 'duration': 4.284}, {'end': 181.665, 'text': "How is this done? Now, you can see there's a lot of problems there.", 'start': 177.68, 'duration': 3.985}, {'end': 184.006, 'text': "It wasn't so perfectly accurate.", 'start': 181.685, 'duration': 2.321}, {'end': 188.267, 'text': "There's a lot of issues here that I want to take a minute to discuss, too.", 'start': 184.686, 'duration': 3.581}, {'end': 192.869, 'text': "But let's just say, for the sake of argument, that is the be-all, end-all of computer vision.", 'start': 188.307, 'duration': 4.562}], 'summary': 'Demonstrating tracking of blue color in an image from a camera, with some accuracy issues.', 'duration': 29.099, 'max_score': 163.77, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44163770.jpg'}, {'end': 418.326, 'src': 'embed', 'start': 393.931, 'weight': 0, 'content': [{'end': 403.137, 'text': "If we're trying to find the color that is the most blue, what if I look at this pixel and find its distance from blue?", 'start': 393.931, 'duration': 9.206}, {'end': 405.639, 'text': 'Then the next pixel and find its distance from blue.', 'start': 403.417, 'duration': 2.222}, {'end': 407.56, 'text': 'Then the next pixel and find its distance from blue.', 'start': 405.699, 'duration': 1.861}, {'end': 413.524, 'text': 'One of those pixels will hold the world record for smallest distance to blue.', 'start': 408.14, 'duration': 5.384}, {'end': 418.326, 'text': 'And if I keep track of that record, does this pixel break the record? No, throw it away.', 'start': 413.904, 'duration': 4.422}], 'summary': 'Using pixel distances to determine the bluest color, discarding those not breaking the record.', 'duration': 24.395, 'max_score': 393.931, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44393931.jpg'}, {'end': 549.497, 'src': 'heatmap', 'start': 521.14, 'weight': 0.908, 'content': [{'end': 527.964, 'text': 'But while this might not be the most useful application in a kind of real interactive scenario,', 'start': 521.14, 'duration': 6.824}, {'end': 530.465, 'text': 'this demonstrates a lot of the basic principles of computer vision.', 'start': 527.964, 'duration': 2.501}, {'end': 532.585, 'text': "So let's look at a couple of things.", 'start': 530.705, 'duration': 1.88}, {'end': 535.687, 'text': 'One is in the code here.', 'start': 533.026, 'duration': 2.661}, {'end': 540.93, 'text': "one thing I want to point out is this is exactly what I'm talking about.", 'start': 535.687, 'duration': 5.243}, {'end': 547.776, 'text': "we have a current color from the video and I need to get its red, green and blue values and then I have a color that I'm tracking.", 'start': 540.93, 'duration': 6.846}, {'end': 549.497, 'text': "that's the color I'm searching for.", 'start': 547.776, 'duration': 1.721}], 'summary': 'Demonstrates basic principles of computer vision with color tracking.', 'duration': 28.357, 'max_score': 521.14, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44521140.jpg'}, {'end': 595.596, 'src': 'heatmap', 'start': 561.3, 'weight': 0.767, 'content': [{'end': 566.322, 'text': "other thing that I'll point out is we're starting before we go through the loop we keep track of.", 'start': 561.3, 'duration': 5.022}, {'end': 573.025, 'text': "there's what's the world record, the world record, some big number that that's obviously the first pixel is going to beat,", 'start': 566.322, 'duration': 6.703}, {'end': 575.726, 'text': 'and then we need to keep track of that X&Y point.', 'start': 573.025, 'duration': 2.701}, {'end': 581.908, 'text': "so at any moment, if we find a pixel with a distance that's less than the world record,", 'start': 575.726, 'duration': 6.182}, {'end': 586.07, 'text': 'then the new world record is that pixel and that x and y we should save.', 'start': 582.208, 'duration': 3.862}, {'end': 588.052, 'text': 'So this is that second piece.', 'start': 586.351, 'duration': 1.701}, {'end': 595.596, 'text': 'The first piece of something new here is that we need to figure out how are two colors different or similar using the distance function.', 'start': 588.332, 'duration': 7.264}], 'summary': 'The program tracks the world record pixel and its x&y point using a distance function.', 'duration': 34.296, 'max_score': 561.3, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44561300.jpg'}], 'start': 79.272, 'title': 'Computer vision basics', 'summary': 'Introduces finding the brightest pixel in an image and tracking a specific color in a video feed using the distance formula, highlighting the flaws and basic principles of computer vision.', 'chapters': [{'end': 639.026, 'start': 79.272, 'title': 'Computer vision basics', 'summary': 'Introduces the concept of finding the brightest pixel in an image and tracking a specific color in a video feed using the distance formula, while highlighting the flaws and basic principles of computer vision.', 'duration': 559.754, 'highlights': ['The process of finding the brightest pixel in an image is explained, involving searching through pixels and determining the pixel with the highest brightness value (255) as the brightest. The explanation covers the method of sequentially examining pixels to identify the one with the highest brightness value (255) and remember its xy location.', "Tracking a specific color in a video feed is demonstrated through the use of a distance function to measure the similarity between colors and the process of continuously updating the 'world record' pixel based on its distance from the tracked color. The example showcases the use of the distance function to evaluate color similarity and the iterative process of identifying and saving the closest pixel to the tracked color.", 'The flaws and limitations of tracking a single pixel in computer vision are discussed, emphasizing the challenges of tracking objects with similar colors and the need to consider larger areas of similar pixels for more practical applications. The discussion highlights the limitations of tracking a single pixel, such as the difficulty in distinguishing objects with similar colors, and suggests the consideration of larger areas of similar pixels for more practical applications.']}], 'duration': 559.754, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB4479272.jpg', 'highlights': ['The process of finding the brightest pixel in an image involves sequentially examining pixels to identify the one with the highest brightness value (255) and remember its xy location.', "Tracking a specific color in a video feed is demonstrated through the use of a distance function to measure the similarity between colors and the process of continuously updating the 'world record' pixel based on its distance from the tracked color.", 'The flaws and limitations of tracking a single pixel in computer vision are discussed, emphasizing the challenges of tracking objects with similar colors and the need to consider larger areas of similar pixels for more practical applications.']}, {'end': 859.294, 'segs': [{'end': 720.801, 'src': 'heatmap', 'start': 640.208, 'weight': 0, 'content': [{'end': 644.529, 'text': 'So one thing to recognize here is the camera is really acting as a sensor.', 'start': 640.208, 'duration': 4.321}, {'end': 647.67, 'text': "There's nothing about it that we're using for display purposes.", 'start': 644.609, 'duration': 3.061}, {'end': 653.171, 'text': "We're just reading the image and analyzing it for some piece of information.", 'start': 647.79, 'duration': 5.381}, {'end': 662.834, 'text': 'So just recently at the winter show here at ITP, students built a spray, an interactive paint application,', 'start': 653.511, 'duration': 9.323}, {'end': 669.778, 'text': 'where you would hold a spray can up to the wall and inside that spray can was a very bright green LED and there was a camera on the wall.', 'start': 662.834, 'duration': 6.944}, {'end': 675.021, 'text': 'So the camera was able to track where that spray can was simply by tracking a very bright LED.', 'start': 670.198, 'duration': 4.823}, {'end': 680.924, 'text': "So there's a lot of cases where you might want to track something, find it simply with color.", 'start': 675.381, 'duration': 5.543}, {'end': 689.803, 'text': 'I should point out that another Gonna just jump to this.', 'start': 682.785, 'duration': 7.018}, {'end': 696.93, 'text': 'Another scenario where you might like to look at the difference between pixels is for looking for motion.', 'start': 690.784, 'duration': 6.146}, {'end': 699.452, 'text': "So I'm gonna stand here and try to stand very, very still.", 'start': 697.19, 'duration': 2.262}, {'end': 706.937, 'text': "And you can see when I stand very, very still, except for my mouth moving, You're not seeing any black pixels.", 'start': 700.033, 'duration': 6.904}, {'end': 710.358, 'text': "But if I move around a lot, you're seeing a lot of black pixels.", 'start': 707.177, 'duration': 3.181}, {'end': 718.9, 'text': 'So one thing that you can do is you can find motion by saying I have one frame of video and I have the next frame of video.', 'start': 710.638, 'duration': 8.262}, {'end': 720.801, 'text': 'Let me compare each pixel.', 'start': 719.14, 'duration': 1.661}], 'summary': 'Using cameras as sensors for tracking objects and detecting motion with color and pixel differences.', 'duration': 70.15, 'max_score': 640.208, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44640208.jpg'}, {'end': 783.075, 'src': 'embed', 'start': 753.9, 'weight': 3, 'content': [{'end': 760.003, 'text': 'So this is another scenario and all these examples are in the learning processing GitHub repository.', 'start': 753.9, 'duration': 6.103}, {'end': 761.543, 'text': 'This one is example 16.13.', 'start': 760.023, 'duration': 1.52}, {'end': 768.146, 'text': 'So this is a little bit about computer vision here.', 'start': 761.543, 'duration': 6.603}, {'end': 769.987, 'text': "I guess I'll just show you one more here.", 'start': 768.166, 'duration': 1.821}, {'end': 775.029, 'text': "There's another example which we can look at just how many pixels have changed per frame.", 'start': 770.007, 'duration': 5.022}, {'end': 783.075, 'text': 'So, if very few pixels have changed per frame, you see this small dot in the center, and as I move around a lot, a lot of pixels are changing.', 'start': 775.589, 'duration': 7.486}], 'summary': 'Computer vision example 16.13 measures pixel changes per frame.', 'duration': 29.175, 'max_score': 753.9, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44753900.jpg'}], 'start': 640.208, 'title': 'Using cameras for interactive paint and analyzing motion in computer vision', 'summary': 'Covers using a camera as a sensor for interactive paint application, involving students building a spray with a bright led and a camera to track its position on the wall, and explores motion analysis in computer vision, demonstrating motion detection and potential applications like motion tracking and edge detection with an example in learning processing github repository.', 'chapters': [{'end': 689.803, 'start': 640.208, 'title': 'Camera as a sensor for interactive paint application', 'summary': 'Discusses the use of a camera as a sensor for an interactive paint application, where students built a spray with a bright led and a camera to track its position on the wall.', 'duration': 49.595, 'highlights': ['The camera is used as a sensor to read and analyze the image for information, such as tracking a bright LED spray can on a wall.', 'Students at ITP built an interactive paint application using a spray can with a bright green LED and a camera to track its position on the wall.', "The camera tracked the position of the spray can by detecting the bright LED, showcasing the camera's capability to track based on color."]}, {'end': 859.294, 'start': 690.784, 'title': 'Analyzing motion in computer vision', 'summary': 'Explores the concept of analyzing motion in computer vision, demonstrating how to detect motion by comparing pixels between frames and highlighting the potential applications, such as motion tracking and edge detection, with an example in learning processing github repository.', 'duration': 168.51, 'highlights': ['The ability to detect motion by comparing pixels between frames is showcased, with the demonstration of how pixel changes indicate motion, as evidenced by the increase in changed pixels when movement is introduced.', 'The concept of edge detection in computer vision is explained, emphasizing how subtle movements allow for identifying the edges of objects based on pixel color changes, as depicted by the example of finding the edges of a hand in the video frames.', 'The potential applications of the discussed concepts, such as motion tracking and finding the brightest or darkest pixel, are highlighted, with the suggestion of creating applications that can follow moving objects and the recommendation of OpenCV for processing as a comprehensive library for computer vision functionality.']}], 'duration': 219.086, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44640208.jpg', 'highlights': ['Students at ITP built an interactive paint application using a spray can with a bright green LED and a camera to track its position on the wall.', "The camera tracked the position of the spray can by detecting the bright LED, showcasing the camera's capability to track based on color.", 'The ability to detect motion by comparing pixels between frames is showcased, with the demonstration of how pixel changes indicate motion, as evidenced by the increase in changed pixels when movement is introduced.', 'The concept of edge detection in computer vision is explained, emphasizing how subtle movements allow for identifying the edges of objects based on pixel color changes, as depicted by the example of finding the edges of a hand in the video frames.', 'The potential applications of the discussed concepts, such as motion tracking and finding the brightest or darkest pixel, are highlighted, with the suggestion of creating applications that can follow moving objects and the recommendation of OpenCV for processing as a comprehensive library for computer vision functionality.']}, {'end': 1054.343, 'segs': [{'end': 886.657, 'src': 'embed', 'start': 860.155, 'weight': 3, 'content': [{'end': 867.961, 'text': 'OpenCV is an open source computer vision library originally developed by Intel, now maintained by open source collectives.', 'start': 860.155, 'duration': 7.806}, {'end': 872.144, 'text': "And you can see here that there's a lot of functionality.", 'start': 869.462, 'duration': 2.682}, {'end': 873.505, 'text': 'It can find your face.', 'start': 872.444, 'duration': 1.061}, {'end': 876.147, 'text': 'It can do all sorts of image processing.', 'start': 874.245, 'duration': 1.902}, {'end': 878.349, 'text': 'It can look for blobs and contours.', 'start': 876.187, 'duration': 2.162}, {'end': 886.657, 'text': 'So a lot of the stuff that we might spend hours or weeks or days or months trying to program from scratch are features that are built into a library.', 'start': 878.649, 'duration': 8.008}], 'summary': 'Opencv is an open source computer vision library with various functionalities like face detection, image processing, blob and contour detection, saving time and effort in programming.', 'duration': 26.502, 'max_score': 860.155, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44860155.jpg'}, {'end': 973.063, 'src': 'embed', 'start': 941.823, 'weight': 1, 'content': [{'end': 945.763, 'text': 'But you can imagine some applications you might do here is how many people are there?', 'start': 941.823, 'duration': 3.94}, {'end': 948.224, 'text': 'Is somebody looking straight versus looking to the side??', 'start': 945.843, 'duration': 2.381}, {'end': 955.71, 'text': 'There was a project just here in the ITP winter show where two people are standing in scene and their faces are swapped.', 'start': 948.844, 'duration': 6.866}, {'end': 958.373, 'text': "So there's a lot of creative possibilities here.", 'start': 956.091, 'duration': 2.282}, {'end': 962.917, 'text': "You know, if you use Google Hangout, you can see that, you know, you can add, there's all these features.", 'start': 958.393, 'duration': 4.524}, {'end': 964.818, 'text': 'We can add a hat or a mustache on somebody.', 'start': 962.957, 'duration': 1.861}, {'end': 967.661, 'text': 'You could do this with the OpenCV library as well.', 'start': 964.858, 'duration': 2.803}, {'end': 973.063, 'text': 'So this is something that I might encourage you to take a look into as a possibility.', 'start': 968.061, 'duration': 5.002}], 'summary': 'Creative possibilities with opencv for face swapping and adding features to images.', 'duration': 31.24, 'max_score': 941.823, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44941823.jpg'}, {'end': 1069.803, 'src': 'embed', 'start': 1039.368, 'weight': 0, 'content': [{'end': 1041.031, 'text': "But then there's also this image over here.", 'start': 1039.368, 'duration': 1.663}, {'end': 1042.771, 'text': 'This image is a depth map.', 'start': 1041.431, 'duration': 1.34}, {'end': 1046.355, 'text': "Now the way that the screen captures work, it's very hard to see what's going on here.", 'start': 1042.832, 'duration': 3.523}, {'end': 1051.58, 'text': "But you can see that my hand is a little bit brighter, and now it's quite a bit darker.", 'start': 1046.795, 'duration': 4.785}, {'end': 1054.343, 'text': "And you can see as I put it in here, we can't see it anymore.", 'start': 1051.921, 'duration': 2.422}, {'end': 1062.533, 'text': 'So what the Microsoft Kinect, and this is the old Kinect, this is the original model from a few years ago, 1414 is the model number.', 'start': 1054.644, 'duration': 7.889}, {'end': 1069.803, 'text': "What it does is it's able to not just say, like, here's a pixel, and here's its red, green, and blue value.", 'start': 1064.44, 'duration': 5.363}], 'summary': 'The microsoft kinect captures depth map images with the original model being 1414, enabling precise hand tracking.', 'duration': 30.435, 'max_score': 1039.368, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB441039367.jpg'}], 'start': 860.155, 'title': 'Opencv for computer vision', 'summary': 'Introduces opencv, an open source computer vision library with built-in functionalities for face detection, image processing, blob and contour detection, and advanced features like recognizing cards and markers in images.', 'chapters': [{'end': 901.29, 'start': 860.155, 'title': 'Opencv for computer vision', 'summary': 'Introduces opencv, an open source computer vision library with a multitude of built-in functionalities, including face detection, image processing, blob and contour detection, and even more advanced features like recognizing cards and markers in images.', 'duration': 41.135, 'highlights': ['The chapter introduces OpenCV, an open source computer vision library with a multitude of built-in functionalities, including face detection, image processing, blob and contour detection, and even more advanced features like recognizing cards and markers in images.', 'OpenCV can save significant development time by providing pre-built features, such as face detection, image processing, blob and contour detection, and advanced functionalities like recognizing cards and markers in images.', 'The library offers a wide range of capabilities, from basic functionalities like face detection to more advanced features like recognizing cards and markers in images.']}, {'end': 1054.343, 'start': 901.37, 'title': 'Opencv for face detection and blob detection', 'summary': 'Discusses the usage of opencv for face detection, including its limitations and creative possibilities, and also highlights blob detection and the capabilities of the microsoft connect camera for depth sensing.', 'duration': 152.973, 'highlights': ['OpenCV can be used for face detection, providing a rectangle around detected faces and enabling creative applications such as face swapping and adding virtual accessories like hats or mustaches.', 'OpenCV also offers blob detection for finding areas of brightness or darkness, useful in tracking applications.', 'The Microsoft Connect camera features a regular RGB camera, an infrared camera, and an infrared projector, providing depth sensing capabilities and the ability to capture depth maps.']}], 'duration': 194.188, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB44860155.jpg', 'highlights': ['OpenCV offers a wide range of capabilities, from basic functionalities like face detection to more advanced features like recognizing cards and markers in images.', 'OpenCV can save significant development time by providing pre-built features, such as face detection, image processing, blob and contour detection, and advanced functionalities like recognizing cards and markers in images.', 'OpenCV can be used for face detection, providing a rectangle around detected faces and enabling creative applications such as face swapping and adding virtual accessories like hats or mustaches.', 'The Microsoft Connect camera features a regular RGB camera, an infrared camera, and an infrared projector, providing depth sensing capabilities and the ability to capture depth maps.']}, {'end': 1370.955, 'segs': [{'end': 1079.068, 'src': 'embed', 'start': 1054.644, 'weight': 3, 'content': [{'end': 1062.533, 'text': 'So what the Microsoft Kinect, and this is the old Kinect, this is the original model from a few years ago, 1414 is the model number.', 'start': 1054.644, 'duration': 7.889}, {'end': 1069.803, 'text': "What it does is it's able to not just say, like, here's a pixel, and here's its red, green, and blue value.", 'start': 1064.44, 'duration': 5.363}, {'end': 1076.066, 'text': "What it says is, here's a pixel, and here's how far it is from the Kinect, from the camera, from the sensor in millimeters.", 'start': 1070.163, 'duration': 5.903}, {'end': 1079.068, 'text': "So you can know what's close, what's far away.", 'start': 1076.406, 'duration': 2.662}], 'summary': 'Microsoft kinect (model 1414) measures distance from camera in millimeters, distinguishing close and far objects.', 'duration': 24.424, 'max_score': 1054.644, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB441054644.jpg'}, {'end': 1114.904, 'src': 'heatmap', 'start': 1083.47, 'weight': 0.757, 'content': [{'end': 1085.811, 'text': 'Oops, sorry, I had a technical malfunction.', 'start': 1083.47, 'duration': 2.341}, {'end': 1092.515, 'text': "So what you can see here in this scenario is that I'm drawing each pixel as a point in three-dimensional space.", 'start': 1086.172, 'duration': 6.343}, {'end': 1096.317, 'text': "And it's a little hard to see probably where we are based on this orientation.", 'start': 1092.976, 'duration': 3.341}, {'end': 1097.938, 'text': "I'm looking at 12 different places.", 'start': 1096.397, 'duration': 1.541}, {'end': 1102.4, 'text': 'you can see that this is essentially the data visualized in three dimensions.', 'start': 1099.119, 'duration': 3.281}, {'end': 1106.621, 'text': 'So some really basic 3D scanning of terms of a scene.', 'start': 1102.46, 'duration': 4.161}, {'end': 1114.904, 'text': "What I'm not using here is there's a library called OpenNI, Simple OpenNI, which is actually now OpenNI was purchased by Apple.", 'start': 1107.301, 'duration': 7.603}], 'summary': '3d scanning of 12 places using openni library for data visualization.', 'duration': 31.434, 'max_score': 1083.47, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB441083470.jpg'}, {'end': 1197.928, 'src': 'embed', 'start': 1171.335, 'weight': 0, 'content': [{'end': 1176.88, 'text': "And so there is an official Microsoft developer's kit, and you can use that with processing.", 'start': 1171.335, 'duration': 5.545}, {'end': 1178.721, 'text': "However, it's only for Windows.", 'start': 1177.16, 'duration': 1.561}, {'end': 1188.746, 'text': 'And at the moment there are a lot of people here at ITP and probably other places in the world working on ways to pass the information from the Kinect and a PC over the network.', 'start': 1178.881, 'duration': 9.865}, {'end': 1192.667, 'text': 'What you have to realize is the Kinect itself is just sending this raw data.', 'start': 1188.766, 'duration': 3.901}, {'end': 1197.928, 'text': 'What is the distance for each pixel from the Kinect? What you can do with that information is incredibly powerful.', 'start': 1193.267, 'duration': 4.661}], 'summary': "Microsoft kinect developer's kit is available for windows, with ongoing efforts to network kinect and pc for powerful data processing.", 'duration': 26.593, 'max_score': 1171.335, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB441171335.jpg'}], 'start': 1054.644, 'title': 'Microsoft kinect capabilities', 'summary': "Covers the original microsoft kinect model's 3d scanning abilities, capturing depth data in millimeters, and its potential for computer vision, including raw data capture and body movement tracking.", 'chapters': [{'end': 1170.935, 'start': 1054.644, 'title': 'Microsoft kinect 3d scanning', 'summary': 'Discusses the capabilities of the original microsoft kinect model, including its ability to capture depth data in millimeters, its use for 3d scanning, and the availability of libraries for different operating systems and versions of kinect.', 'duration': 116.291, 'highlights': ['The original Microsoft Kinect model (1414) captures depth data in millimeters, enabling 3D scanning of scenes and objects.', 'Libraries such as OpenKinect and SimpleOpenNI are available for the original Kinect, while the newer Kinect version 2 offers higher resolution and more accurate skeleton tracking but lacks easily available open source drivers.', 'The chapter provides insights into the availability of libraries for different operating systems and versions of Kinect, emphasizing the suitability of the newer Kinect version 2 for enhanced capabilities.']}, {'end': 1370.955, 'start': 1171.335, 'title': 'Using kinect for computer vision', 'summary': 'Discusses the potential of using kinect for computer vision, highlighting its ability to capture raw depth data and track body movements, along with the possibilities of using third-party applications for message passing to processing.', 'duration': 199.62, 'highlights': ["Kinect's ability to capture raw depth data and track body movements is discussed, showcasing its potential for computer vision. The Kinect can recognize the form of the body, track the position of body parts, and provide raw depth data, allowing for powerful information processing and tracking.", 'The importance of the for loop in looking through all pixels for image processing and computer vision applications is emphasized. The for loop is crucial for image processing and computer vision tasks, enabling the identification of closest pixels to the Kinect and facilitating various computer vision applications.', 'Mention of third-party applications like community core vision and open TSPS for passing messages about tracking to processing is made. Third-party applications like community core vision and open TSPS can pass messages about tracking to processing, providing additional resources for developing computer vision applications.']}], 'duration': 316.311, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/h8tk0hmWB44/pics/h8tk0hmWB441054644.jpg', 'highlights': ['The original Microsoft Kinect model (1414) captures depth data in millimeters, enabling 3D scanning of scenes and objects.', "Kinect's ability to capture raw depth data and track body movements is discussed, showcasing its potential for computer vision.", 'The for loop is crucial for image processing and computer vision tasks, enabling the identification of closest pixels to the Kinect and facilitating various computer vision applications.', 'Libraries such as OpenKinect and SimpleOpenNI are available for the original Kinect, while the newer Kinect version 2 offers higher resolution and more accurate skeleton tracking but lacks easily available open source drivers.', 'The chapter provides insights into the availability of libraries for different operating systems and versions of Kinect, emphasizing the suitability of the newer Kinect version 2 for enhanced capabilities.', 'Mention of third-party applications like community core vision and open TSPS for passing messages about tracking to processing is made.']}], 'highlights': ['OpenCV offers a wide range of capabilities, from basic functionalities like face detection to more advanced features like recognizing cards and markers in images.', 'The process of finding the brightest pixel in an image involves sequentially examining pixels to identify the one with the highest brightness value (255) and remember its xy location.', 'The ability to detect motion by comparing pixels between frames is showcased, with the demonstration of how pixel changes indicate motion, as evidenced by the increase in changed pixels when movement is introduced.', 'The chapter discusses potential applications of computer vision, such as user presence detection, hand gesture recognition, and clothing color identification.', "The camera tracked the position of the spray can by detecting the bright LED, showcasing the camera's capability to track based on color.", "Introduces computer vision basics and applications such as user presence detection, hand gesture recognition, and clothing color identification, covering topics like finding the brightest pixel, tracking specific colors, interactive paint using cameras, motion analysis, and opencv functionalities including face detection and image processing, along with microsoft kinect's 3d scanning capabilities.", 'OpenCV can save significant development time by providing pre-built features, such as face detection, image processing, blob and contour detection, and advanced functionalities like recognizing cards and markers in images.', 'The flaws and limitations of tracking a single pixel in computer vision are discussed, emphasizing the challenges of tracking objects with similar colors and the need to consider larger areas of similar pixels for more practical applications.', 'The concept of edge detection in computer vision is explained, emphasizing how subtle movements allow for identifying the edges of objects based on pixel color changes, as depicted by the example of finding the edges of a hand in the video frames.', 'The for loop is crucial for image processing and computer vision tasks, enabling the identification of closest pixels to the Kinect and facilitating various computer vision applications.']}