title
12.3: Raw Depth Data - Point Clouds and Thresholds - Kinect and Processing Tutorial

description
In this video I look at how to iterate over the raw depth data array. I show how to render the depth as a point cloud in 3d space. I also look at how to set a minimum and maximum depth threshold to look only a particular set of pixels. This can be useful for isolating a person's silhouette or a hand stretched out in front. All examples: https://github.com/shiffman/Video-Lesson-Materials Open Kinect links: https://github.com/shiffman/OpenKinect-for-Processing http://shiffman.net/p5/kinect/ Contact: https://twitter.com/shiffman Next video: https://youtu.be/Kr4s5sLoROY Learn Processing from scratch: https://www.youtube.com/user/shiffman/playlists?sort=dd&shelf_id=2&view=50 More about image processing in Processing: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6YB9x6f23CBftiyx0u_5sO9 Help us caption & translate this video! http://amara.org/v/Qbtg/ 📄 Code of Conduct: https://github.com/CodingTrain/Code-of-Conduct

detail
{'title': '12.3: Raw Depth Data - Point Clouds and Thresholds - Kinect and Processing Tutorial', 'heatmap': [{'end': 141.919, 'start': 119.468, 'weight': 0.783}, {'end': 242.419, 'start': 217.526, 'weight': 0.786}, {'end': 320.883, 'start': 281.924, 'weight': 0.72}], 'summary': 'This tutorial delves into differences in depth values between version 1 and version 2 kinect, generating a point cloud in 3d space with depth values between 0 and 4500, 3d visualization, rgb manipulation, threshold calibration for hand tracking, and obstacle handling.', 'chapters': [{'end': 332.512, 'segs': [{'end': 25.472, 'src': 'embed', 'start': 1.087, 'weight': 0, 'content': [{'end': 10.549, 'text': 'Hello, In this video I plan and hope and am excited to look at the raw depth data, meaning not the depth image,', 'start': 1.087, 'duration': 9.462}, {'end': 17.791, 'text': "not the depth values converted to a grayscale image, but actually the raw depth data that's coming out of the Connect itself.", 'start': 10.549, 'duration': 7.242}, {'end': 22.571, 'text': "So again, with the version 2 Connect, you're getting numbers between 0 and 4, 500.", 'start': 18.111, 'duration': 4.46}, {'end': 25.472, 'text': "With the version 1 Connect, you're getting numbers between 0 and 2, 048.", 'start': 22.572, 'duration': 2.9}], 'summary': 'The connect version 2 produces depth data between 0 and 4,500, while version 1 produces data between 0 and 2,048.', 'duration': 24.385, 'max_score': 1.087, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo1087.jpg'}, {'end': 98.22, 'src': 'embed', 'start': 66.777, 'weight': 1, 'content': [{'end': 69.199, 'text': 'yet I insist on trying to explain it in this weird way.', 'start': 66.777, 'duration': 2.422}, {'end': 75.465, 'text': 'But I want to take all the points that the Kinect are seeing in this physical three-dimensional space where I am,', 'start': 70.04, 'duration': 5.425}, {'end': 81.771, 'text': 'and I want to move these virtual dots which are on the screen in this virtual 3D space.', 'start': 75.465, 'duration': 6.306}, {'end': 83.212, 'text': "And that's known as a point cloud.", 'start': 82.011, 'duration': 1.201}, {'end': 88.237, 'text': 'This is how you might start to build a 3D model of what the Kinect is seeing in the space.', 'start': 83.592, 'duration': 4.645}, {'end': 92.198, 'text': 'So the key difference here.', 'start': 88.837, 'duration': 3.361}, {'end': 98.22, 'text': 'so one thing that I had before, in the previous video, is we were looking at this pixel-based image right?', 'start': 92.198, 'duration': 6.022}], 'summary': "Explaining the process of creating a 3d model using kinect's point cloud data.", 'duration': 31.443, 'max_score': 66.777, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo66777.jpg'}, {'end': 141.919, 'src': 'heatmap', 'start': 119.468, 'weight': 0.783, 'content': [{'end': 127.872, 'text': "So let's come over here and I'm going to do a couple things in this video, but this first point cloud example I mostly have the code already.", 'start': 119.468, 'duration': 8.404}, {'end': 134.435, 'text': "So you can see here that what I'm doing is looping through the connects width and height.", 'start': 128.172, 'duration': 6.263}, {'end': 138.697, 'text': "Again, I'm skipping because I don't need to do every single point.", 'start': 134.535, 'duration': 4.162}, {'end': 141.919, 'text': "I don't need to do all of the points just to visually get this effect.", 'start': 138.757, 'duration': 3.162}], 'summary': 'Code loops through the connects width and height, skipping unnecessary points for visual effect.', 'duration': 22.451, 'max_score': 119.468, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo119468.jpg'}, {'end': 247.642, 'src': 'heatmap', 'start': 217.526, 'weight': 0.786, 'content': [{'end': 222.128, 'text': 'Like where is the actual x, where is the actual y, based on how the camera is seeing it.', 'start': 217.526, 'duration': 4.602}, {'end': 230.572, 'text': 'So in order to do that, this particular example has just this function, which essentially you want to download these examples and copy this verbatim.', 'start': 222.748, 'duration': 7.824}, {'end': 238.077, 'text': 'But this function is using all of these kind of parameters that are built into the hardware itself.', 'start': 231.533, 'duration': 6.544}, {'end': 242.419, 'text': "So these are like a whole set of numbers and values that are just part of the Kinect's calibration.", 'start': 238.357, 'duration': 4.062}, {'end': 247.642, 'text': 'And you kind of multiply and divide by these numbers, and you get the actual value of where it is in space.', 'start': 242.779, 'duration': 4.863}], 'summary': "Function uses kinect's calibration to determine actual x, y coordinates.", 'duration': 30.116, 'max_score': 217.526, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo217526.jpg'}, {'end': 281.324, 'src': 'embed', 'start': 250.884, 'weight': 2, 'content': [{'end': 257.889, 'text': "But right now I'm sort of inclined to sort of skip it and say the interesting thing is, what you're getting is, if you give the raw depth value,", 'start': 250.884, 'duration': 7.005}, {'end': 262.493, 'text': "the pixel x and the pixel y, and use that function, you're going to get the x,", 'start': 257.889, 'duration': 4.604}, {'end': 267.216, 'text': 'y and depth values in millimeters back of where those things are in physical space.', 'start': 262.493, 'duration': 4.723}, {'end': 269.878, 'text': "So I don't want to in fact draw the.", 'start': 267.636, 'duration': 2.242}, {'end': 281.324, 'text': "this is what you're seeing in this particular visualization right now is just all of these pixels at their exact x y value, with zero depth.", 'start': 269.878, 'duration': 11.446}], 'summary': 'Using raw depth value, pixel x, and pixel y, function returns x, y, and depth values in millimeters.', 'duration': 30.44, 'max_score': 250.884, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo250884.jpg'}, {'end': 320.883, 'src': 'heatmap', 'start': 281.924, 'weight': 0.72, 'content': [{'end': 289.907, 'text': 'So what I want to do is change this program to say, this actual physical point, this p vector, a p vector is an object that has an x, a y, and a z.', 'start': 281.924, 'duration': 7.983}, {'end': 295.229, 'text': 'I want to draw the vertex at point.x, point.y, and now point.y.', 'start': 289.907, 'duration': 5.322}, {'end': 300.725, 'text': "Point.z And in order to make this a little bit better, I'm going to skip fewer pixels.", 'start': 296.581, 'duration': 4.144}, {'end': 301.826, 'text': "I'm going to skip only four.", 'start': 300.745, 'duration': 1.081}, {'end': 303.327, 'text': "And I'm going to run this again.", 'start': 302.206, 'duration': 1.121}, {'end': 306.59, 'text': "And now you'll see here I am.", 'start': 303.347, 'duration': 3.243}, {'end': 307.631, 'text': 'This is the point cloud.', 'start': 306.79, 'duration': 0.841}, {'end': 309.452, 'text': 'This is me in three-dimensional space.', 'start': 307.651, 'duration': 1.801}, {'end': 313.476, 'text': "So if I zoom in on this, you can start to see what's going on.", 'start': 309.813, 'duration': 3.663}, {'end': 315.218, 'text': 'This over here, by the way, is the wall.', 'start': 313.536, 'duration': 1.682}, {'end': 317.48, 'text': "It's funny how I can put my hands on the wall.", 'start': 315.438, 'duration': 2.042}, {'end': 319.141, 'text': "It's almost as if I'm distorting the wall.", 'start': 317.5, 'duration': 1.641}, {'end': 320.883, 'text': "But really what I'm doing is I'm casting a shadow.", 'start': 319.181, 'duration': 1.702}], 'summary': "Program modified to draw a 3d point cloud, skipping only four pixels, resulting in a clearer visualization of the wall and the caster's shadow.", 'duration': 38.959, 'max_score': 281.924, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo281924.jpg'}], 'start': 1.087, 'title': 'Raw depth data exploration', 'summary': 'Explores differences in depth values between version 1 and version 2 kinect, and illustrates the generation of a point cloud in physical 3d space with depth values between 0 and 4500, to build a 3d model.', 'chapters': [{'end': 332.512, 'start': 1.087, 'title': 'Kinect raw depth data exploration', 'summary': 'Explores the raw depth data from the kinect, demonstrating the differences in depth values between the version 1 and version 2 kinect, and it illustrates the generation of a point cloud in physical 3d space, with depth values between 0 and 4500, to build a 3d model.', 'duration': 331.425, 'highlights': ['The Kinect version 2 provides depth values between 0 and 4,500, while the version 1 provides values between 0 and 2,048, showcasing the differences in depth data between the two versions.', 'The demonstration involves the creation of a point cloud in physical 3D space, using the raw depth data to represent the actual physical space seen by the Kinect, enabling the building of a 3D model.', 'The process involves converting raw depth values into actual physical space measurements using a function that utilizes parameters built into the hardware, yielding x, y, and depth values in millimeters.', 'The visualization of the point cloud reveals the physical representation of the space being observed by the Kinect, with the ability to manipulate and visualize the 3D model, providing a practical understanding of the raw depth data.', "The chapter illustrates the utilization of the raw depth data to generate a point cloud, enabling the visualization of physical 3D space and the creation of a 3D model, showcasing the practical application of the Kinect's capabilities."]}], 'duration': 331.425, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo1087.jpg', 'highlights': ['The Kinect version 2 provides depth values between 0 and 4,500, while the version 1 provides values between 0 and 2,048, showcasing the differences in depth data between the two versions.', 'The demonstration involves the creation of a point cloud in physical 3D space, using the raw depth data to represent the actual physical space seen by the Kinect, enabling the building of a 3D model.', 'The process involves converting raw depth values into actual physical space measurements using a function that utilizes parameters built into the hardware, yielding x, y, and depth values in millimeters.', 'The visualization of the point cloud reveals the physical representation of the space being observed by the Kinect, with the ability to manipulate and visualize the 3D model, providing a practical understanding of the raw depth data.', "The chapter illustrates the utilization of the raw depth data to generate a point cloud, enabling the visualization of physical 3D space and the creation of a 3D model, showcasing the practical application of the Kinect's capabilities."]}, {'end': 658.603, 'segs': [{'end': 359.97, 'src': 'embed', 'start': 332.753, 'weight': 0, 'content': [{'end': 336.576, 'text': 'But you can see here, this is now a visualization in three dimensional space.', 'start': 332.753, 'duration': 3.823}, {'end': 339.618, 'text': 'So you could connect these points with lines, you could color them.', 'start': 336.596, 'duration': 3.022}, {'end': 341.94, 'text': "There's a way of actually getting the RGB values.", 'start': 339.638, 'duration': 2.302}, {'end': 346.364, 'text': 'And so you could see like the colors that are on my shirt on these points as well.', 'start': 342.321, 'duration': 4.043}, {'end': 348.045, 'text': 'This is a road you could go down.', 'start': 346.624, 'duration': 1.421}, {'end': 351.446, 'text': 'And I find this road to be particularly interesting.', 'start': 349.065, 'duration': 2.381}, {'end': 356.528, 'text': "And you can see that I'm using just a simple y rotation.", 'start': 352.647, 'duration': 3.881}, {'end': 359.97, 'text': "So now I'm kind of like spinning around this image, which has now gone off screen.", 'start': 356.568, 'duration': 3.402}], 'summary': 'Visualization of data points in 3d space, with potential for color and rgb analysis.', 'duration': 27.217, 'max_score': 332.753, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo332753.jpg'}, {'end': 413.219, 'src': 'embed', 'start': 387.826, 'weight': 1, 'content': [{'end': 393.967, 'text': 'So what if I just wanted to even only get my hand right here? What I want to do is try to calibrate a threshold.', 'start': 387.826, 'duration': 6.141}, {'end': 398.689, 'text': "So what if I want the Kinect only to see the Kinect's over here, remember? So it's to the left of me.", 'start': 394.327, 'duration': 4.362}, {'end': 401.31, 'text': 'side that is your viewing.', 'start': 400.049, 'duration': 1.261}, {'end': 405.393, 'text': 'But what if I want to say only look at the pixels in between here and here?', 'start': 401.67, 'duration': 3.723}, {'end': 407.675, 'text': 'And that would conceivably get my hand right?', 'start': 405.674, 'duration': 2.001}, {'end': 408.636, 'text': 'How would I do that?', 'start': 407.875, 'duration': 0.761}, {'end': 413.219, 'text': 'How would I only look at the pixels between a certain minimum and a certain maximum?', 'start': 408.716, 'duration': 4.503}], 'summary': 'Desire to calibrate a threshold for kinect, focusing on specific pixels for hand detection.', 'duration': 25.393, 'max_score': 387.826, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo387826.jpg'}, {'end': 531.047, 'src': 'embed', 'start': 500.376, 'weight': 2, 'content': [{'end': 506.103, 'text': "And I'm going to create a blank image, which is the same as the width and height of the connect.", 'start': 500.376, 'duration': 5.727}, {'end': 510.954, 'text': "And it's an RGB image.", 'start': 509.673, 'duration': 1.281}, {'end': 515.177, 'text': 'So this is a function in Processing Create Image that just makes a blank image.', 'start': 511.374, 'duration': 3.803}, {'end': 524.943, 'text': "And then what I'm going to do right now is I am going to in here right here.", 'start': 516.236, 'duration': 8.707}, {'end': 531.047, 'text': "I'm going to say image.loadPixels, because I need to operate on the pixels of that image.", 'start': 524.943, 'duration': 6.104}], 'summary': 'Creating a blank rgb image with the same width and height as the connect and operating on its pixels.', 'duration': 30.671, 'max_score': 500.376, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo500376.jpg'}], 'start': 332.753, 'title': '3d visualization and image processing in depth', 'summary': 'Discusses visualization of 3d space, rgb manipulation, and calibrating a threshold to focus on specific pixels, aiming to extract information. it also covers creating images using processing create image, setting pixels based on raw depth, and filling pixels with specific color based on depth values to visualize the output.', 'chapters': [{'end': 478.238, 'start': 332.753, 'title': '3d visualization and depth values', 'summary': 'Discusses the visualization of 3d space, manipulation of rgb values, and the process of calibrating a threshold to focus on specific pixels, aiming to extract relevant information from a raw depth field.', 'duration': 145.485, 'highlights': ['The visualization of 3D space, manipulation of RGB values, and the potential to extract relevant information from a raw depth field.', 'Calibrating a threshold to focus on specific pixels, potentially isolating certain objects or body parts for further analysis.', 'The process of using a double nested loop to go through every depth value within a certain range, aiming to isolate specific pixels based on their depth values.']}, {'end': 658.603, 'start': 478.238, 'title': 'Image processing in depth test', 'summary': 'Discusses the process of creating an image using processing create image, setting pixels based on raw depth, and filling pixels on the screen with a specific color based on depth values, aiming to visualize the output of the code.', 'duration': 180.365, 'highlights': ["Creating a blank image using the same width and height of the connect and setting pixels based on the raw depth. The speaker creates a blank image with the same width and height of the connect using the function 'Create Image' in Processing and sets pixels in that image based on the raw depth.", 'Filling every pixel on the screen with a specific color based on the depth values to visualize the output of the code. The speaker fills every pixel on the screen based on the depth values, using conditional statements to set a purple color if the distance is between 300 and 1500, and black otherwise, aiming to visualize the output of the code.', "Discussing the process of creating an image and setting pixels based on the raw depth. The chapter covers the process of creating an image with the same width and height of the connect and setting pixels in that image based on the raw depth, using the 'Create Image' function in Processing."]}], 'duration': 325.85, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo332753.jpg', 'highlights': ['The visualization of 3D space, manipulation of RGB values, and the potential to extract relevant information from a raw depth field.', 'Calibrating a threshold to focus on specific pixels, potentially isolating certain objects or body parts for further analysis.', 'Creating a blank image using the same width and height of the connect and setting pixels based on the raw depth.']}, {'end': 997.121, 'segs': [{'end': 736.157, 'src': 'embed', 'start': 697.001, 'weight': 0, 'content': [{'end': 702.266, 'text': "So one thing you'll notice here is that the hands go away once you're about a foot and a half from the Kinect.", 'start': 697.001, 'duration': 5.265}, {'end': 707.031, 'text': "So what I really want is between about, I don't know, between 0 and maybe like 500.", 'start': 702.567, 'duration': 4.464}, {'end': 710.334, 'text': "So there's probably a better way for me to calibrate this.", 'start': 707.031, 'duration': 3.303}, {'end': 712.856, 'text': 'than just randomly picking numbers.', 'start': 711.235, 'duration': 1.621}, {'end': 714.237, 'text': "But let's give this a try.", 'start': 713.196, 'duration': 1.041}, {'end': 717.438, 'text': 'You can see nothing, nothing, nothing, nothing, nothing, nothing, nothing, nothing, nothing.', 'start': 714.257, 'duration': 3.181}, {'end': 718.679, 'text': "Oh, that didn't do us any good.", 'start': 717.558, 'duration': 1.121}, {'end': 723.902, 'text': "Whoops, that's not going to do me any good either.", 'start': 720.52, 'duration': 3.382}, {'end': 725.363, 'text': "Let's do between 200 and 1, 000.", 'start': 723.922, 'duration': 1.441}, {'end': 728.965, 'text': "So you can see here how I'm able to pick out only my hand.", 'start': 725.363, 'duration': 3.602}, {'end': 736.157, 'text': "Again, I've got this problem with the wall.", 'start': 734.376, 'duration': 1.781}], 'summary': 'Calibrating kinect for hand detection between 200 and 1000; issue with wall', 'duration': 39.156, 'max_score': 697.001, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo697001.jpg'}, {'end': 867.59, 'src': 'embed', 'start': 841.036, 'weight': 3, 'content': [{'end': 845.08, 'text': "and let's calibrate the maximum threshold like how far back am I seeing?", 'start': 841.036, 'duration': 4.044}, {'end': 848.363, 'text': "but the minimum needs to be higher and then I don't want to see too far back.", 'start': 845.08, 'duration': 3.283}, {'end': 850.205, 'text': 'So there we go.', 'start': 849.505, 'duration': 0.7}, {'end': 851.426, 'text': 'So this I feel like is good.', 'start': 850.225, 'duration': 1.201}, {'end': 855.727, 'text': "If I'm getting my hand right now, it's between about 480 and 827.", 'start': 851.846, 'duration': 3.881}, {'end': 859.488, 'text': "Only if I'm standing right here, of course.", 'start': 855.727, 'duration': 3.761}, {'end': 864.569, 'text': 'But you could design an interactive exhibit where you put some footprints on the floor and the person has to stand there.', 'start': 859.648, 'duration': 4.921}, {'end': 866.63, 'text': "So I'm now going to keep my hand.", 'start': 864.829, 'duration': 1.801}, {'end': 867.59, 'text': 'Boy, this is a long video.', 'start': 866.67, 'duration': 0.92}], 'summary': 'Calibrating maximum threshold for interactive exhibit, range: 480-827, potential footprints for positioning.', 'duration': 26.554, 'max_score': 841.036, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo841036.jpg'}, {'end': 912.738, 'src': 'embed', 'start': 882.917, 'weight': 4, 'content': [{'end': 884.218, 'text': "And I'm going to run this again.", 'start': 882.917, 'duration': 1.301}, {'end': 890.403, 'text': "And we can see I'm getting a pretty good tracking of my hand.", 'start': 884.238, 'duration': 6.165}, {'end': 897.389, 'text': "So one thing that I'm going to do now, of course, which I think would be useful, is try to get rid of this wall over here.", 'start': 891.184, 'duration': 6.205}, {'end': 904.394, 'text': 'So the wall is a bit of a problem, but I can do a little bit of a cheat here, I think, which is also to say if and.', 'start': 897.649, 'duration': 6.745}, {'end': 912.738, 'text': "And x is greater than, I don't know what, how many pixels do you think that was? That was probably about 75 pixels.", 'start': 907.001, 'duration': 5.737}], 'summary': 'Improving hand tracking by removing the wall, estimated 75 pixels', 'duration': 29.821, 'max_score': 882.917, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo882917.jpg'}, {'end': 990.6, 'src': 'embed', 'start': 958.231, 'weight': 5, 'content': [{'end': 960.793, 'text': 'How would I actually just find the center of my hand?', 'start': 958.231, 'duration': 2.562}, {'end': 966.916, 'text': 'So I could control a processing sketch now by moving my hand around or moving this hand around, or what if I do both hands?', 'start': 961.073, 'duration': 5.843}, {'end': 968.296, 'text': 'So how would I do that?', 'start': 967.176, 'duration': 1.12}, {'end': 971.157, 'text': "I feel like I'm some sort of magic person here.", 'start': 968.616, 'duration': 2.541}, {'end': 973.677, 'text': "So that's what I'm going to look at in the next video.", 'start': 971.797, 'duration': 1.88}, {'end': 979.218, 'text': "How do I find the center of my hand and control something else, like a little snake that's moving around the screen,", 'start': 973.937, 'duration': 5.281}, {'end': 980.958, 'text': 'or make a particle system come out of my hand?', 'start': 979.218, 'duration': 1.74}, {'end': 982.899, 'text': "We'll look at that in the next video.", 'start': 981.218, 'duration': 1.681}, {'end': 986.339, 'text': 'And another thing I want to look at is how would I find the top of my head?', 'start': 983.199, 'duration': 3.14}, {'end': 990.6, 'text': "So if I'm the human being here, how do I know if I'm bending down or standing up??", 'start': 986.699, 'duration': 3.901}], 'summary': 'Exploring how to find center of hand and control objects in processing sketch, next video will cover finding top of head.', 'duration': 32.369, 'max_score': 958.231, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo958231.jpg'}], 'start': 658.963, 'title': 'Calibrating kinect and hand tracking', 'summary': 'Delves into calibrating kinect for computer vision, addressing challenges in contour finding and obstacle handling, and discusses threshold calibration for hand tracking to achieve a clean outline and plan for future concepts.', 'chapters': [{'end': 736.157, 'start': 658.963, 'title': 'Calibrating kinect for computer vision', 'summary': 'Discusses the process of calibrating a kinect for computer vision, highlighting the challenges of finding contours and dealing with obstacles, such as walls, and the need for better setup and calibration for accurate detection.', 'duration': 77.194, 'highlights': ['The challenge of finding contours and dealing with obstacles, such as walls, is highlighted, emphasizing the need for a better setup and calibration for accurate detection.', 'The difficulty of maintaining detection of hands within a specific range from the Kinect, between 0 and 500, is discussed, indicating the need for improved calibration methods.', 'The process of calibrating the Kinect is described, including the trial and error of selecting specific ranges, such as between 200 and 1000, to accurately detect and isolate specific objects, like hands.']}, {'end': 997.121, 'start': 736.537, 'title': 'Threshold calibration and hand tracking', 'summary': "Discusses the process of calibrating minimum and maximum thresholds for hand tracking, achieving a clean outline of the hand and planning to explore further concepts like finding the hand's center and detecting body movements in the next video.", 'duration': 260.584, 'highlights': ['The narrator calibrates the minimum and maximum thresholds to 480 and 830 to achieve a good tracking of the hand.', 'A cheat method is used to eliminate a wall by restricting tracking beyond a certain distance, improving the accuracy of hand tracking.', 'The concept of finding the center of the hand and controlling elements on the screen is planned to be explored in the next video.', 'The narrator also intends to explore detecting body movements, such as determining if a person is bending down or standing up, in the upcoming video.']}], 'duration': 338.158, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/E1eIg54clGo/pics/E1eIg54clGo658963.jpg', 'highlights': ['Calibrating Kinect for accurate contour finding and obstacle handling.', 'Improving calibration methods for maintaining hand detection within a specific range.', 'Selecting specific ranges for accurate detection and isolation of objects like hands.', 'Calibrating minimum and maximum thresholds to 480 and 830 for hand tracking.', 'Using a cheat method to improve accuracy of hand tracking by eliminating obstacles.', 'Planning to explore finding the center of the hand and controlling elements on the screen.', 'Intending to explore detecting body movements like bending down or standing up.']}], 'highlights': ['The Kinect version 2 provides depth values between 0 and 4,500, while the version 1 provides values between 0 and 2,048, showcasing the differences in depth data between the two versions.', 'The demonstration involves the creation of a point cloud in physical 3D space, using the raw depth data to represent the actual physical space seen by the Kinect, enabling the building of a 3D model.', 'The process involves converting raw depth values into actual physical space measurements using a function that utilizes parameters built into the hardware, yielding x, y, and depth values in millimeters.', 'The visualization of the point cloud reveals the physical representation of the space being observed by the Kinect, with the ability to manipulate and visualize the 3D model, providing a practical understanding of the raw depth data.', "The chapter illustrates the utilization of the raw depth data to generate a point cloud, enabling the visualization of physical 3D space and the creation of a 3D model, showcasing the practical application of the Kinect's capabilities.", 'Calibrating a threshold to focus on specific pixels, potentially isolating certain objects or body parts for further analysis.', 'Creating a blank image using the same width and height of the connect and setting pixels based on the raw depth.', 'Calibrating Kinect for accurate contour finding and obstacle handling.', 'Improving calibration methods for maintaining hand detection within a specific range.', 'Selecting specific ranges for accurate detection and isolation of objects like hands.', 'Calibrating minimum and maximum thresholds to 480 and 830 for hand tracking.', 'Using a cheat method to improve accuracy of hand tracking by eliminating obstacles.', 'Planning to explore finding the center of the hand and controlling elements on the screen.', 'Intending to explore detecting body movements like bending down or standing up.']}