title
12.1: What is the Kinect? - Kinect and Processing Tutorial

description
In this video I discuss how to get started working with the Microsoft Kinect in Processing using the Open Kinect for Processing library. I look at the different versions of the kinect -- 1414, 1473, v1 vs v2, and discuss in basic terms how the kinect itself works. I discuss how the kinect works in the context of computer vision applications. All examples: https://github.com/shiffman/Video-Lesson-Materials Open Kinect links: https://github.com/shiffman/OpenKinect-for-Processing http://shiffman.net/p5/kinect/ Contact: https://twitter.com/shiffman Next video: https://youtu.be/FBmxc4EyVjs Learn Processing from scratch: https://www.youtube.com/user/shiffman/playlists?sort=dd&shelf_id=2&view=50 More about image processing in Processing: https://www.youtube.com/playlist?list=PLRqwX-V7Uu6YB9x6f23CBftiyx0u_5sO9 Help us caption & translate this video! http://amara.org/v/Qbti/ 📄 Code of Conduct: https://github.com/CodingTrain/Code-of-Conduct

detail
{'title': '12.1: What is the Kinect? - Kinect and Processing Tutorial', 'heatmap': [{'end': 835.372, 'start': 784.69, 'weight': 0.708}], 'summary': 'This tutorial series introduces microsoft kinect, covering its different versions, programming environments, and the intention to delve into code in the next video. it also discusses the development of open connect, depth-sensing capabilities, rgb camera, and its applications in computer vision. additionally, it explores kinect for windows, issues with model 1473, and the significant upgrades in kinect version 2, including increased accuracy, faster processing, and higher resolution rgb cameras. the chapter also covers running the schiffman open connect library, setting up the kinect library for processing, and capabilities of the microsoft official sdk for depth and rgb data analysis.', 'chapters': [{'end': 136.759, 'segs': [{'end': 35.289, 'src': 'embed', 'start': 1.533, 'weight': 0, 'content': [{'end': 6.109, 'text': "Hello, this is the first video in a series of videos I'm making about the Microsoft Connect.", 'start': 1.533, 'duration': 4.576}, {'end': 7.545, 'text': 'This thing.', 'start': 7.065, 'duration': 0.48}, {'end': 9.427, 'text': 'So what is this thing??', 'start': 8.246, 'duration': 1.181}, {'end': 10.088, 'text': 'How does it work?', 'start': 9.487, 'duration': 0.601}, {'end': 12.87, 'text': 'And how do you write your own software that makes use of this thing??', 'start': 10.208, 'duration': 2.662}, {'end': 15.272, 'text': 'How can you do all sorts of creative coding projects?', 'start': 12.91, 'duration': 2.362}, {'end': 21.477, 'text': "Now there's a lot of different programming languages and environments, and frameworks and libraries for how you might use the Connect.", 'start': 15.692, 'duration': 5.785}, {'end': 30.725, 'text': "I'm going to use this thing called Processing, Processing 3, the third edition version of Processing, which is a Java-based programming environment,", 'start': 22.198, 'duration': 8.527}, {'end': 35.289, 'text': 'open source environment, that there is a Connect several different Connect libraries for.', 'start': 30.725, 'duration': 4.564}], 'summary': 'Introduction to microsoft connect and how to utilize it for creative coding projects using processing 3, a java-based programming environment.', 'duration': 33.756, 'max_score': 1.533, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM1533.jpg'}, {'end': 155.405, 'src': 'embed', 'start': 120.588, 'weight': 2, 'content': [{'end': 122.329, 'text': "But hopefully, I'll get things loosely right.", 'start': 120.588, 'duration': 1.741}, {'end': 131.475, 'text': 'So the original Kinect version 1 model 1414 is the one that came out, I think it was November 2011.', 'start': 122.97, 'duration': 8.505}, {'end': 132.175, 'text': 'somewhere around there.', 'start': 131.475, 'duration': 0.7}, {'end': 133.616, 'text': 'I remember the weekend it came out.', 'start': 132.175, 'duration': 1.441}, {'end': 136.759, 'text': 'people were quote, unquote, hacking it, but really just making.', 'start': 133.616, 'duration': 3.143}, {'end': 144.685, 'text': 'by hacking it I mean making open source drivers to read the data, a driver being a thing that your computer runs to talk to the hardware device.', 'start': 136.759, 'duration': 7.926}, {'end': 155.405, 'text': 'And so when that came out, a library, I worked on a library called Open Connect for processing,', 'start': 145.846, 'duration': 9.559}], 'summary': 'The original kinect version 1 model 1414 was released in november 2011, and people started making open source drivers for it, such as the open connect library.', 'duration': 34.817, 'max_score': 120.588, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM120588.jpg'}], 'start': 1.533, 'title': 'Introduction to microsoft connect', 'summary': 'Provides an overview of microsoft connect, discussing different versions, programming environments, and the intention to delve into code in the next video.', 'chapters': [{'end': 136.759, 'start': 1.533, 'title': 'Introduction to microsoft connect', 'summary': 'Provides an overview of the microsoft connect, discussing the different versions available, the programming environments for using the connect, and the intention to delve into code in the next video.', 'duration': 135.226, 'highlights': ['The chapter provides an overview of the Microsoft Connect, discussing the different versions available, the programming environments for using the Connect, and the intention to delve into code in the next video. The chapter covers the overview of Microsoft Connect, including different versions, programming environments, and plans for future video content.', 'The speaker introduces Processing 3, a Java-based programming environment for the Connect, and mentions plans to explore p5.js, a JavaScript framework for creative coding in the browser. Introduction of Processing 3 as a Java-based programming environment for the Connect and plans to explore p5.js for creative coding in the browser.', 'The speaker discusses the two key versions of the Kinect - version 1 and version 2 - and acknowledges the possibility of inaccuracies in the information provided. Discussion of the two key versions of Kinect - version 1 and version 2 - and acknowledgment of potential inaccuracies.']}], 'duration': 135.226, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM1533.jpg', 'highlights': ['The chapter provides an overview of the Microsoft Connect, including different versions, programming environments, and plans for future video content.', 'Introduction of Processing 3 as a Java-based programming environment for the Connect and plans to explore p5.js for creative coding in the browser.', 'Discussion of the two key versions of Kinect - version 1 and version 2 - and acknowledgment of potential inaccuracies.']}, {'end': 478.595, 'segs': [{'end': 187.932, 'src': 'embed', 'start': 162.332, 'weight': 3, 'content': [{'end': 170.2, 'text': 'an open source driver for connecting to the Kinect, which is also known as a libfreenet.', 'start': 162.332, 'duration': 7.868}, {'end': 175.324, 'text': 'So this is sort of the genesis of all of this.', 'start': 172.842, 'duration': 2.482}, {'end': 182.489, 'text': 'The thing that I built for processing is just a thin layer on top of work that lots and lots of other people did,', 'start': 176.244, 'duration': 6.245}, {'end': 184.83, 'text': 'which allows you to get the data from the Kinect.', 'start': 182.489, 'duration': 2.341}, {'end': 186.511, 'text': "Now, let's come back to the Kinect.", 'start': 184.89, 'duration': 1.621}, {'end': 187.932, 'text': "It's over here.", 'start': 186.571, 'duration': 1.361}], 'summary': 'Open source driver libfreenect connects to kinect for data access.', 'duration': 25.6, 'max_score': 162.332, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM162332.jpg'}, {'end': 328.507, 'src': 'embed', 'start': 300.626, 'weight': 0, 'content': [{'end': 307.591, 'text': 'By analyzing that distortion, the connect can recognize what things are closer and what things are further away.', 'start': 300.626, 'duration': 6.965}, {'end': 313.996, 'text': 'So the value of this is often referred to, you can think of it as a depth camera or a depth sensor.', 'start': 307.952, 'duration': 6.044}, {'end': 317.379, 'text': 'This is what this infrared projector and infrared camera are doing.', 'start': 314.356, 'duration': 3.023}, {'end': 321.802, 'text': "They're measuring the depth of each pixel in the room.", 'start': 317.719, 'duration': 4.083}, {'end': 328.507, 'text': "So while a regular web camera says, here's a 640 by 480 image, each pixel has a red, green, and blue value.", 'start': 321.842, 'duration': 6.665}], 'summary': 'Infrared projector and camera measure depth of each pixel, enabling recognition of closeness and distance.', 'duration': 27.881, 'max_score': 300.626, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM300626.jpg'}, {'end': 478.595, 'src': 'embed', 'start': 431.826, 'weight': 1, 'content': [{'end': 439.53, 'text': 'So the infrared camera sees the depth of a given pixel at a different place that the RGB camera sees that color.', 'start': 431.826, 'duration': 7.704}, {'end': 447.334, 'text': "So this is an alignment problem, a calibration problem, where the color pixels don't necessarily line up exactly with the depth pixels.", 'start': 439.83, 'duration': 7.504}, {'end': 456.721, 'text': 'And there are lots of strategies for solving this problem and lots of frameworks and libraries, in particular the official Microsoft SDK,', 'start': 447.834, 'duration': 8.887}, {'end': 459.863, 'text': 'which has things baked into it that do this for you.', 'start': 456.721, 'duration': 3.142}, {'end': 467.428, 'text': "But one of the nice things that we'll see once we get to the Kinect version 2 is it has something called a registered image,", 'start': 460.083, 'duration': 7.345}, {'end': 471.872, 'text': 'which is an image that aligns the depth pixels with the color pixels.', 'start': 467.428, 'duration': 4.444}, {'end': 473.833, 'text': 'OK, so this is what the Kinect does.', 'start': 472.092, 'duration': 1.741}, {'end': 478.595, 'text': 'And I really described here what the Connect version 1 does.', 'start': 475.294, 'duration': 3.301}], 'summary': 'The alignment problem between color and depth pixels in kinect is addressed by the registered image in kinect version 2.', 'duration': 46.769, 'max_score': 431.826, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM431826.jpg'}], 'start': 136.759, 'title': 'Kinect sensor', 'summary': 'Describes the development of open connect, an open-source driver for connecting to the kinect, explaining its depth-sensing capabilities and rgb camera, highlighting its applications in computer vision and alignment challenges between depth and color pixels.', 'chapters': [{'end': 478.595, 'start': 136.759, 'title': 'Kinect sensor: depth and color analysis', 'summary': 'Describes the development of open connect, an open-source driver for connecting to the kinect, explaining the functionality of the kinect sensor, including its depth-sensing capabilities and rgb camera, highlighting its applications in computer vision and the alignment challenges between depth and color pixels.', 'duration': 341.836, 'highlights': ['The development of Open Connect, an open-source driver for connecting to the Kinect, enabling data reading and processing. N/A', "Explanation of the Kinect sensor's depth-sensing capabilities through its infrared projector and camera, allowing measurement of the depth of each pixel in the room. N/A", "Highlighting the value of Kinect's depth-sensing capabilities in computer vision, including background removal, shape recognition, and 3D scanning. N/A", 'Discussion of the alignment problem between the RGB camera and the depth pixels in the Kinect sensor, and strategies for solving this challenge. N/A']}], 'duration': 341.836, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM136759.jpg', 'highlights': ["Explanation of the Kinect sensor's depth-sensing capabilities through its infrared projector and camera, allowing measurement of the depth of each pixel in the room.", "Highlighting the value of Kinect's depth-sensing capabilities in computer vision, including background removal, shape recognition, and 3D scanning.", 'Discussion of the alignment problem between the RGB camera and the depth pixels in the Kinect sensor, and strategies for solving this challenge.', 'The development of Open Connect, an open-source driver for connecting to the Kinect, enabling data reading and processing.']}, {'end': 748.328, 'segs': [{'end': 523.159, 'src': 'embed', 'start': 478.855, 'weight': 4, 'content': [{'end': 483.817, 'text': "There was also model 1473 that came out, I don't know, a year or two later.", 'start': 478.855, 'duration': 4.962}, {'end': 486.158, 'text': 'This one has some problems.', 'start': 483.837, 'duration': 2.321}, {'end': 491.54, 'text': "In particular, there's a little bit of a bug currently with running it with the processing library, although it does work.", 'start': 486.238, 'duration': 5.302}, {'end': 493.641, 'text': 'Kind of only will work every other time.', 'start': 491.84, 'duration': 1.801}, {'end': 495.141, 'text': "Can't figure it out for the life of me.", 'start': 493.741, 'duration': 1.4}, {'end': 499.123, 'text': 'But both of these will work with the library.', 'start': 496.762, 'duration': 2.361}, {'end': 502.184, 'text': 'What you need to look for with the library is the version 1 examples.', 'start': 499.283, 'duration': 2.901}, {'end': 504.345, 'text': "So that's that.", 'start': 502.984, 'duration': 1.361}, {'end': 508.168, 'text': 'Now, in between here, there was this Kinect for Windows.', 'start': 504.625, 'duration': 3.543}, {'end': 514.251, 'text': 'And I think this was a version of the Kinect that Microsoft made to plug into Windows computers.', 'start': 509.249, 'duration': 5.002}, {'end': 520.236, 'text': 'Originally, this was designed for use with the Xbox for games that you would play by dancing.', 'start': 514.513, 'duration': 5.723}, {'end': 523.159, 'text': "I'm kicking my leg, by the way, if you can't see that.", 'start': 521.298, 'duration': 1.861}], 'summary': 'Model 1473 had some issues with the processing library, and kinect for windows was designed for xbox games before being adapted for windows computers.', 'duration': 44.304, 'max_score': 478.855, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM478855.jpg'}, {'end': 589.538, 'src': 'embed', 'start': 546.312, 'weight': 0, 'content': [{'end': 551.755, 'text': 'The Kinect version two is a newer and quite significant upgrade from the first Kinect.', 'start': 546.312, 'duration': 5.443}, {'end': 555.977, 'text': 'and it actually uses a completely different technique.', 'start': 553.255, 'duration': 2.722}, {'end': 559.399, 'text': 'It uses infrared light, but it uses a technique called time of flight.', 'start': 556.037, 'duration': 3.362}, {'end': 566.062, 'text': 'So it sends the infrared light out, measures how long it takes for it to bounce back and how long that takes.', 'start': 559.779, 'duration': 6.283}, {'end': 568.363, 'text': 'lets the sensor know how far away things are.', 'start': 566.062, 'duration': 2.301}, {'end': 574.346, 'text': "Kind of like a bat maybe does stuff with sound to see, I don't know, dolphins do stuff like that, but all with sound.", 'start': 568.423, 'duration': 5.923}, {'end': 577.728, 'text': 'So with light bouncing it back and forth, the new Kinect does that.', 'start': 574.586, 'duration': 3.142}, {'end': 586.575, 'text': "And I suppose it's, a bit more accurate, it's faster, and the RGB cameras also in the new Kinect is higher resolution.", 'start': 577.948, 'duration': 8.627}, {'end': 589.538, 'text': "Okay, so that's the basics overview.", 'start': 587.276, 'duration': 2.262}], 'summary': 'The kinect v2 uses time of flight technique with infrared light, providing higher accuracy and faster performance.', 'duration': 43.226, 'max_score': 546.312, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM546312.jpg'}, {'end': 629.522, 'src': 'embed', 'start': 608.153, 'weight': 7, 'content': [{'end': 619.258, 'text': "And you can see now that what you're seeing in this particular image is an example of a processing sketch which is rendering all of the pieces of what the library,", 'start': 608.153, 'duration': 11.105}, {'end': 620.198, 'text': 'what the Kinect offers.', 'start': 619.258, 'duration': 0.94}, {'end': 624.1, 'text': "Now, oh, but I have something more to mention about this, but I'll get to that in a second.", 'start': 620.498, 'duration': 3.602}, {'end': 627.781, 'text': "So up top, you can see that's just the RGB image.", 'start': 624.34, 'duration': 3.441}, {'end': 629.522, 'text': "So it's like I have a webcam over here.", 'start': 628.021, 'duration': 1.501}], 'summary': "An example of a processing sketch rendering kinect's features, including the rgb image.", 'duration': 21.369, 'max_score': 608.153, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM608153.jpg'}, {'end': 677.96, 'src': 'embed', 'start': 650.667, 'weight': 2, 'content': [{'end': 655.071, 'text': 'So mostly it just looks like this creepy thing, but you can make use of that, you can get that image as well.', 'start': 650.667, 'duration': 4.404}, {'end': 659.754, 'text': "Up top, up the top right, this is what's known as the depth image.", 'start': 656.151, 'duration': 3.603}, {'end': 663.315, 'text': 'So what the connect is measuring is in millimeters.', 'start': 660.234, 'duration': 3.081}, {'end': 666.956, 'text': "It's measuring a value between 0 and 4500.", 'start': 663.615, 'duration': 3.341}, {'end': 673.519, 'text': 'How far is the thing away from the camera? And then often a depth image is used to visualize that data.', 'start': 666.956, 'duration': 6.563}, {'end': 677.96, 'text': 'So in this case you can see as I start to go further and further back, I get brighter.', 'start': 673.739, 'duration': 4.221}], 'summary': 'The kinect measures depth in millimeters, ranging from 0 to 4500, to determine the distance from the camera.', 'duration': 27.293, 'max_score': 650.667, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM650667.jpg'}, {'end': 736.639, 'src': 'embed', 'start': 709.879, 'weight': 3, 'content': [{'end': 713.902, 'text': 'And then down in the bottom right, this is the registered image.', 'start': 709.879, 'duration': 4.023}, {'end': 719.386, 'text': 'So this is not part of, if you use the version one, connect with the Open Connect library.', 'start': 714.142, 'duration': 5.244}, {'end': 720.947, 'text': 'This is not part of that.', 'start': 719.727, 'duration': 1.22}, {'end': 730.635, 'text': 'However, with version 2, this is the image that aligns all of the RGB values from the webcam with the depth values.', 'start': 721.268, 'duration': 9.367}, {'end': 736.639, 'text': 'So, if you wanted to hopefully something I might be able to demonstrate in some video is just do background removal, where you see only me,', 'start': 730.675, 'duration': 5.964}], 'summary': 'In version 2, the registered image aligns rgb webcam values with depth values, enabling background removal.', 'duration': 26.76, 'max_score': 709.879, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM709879.jpg'}], 'start': 478.855, 'title': 'Kinect for windows and kinect version 2', 'summary': "Discusses the issues and compatibility of model 1473 with kinect for windows, microsoft's shift in design, and the significant upgrades in kinect version 2, including increased accuracy, faster processing, and higher resolution rgb cameras.", 'chapters': [{'end': 545.892, 'start': 478.855, 'title': 'Kinect for windows and model 1473', 'summary': "Discusses the issues and compatibility of model 1473 and the kinect for windows with the processing library, and microsoft's shift in design for use with regular computers.", 'duration': 67.037, 'highlights': ['The Kinect for Windows was originally designed for the Xbox but was later adapted for regular computers, though its compatibility with the processing library is uncertain.', 'The model 1473 has some issues when running with the processing library, working only every other time, indicating compatibility problems.', 'The Kinect for Windows was initially designed for the Xbox for dancing games, showing its original purpose before being adapted for regular computers.']}, {'end': 748.328, 'start': 546.312, 'title': 'Upgrade to kinect version 2', 'summary': 'Discusses the significant upgrade from the first kinect to kinect version 2, which uses infrared light and a technique called time of flight, resulting in increased accuracy, faster processing, and higher resolution rgb cameras, while also providing detailed insights into the different types of images and functionalities of the new kinect.', 'duration': 202.016, 'highlights': ['The new Kinect version 2 uses infrared light and time of flight technique to measure how long it takes for the light to bounce back, resulting in increased accuracy and faster processing.', 'The RGB cameras in Kinect version 2 offer higher resolution compared to the previous version, enhancing the overall visual quality.', 'The chapter provides a detailed explanation of the various types of images captured by Kinect version 2, including RGB, raw infrared feed, depth image, and registered image, along with their functionalities and applications.', 'The depth image captured by Kinect version 2 measures values in millimeters between 0 and 4500, allowing visualization of distances and facilitating tasks such as background removal.', 'The registered image in Kinect version 2 aligns RGB values with depth values, enabling functionalities like background removal for enhanced visual applications.']}], 'duration': 269.473, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM478855.jpg', 'highlights': ['The new Kinect version 2 uses infrared light and time of flight technique to measure how long it takes for the light to bounce back, resulting in increased accuracy and faster processing.', 'The RGB cameras in Kinect version 2 offer higher resolution compared to the previous version, enhancing the overall visual quality.', 'The depth image captured by Kinect version 2 measures values in millimeters between 0 and 4500, allowing visualization of distances and facilitating tasks such as background removal.', 'The registered image in Kinect version 2 aligns RGB values with depth values, enabling functionalities like background removal for enhanced visual applications.', 'The model 1473 has some issues when running with the processing library, working only every other time, indicating compatibility problems.', 'The Kinect for Windows was originally designed for the Xbox for dancing games, showing its original purpose before being adapted for regular computers.', 'The Kinect for Windows was initially designed for the Xbox but was later adapted for regular computers, though its compatibility with the processing library is uncertain.', 'The chapter provides a detailed explanation of the various types of images captured by Kinect version 2, including RGB, raw infrared feed, depth image, and registered image, along with their functionalities and applications.']}, {'end': 991.567, 'segs': [{'end': 835.372, 'src': 'heatmap', 'start': 749.949, 'weight': 2, 'content': [{'end': 754.572, 'text': 'So what I want to show you now is how do you get this library to run this particular example.', 'start': 749.949, 'duration': 4.623}, {'end': 755.813, 'text': 'So a couple of things.', 'start': 755.092, 'duration': 0.721}, {'end': 759.135, 'text': "One is, I'll put all this in the description of the video.", 'start': 756.073, 'duration': 3.062}, {'end': 760.255, 'text': 'This is the URL.', 'start': 759.395, 'duration': 0.86}, {'end': 765.258, 'text': 'The library is at github.com slash Schiffman open connect for processing.', 'start': 760.976, 'duration': 4.282}, {'end': 767.82, 'text': "You don't need to go to that URL, but that's where the source code is.", 'start': 765.278, 'duration': 2.542}, {'end': 769.921, 'text': "There's a little bit of documentation there.", 'start': 768.24, 'duration': 1.681}, {'end': 774.884, 'text': 'I want to give a big thank you to Thomas Sanchez Lengeling.', 'start': 770.221, 'duration': 4.663}, {'end': 776.665, 'text': 'I might not have pronounced his last name correctly.', 'start': 774.984, 'duration': 1.681}, {'end': 780.747, 'text': 'He wrote all the code for making this library work with the Kinect version 2.', 'start': 776.805, 'duration': 3.942}, {'end': 784.63, 'text': 'So I worked on the version 1 a number of years ago and sort of floundered.', 'start': 780.747, 'duration': 3.883}, {'end': 788.072, 'text': 'And Thomas came back and revived this and really helped over the summer.', 'start': 784.69, 'duration': 3.382}, {'end': 793.255, 'text': 'And there is also, I have a page that has some additional documentation.', 'start': 789.112, 'duration': 4.143}, {'end': 796.236, 'text': "It's Schiffman.net slash P5 slash Kinect.", 'start': 793.695, 'duration': 2.541}, {'end': 804.964, 'text': "This is some text that kind of goes through the different versions and some of these examples as well that I'm going to cover in the videos.", 'start': 797.617, 'duration': 7.347}, {'end': 813.052, 'text': "Now, in order to get the actual library itself, what you need to do is go to, first you need to download processing if you don't have that already.", 'start': 805.365, 'duration': 7.687}, {'end': 814.413, 'text': "That's at processing.org.", 'start': 813.092, 'duration': 1.321}, {'end': 820.88, 'text': "Then what you'll need to do is once you have processing, it might look just like this to you, something empty.", 'start': 814.914, 'duration': 5.966}, {'end': 821.861, 'text': "You're going to go to Sketch.", 'start': 820.9, 'duration': 0.961}, {'end': 825.566, 'text': 'Import Library, Add Library.', 'start': 823.405, 'duration': 2.161}, {'end': 828.028, 'text': 'Now, you can see that I have three libraries here.', 'start': 825.626, 'duration': 2.402}, {'end': 831.45, 'text': "I already have that library, but I'm going to pretend that I don't for a second.", 'start': 828.048, 'duration': 3.402}, {'end': 835.372, 'text': "I'm going to go to Add Library, which opens up this Contributions Manager.", 'start': 831.79, 'duration': 3.582}], 'summary': 'Demonstration of setting up library for kinect with processing, including urls and instructions.', 'duration': 38.123, 'max_score': 749.949, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM749949.jpg'}, {'end': 828.028, 'src': 'embed', 'start': 805.365, 'weight': 1, 'content': [{'end': 813.052, 'text': "Now, in order to get the actual library itself, what you need to do is go to, first you need to download processing if you don't have that already.", 'start': 805.365, 'duration': 7.687}, {'end': 814.413, 'text': "That's at processing.org.", 'start': 813.092, 'duration': 1.321}, {'end': 820.88, 'text': "Then what you'll need to do is once you have processing, it might look just like this to you, something empty.", 'start': 814.914, 'duration': 5.966}, {'end': 821.861, 'text': "You're going to go to Sketch.", 'start': 820.9, 'duration': 0.961}, {'end': 825.566, 'text': 'Import Library, Add Library.', 'start': 823.405, 'duration': 2.161}, {'end': 828.028, 'text': 'Now, you can see that I have three libraries here.', 'start': 825.626, 'duration': 2.402}], 'summary': 'To obtain the library, download processing from processing.org and import the library through sketch.', 'duration': 22.663, 'max_score': 805.365, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM805365.jpg'}, {'end': 872.391, 'src': 'embed', 'start': 848.398, 'weight': 4, 'content': [{'end': 856.544, 'text': 'OpenNI was an open source platform, open source framework for doing skeleton tracking, meaning finding the human form where the hands are,', 'start': 848.398, 'duration': 8.146}, {'end': 859.927, 'text': 'where the head is, which is very, very powerful, and things that you can do with the Connect.', 'start': 856.544, 'duration': 3.383}, {'end': 861.929, 'text': "I'm starting with just the raw depth data.", 'start': 860.007, 'duration': 1.922}, {'end': 866.91, 'text': 'But OpenAI, I think, was purchased by Apple and then shut down as an open thing.', 'start': 863.069, 'duration': 3.841}, {'end': 868.911, 'text': 'But there are some efforts to revive it.', 'start': 867.19, 'duration': 1.721}, {'end': 872.391, 'text': "And so you could Google around, and that's something that you could possibly use.", 'start': 869.291, 'duration': 3.1}], 'summary': 'Openni, an open source platform for skeleton tracking, was powerful. openai was purchased by apple and shut down, but there are efforts to revive it.', 'duration': 23.993, 'max_score': 848.398, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM848398.jpg'}, {'end': 914.63, 'src': 'embed', 'start': 885.475, 'weight': 0, 'content': [{'end': 889.096, 'text': "And I'm going to demonstrate that using a PC in a later video.", 'start': 885.475, 'duration': 3.621}, {'end': 899.201, 'text': 'This is a key, this is a really great thing to use if you want to get all of the magic that Microsoft has spent all this time developing.', 'start': 891.236, 'duration': 7.965}, {'end': 904.224, 'text': 'So what the Kinect just gives you is raw depth data, raw RGB data.', 'start': 899.521, 'duration': 4.703}, {'end': 911.589, 'text': "but what the Microsoft SDK does is it pulls that data in and on and it analyzes it and finds where's the human being like.", 'start': 904.224, 'duration': 7.365}, {'end': 914.63, 'text': 'what kind of muscle are they making? Like, where is their head??', 'start': 911.589, 'duration': 3.041}], 'summary': 'Demonstrating use of pc, kinect provides raw depth and rgb data, microsoft sdk analyzes and identifies human movements.', 'duration': 29.155, 'max_score': 885.475, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM885475.jpg'}], 'start': 749.949, 'title': 'Running and setting up kinect libraries', 'summary': 'Covers running the schiffman open connect library and setting up the kinect library for processing. it includes the source code url, installation process, and capabilities of the microsoft official sdk for depth and rgb data analysis.', 'chapters': [{'end': 804.964, 'start': 749.949, 'title': 'Running schiffman open connect library', 'summary': 'Discusses running the schiffman open connect library for processing, including the url for the source code, a thank you to thomas sanchez lengeling for writing the code, and additional documentation available at schiffman.net/p5/kinect.', 'duration': 55.015, 'highlights': ['Thomas Sanchez Lengeling wrote all the code for making the library work with the Kinect version 2, providing significant help over the summer.', "The library's source code is available at github.com/Schiffman/openconnect for processing, and additional documentation can be found at Schiffman.net/p5/Kinect."]}, {'end': 991.567, 'start': 805.365, 'title': 'Setting up kinect library for processing', 'summary': 'Explains how to set up the kinect library for processing, including the process of downloading, installing, and using the library, and the capabilities of the microsoft official sdk for analyzing raw depth and rgb data.', 'duration': 186.202, 'highlights': ['The process of downloading, installing, and using the Kinect library for Processing is explained, with a demonstration of the Contributions Manager and the installation steps. Demonstrates the process of downloading and installing the library.', 'The capabilities of the Microsoft official SDK for analyzing raw depth and RGB data are highlighted, including analyzing human form, muscle movements, and hand gestures. Highlights the advanced analysis capabilities of the Microsoft official SDK.', 'The potential use of the Simple OpenNI library for skeleton tracking and its compatibility issues with Processing 3 are mentioned. Mentions the potential use and compatibility issues of the Simple OpenNI library.']}], 'duration': 241.618, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/QmVNgdapJJM/pics/QmVNgdapJJM749949.jpg', 'highlights': ['The capabilities of the Microsoft official SDK for analyzing raw depth and RGB data are highlighted, including analyzing human form, muscle movements, and hand gestures.', 'The process of downloading, installing, and using the Kinect library for Processing is explained, with a demonstration of the Contributions Manager and the installation steps.', 'Thomas Sanchez Lengeling wrote all the code for making the library work with the Kinect version 2, providing significant help over the summer.', "The library's source code is available at github.com/Schiffman/openconnect for processing, and additional documentation can be found at Schiffman.net/p5/Kinect.", 'The potential use of the Simple OpenNI library for skeleton tracking and its compatibility issues with Processing 3 are mentioned.']}], 'highlights': ['The new Kinect version 2 uses infrared light and time of flight technique to measure how long it takes for the light to bounce back, resulting in increased accuracy and faster processing.', 'The RGB cameras in Kinect version 2 offer higher resolution compared to the previous version, enhancing the overall visual quality.', 'The depth image captured by Kinect version 2 measures values in millimeters between 0 and 4500, allowing visualization of distances and facilitating tasks such as background removal.', 'The registered image in Kinect version 2 aligns RGB values with depth values, enabling functionalities like background removal for enhanced visual applications.', 'The capabilities of the Microsoft official SDK for analyzing raw depth and RGB data are highlighted, including analyzing human form, muscle movements, and hand gestures.', 'The process of downloading, installing, and using the Kinect library for Processing is explained, with a demonstration of the Contributions Manager and the installation steps.', 'The development of Open Connect, an open-source driver for connecting to the Kinect, enabling data reading and processing.', 'Introduction of Processing 3 as a Java-based programming environment for the Connect and plans to explore p5.js for creative coding in the browser.', 'The chapter provides a detailed explanation of the various types of images captured by Kinect version 2, including RGB, raw infrared feed, depth image, and registered image, along with their functionalities and applications.', 'The chapter provides an overview of the Microsoft Connect, including different versions, programming environments, and plans for future video content.']}