title
Python Requests Tutorial: Request Web Pages, Download Images, POST Data, Read JSON, and More

description
In this Python Programming Tutorial, we will be learning how to use the Requests library. The Requests library allows us to send HTTP requests and interact with web pages. We will be learning how to grab the source code of a site, download images, POST form data to routes, read JSON responses, perform authentication, and more. Let's get started... BeautifulSoup Tutorial - https://youtu.be/ng2o98k983k File Objects Tutorial - https://youtu.be/Uh2ebFW8OYM ✅ Support My Channel Through Patreon: https://www.patreon.com/coreyms ✅ Become a Channel Member: https://www.youtube.com/channel/UCCezIgC97PvUuR4_gbFUs5g/join ✅ One-Time Contribution Through PayPal: https://goo.gl/649HFY ✅ Cryptocurrency Donations: Bitcoin Wallet - 3MPH8oY2EAgbLVy7RBMinwcBntggi7qeG3 Ethereum Wallet - 0x151649418616068fB46C3598083817101d3bCD33 Litecoin Wallet - MPvEBY5fxGkmPQgocfJbxP6EmTo5UUXMot ✅ Corey's Public Amazon Wishlist http://a.co/inIyro1 ✅ Equipment I Use and Books I Recommend: https://www.amazon.com/shop/coreyschafer ▶️ You Can Find Me On: My Website - http://coreyms.com/ My Second Channel - https://www.youtube.com/c/coreymschafer Facebook - https://www.facebook.com/CoreyMSchafer Twitter - https://twitter.com/CoreyMSchafer Instagram - https://www.instagram.com/coreymschafer/ #Python #Requests

detail
{'title': 'Python Requests Tutorial: Request Web Pages, Download Images, POST Data, Read JSON, and More', 'heatmap': [{'end': 121.389, 'start': 87.761, 'weight': 0.918}, {'end': 152.925, 'start': 133.934, 'weight': 1}, {'end': 220.445, 'start': 174.939, 'weight': 0.806}, {'end': 496.052, 'start': 476.815, 'weight': 0.789}, {'end': 632.822, 'start': 582.226, 'weight': 0.832}, {'end': 722.138, 'start': 657.538, 'weight': 0.721}, {'end': 796.34, 'start': 764.078, 'weight': 0.72}, {'end': 918.516, 'start': 880.317, 'weight': 0.836}, {'end': 946.007, 'start': 930.278, 'weight': 0.744}, {'end': 1006.204, 'start': 967.943, 'weight': 0.889}, {'end': 1115.578, 'start': 1078.296, 'weight': 0.85}], 'summary': 'Tutorial explores python request library, demonstrating its capability to simplify http requests for getting and posting information, image download, authentication data, and following redirects. it also covers website content retrieval, monitoring website responses, testing http methods, setting timeouts, and basic authentication using the request library with practical examples.', 'chapters': [{'end': 91.402, 'segs': [{'end': 29.964, 'src': 'embed', 'start': 0.189, 'weight': 0, 'content': [{'end': 4.471, 'text': "Hey there, how's it going everybody? In this video we're going to take an in-depth look at the request library.", 'start': 0.189, 'duration': 4.282}, {'end': 10.274, 'text': 'So the request library allows us to easily make HTTP requests to get information from websites.', 'start': 4.831, 'duration': 5.443}, {'end': 14.416, 'text': "Now I believe it's the most downloaded Python package and there's a good reason for that.", 'start': 10.614, 'duration': 3.802}, {'end': 20.659, 'text': 'It does a really good job of taking tasks that used to be complicated and confusing and instead makes them easy and intuitive.', 'start': 14.776, 'duration': 5.883}, {'end': 24.841, 'text': "So that's why the tagline for the request library is HTTP for humans.", 'start': 20.979, 'duration': 3.862}, {'end': 29.964, 'text': "Now we've used the request library and a lot of previous videos to get information from websites,", 'start': 25.261, 'duration': 4.703}], 'summary': 'The request library is the most downloaded python package, simplifying http requests and making tasks easy and intuitive.', 'duration': 29.775, 'max_score': 0.189, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs189.jpg'}], 'start': 0.189, 'title': 'Python request library', 'summary': 'Explores the request library, highlighting its ability to simplify http requests, including getting and posting information, downloading images, sending authentication data, and following redirects.', 'chapters': [{'end': 91.402, 'start': 0.189, 'title': 'Understanding the python request library', 'summary': 'Explores the request library, the most downloaded python package, highlighting its ability to simplify http requests, including getting and posting information, downloading images, sending authentication data, and following redirects.', 'duration': 91.213, 'highlights': ['The request library is the most downloaded Python package, simplifying tasks that used to be complicated and confusing, making them easy and intuitive.', 'The library allows us to make HTTP requests to get information from websites and can perform tasks such as getting and posting information, downloading images, sending authentication information for basic logins and forms, and following redirects.', 'While the request library is excellent for getting information from a website, it is not intended for parsing that information, for which tools like Beautiful Soup or request-html can be used.']}], 'duration': 91.213, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs189.jpg', 'highlights': ['The request library simplifies tasks, making them easy and intuitive.', 'It allows making HTTP requests to get and post information, download images, send authentication data, and follow redirects.', 'The request library is not intended for parsing information; tools like Beautiful Soup or request-html can be used for that.']}, {'end': 450.038, 'segs': [{'end': 156.227, 'src': 'heatmap', 'start': 117.949, 'weight': 2, 'content': [{'end': 121.389, 'text': "So first, let's simply grab the contents of a website.", 'start': 117.949, 'duration': 3.44}, {'end': 125.71, 'text': "So I'm going to use the URL for an XKCD comic about Python.", 'start': 121.749, 'duration': 3.961}, {'end': 128.511, 'text': 'So I have this pulled up in my browser here.', 'start': 126.11, 'duration': 2.401}, {'end': 130.652, 'text': 'So if I go over to my page here.', 'start': 128.651, 'duration': 2.001}, {'end': 133.474, 'text': 'So this is just a funny web comic about Python.', 'start': 131.173, 'duration': 2.301}, {'end': 139.698, 'text': 'And if any of you have ever done import anti-gravity from a Python script, then this is the comic that pops up.', 'start': 133.934, 'duration': 5.764}, {'end': 144.42, 'text': "So let's say that I wanted to request this page through Python using requests.", 'start': 140.198, 'duration': 4.222}, {'end': 147.142, 'text': "So first, let's just copy this URL.", 'start': 144.82, 'duration': 2.322}, {'end': 149.563, 'text': "So I'm going to go up here and copy the URL.", 'start': 147.362, 'duration': 2.201}, {'end': 152.925, 'text': "And now let's go back to our empty module.", 'start': 150.123, 'duration': 2.802}, {'end': 156.227, 'text': 'and now I can first import the request library.', 'start': 153.345, 'duration': 2.882}], 'summary': 'Using requests to fetch the xkcd python comic from its url.', 'duration': 38.278, 'max_score': 117.949, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs117949.jpg'}, {'end': 220.445, 'src': 'heatmap', 'start': 174.939, 'weight': 0.806, 'content': [{'end': 180.281, 'text': 'So if I run that, then we can see that we get this response object.', 'start': 174.939, 'duration': 5.342}, {'end': 183.441, 'text': 'And that tells us that we got a 200 response.', 'start': 180.581, 'duration': 2.86}, {'end': 187.042, 'text': "So now let's see how we can get the content from this web page.", 'start': 183.882, 'duration': 3.16}, {'end': 192.263, 'text': 'So if you ever wanted to see all of the attributes and methods available to you from a specific object,', 'start': 187.422, 'duration': 4.841}, {'end': 196.224, 'text': 'you can simply run it through this dir function here.', 'start': 192.263, 'duration': 3.961}, {'end': 202.265, 'text': 'So if we run that object into dir, save that and run it.', 'start': 196.664, 'duration': 5.601}, {'end': 204.706, 'text': 'Let me make this a little larger here.', 'start': 202.445, 'duration': 2.261}, {'end': 210.012, 'text': 'then it should show us the attributes and methods that we can access within this response object.', 'start': 205.166, 'duration': 4.846}, {'end': 220.445, 'text': 'So you can see here that we have content, headers, isRedirect, JSON links, an OK status, and status code, and things like that.', 'start': 210.453, 'duration': 9.992}], 'summary': 'Demonstrating accessing attributes and methods from a response object, including 200 response status.', 'duration': 45.506, 'max_score': 174.939, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs174939.jpg'}, {'end': 244.749, 'src': 'embed', 'start': 220.865, 'weight': 3, 'content': [{'end': 227.046, 'text': "Now if you'd ever like to see a more detailed explanation of an object's attributes and methods, then you can run it through the help function.", 'start': 220.865, 'duration': 6.181}, {'end': 234.327, 'text': 'So if I come up here and instead of dir, we say help and print out the help of this response object.', 'start': 227.466, 'duration': 6.861}, {'end': 235.548, 'text': 'So let me run that.', 'start': 234.728, 'duration': 0.82}, {'end': 237.588, 'text': "Now I'll scroll up here to the top.", 'start': 235.968, 'duration': 1.62}, {'end': 244.749, 'text': 'You can see we have a lot more to scroll through, but it gives us a much more detailed explanation of everything from this object.', 'start': 237.908, 'duration': 6.841}], 'summary': "The help function provides detailed explanation of an object's attributes and methods.", 'duration': 23.884, 'max_score': 220.865, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs220865.jpg'}, {'end': 337.786, 'src': 'embed', 'start': 308.533, 'weight': 1, 'content': [{'end': 310.755, 'text': "But let's say that you wanted to download an image.", 'start': 308.533, 'duration': 2.222}, {'end': 317.438, 'text': "So if you wanted to download an image, then you'd want to use content, which gives you the content of the response in bytes.", 'start': 311.155, 'duration': 6.283}, {'end': 324.041, 'text': 'So if we go back to that web page in the browser, if I look here at the bottom of this comic,', 'start': 317.878, 'duration': 6.163}, {'end': 327.502, 'text': 'then we can see that it gives us an image URL for this comic.', 'start': 324.041, 'duration': 3.461}, {'end': 334.225, 'text': 'So let me copy this and paste it in here into a new tab.', 'start': 327.862, 'duration': 6.363}, {'end': 337.786, 'text': 'And we can see that this is just the image of the comic.', 'start': 334.905, 'duration': 2.881}], 'summary': 'Downloading an image using content gives the response in bytes, as seen in the comic url.', 'duration': 29.253, 'max_score': 308.533, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs308533.jpg'}, {'end': 450.038, 'src': 'embed', 'start': 410.924, 'weight': 0, 'content': [{'end': 415.946, 'text': 'but this code that we wrote is going to save this file in the same directory as our Python module,', 'start': 410.924, 'duration': 5.022}, {'end': 419.649, 'text': "since we didn't specify a full path in the open function.", 'start': 416.306, 'duration': 3.343}, {'end': 426.175, 'text': 'so if we run this and open the directory where this python module is located, then there should be an image saved there as well.', 'start': 419.649, 'duration': 6.526}, {'end': 432.54, 'text': "so i'm going to save this and run it and we don't get any output down here because we didn't print anything out.", 'start': 426.175, 'duration': 6.365}, {'end': 440.528, 'text': 'but i have the directory open where i wrote this module And we can see that we have the comic.png right here.', 'start': 432.54, 'duration': 7.988}, {'end': 445.053, 'text': 'And if I open that up, we can see that it is the comic from that website.', 'start': 440.808, 'duration': 4.245}, {'end': 450.038, 'text': 'Okay, so now let me close that down and go back to our module here.', 'start': 445.433, 'duration': 4.605}], 'summary': "The code saved an image file named 'comic.png' in the same directory as the python module.", 'duration': 39.114, 'max_score': 410.924, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs410924.jpg'}], 'start': 91.722, 'title': 'Python requests for web content retrieval', 'summary': "Demonstrates installing 'request' library, creating a module, and fetching website content using python requests. it also covers accessing object attributes, image download, using functions like dir and help, and writing bytes to a file.", 'chapters': [{'end': 169.175, 'start': 91.722, 'title': 'Using python requests to fetch website content', 'summary': "Demonstrates using the pip command to install the 'request' library, creating a new module to work with it, and fetching the contents of a website using python requests.", 'duration': 77.453, 'highlights': ["Demonstrates using the pip command to install the 'request' library The speaker explains how to use the pip command to install the 'request' library, emphasizing its simplicity and quick download process.", "Creating a new module to work with the 'request' library The speaker instructs on creating a new module, 'rdemo.py', to work with the 'request' library for website content retrieval.", "Fetching the contents of a website using Python requests The speaker demonstrates the process of using the 'request' library to fetch the contents of a website by importing the library and using the request.get method with a specified URL."]}, {'end': 450.038, 'start': 169.716, 'title': 'Web page content retrieval and image download', 'summary': 'Demonstrates how to retrieve web page content, access object attributes and methods, and download an image using python, including using functions like dir, help, and open, and techniques like writing bytes to a file.', 'duration': 280.322, 'highlights': ['The response object from a web page shows a 200 response, and attributes and methods like content, headers, isRedirect, JSON links, OK status, and status code can be accessed using dir function. The response object from a web page shows a 200 response, and attributes and methods like content, headers, isRedirect, JSON links, OK status, and status code can be accessed using dir function.', "The help function provides a more detailed explanation of an object's attributes and methods, including details about content in bytes and content in Unicode. The help function provides a more detailed explanation of an object's attributes and methods, including details about content in bytes and content in Unicode.", 'Accessing the content of the response in Unicode can be achieved by printing the response object, useful for parsing out information. Accessing the content of the response in Unicode can be achieved by printing the response object, useful for parsing out information.', "To download an image, the content of the response in bytes can be used, and the bytes can be saved to a file using the open function with 'write bytes' mode. To download an image, the content of the response in bytes can be used, and the bytes can be saved to a file using the open function with 'write bytes' mode."]}], 'duration': 358.316, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs91722.jpg', 'highlights': ["Creating a new module to work with the 'request' library The speaker instructs on creating a new module, 'rdemo.py', to work with the 'request' library for website content retrieval.", "Demonstrates using the pip command to install the 'request' library The speaker explains how to use the pip command to install the 'request' library, emphasizing its simplicity and quick download process.", 'The response object from a web page shows a 200 response, and attributes and methods like content, headers, isRedirect, JSON links, OK status, and status code can be accessed using dir function.', "The help function provides a more detailed explanation of an object's attributes and methods, including details about content in bytes and content in Unicode.", "Fetching the contents of a website using Python requests The speaker demonstrates the process of using the 'request' library to fetch the contents of a website by importing the library and using the request.get method with a specified URL."]}, {'end': 1131.265, 'segs': [{'end': 504.182, 'src': 'heatmap', 'start': 476.815, 'weight': 0.789, 'content': [{'end': 482.057, 'text': 'So if I run that, then we can see that we got a 200 response from the site.', 'start': 476.815, 'duration': 5.242}, {'end': 487.281, 'text': "Now, if you don't know your HTTP status codes, Basically, the 200s are a success.", 'start': 482.417, 'duration': 4.864}, {'end': 489.544, 'text': 'The 300s are redirects.', 'start': 487.882, 'duration': 1.662}, {'end': 496.052, 'text': "The 400s are client errors, like if you try to access a page that you don't have permission to view or something like that.", 'start': 490.124, 'duration': 5.928}, {'end': 499.035, 'text': 'And 500 errors are server errors.', 'start': 496.813, 'duration': 2.222}, {'end': 504.182, 'text': "Those are the errors that you'll see when a site just crashes and you can't access it for some time.", 'start': 499.115, 'duration': 5.067}], 'summary': 'Http status codes: 200 success, 300 redirects, 400 client errors, 500 server errors', 'duration': 27.367, 'max_score': 476.815, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs476815.jpg'}, {'end': 544.049, 'src': 'embed', 'start': 519.857, 'weight': 6, 'content': [{'end': 526.522, 'text': 'So if I save that and run it, then that will return true for anything that is less than a 400 response.', 'start': 519.857, 'duration': 6.665}, {'end': 531.004, 'text': 'So if you get any kind of client or server error, then that would return false.', 'start': 526.882, 'duration': 4.122}, {'end': 535.786, 'text': "So for example, I'll probably write a script soon that monitors my personal website.", 'start': 531.744, 'duration': 4.042}, {'end': 541.028, 'text': "And if I get a response that isn't okay, then I'll send myself an email that something is up with that.", 'start': 536.166, 'duration': 4.862}, {'end': 544.049, 'text': "And I'll use something like this in that script.", 'start': 541.608, 'duration': 2.441}], 'summary': 'Code will return true for responses less than 400, false for errors. will be used to monitor personal website.', 'duration': 24.192, 'max_score': 519.857, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs519857.jpg'}, {'end': 650.834, 'src': 'heatmap', 'start': 582.226, 'weight': 4, 'content': [{'end': 589.11, 'text': "Now for the rest of the video, I'm going to use a site called httpbin.org that allows us to test different queries.", 'start': 582.226, 'duration': 6.884}, {'end': 593.093, 'text': 'So let me open this up in the browser so we can see what this looks like.', 'start': 589.47, 'duration': 3.623}, {'end': 597.175, 'text': 'So this site was actually written by the same guy who wrote the request library,', 'start': 593.533, 'duration': 3.642}, {'end': 602.218, 'text': "and I'll never understand how some of these people are so productive and can create so many awesome tools,", 'start': 597.175, 'duration': 5.043}, {'end': 605.62, 'text': "but I'm really glad that these people exist because they do some awesome work.", 'start': 602.218, 'duration': 3.402}, {'end': 610.883, 'text': 'So this site will show us the different routes available to us and what it allows us to test.', 'start': 606.1, 'duration': 4.783}, {'end': 617.667, 'text': 'So we can test different HTTP methods like get, post, put, delete, and things like that.', 'start': 611.283, 'duration': 6.384}, {'end': 621.891, 'text': 'We can test redirects if we want to test redirects.', 'start': 618.628, 'duration': 3.263}, {'end': 626.195, 'text': 'We can see here that we can test authentication and all kinds of different stuff.', 'start': 622.151, 'duration': 4.044}, {'end': 632.822, 'text': "So for example, let's say that you wanted to do a get request on a route that has some URL parameters.", 'start': 626.596, 'duration': 6.226}, {'end': 637.687, 'text': "So I'm going to use the HTTP bin get route to test this.", 'start': 633.142, 'duration': 4.545}, {'end': 647.813, 'text': "And the way that he's set this site up is that HTTP bin is going to respond with JSON information of what we sent in the request,", 'start': 638.207, 'duration': 9.606}, {'end': 650.834, 'text': 'so that we can make sure that we set all of the parameters properly.', 'start': 647.813, 'duration': 3.021}], 'summary': 'Using httpbin.org to test various http methods and features for requests and responses.', 'duration': 24.238, 'max_score': 582.226, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs582226.jpg'}, {'end': 722.138, 'src': 'heatmap', 'start': 657.538, 'weight': 0.721, 'content': [{'end': 662.142, 'text': "So like I said, I'm going to use request to pass some URL parameters with our request.", 'start': 657.538, 'duration': 4.604}, {'end': 675.213, 'text': 'So the HTTP bin route for their git request is going to be https http bin.org forward slash git.', 'start': 662.582, 'duration': 12.631}, {'end': 681.335, 'text': "Now, usually you can pass URL parameters directly in the URL, and you've probably seen these before.", 'start': 675.633, 'duration': 5.702}, {'end': 685.256, 'text': 'So these are things in the URL that look like you know,', 'start': 681.715, 'duration': 3.541}, {'end': 694.06, 'text': 'question mark page equals two and then an ampersand sign there count equals 25 or something like that.', 'start': 685.256, 'duration': 8.804}, {'end': 702.125, 'text': 'Now we can still add parameters to the URL directly like this if we wanted to, but adding them to the URL directly like that can be prone to mistakes.', 'start': 694.5, 'duration': 7.625}, {'end': 708.889, 'text': 'So the request library actually lets us pass these in as a dictionary and then generates the appropriate URL for us.', 'start': 702.485, 'duration': 6.404}, {'end': 713.472, 'text': 'So what we can do, let me remove these parameters here from the URL.', 'start': 709.329, 'duration': 4.143}, {'end': 715.833, 'text': 'Let me go up a line here.', 'start': 714.332, 'duration': 1.501}, {'end': 722.138, 'text': "I'm just going to say payload is equal to, and now let's create a dictionary of those parameters.", 'start': 716.254, 'duration': 5.884}], 'summary': 'Using request library to pass url parameters as a dictionary to avoid mistakes and generate appropriate url.', 'duration': 64.6, 'max_score': 657.538, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs657538.jpg'}, {'end': 796.34, 'src': 'heatmap', 'start': 764.078, 'weight': 0.72, 'content': [{'end': 768.44, 'text': 'And we can see that the response said that our request had arguments.', 'start': 764.078, 'duration': 4.362}, {'end': 780.788, 'text': 'of a page equal to 2 and a count equal to 25, and we can look at the request URL also and see that the library created the correct URL for us.', 'start': 769.4, 'duration': 11.388}, {'end': 787.473, 'text': 'so if I print out instead of r.text, I print out r.url and run that,', 'start': 780.788, 'duration': 6.685}, {'end': 796.34, 'text': 'then we can see that the URL that it requested was this right here and it has those correct URL parameters in the URL itself.', 'start': 787.473, 'duration': 8.867}], 'summary': 'The response included arguments of page=2 and count=25, generating the correct url.', 'duration': 32.262, 'max_score': 764.078, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs764078.jpg'}, {'end': 946.007, 'src': 'heatmap', 'start': 880.317, 'weight': 0, 'content': [{'end': 888.862, 'text': "if I scroll up, then we can see that our args we don't have any args because those are URL parameters in our form here.", 'start': 880.317, 'duration': 8.545}, {'end': 890.283, 'text': 'this is the data that was posted.', 'start': 888.862, 'duration': 1.421}, {'end': 895.186, 'text': 'So we have a username set to Corey and a password set to testing.', 'start': 890.683, 'duration': 4.503}, {'end': 897.888, 'text': "And you can make those key value pairs whatever you'd like.", 'start': 895.506, 'duration': 2.382}, {'end': 900.97, 'text': "So if you're trying to pass in certain form data,", 'start': 898.228, 'duration': 2.742}, {'end': 907.694, 'text': "then you'll have to actually look at the HTML source code of the URL to see what values that form actually expects.", 'start': 900.97, 'duration': 6.724}, {'end': 914.375, 'text': "Now, like I said, we're getting a JSON response back from the httpbin website.", 'start': 908.374, 'duration': 6.001}, {'end': 918.516, 'text': 'So JSON responses are very common when working with certain APIs.', 'start': 914.775, 'duration': 3.741}, {'end': 923.297, 'text': 'So if you get a JSON response, then instead of using .', 'start': 918.896, 'duration': 4.401}, {'end': 927.657, 'text': 'text as the method here, you could also use a JSON method instead.', 'start': 923.297, 'duration': 4.36}, {'end': 929.958, 'text': "So let's see what that does.", 'start': 928.217, 'duration': 1.741}, {'end': 933.618, 'text': 'So I will print out r.json, and that is a method.', 'start': 930.278, 'duration': 3.34}, {'end': 937.359, 'text': 'So we have to put parameters after that so that we execute that.', 'start': 933.658, 'duration': 3.701}, {'end': 940.101, 'text': 'So let me save that and run it.', 'start': 937.859, 'duration': 2.242}, {'end': 946.007, 'text': 'So what this did is it created a Python dictionary from that JSON response.', 'start': 940.862, 'duration': 5.145}], 'summary': 'The transcript discusses receiving a json response and converting it into a python dictionary.', 'duration': 59.784, 'max_score': 880.317, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs880317.jpg'}, {'end': 1006.204, 'src': 'heatmap', 'start': 967.943, 'weight': 0.889, 'content': [{'end': 981.988, 'text': 'so, for example, you know i could just access the form key of that response and say print r dict and then access that form key.', 'start': 967.943, 'duration': 14.045}, {'end': 988.166, 'text': 'so if i save that and run it, then we can see that we get the posted form values from that response.', 'start': 981.988, 'duration': 6.178}, {'end': 991.551, 'text': "Now, if you're used to using JSON in Python,", 'start': 988.666, 'duration': 2.885}, {'end': 999.763, 'text': 'then this is basically the same as importing the JSON module and then using JSON.loadS on the response text,', 'start': 991.551, 'duration': 8.212}, {'end': 1002.984, 'text': 'But this is just a nice little shortcut for that functionality.', 'start': 1000.263, 'duration': 2.721}, {'end': 1006.204, 'text': 'Okay, so we saw how to do a POST request with some data.', 'start': 1003.424, 'duration': 2.78}], 'summary': 'Demonstrated accessing form key and using json module in python for post request.', 'duration': 38.261, 'max_score': 967.943, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs967943.jpg'}, {'end': 1131.265, 'src': 'heatmap', 'start': 1078.296, 'weight': 1, 'content': [{'end': 1090.298, 'text': "So I'm at httpbin.org forward slash Basic dash auth forward slash and now I'll do a user of Cori forward slash and a password of testing.", 'start': 1078.296, 'duration': 12.002}, {'end': 1098.465, 'text': 'So if I run that and go to that route, then we can see that when we run this, it pops up a basic authentication form here,', 'start': 1090.719, 'duration': 7.746}, {'end': 1103.249, 'text': 'where the correct credentials are going to be the user and password that we specified in the URL.', 'start': 1098.465, 'duration': 4.784}, {'end': 1109.874, 'text': "So if I put in the wrong username, so I'll say Corey MS and then a password of testing.", 'start': 1103.809, 'duration': 6.065}, {'end': 1115.578, 'text': "If I try to sign in, then we can see that it shakes that off, telling us that it doesn't work.", 'start': 1110.414, 'duration': 5.164}, {'end': 1121.823, 'text': "But if I put in the correct username and password, so I'll put in a username of Corey and a password.", 'start': 1115.938, 'duration': 5.885}, {'end': 1131.265, 'text': 'of testing and sign in, then we can see that we get redirected to this page that says authenticated is true and our user is corey.', 'start': 1122.683, 'duration': 8.582}], 'summary': 'Using httpbin.org, basic authentication is tested with correct and incorrect credentials, resulting in successful and failed logins.', 'duration': 20.851, 'max_score': 1078.296, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs1078296.jpg'}], 'start': 450.439, 'title': 'Monitoring websites and testing http methods', 'summary': 'Introduces the use of python request library to monitor website responses, including checking status codes, identifying client and server errors, and extracting response header information. it also discusses testing different http methods like get, post, put, delete with httpbin.org, and covers authentication methods using basic authentication, showcasing examples and responses.', 'chapters': [{'end': 581.345, 'start': 450.439, 'title': 'Using python request library for website monitoring', 'summary': 'Introduces the use of python request library to monitor website responses, including checking status codes, identifying client and server errors, and extracting response header information.', 'duration': 130.906, 'highlights': ['The chapter emphasizes the importance of monitoring website responses by using Python Request Library, including checking status codes to ensure successful responses, with a demonstration of obtaining a 200 response from a site.', 'The chapter explains the significance of HTTP status codes, categorizing 200s as success, 300s as redirects, 400s as client errors, and 500s as server errors, providing a clear understanding of different types of errors and their implications for website monitoring.', 'The chapter demonstrates how to easily check for client or server errors using Request Library, enabling the user to print out a boolean value for responses less than 400, and illustrates a practical application of this functionality in monitoring a personal website and triggering email notifications for non-OK responses.', 'The chapter introduces the capability of Request Library to extract in-depth response information by accessing and printing out response headers, allowing users to obtain details such as server type, content type, and other relevant information for website monitoring.']}, {'end': 1131.265, 'start': 582.226, 'title': 'Testing http methods with httpbin.org', 'summary': 'Discusses testing different http methods like get, post, put, delete with httpbin.org, and also covers authentication methods using basic authentication, showcasing examples and responses.', 'duration': 549.039, 'highlights': ['The chapter discusses testing different HTTP methods like get, post, put, delete with httpbin.org. The speaker explains how httpbin.org allows testing of different queries, including HTTP methods like get, post, put, delete, and redirects, and also demonstrates passing URL parameters and posting form data.', 'The chapter covers authentication methods using basic authentication, showcasing examples and responses. The chapter demonstrates using httpbin.org to test basic authentication routes by specifying credentials in the URL, and provides examples of successful and unsuccessful authentication attempts.', 'The chapter showcases examples of responses for different requests. The speaker demonstrates examining the responses from httpbin.org, including JSON responses, and provides examples of capturing JSON responses in Python dictionaries for further processing.']}], 'duration': 680.826, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs450439.jpg', 'highlights': ['The chapter emphasizes the importance of monitoring website responses by using Python Request Library, including checking status codes to ensure successful responses, with a demonstration of obtaining a 200 response from a site.', 'The chapter introduces the capability of Request Library to extract in-depth response information by accessing and printing out response headers, allowing users to obtain details such as server type, content type, and other relevant information for website monitoring.', 'The chapter demonstrates how to easily check for client or server errors using Request Library, enabling the user to print out a boolean value for responses less than 400, and illustrates a practical application of this functionality in monitoring a personal website and triggering email notifications for non-OK responses.', 'The chapter explains the significance of HTTP status codes, categorizing 200s as success, 300s as redirects, 400s as client errors, and 500s as server errors, providing a clear understanding of different types of errors and their implications for website monitoring.', 'The chapter discusses testing different HTTP methods like get, post, put, delete with httpbin.org. The speaker explains how httpbin.org allows testing of different queries, including HTTP methods like get, post, put, delete, and redirects, and also demonstrates passing URL parameters and posting form data.', 'The chapter covers authentication methods using basic authentication, showcasing examples and responses. The chapter demonstrates using httpbin.org to test basic authentication routes by specifying credentials in the URL, and provides examples of successful and unsuccessful authentication attempts.', 'The chapter showcases examples of responses for different requests. The speaker demonstrates examining the responses from httpbin.org, including JSON responses, and provides examples of capturing JSON responses in Python dictionaries for further processing.']}, {'end': 1487.764, 'segs': [{'end': 1248.182, 'src': 'embed', 'start': 1176.008, 'weight': 1, 'content': [{'end': 1179.81, 'text': 'All we have to do is pass in some additional parameters here.', 'start': 1176.008, 'duration': 3.802}, {'end': 1183.253, 'text': "And so I'll say comma for our get method.", 'start': 1180.43, 'duration': 2.823}, {'end': 1185.835, 'text': "And I'll say auth is equal to.", 'start': 1183.673, 'duration': 2.162}, {'end': 1187.897, 'text': "And we're going to pass in a tuple.", 'start': 1186.295, 'duration': 1.602}, {'end': 1190.619, 'text': 'And the tuple will be the username and the password.', 'start': 1188.257, 'duration': 2.362}, {'end': 1193.082, 'text': 'So the first value here is the username.', 'start': 1191.06, 'duration': 2.022}, {'end': 1193.962, 'text': "So I'll say Corey.", 'start': 1193.222, 'duration': 0.74}, {'end': 1197.646, 'text': 'And for the password, I will say testing.', 'start': 1194.403, 'duration': 3.243}, {'end': 1206.633, 'text': 'So now, if I come down here and do a print of r.txt, if I save that and run it,', 'start': 1198.286, 'duration': 8.347}, {'end': 1214.359, 'text': 'then we can see that we get the response that we were expecting and that we were authenticated is set to true and the user is set to Corey.', 'start': 1206.633, 'duration': 7.726}, {'end': 1218.983, 'text': "Now if we were to use the wrong credentials, then let's see what happens.", 'start': 1214.779, 'duration': 4.204}, {'end': 1223.887, 'text': "So instead of Corey as the username, I'm going to pass in CoreyMS instead.", 'start': 1219.163, 'duration': 4.724}, {'end': 1225.688, 'text': 'So I will save that and run it.', 'start': 1224.247, 'duration': 1.441}, {'end': 1230.212, 'text': "So if I run that as is, then we can see that there's no response text.", 'start': 1226.149, 'duration': 4.063}, {'end': 1236.517, 'text': 'So let me remove that text part from the response and just print out the response object itself.', 'start': 1230.632, 'duration': 5.885}, {'end': 1239.78, 'text': "So now I'm just printing out R, which is the response.", 'start': 1236.938, 'duration': 2.842}, {'end': 1241.64, 'text': "So I'll save that and run it.", 'start': 1240.12, 'duration': 1.52}, {'end': 1248.182, 'text': 'And when I do that, we can see that we get a response, but it has a status code of 401.', 'start': 1242.241, 'duration': 5.941}], 'summary': "Passing authentication parameters for get method. successful authentication with username 'corey' and password 'testing' and failure with wrong credentials.", 'duration': 72.174, 'max_score': 1176.008, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs1176008.jpg'}, {'end': 1390.921, 'src': 'embed', 'start': 1358.784, 'weight': 0, 'content': [{'end': 1364.37, 'text': "So if I delay by one second here, so I'm going to copy that URL.", 'start': 1358.784, 'duration': 5.586}, {'end': 1368.192, 'text': 'and paste this in to my script.', 'start': 1365.371, 'duration': 2.821}, {'end': 1374.435, 'text': "so we're getting a URL that has a delay of one second and a timeout of three seconds.", 'start': 1368.192, 'duration': 6.243}, {'end': 1379.917, 'text': "so I'm going to run that and we can see that the response is a 200 response.", 'start': 1374.435, 'duration': 5.482}, {'end': 1382.578, 'text': 'now what if I delay for six seconds?', 'start': 1379.917, 'duration': 2.661}, {'end': 1390.921, 'text': 'so we can imagine if it takes six seconds to get a URL response that you know usually is going to time out for most people.', 'start': 1382.578, 'duration': 8.343}], 'summary': 'Testing url response times: 1s delay, 200 response; 6s delay likely to cause timeouts.', 'duration': 32.137, 'max_score': 1358.784, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs1358784.jpg'}], 'start': 1131.265, 'title': 'Python requests and setting timeout', 'summary': 'Covers python requests for basic authentication, including passing credentials and handling responses, with examples of successful and unsuccessful authentication. it also emphasizes the importance of setting a timeout when monitoring a website using the requests library, with an example using http bin and the necessity of setting timeouts for every request to avoid indefinite hanging.', 'chapters': [{'end': 1271.427, 'start': 1131.265, 'title': 'Python requests for basic authentication', 'summary': 'Explains how to use python requests to programmatically perform basic authentication, including passing credentials and handling responses, with examples of successful and unsuccessful authentication.', 'duration': 140.162, 'highlights': ['The chapter demonstrates creating a GET request in Python using the requests library, including passing basic authentication credentials and handling response codes.', 'The tutorial provides a clear example of successfully authenticating with the correct credentials, resulting in a 200 response.', 'It also illustrates the behavior of unsuccessful authentication, displaying a 401 unauthorized response code.']}, {'end': 1487.764, 'start': 1271.427, 'title': 'Setting timeout for website monitoring', 'summary': 'Discusses the importance of setting a timeout when monitoring a website using the requests library, highlighting how to implement it with an example using http bin and stressing the necessity of setting timeouts for every request to avoid indefinite hanging, with the speaker emphasizing its use for personal website monitoring.', 'duration': 216.337, 'highlights': ['The necessity of setting timeouts for every request is emphasized, with the speaker stressing the importance of avoiding indefinite hanging and ensuring timely notifications for website monitoring.', 'An example of implementing a timeout using the requests library with HTTP bin is provided, demonstrating the use of timeout parameters to handle delayed responses.', 'The speaker highlights the specific use case of setting a timeout for personal website monitoring, ensuring timely notifications if the website takes longer than a specified time to respond.']}], 'duration': 356.499, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/tb8gHvYlCFs/pics/tb8gHvYlCFs1131265.jpg', 'highlights': ['The tutorial provides a clear example of successfully authenticating with the correct credentials, resulting in a 200 response.', 'An example of implementing a timeout using the requests library with HTTP bin is provided, demonstrating the use of timeout parameters to handle delayed responses.', 'The necessity of setting timeouts for every request is emphasized, with the speaker stressing the importance of avoiding indefinite hanging and ensuring timely notifications for website monitoring.', 'The chapter demonstrates creating a GET request in Python using the requests library, including passing basic authentication credentials and handling response codes.', 'It also illustrates the behavior of unsuccessful authentication, displaying a 401 unauthorized response code.', 'The speaker highlights the specific use case of setting a timeout for personal website monitoring, ensuring timely notifications if the website takes longer than a specified time to respond.']}], 'highlights': ['The request library simplifies tasks, making them easy and intuitive.', 'It allows making HTTP requests to get and post information, download images, send authentication data, and follow redirects.', "Creating a new module to work with the 'request' library The speaker instructs on creating a new module, 'rdemo.py', to work with the 'request' library for website content retrieval.", "Demonstrates using the pip command to install the 'request' library The speaker explains how to use the pip command to install the 'request' library, emphasizing its simplicity and quick download process.", 'The response object from a web page shows a 200 response, and attributes and methods like content, headers, isRedirect, JSON links, OK status, and status code can be accessed using dir function.', "The help function provides a more detailed explanation of an object's attributes and methods, including details about content in bytes and content in Unicode.", "Fetching the contents of a website using Python requests The speaker demonstrates the process of using the 'request' library to fetch the contents of a website by importing the library and using the request.get method with a specified URL.", 'The chapter emphasizes the importance of monitoring website responses by using Python Request Library, including checking status codes to ensure successful responses, with a demonstration of obtaining a 200 response from a site.', 'The chapter introduces the capability of Request Library to extract in-depth response information by accessing and printing out response headers, allowing users to obtain details such as server type, content type, and other relevant information for website monitoring.', 'The chapter demonstrates how to easily check for client or server errors using Request Library, enabling the user to print out a boolean value for responses less than 400, and illustrates a practical application of this functionality in monitoring a personal website and triggering email notifications for non-OK responses.', 'The chapter explains the significance of HTTP status codes, categorizing 200s as success, 300s as redirects, 400s as client errors, and 500s as server errors, providing a clear understanding of different types of errors and their implications for website monitoring.', 'The chapter discusses testing different HTTP methods like get, post, put, delete with httpbin.org. The speaker explains how httpbin.org allows testing of different queries, including HTTP methods like get, post, put, delete, and redirects, and also demonstrates passing URL parameters and posting form data.', 'The chapter covers authentication methods using basic authentication, showcasing examples and responses. The chapter demonstrates using httpbin.org to test basic authentication routes by specifying credentials in the URL, and provides examples of successful and unsuccessful authentication attempts.', 'The chapter showcases examples of responses for different requests. The speaker demonstrates examining the responses from httpbin.org, including JSON responses, and provides examples of capturing JSON responses in Python dictionaries for further processing.', 'The tutorial provides a clear example of successfully authenticating with the correct credentials, resulting in a 200 response.', 'An example of implementing a timeout using the requests library with HTTP bin is provided, demonstrating the use of timeout parameters to handle delayed responses.', 'The necessity of setting timeouts for every request is emphasized, with the speaker stressing the importance of avoiding indefinite hanging and ensuring timely notifications for website monitoring.', 'The chapter demonstrates creating a GET request in Python using the requests library, including passing basic authentication credentials and handling response codes.', 'It also illustrates the behavior of unsuccessful authentication, displaying a 401 unauthorized response code.', 'The speaker highlights the specific use case of setting a timeout for personal website monitoring, ensuring timely notifications if the website takes longer than a specified time to respond.']}