title
Xgboost Classification Indepth Maths Intuition- Machine Learning Algorithms🔥🔥🔥🔥

description
XGBoost is a decision-tree-based ensemble Machine Learning algorithm that uses a gradient boosting framework. In prediction problems involving unstructured data (images, text, etc.) artificial neural networks tend to outperform all other algorithms or frameworks. All Playlist In My channel Complete ML Playlist :https://www.youtube.com/playlist?list=PLZoTAELRMXVPBTrWtJkn3wWQxZkmTXGwe Complete NLP Playlist:https://www.youtube.com/playlist?list=PLZoTAELRMXVMdJ5sqbCK2LiM0HhQVWNzm Docker End To End Implementation: https://www.youtube.com/playlist?list=PLZoTAELRMXVNKtpy0U_Mx9N26w8n0hIbs Live stream Playlist: https://www.youtube.com/playlist?list=PLZoTAELRMXVNxYFq_9MuiUdn2YnlFqmMK Machine Learning Pipelines: https://www.youtube.com/playlist?list=PLZoTAELRMXVNKtpy0U_Mx9N26w8n0hIbs Pytorch Playlist: https://www.youtube.com/playlist?list=PLZoTAELRMXVNxYFq_9MuiUdn2YnlFqmMK Feature Engineering :https://www.youtube.com/playlist?list=PLZoTAELRMXVPwYGE2PXD3x0bfKnR0cJjN Live Projects :https://www.youtube.com/playlist?list=PLZoTAELRMXVOFnfSwkB_uyr4FT-327noK Kaggle competition :https://www.youtube.com/playlist?list=PLZoTAELRMXVPiKOxbwaniXjHJ02bdkLWy Mongodb with Python :https://www.youtube.com/playlist?list=PLZoTAELRMXVN_8zzsevm1bm6G-plsiO1I MySQL With Python :https://www.youtube.com/playlist?list=PLZoTAELRMXVMd3RF7p-u7ezEysGaG9JmO Deployment Architectures:https://www.youtube.com/playlist?list=PLZoTAELRMXVOPzVJiSJAn9Ly27Fi1-8ac Amazon sagemaker :https://www.youtube.com/playlist?list=PLZoTAELRMXVONh5mHrXowH6-dgyWoC_Ew Please donate if you want to support the channel through GPay UPID, Gpay: krishnaik06@okicici Discord Server Link: https://discord.gg/tvAJuuy Telegram link: https://t.me/joinchat/N77M7xRvYUd403DgfE4TWw Please join as a member in my channel to get additional benefits like materials in Data Science, live streaming for Members and many more https://www.youtube.com/channel/UCNU_lfiiWBdtULKOw6X0Dig/join Please do subscribe my other channel too https://www.youtube.com/channel/UCjWY5hREA6FFYrthD0rZNIw Connect with me here: Twitter: https://twitter.com/Krishnaik06 Facebook: https://www.facebook.com/krishnaik06 instagram: https://www.instagram.com/krishnaik06 #xgboostclassifier #xgboost

detail
{'title': 'Xgboost Classification Indepth Maths Intuition- Machine Learning Algorithms🔥🔥🔥🔥', 'heatmap': [{'end': 1051.913, 'start': 1008.277, 'weight': 0.844}, {'end': 1170.795, 'start': 1101.172, 'weight': 0.843}, {'end': 1226.847, 'start': 1208.959, 'weight': 0.701}], 'summary': "Delves into xgboost classifier, highlighting its significance in ensemble techniques and differentiating between bagging and boosting algorithms. it covers practical implementation and in-depth explanation, expected to last 20 to 25 minutes. the video further explores hgboost's classification application, xgboost tree construction, decision tree split weight calculation, credit split analysis, and xgboost tree training techniques including pruning and model output calculation.", 'chapters': [{'end': 91.674, 'segs': [{'end': 33.031, 'src': 'embed', 'start': 0.129, 'weight': 0, 'content': [{'end': 2.432, 'text': 'Hello all my name is Krishnayak and welcome to my YouTube channel.', 'start': 0.129, 'duration': 2.303}, {'end': 6.156, 'text': 'So guys from past many days many people are talking about XGBoost.', 'start': 2.452, 'duration': 3.704}, {'end': 9.88, 'text': 'It was one of the most requested video by you all.', 'start': 7.117, 'duration': 2.763}, {'end': 16.929, 'text': "So in this video I'm going to discuss about XGBoost classifier and in the upcoming videos also I'll be discussing about XGBoost regression.", 'start': 10.381, 'duration': 6.548}, {'end': 20.894, 'text': 'I have been uploading a lot of practical implementation with respect to XGBoost.', 'start': 17.489, 'duration': 3.405}, {'end': 26.101, 'text': 'So please make sure that you focus on this session and yes, we will be following step by step.', 'start': 21.214, 'duration': 4.887}, {'end': 29.486, 'text': 'This probably will be a 20 minutes video, 20 to 25 minutes video.', 'start': 26.442, 'duration': 3.044}, {'end': 33.031, 'text': "I'll try to keep it as short as possible and I'll try to explain you all the things.", 'start': 29.506, 'duration': 3.525}], 'summary': 'Krishnayak introduces xgboost classifier and promises practical implementations in 20-25 minute video.', 'duration': 32.902, 'max_score': 0.129, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ129.jpg'}, {'end': 96.558, 'src': 'embed', 'start': 70.737, 'weight': 1, 'content': [{'end': 76.722, 'text': 'you need to know gradient boosting algorithm if you want to really understand how does extreme gradient boosting actually work?', 'start': 70.737, 'duration': 5.985}, {'end': 80.725, 'text': 'So XG, right, the full form is extreme gradient.', 'start': 77.202, 'duration': 3.523}, {'end': 85.649, 'text': "So let's go ahead and try to understand how does XG boost classifier get strain.", 'start': 81.186, 'duration': 4.463}, {'end': 88.312, 'text': 'I will try to discuss a whole lot in complete depth.', 'start': 85.95, 'duration': 2.362}, {'end': 91.674, 'text': 'So let me take a very good example, a simple example.', 'start': 88.812, 'duration': 2.862}, {'end': 93.736, 'text': 'So on the first feature, you have salary.', 'start': 92.074, 'duration': 1.662}, {'end': 96.558, 'text': 'On the second feature, you have credit score.', 'start': 94.736, 'duration': 1.822}], 'summary': 'Understanding extreme gradient boosting algorithm is crucial for grasping xg boost classifier, as demonstrated through a simple example with salary and credit score features.', 'duration': 25.821, 'max_score': 70.737, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ70737.jpg'}], 'start': 0.129, 'title': 'Xgboost classifier', 'summary': 'Introduces xgboost classifier, emphasizing its importance, relationship with ensemble techniques, and covering differences between bagging and boosting algorithms. the video is expected to be 20 to 25 minutes long, focusing on practical implementation and in-depth explanation.', 'chapters': [{'end': 91.674, 'start': 0.129, 'title': 'Xgboost classifier overview', 'summary': 'Introduces xgboost classifier, covering its importance, relationship with ensemble techniques, and planned content for future videos. it also outlines the differences between bagging and boosting algorithms and emphasizes the need to understand gradient boosting for comprehending xgboost. the video is expected to be 20 to 25 minutes long, focusing on practical implementation and in-depth explanation.', 'duration': 91.545, 'highlights': ["The chapter introduces XGBoost classifier, covering its importance, relationship with ensemble techniques, and planned content for future videos. It was one of the most requested video by you all. In the upcoming videos also I'll be discussing about XGBoost regression.", 'It outlines the differences between bagging and boosting algorithms and emphasizes the need to understand gradient boosting for comprehending XGBoost. In the ensemble techniques, you basically have two different kinds. One is the bagging, one is the boosting. And XGBoost, AdaBoost, GradientBoosting, those are all kind of boosting algorithms.', "The video is expected to be 20 to 25 minutes long, focusing on practical implementation and in-depth explanation. This probably will be a 20 minutes video, 20 to 25 minutes video. I'll try to keep it as short as possible and I'll try to explain you all the things."]}], 'duration': 91.545, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ129.jpg', 'highlights': ['The video is expected to be 20 to 25 minutes long, focusing on practical implementation and in-depth explanation.', 'It outlines the differences between bagging and boosting algorithms and emphasizes the need to understand gradient boosting for comprehending XGBoost.', 'The chapter introduces XGBoost classifier, covering its importance, relationship with ensemble techniques, and planned content for future videos.']}, {'end': 340.895, 'segs': [{'end': 140.236, 'src': 'embed', 'start': 111.963, 'weight': 0, 'content': [{'end': 116.527, 'text': 'Now I have data set like less than or equal to 50k.', 'start': 111.963, 'duration': 4.564}, {'end': 117.988, 'text': 'The credit score is bad.', 'start': 116.767, 'duration': 1.221}, {'end': 120.25, 'text': 'I have classified credit score in three categories.', 'start': 118.008, 'duration': 2.242}, {'end': 123.032, 'text': 'Bad, good, normal.', 'start': 120.71, 'duration': 2.322}, {'end': 125.274, 'text': 'Bad, good, normal.', 'start': 123.353, 'duration': 1.921}, {'end': 129.176, 'text': 'And the output is either 0 and 1 since we are solving a classification problem.', 'start': 125.594, 'duration': 3.582}, {'end': 134.202, 'text': 'And again HGBoost is also useful in solving multi-class classification problem also.', 'start': 129.518, 'duration': 4.684}, {'end': 135.754, 'text': "So let's go ahead.", 'start': 135.154, 'duration': 0.6}, {'end': 140.236, 'text': "Now in the first step, here I've actually written some of the steps over here.", 'start': 136.154, 'duration': 4.082}], 'summary': 'Data set has less than or equal to 50k, classified into bad, good, normal credit scores. output is 0 or 1, and hgbboost can solve multi-class classification problems.', 'duration': 28.273, 'max_score': 111.963, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ111963.jpg'}, {'end': 293.087, 'src': 'embed', 'start': 265.694, 'weight': 1, 'content': [{'end': 268.337, 'text': 'Less than or equal to 50k, greater than or equal to 50k.', 'start': 265.694, 'duration': 2.643}, {'end': 275.416, 'text': 'Always remember in XGBoost, whenever you construct a tree, you always need to do a binary classifier.', 'start': 268.797, 'duration': 6.619}, {'end': 277.838, 'text': 'let it be having three categories also.', 'start': 275.416, 'duration': 2.422}, {'end': 280.739, 'text': 'let it be having four categories also, so one at once.', 'start': 277.838, 'duration': 2.901}, {'end': 283.441, 'text': 'what you can do is that you create a binary classifier.', 'start': 280.739, 'duration': 2.702}, {'end': 286.463, 'text': 'you basically have to divide in.', 'start': 283.441, 'duration': 3.022}, {'end': 289.325, 'text': 'the leaf node will be actually two every time, okay.', 'start': 286.463, 'duration': 2.862}, {'end': 293.087, 'text': 'so suppose, if I have three categories, less than or equal to 50 will go over here.', 'start': 289.325, 'duration': 3.762}], 'summary': 'Xgboost requires binary classifiers for tree construction with multiple categories. leaf nodes will be two for every category.', 'duration': 27.393, 'max_score': 265.694, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ265694.jpg'}, {'end': 328.944, 'src': 'embed', 'start': 303.432, 'weight': 2, 'content': [{'end': 310.594, 'text': 'we have to basically split the leaf node into two splits and then we basically need to compute the similarity weight.', 'start': 303.432, 'duration': 7.162}, {'end': 313.035, 'text': "I'll define about what is this similarity weight.", 'start': 310.594, 'duration': 2.441}, {'end': 318.396, 'text': 'it is pretty much important and based on the similarity weight, okay, based on the similarity weight,', 'start': 313.035, 'duration': 5.361}, {'end': 320.777, 'text': 'and then we will also find out what is information gain.', 'start': 318.396, 'duration': 2.381}, {'end': 328.944, 'text': 'If you remember, in decision tree guys, we use entropy and then we use the information gain to basically do the splitting thing.', 'start': 321.818, 'duration': 7.126}], 'summary': 'Split leaf node into two, compute similarity weight, find information gain.', 'duration': 25.512, 'max_score': 303.432, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ303432.jpg'}], 'start': 92.074, 'title': 'Hgboost and xgboost tree construction', 'summary': 'Covers the use of hgboost for classification and demonstrates its application in a simple example with salary, credit score, and approval. it also discusses the construction of decision trees in xgboost using residuals and the construction of a tree using binary classifiers for multiple categories.', 'chapters': [{'end': 194.362, 'start': 92.074, 'title': 'Hgboost for classification', 'summary': 'Explains the use of hgboost for classification, demonstrating its application in a simple example with salary, credit score, and approval, and detailing the process of constructing a base model for a classification problem.', 'duration': 102.288, 'highlights': ['The chapter demonstrates the application of HGBoost in a simple example with salary, credit score, and approval, showcasing its usefulness in solving multi-class classification problems.', 'It details the process of constructing a base model for a classification problem, including the calculation of initial probability and residuals.', 'The example includes the classification of credit scores into three categories: bad, good, and normal, with the output being either 0 or 1.', 'The initial probability for the base model is calculated as 0.5, as the output in the classification problem is only 0 and 1.']}, {'end': 265.654, 'start': 194.522, 'title': 'Xgboost tree construction', 'summary': 'Discusses the construction of a decision tree in xgboost using residuals, where the residuals are calculated as -0.5, 0.5, and 0, and the decision tree is constructed based on input features such as salary.', 'duration': 71.132, 'highlights': ['The residuals are calculated as -0.5, 0.5, and 0. The speaker explains the calculation of residuals as -0.5, 0.5, and 0, which are used to construct the decision tree in XGBoost.', 'The decision tree is constructed based on input features such as salary. The chapter discusses the construction of the decision tree in XGBoost based on input features, specifically focusing on the column of salary.']}, {'end': 340.895, 'start': 265.694, 'title': 'Xgboost binary classifier', 'summary': 'Explains how xgboost constructs a tree using binary classifiers for multiple categories, emphasizing the computation of similarity weight and information gain.', 'duration': 75.201, 'highlights': ['XGBoost requires constructing a tree using binary classifiers for multiple categories, ensuring that each leaf node is divided into two splits, such as less than or equal to 50k and greater than 50k, and computing similarity weight and information gain.', 'The computation of similarity weight is crucial in XGBoost, as it determines the basis for splitting the leaf nodes, influencing the overall decision-making process.', 'In XGBoost, the splitting process involves using similarity weight and information gain, similar to decision trees using entropy, to effectively categorize the data points and make informed decisions.']}], 'duration': 248.821, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ92074.jpg', 'highlights': ['The chapter demonstrates the application of HGBoost in a simple example with salary, credit score, and approval, showcasing its usefulness in solving multi-class classification problems.', 'XGBoost requires constructing a tree using binary classifiers for multiple categories, ensuring that each leaf node is divided into two splits, such as less than or equal to 50k and greater than 50k, and computing similarity weight and information gain.', 'The computation of similarity weight is crucial in XGBoost, as it determines the basis for splitting the leaf nodes, influencing the overall decision-making process.', 'The example includes the classification of credit scores into three categories: bad, good, and normal, with the output being either 0 or 1.']}, {'end': 733.097, 'segs': [{'end': 389.316, 'src': 'embed', 'start': 364.369, 'weight': 0, 'content': [{'end': 371.712, 'text': 'okay, how we have to calculate the similarity weight, understand, summation of all the residuals, summation of all the residuals.', 'start': 364.369, 'duration': 7.343}, {'end': 373.192, 'text': 'so what will be over here?', 'start': 371.712, 'duration': 1.48}, {'end': 383.196, 'text': 'I will try to do the summation 0.5 plus 0.5 plus 0.5, and this will be minus 0.5.', 'start': 373.192, 'duration': 10.004}, {'end': 389.316, 'text': 'whole square, divided by, divided by summation of probability.', 'start': 383.196, 'duration': 6.12}], 'summary': 'Calculating similarity weight using residuals, with example 0.5, -0.5, and division by probability.', 'duration': 24.947, 'max_score': 364.369, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ364369.jpg'}, {'end': 660.396, 'src': 'embed', 'start': 613.338, 'weight': 1, 'content': [{'end': 616.199, 'text': "And I've done the computation in front of you with respect to this also.", 'start': 613.338, 'duration': 2.861}, {'end': 619.781, 'text': 'And this is my similarity weight for the root right?', 'start': 616.699, 'duration': 3.082}, {'end': 624.043, 'text': 'Now, with respect to this, what is the gain that I have to compute?', 'start': 620.241, 'duration': 3.802}, {'end': 625.604, 'text': 'The gain will nothing be.', 'start': 624.103, 'duration': 1.501}, {'end': 632.587, 'text': 'it will be the similarity score in the left-hand side that is 0,, plus the similarity score in the right-hand side that is 0.33,', 'start': 625.604, 'duration': 6.983}, {'end': 634.488, 'text': 'minus the similarity score of the root.', 'start': 632.587, 'duration': 1.901}, {'end': 640.489, 'text': 'right. so if i go and subtract 0.14, it will be nothing but 0.21.', 'start': 635.248, 'duration': 5.241}, {'end': 643.17, 'text': 'okay, so 0.21 is my gain.', 'start': 640.489, 'duration': 2.681}, {'end': 647.391, 'text': 'so based on this, the total gain that i get is 0.21.', 'start': 643.17, 'duration': 4.221}, {'end': 650.972, 'text': 'now, reason i am actually split, calculating the gain.', 'start': 647.391, 'duration': 3.581}, {'end': 654.312, 'text': 'understand one thing, guys, why i am actually calculating gain.', 'start': 650.972, 'duration': 3.34}, {'end': 658.494, 'text': 'suppose i have one more category over here, I have one more category.', 'start': 654.312, 'duration': 4.182}, {'end': 660.396, 'text': "or why I'm using this specific gain?", 'start': 658.494, 'duration': 1.902}], 'summary': 'Calculation of gain based on similarity scores: total gain is 0.21.', 'duration': 47.058, 'max_score': 613.338, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ613338.jpg'}], 'start': 340.895, 'title': 'Similarity weight calculation for decision tree split', 'summary': 'Details the process of calculating similarity weight for decision tree splits, including the summation of residuals, probability calculations, and gain computation, resulting in a total gain of 0.21.', 'chapters': [{'end': 733.097, 'start': 340.895, 'title': 'Similarity weight calculation for decision tree split', 'summary': 'Details the process of calculating similarity weight for decision tree splits, including the summation of residuals, probability calculations, and gain computation, resulting in a total gain of 0.21.', 'duration': 392.202, 'highlights': ['The chapter details the process of calculating similarity weight for decision tree splits Explains the steps involved in calculating similarity weight for decision tree splits.', 'The process includes the summation of residuals and probability calculations Describes the steps for calculating the summation of residuals and probability for similarity weight calculation.', 'The gain computation results in a total gain of 0.21 Specifies the total gain obtained from the gain computation for decision tree splits.']}], 'duration': 392.202, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ340895.jpg', 'highlights': ['The process includes the summation of residuals and probability calculations Describes the steps for calculating the summation of residuals and probability for similarity weight calculation.', 'The chapter details the process of calculating similarity weight for decision tree splits Explains the steps involved in calculating similarity weight for decision tree splits.', 'The gain computation results in a total gain of 0.21 Specifies the total gain obtained from the gain computation for decision tree splits.']}, {'end': 1006.648, 'segs': [{'end': 765.914, 'src': 'embed', 'start': 733.637, 'weight': 1, 'content': [{'end': 735.738, 'text': 'Here I will be putting good and normal.', 'start': 733.637, 'duration': 2.101}, {'end': 737.419, 'text': "Here I'll be putting bad.", 'start': 736.199, 'duration': 1.22}, {'end': 743.383, 'text': 'Now with respect to this, how many features you have with respect to bad? Bad is having 0.5 over here.', 'start': 738.14, 'duration': 5.243}, {'end': 751.104, 'text': 'okay and anywhere you are having bad with less than or equal to 50, okay, less than or equal to 50 also.', 'start': 744.54, 'duration': 6.564}, {'end': 751.805, 'text': 'is there.', 'start': 751.104, 'duration': 0.701}, {'end': 753.126, 'text': 'so less than or equal to 50.', 'start': 751.805, 'duration': 1.321}, {'end': 754.567, 'text': 'one bad only is there.', 'start': 753.126, 'duration': 1.441}, {'end': 758.929, 'text': 'remaining. all are good, right and good and normal with respect to less than or equal to 50.', 'start': 754.567, 'duration': 4.362}, {'end': 760.49, 'text': 'how many points we have?', 'start': 758.929, 'duration': 1.561}, {'end': 765.914, 'text': 'so we have one point over here, two point over here and three point over here.', 'start': 760.49, 'duration': 5.424}], 'summary': 'Out of the total points, one is bad and the rest are good and normal.', 'duration': 32.277, 'max_score': 733.637, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ733637.jpg'}, {'end': 803.261, 'src': 'embed', 'start': 775.862, 'weight': 4, 'content': [{'end': 782.746, 'text': 'whether I should go with this credit split of blue green comma normal in the other split, I mean in the other branch,', 'start': 775.862, 'duration': 6.884}, {'end': 787.008, 'text': 'or can I use blue comma green in this that I have to compute in this right?', 'start': 782.746, 'duration': 4.262}, {'end': 794.895, 'text': "So again I'll go and compute the similarity matrix sorry similarity weight in this case when I have just one point that is minus 0.5.", 'start': 787.329, 'duration': 7.566}, {'end': 796.196, 'text': 'right minus, 0.5.', 'start': 794.895, 'duration': 1.301}, {'end': 797.237, 'text': 'how do i compute it again?', 'start': 796.196, 'duration': 1.041}, {'end': 798.798, 'text': "i'll try to use the summation of residual.", 'start': 797.237, 'duration': 1.561}, {'end': 803.261, 'text': 'residual is only one, so this will be nothing but 0.25 divided by 0.5.', 'start': 798.798, 'duration': 4.463}], 'summary': 'The similarity weight computation results in 0.25 divided by 0.5.', 'duration': 27.399, 'max_score': 775.862, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ775862.jpg'}, {'end': 952.766, 'src': 'embed', 'start': 921.166, 'weight': 0, 'content': [{'end': 922.347, 'text': 'Okay? We will try to do the split.', 'start': 921.166, 'duration': 1.181}, {'end': 929.048, 'text': 'Right? We will try to see out of these two which will be giving the highest information gain I will be selecting that split.', 'start': 922.947, 'duration': 6.101}, {'end': 931.369, 'text': 'Understand? In that specific way guys.', 'start': 929.769, 'duration': 1.6}, {'end': 940.664, 'text': "whichever will be giving us the highest split, we will be taking that okay, and with respect to this, i'm going to write this is 0.33.", 'start': 932.662, 'duration': 8.002}, {'end': 942.824, 'text': 'okay, this is 0.33.', 'start': 940.664, 'duration': 2.16}, {'end': 944.284, 'text': 'this is the similarity weight 0.33.', 'start': 942.824, 'duration': 1.46}, {'end': 952.766, 'text': "okay, now, with respect to this, when i do a split on blue green sorry, blue green i'm saying bad and good, right, and remember,", 'start': 944.284, 'duration': 8.482}], 'summary': 'Selecting split with highest information gain, similarity weight 0.33.', 'duration': 31.6, 'max_score': 921.166, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ921166.jpg'}, {'end': 986.002, 'src': 'embed', 'start': 962.549, 'weight': 3, 'content': [{'end': 969.238, 'text': 'Now here when I am doing this split again how many points is basically good and good 1, 2, 3.', 'start': 962.549, 'duration': 6.689}, {'end': 971.62, 'text': 'Right So good and bad 1, 2, 3.', 'start': 969.238, 'duration': 2.382}, {'end': 973.683, 'text': 'Then 1, 2, 3 basically means over here is nothing but 0.5, 0.5, 0.5.', 'start': 971.622, 'duration': 2.061}, {'end': 980.761, 'text': 'Right Then if I try to compute the same similarity matrix for this.', 'start': 973.685, 'duration': 7.076}, {'end': 982.921, 'text': 'How much it will be? It will be same right.', 'start': 981.141, 'duration': 1.78}, {'end': 984.682, 'text': 'Because we got over here same thing.', 'start': 983.442, 'duration': 1.24}, {'end': 986.002, 'text': 'So this will be nothing but 0.33.', 'start': 985.062, 'duration': 0.94}], 'summary': 'A similarity matrix computation resulted in a value of 0.33.', 'duration': 23.453, 'max_score': 962.549, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ962549.jpg'}], 'start': 733.637, 'title': 'Credit split analysis and information gain calculation', 'summary': "Discusses credit split analysis based on 'good', 'bad', and 'normal' features, with 'bad' having a value of 0.5 and occurring only once, while 'good' and 'normal' have three points each. it also explains the computation of similarity weight and information gain, with the highest gain being 1.33 and a split on 'credit' and 'income' attributes yielding the same similarity weight of 0.33.", 'chapters': [{'end': 797.237, 'start': 733.637, 'title': 'Credit split analysis', 'summary': "Discusses the analysis of credit split based on the features 'good', 'bad' and 'normal', where 'bad' has a value of 0.5 and occurs only once, while 'good' and 'normal' have three points each, with the decision of credit split to be determined based on the computed similarity weight.", 'duration': 63.6, 'highlights': ["The value of 'bad' is 0.5, occurring only once, while 'good' and 'normal' have three points each.", 'The decision of credit split is to be determined based on the computed similarity weight.', "The process involves computing similarity weight based on the occurrence and values of the features 'good', 'bad', and 'normal'."]}, {'end': 1006.648, 'start': 797.237, 'title': 'Information gain calculation', 'summary': "Explains the computation of similarity weight and information gain for different splits in a decision tree, with the highest gain being 1.33 and a split on 'credit' and 'income' attributes yielding the same similarity weight of 0.33.", 'duration': 209.411, 'highlights': ['The total gain with respect to the split is 1.33. The highest information gain obtained from the split, indicating the most valuable decision point in the tree.', "A split on 'credit' and 'income' attributes yields a similarity weight of 0.33. The computation of similarity weight for a specific split, providing insight into the relevance of the split in the decision-making process.", 'The similarity weight will be equal to 1. Explanation of the computation process leading to a similarity weight of 1 for a specific split, indicating a strong relevance of the split in the decision tree.']}], 'duration': 273.011, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ733637.jpg', 'highlights': ['The highest information gain obtained from the split is 1.33, indicating the most valuable decision point in the tree.', "The value of 'bad' is 0.5, occurring only once, while 'good' and 'normal' have three points each.", "A split on 'credit' and 'income' attributes yields a similarity weight of 0.33, providing insight into the relevance of the split in the decision-making process.", "The process involves computing similarity weight based on the occurrence and values of the features 'good', 'bad', and 'normal'.", 'The decision of credit split is to be determined based on the computed similarity weight.']}, {'end': 1437.294, 'segs': [{'end': 1036.886, 'src': 'embed', 'start': 1008.277, 'weight': 0, 'content': [{'end': 1009.798, 'text': 'I create this tree now.', 'start': 1008.277, 'duration': 1.521}, {'end': 1012.539, 'text': 'after this, whether we should do the splitting or not.', 'start': 1009.798, 'duration': 2.741}, {'end': 1015.42, 'text': 'that is actually covered by post pruning.', 'start': 1012.539, 'duration': 2.881}, {'end': 1016.7, 'text': 'post pruning.', 'start': 1015.42, 'duration': 1.28}, {'end': 1020.542, 'text': 'post pruning is basically calculated in XGBoost by a cover value.', 'start': 1016.7, 'duration': 3.842}, {'end': 1022.262, 'text': 'what is this cover value?', 'start': 1020.542, 'duration': 1.72}, {'end': 1026.044, 'text': 'whatever thing we have actually taken in this denominator.', 'start': 1022.262, 'duration': 3.782}, {'end': 1027.097, 'text': 'Okay,', 'start': 1026.896, 'duration': 0.201}, {'end': 1031.06, 'text': 'So this denominator is nothing but probability of 1 minus prob.', 'start': 1027.438, 'duration': 3.622}, {'end': 1033.423, 'text': 'So this will be my cover value.', 'start': 1031.681, 'duration': 1.742}, {'end': 1036.886, 'text': 'Now in this case my cover value is nothing but 0.25.', 'start': 1034.163, 'duration': 2.723}], 'summary': 'Xgboost uses post pruning with cover value of 0.25.', 'duration': 28.609, 'max_score': 1008.277, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ1008277.jpg'}, {'end': 1051.913, 'src': 'heatmap', 'start': 1008.277, 'weight': 0.844, 'content': [{'end': 1009.798, 'text': 'I create this tree now.', 'start': 1008.277, 'duration': 1.521}, {'end': 1012.539, 'text': 'after this, whether we should do the splitting or not.', 'start': 1009.798, 'duration': 2.741}, {'end': 1015.42, 'text': 'that is actually covered by post pruning.', 'start': 1012.539, 'duration': 2.881}, {'end': 1016.7, 'text': 'post pruning.', 'start': 1015.42, 'duration': 1.28}, {'end': 1020.542, 'text': 'post pruning is basically calculated in XGBoost by a cover value.', 'start': 1016.7, 'duration': 3.842}, {'end': 1022.262, 'text': 'what is this cover value?', 'start': 1020.542, 'duration': 1.72}, {'end': 1026.044, 'text': 'whatever thing we have actually taken in this denominator.', 'start': 1022.262, 'duration': 3.782}, {'end': 1027.097, 'text': 'Okay,', 'start': 1026.896, 'duration': 0.201}, {'end': 1031.06, 'text': 'So this denominator is nothing but probability of 1 minus prob.', 'start': 1027.438, 'duration': 3.622}, {'end': 1033.423, 'text': 'So this will be my cover value.', 'start': 1031.681, 'duration': 1.742}, {'end': 1036.886, 'text': 'Now in this case my cover value is nothing but 0.25.', 'start': 1034.163, 'duration': 2.723}, {'end': 1049.091, 'text': "That basically means if my, if my probability value is 0.25, it's less than 0.25, or if my gain, if I'm considering that my gain is less than 0.25,", 'start': 1036.886, 'duration': 12.205}, {'end': 1051.913, 'text': "I'll just cut the branch that is called as post pruning.", 'start': 1049.091, 'duration': 2.822}], 'summary': 'Xgboost uses post pruning with cover value to cut branches with gain less than 0.25.', 'duration': 43.636, 'max_score': 1008.277, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ1008277.jpg'}, {'end': 1084.459, 'src': 'embed', 'start': 1057.976, 'weight': 4, 'content': [{'end': 1065.601, 'text': 'okay, and here you can actually do the further split with respect to this also, you can take credit over here and you can do this split also,', 'start': 1057.976, 'duration': 7.625}, {'end': 1067.402, 'text': 'and you can basically construct your decision tree.', 'start': 1065.601, 'duration': 1.801}, {'end': 1076.032, 'text': 'okay, but always remember one point right based on your similarity, weight and based on your cover pruning, you can cut the tree,', 'start': 1068.106, 'duration': 7.926}, {'end': 1079.715, 'text': 'you can branch it out, and that is actually called as pre pruning.', 'start': 1076.032, 'duration': 3.683}, {'end': 1084.459, 'text': 'okay, or you can also say it as post pruning okay, because once you construct it and then you cover the value,', 'start': 1079.715, 'duration': 4.744}], 'summary': 'Decision tree can be split based on similarity, weight, and cover pruning, leading to pre or post pruning.', 'duration': 26.483, 'max_score': 1057.976, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ1057976.jpg'}, {'end': 1170.795, 'src': 'heatmap', 'start': 1101.172, 'weight': 0.843, 'content': [{'end': 1108.037, 'text': 'like that, i may construct any number of decision tree once i calculate the residuals once i calculate the residuals.', 'start': 1101.172, 'duration': 6.865}, {'end': 1115.403, 'text': 'okay, now, with respect to this, whenever i get a new data, new data, suppose i get new data, something like this less than 50k,', 'start': 1108.037, 'duration': 7.366}, {'end': 1116.464, 'text': 'then the credit is bad.', 'start': 1115.403, 'duration': 1.061}, {'end': 1118.825, 'text': 'so less than 50k, the credit is bad.', 'start': 1116.464, 'duration': 2.361}, {'end': 1120.487, 'text': "so i'm getting the similarity weight as one.", 'start': 1118.825, 'duration': 1.662}, {'end': 1122.989, 'text': 'okay, so the output of this will be one.', 'start': 1121.067, 'duration': 1.922}, {'end': 1124.511, 'text': 'but how do I compute it?', 'start': 1122.989, 'duration': 1.522}, {'end': 1125.452, 'text': 'how do I compute it?', 'start': 1124.511, 'duration': 0.941}, {'end': 1126.673, 'text': 'so first of all,', 'start': 1125.452, 'duration': 1.221}, {'end': 1134.783, 'text': 'I really need to find out the base model output with respect to this probability and to calculate that I use this formula log of odds.', 'start': 1126.673, 'duration': 8.11}, {'end': 1135.383, 'text': 'this is just like.', 'start': 1134.783, 'duration': 0.6}, {'end': 1139.165, 'text': 'Probably you have seen this equation also in logistic regression.', 'start': 1136.364, 'duration': 2.801}, {'end': 1142.646, 'text': 'So here I am just going to replace p by 0.5.', 'start': 1139.765, 'duration': 2.881}, {'end': 1146.407, 'text': 'So this will nothing be but log of 0.5 divided by 1 minus p.', 'start': 1142.646, 'duration': 3.761}, {'end': 1148.167, 'text': '1 minus p means 0.5.', 'start': 1146.567, 'duration': 1.6}, {'end': 1150.008, 'text': 'Then again you are going to get 0.5.', 'start': 1148.167, 'duration': 1.841}, {'end': 1152.689, 'text': 'This 0.5 by 0.5 is nothing but 1.', 'start': 1150.008, 'duration': 2.681}, {'end': 1153.969, 'text': 'Log of 1 is nothing but 0.', 'start': 1152.689, 'duration': 1.28}, {'end': 1158.052, 'text': 'So this base model will be giving me the output as 0.', 'start': 1153.969, 'duration': 4.083}, {'end': 1160.353, 'text': 'right, it will be giving me the output as 0.', 'start': 1158.052, 'duration': 2.301}, {'end': 1162.633, 'text': 'so once I get the output as 0,', 'start': 1160.353, 'duration': 2.28}, {'end': 1170.795, 'text': 'if I really wanted to find out what will be the record output or record residual with respect to this right or record probability also with respect to this,', 'start': 1162.633, 'duration': 8.162}], 'summary': 'Decision tree calculated residuals for new data, determining bad credit for income less than 50k.', 'duration': 69.623, 'max_score': 1101.172, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ1101172.jpg'}, {'end': 1152.689, 'src': 'embed', 'start': 1122.989, 'weight': 1, 'content': [{'end': 1124.511, 'text': 'but how do I compute it?', 'start': 1122.989, 'duration': 1.522}, {'end': 1125.452, 'text': 'how do I compute it?', 'start': 1124.511, 'duration': 0.941}, {'end': 1126.673, 'text': 'so first of all,', 'start': 1125.452, 'duration': 1.221}, {'end': 1134.783, 'text': 'I really need to find out the base model output with respect to this probability and to calculate that I use this formula log of odds.', 'start': 1126.673, 'duration': 8.11}, {'end': 1135.383, 'text': 'this is just like.', 'start': 1134.783, 'duration': 0.6}, {'end': 1139.165, 'text': 'Probably you have seen this equation also in logistic regression.', 'start': 1136.364, 'duration': 2.801}, {'end': 1142.646, 'text': 'So here I am just going to replace p by 0.5.', 'start': 1139.765, 'duration': 2.881}, {'end': 1146.407, 'text': 'So this will nothing be but log of 0.5 divided by 1 minus p.', 'start': 1142.646, 'duration': 3.761}, {'end': 1148.167, 'text': '1 minus p means 0.5.', 'start': 1146.567, 'duration': 1.6}, {'end': 1150.008, 'text': 'Then again you are going to get 0.5.', 'start': 1148.167, 'duration': 1.841}, {'end': 1152.689, 'text': 'This 0.5 by 0.5 is nothing but 1.', 'start': 1150.008, 'duration': 2.681}], 'summary': 'The base model output can be calculated using the formula log of odds, replacing p with 0.5, resulting in an output of 1.', 'duration': 29.7, 'max_score': 1122.989, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ1122989.jpg'}, {'end': 1238.194, 'src': 'heatmap', 'start': 1208.959, 'weight': 0.701, 'content': [{'end': 1212.427, 'text': 'And again, remember, guys, this learning rate will be between 0 to 1..', 'start': 1208.959, 'duration': 3.468}, {'end': 1214.792, 'text': 'Suppose I take this learning rate as 0.1.', 'start': 1212.427, 'duration': 2.365}, {'end': 1215.935, 'text': 'And I take this value of.', 'start': 1214.792, 'duration': 1.143}, {'end': 1222.324, 'text': 'similarity weight as 1, how much is this value? This is nothing but 0.1.', 'start': 1217.04, 'duration': 5.284}, {'end': 1226.847, 'text': 'And then I apply my sigmoid activation function or sigmoid function.', 'start': 1222.324, 'duration': 4.523}, {'end': 1234.892, 'text': 'This sigmoid function is nothing but, if I write it over here, 1 divided by 1 plus e to the power of minus 0.1.', 'start': 1227.207, 'duration': 7.685}, {'end': 1238.194, 'text': 'So whatever value I get, suppose I get it as 0.6.', 'start': 1234.892, 'duration': 3.302}], 'summary': 'Learning rate ranges from 0 to 1; with rate 0.1 and similarity weight 1, sigmoid function yields 0.6.', 'duration': 29.235, 'max_score': 1208.959, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ1208959.jpg'}, {'end': 1407.93, 'src': 'embed', 'start': 1366.752, 'weight': 2, 'content': [{'end': 1368.573, 'text': 'Then again, it will be like this, alpha of T3.', 'start': 1366.752, 'duration': 1.821}, {'end': 1372.075, 'text': "Like this, we'll try to do till alpha of T of n.", 'start': 1369.053, 'duration': 3.022}, {'end': 1374.517, 'text': 'And once we do, once we apply the sigmoid function.', 'start': 1372.075, 'duration': 2.442}, {'end': 1379.64, 'text': "finally, you'll be able to see that my probability will be giving a good value where my residual will vary very less.", 'start': 1374.517, 'duration': 5.123}, {'end': 1384.944, 'text': 'At that time, I can consider that XGBoost tree has been trained in a proper way.', 'start': 1380.341, 'duration': 4.603}, {'end': 1389.307, 'text': 'okay?. One more thing that I missed to tell you is, guys this about this particular alpha value', 'start': 1384.944, 'duration': 4.363}, {'end': 1396.293, 'text': 'Over here in all the time I have taken when I am computing the similarity weight I have taken alpha is equal to 0.', 'start': 1390.227, 'duration': 6.066}, {'end': 1399.896, 'text': 'You can always select this alpha value which is like an hyperparameter.', 'start': 1396.293, 'duration': 3.603}, {'end': 1407.93, 'text': 'So this is in short how the XGBoost tree actually gets trained.', 'start': 1403.947, 'duration': 3.983}], 'summary': 'Xgboost tree trained with alpha value 0 leads to low residual variance, indicating proper training.', 'duration': 41.178, 'max_score': 1366.752, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ1366752.jpg'}], 'start': 1008.277, 'title': 'Xgboost tree training', 'summary': 'Covers xgboost pruning techniques including post-pruning, pre-pruning with a cover value of 0.25, and construction of decision trees. it also explains the process of training an xgboost tree involving base model output calculation, updating probabilities, and minimizing residuals.', 'chapters': [{'end': 1122.989, 'start': 1008.277, 'title': 'Xgboost pruning techniques', 'summary': 'Explains the concepts of post pruning and pre pruning in xgboost, with a cover value of 0.25 determining the branch cutting. it also highlights the process of constructing decision trees and handling new data.', 'duration': 114.712, 'highlights': ['The cover value in XGBoost is calculated as the probability of 1 minus prob, with a specific value of 0.25 determining whether to cut a branch during post pruning.', 'Constructing decision trees in XGBoost involves considering similarity weight and cover pruning to determine tree cutting, referred to as pre pruning or post pruning.', 'New data in XGBoost is processed through the constructed decision trees, enabling the determination of outputs based on the given input.']}, {'end': 1437.294, 'start': 1122.989, 'title': 'Xgboost tree training', 'summary': 'Explains the process of training an xgboost tree, involving the calculation of base model output, updating probabilities using learning rates and constructing decision trees to minimize residuals, ultimately leading to a trained xgboost tree.', 'duration': 314.305, 'highlights': ['The process starts with calculating the base model output using the log of odds formula, which involves replacing p by 0.5 and results in an output of 0 (relevant for initial probability calculation).', 'The training involves updating the probability using a learning rate and the sigmoid function, leading to the computation of new probabilities for each record (quantifying the impact of learning rate on probability update).', 'Constructing decision trees and updating probabilities based on residuals is a key step in the training process, with the aim of minimizing the residuals and ensuring the XGBoost tree is trained properly (highlighting the iterative nature of decision tree construction and probability update).', 'The alpha value, serving as a learning rate hyperparameter, plays a crucial role in computing the similarity weight and updating probabilities, impacting the training process of the XGBoost tree (emphasizing the significance of the alpha value as a hyperparameter).']}], 'duration': 429.017, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/gPciUPwWJQQ/pics/gPciUPwWJQQ1008277.jpg', 'highlights': ['The cover value in XGBoost is calculated as the probability of 1 minus prob, with a specific value of 0.25 determining whether to cut a branch during post pruning.', 'The process starts with calculating the base model output using the log of odds formula, which involves replacing p by 0.5 and results in an output of 0 (relevant for initial probability calculation).', 'Constructing decision trees and updating probabilities based on residuals is a key step in the training process, with the aim of minimizing the residuals and ensuring the XGBoost tree is trained properly (highlighting the iterative nature of decision tree construction and probability update).', 'The alpha value, serving as a learning rate hyperparameter, plays a crucial role in computing the similarity weight and updating probabilities, impacting the training process of the XGBoost tree (emphasizing the significance of the alpha value as a hyperparameter).', 'Constructing decision trees in XGBoost involves considering similarity weight and cover pruning to determine tree cutting, referred to as pre pruning or post pruning.']}], 'highlights': ['The video is expected to be 20 to 25 minutes long, focusing on practical implementation and in-depth explanation.', 'It outlines the differences between bagging and boosting algorithms and emphasizes the need to understand gradient boosting for comprehending XGBoost.', 'The chapter introduces XGBoost classifier, covering its importance, relationship with ensemble techniques, and planned content for future videos.', 'The chapter demonstrates the application of HGBoost in a simple example with salary, credit score, and approval, showcasing its usefulness in solving multi-class classification problems.', 'XGBoost requires constructing a tree using binary classifiers for multiple categories, ensuring that each leaf node is divided into two splits, such as less than or equal to 50k and greater than 50k, and computing similarity weight and information gain.', 'The computation of similarity weight is crucial in XGBoost, as it determines the basis for splitting the leaf nodes, influencing the overall decision-making process.', 'The process includes the summation of residuals and probability calculations Describes the steps for calculating the summation of residuals and probability for similarity weight calculation.', 'The chapter details the process of calculating similarity weight for decision tree splits Explains the steps involved in calculating similarity weight for decision tree splits.', 'The highest information gain obtained from the split is 1.33, indicating the most valuable decision point in the tree.', "The value of 'bad' is 0.5, occurring only once, while 'good' and 'normal' have three points each.", "A split on 'credit' and 'income' attributes yields a similarity weight of 0.33, providing insight into the relevance of the split in the decision-making process.", 'The cover value in XGBoost is calculated as the probability of 1 minus prob, with a specific value of 0.25 determining whether to cut a branch during post pruning.', 'The process starts with calculating the base model output using the log of odds formula, which involves replacing p by 0.5 and results in an output of 0 (relevant for initial probability calculation).', 'Constructing decision trees and updating probabilities based on residuals is a key step in the training process, with the aim of minimizing the residuals and ensuring the XGBoost tree is trained properly (highlighting the iterative nature of decision tree construction and probability update).', 'The alpha value, serving as a learning rate hyperparameter, plays a crucial role in computing the similarity weight and updating probabilities, impacting the training process of the XGBoost tree (emphasizing the significance of the alpha value as a hyperparameter).', 'Constructing decision trees in XGBoost involves considering similarity weight and cover pruning to determine tree cutting, referred to as pre pruning or post pruning.']}