title

Tutorial 27- Ridge and Lasso Regression Indepth Intuition- Data Science

description

Please join as a member in my channel to get additional benefits like materials in Data Science, live streaming for Members and many more
https://www.youtube.com/channel/UCNU_lfiiWBdtULKOw6X0Dig/join
#Regularization
⭐ Kite is a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I've been using Kite for a few months and I love it! https://www.kite.com/get-kite/?utm_medium=referral&utm_source=youtube&utm_campaign=krishnaik&utm_content=description-only
Please do subscribe my other channel too
https://www.youtube.com/channel/UCjWY5hREA6FFYrthD0rZNIw
Connect with me here:
Twitter: https://twitter.com/Krishnaik06
Facebook: https://www.facebook.com/krishnaik06
instagram: https://www.instagram.com/krishnaik06

detail

{'title': 'Tutorial 27- Ridge and Lasso Regression Indepth Intuition- Data Science', 'heatmap': [{'end': 1123.5, 'start': 1089.774, 'weight': 0.803}], 'summary': 'Tutorial delves into ridge and lasso regression, addressing overfitting and underfitting in machine learning, emphasizing the significance of low bias and low variance models, and explaining the iterative process of optimizing the cost function in linear regression using ridge regression to minimize errors and overfitting.', 'chapters': [{'end': 171.303, 'segs': [{'end': 29.285, 'src': 'embed', 'start': 0.169, 'weight': 0, 'content': [{'end': 2.851, 'text': 'hello. all my name is krishna and welcome to my youtube channel today.', 'start': 0.169, 'duration': 2.682}, {'end': 7.473, 'text': "in this particular video we'll be discussing about ridge and lasso regression.", 'start': 2.851, 'duration': 4.622}, {'end': 14.377, 'text': "these are some kind of regularization, hyper tuning techniques, uh, and i'll try to explain you in this particular video completely.", 'start': 7.473, 'duration': 6.904}, {'end': 18.939, 'text': "uh, we'll just go in depth, understanding all the techniques and why exactly it is used.", 'start': 14.377, 'duration': 4.562}, {'end': 23.522, 'text': 'in my previous video, i have already uploaded a tutorial regarding linear regression.', 'start': 18.939, 'duration': 4.583}, {'end': 29.285, 'text': 'guys, Please make sure that you know how linear regression is implemented in order to follow this particular video.', 'start': 23.522, 'duration': 5.763}], 'summary': 'Krishna discusses ridge and lasso regression for hyper tuning techniques in this video.', 'duration': 29.116, 'max_score': 0.169, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s169.jpg'}, {'end': 59.004, 'src': 'embed', 'start': 33.208, 'weight': 1, 'content': [{'end': 41.973, 'text': 'I have already uploaded many videos regarding bias and variance and also in my previous tutorial of this specific playlist I have also uploaded linear regression.', 'start': 33.208, 'duration': 8.765}, {'end': 43.514, 'text': 'I have explained the maths behind it.', 'start': 41.994, 'duration': 1.52}, {'end': 50.799, 'text': 'Now let us go and understand how exactly ridge and lasso regression works, and make sure, guys, you watch this video till the end,', 'start': 43.875, 'duration': 6.924}, {'end': 52.8, 'text': "because I'm going to explain various things.", 'start': 50.799, 'duration': 2.001}, {'end': 59.004, 'text': "I'm going to cover both the topic in this same video, because there is some minor difference between ridge and lasso regression.", 'start': 52.8, 'duration': 6.204}], 'summary': 'Explains ridge and lasso regression in tutorial videos.', 'duration': 25.796, 'max_score': 33.208, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s33208.jpg'}, {'end': 121.809, 'src': 'embed', 'start': 93.82, 'weight': 2, 'content': [{'end': 96.643, 'text': 'Now, when do we use ridge and lasso regression?', 'start': 93.82, 'duration': 2.823}, {'end': 97.484, 'text': 'we need to find out.', 'start': 96.643, 'duration': 0.841}, {'end': 98.746, 'text': 'We need to understand.', 'start': 97.664, 'duration': 1.082}, {'end': 101.028, 'text': 'Let me just give you a very small example guys.', 'start': 99.106, 'duration': 1.922}, {'end': 106.654, 'text': 'Suppose I consider that I am training a model wherein, based on the years of experience,', 'start': 101.449, 'duration': 5.205}, {'end': 112.24, 'text': 'I need to predict salary and just understand that my training data has only two points, which is like this', 'start': 106.654, 'duration': 5.586}, {'end': 118.006, 'text': 'okay, now, when i am actually creating the best fit line with the help of linear regression,', 'start': 112.961, 'duration': 5.045}, {'end': 121.809, 'text': 'i usually create this best fit line which passes through this particular point.', 'start': 118.006, 'duration': 3.803}], 'summary': 'Ridge and lasso regression comparison in small data scenario.', 'duration': 27.989, 'max_score': 93.82, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s93820.jpg'}], 'start': 0.169, 'title': 'Ridge and lasso regression', 'summary': 'Covers the concepts of ridge and lasso regression, explaining their differences and use cases, and provides an example of how they are used in comparison to linear regression, emphasizing the importance of reducing the cost function in model training.', 'chapters': [{'end': 171.303, 'start': 0.169, 'title': 'Understanding ridge and lasso regression', 'summary': 'Covers the concepts of ridge and lasso regression, explaining their differences and use cases, and provides an example of how they are used in comparison to linear regression, emphasizing the importance of reducing the cost function in model training.', 'duration': 171.134, 'highlights': ['The chapter explains the concepts of ridge and lasso regression, highlighting their use as regularization and hyper tuning techniques to reduce the cost function during model training.', 'It emphasizes the importance of understanding linear regression and bias and variance before delving into ridge and lasso regression for better comprehension.', 'The speaker provides an example of using linear regression to predict salary based on years of experience and contrasts it with the scenarios where ridge and lasso regression are more suitable, showcasing the need to minimize the cost function in these cases.']}], 'duration': 171.134, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s169.jpg', 'highlights': ['The chapter explains ridge and lasso regression as regularization techniques to reduce cost function.', 'Understanding linear regression, bias, and variance is emphasized before delving into ridge and lasso regression.', 'An example of using linear regression to predict salary is contrasted with scenarios where ridge and lasso regression are more suitable.']}, {'end': 294.969, 'segs': [{'end': 203.768, 'src': 'embed', 'start': 171.723, 'weight': 2, 'content': [{'end': 174.327, 'text': 'Now, understand guys, this is for my training data set.', 'start': 171.723, 'duration': 2.604}, {'end': 176.53, 'text': 'This is for my training.', 'start': 175.508, 'duration': 1.022}, {'end': 177.311, 'text': 'data. set right?', 'start': 176.53, 'duration': 0.781}, {'end': 181.043, 'text': 'Now, imagine and always remember.', 'start': 178.803, 'duration': 2.24}, {'end': 187.805, 'text': 'whenever we want to create any model, let it be a classification model or regression model, we need to create a generalized model.', 'start': 181.043, 'duration': 6.762}, {'end': 193.406, 'text': 'That basically means that, now suppose for my test data, I have some of the points over here.', 'start': 189.005, 'duration': 4.401}, {'end': 196.726, 'text': 'Suppose this is my test data.', 'start': 194.146, 'duration': 2.58}, {'end': 203.768, 'text': "Now what will happen? Now I have got with the help of this particular training data set, I've got this as my best fit line.", 'start': 197.307, 'duration': 6.461}], 'summary': 'Training data used to create a generalized model for classification or regression, resulting in a best fit line.', 'duration': 32.045, 'max_score': 171.723, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s171723.jpg'}, {'end': 242.394, 'src': 'embed', 'start': 217.893, 'weight': 0, 'content': [{'end': 224.561, 'text': 'Overfitting That basically means for my training data set, my model has given me a wonderful result.', 'start': 217.893, 'duration': 6.668}, {'end': 227.023, 'text': 'That is completely low bias result.', 'start': 225.061, 'duration': 1.962}, {'end': 230.347, 'text': 'You know, low bias or less error result you can say.', 'start': 227.564, 'duration': 2.783}, {'end': 235.292, 'text': 'But when I go and test for test data set now you can see my test data set is over here, right?', 'start': 230.787, 'duration': 4.505}, {'end': 238.076, 'text': 'My new point is over here and it is predicting somewhere here.', 'start': 235.413, 'duration': 2.663}, {'end': 242.394, 'text': 'Now, when I do that, there is a huge difference.', 'start': 239.772, 'duration': 2.622}], 'summary': 'Overfitting occurs when the model performs well on training data but poorly on test data.', 'duration': 24.501, 'max_score': 217.893, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s217893.jpg'}, {'end': 287.223, 'src': 'embed', 'start': 257.764, 'weight': 1, 'content': [{'end': 260.285, 'text': 'So, that is the basic difference between overfitting.', 'start': 257.764, 'duration': 2.521}, {'end': 264.686, 'text': 'In case of underfitting, what will happen? For the training dataset also, you are getting high error.', 'start': 260.605, 'duration': 4.081}, {'end': 267.727, 'text': 'For the test dataset also, you are getting high error.', 'start': 264.966, 'duration': 2.761}, {'end': 268.827, 'text': 'That is what is underfitting.', 'start': 267.747, 'duration': 1.08}, {'end': 270.848, 'text': 'Now, I hope I made you understand.', 'start': 269.147, 'duration': 1.701}, {'end': 274.849, 'text': 'Let me just draw this particular point once again so that you will be able to understand.', 'start': 270.868, 'duration': 3.981}, {'end': 276.669, 'text': 'Now, let me make you understand once again.', 'start': 275.009, 'duration': 1.66}, {'end': 278.97, 'text': 'Suppose this is my dataset.', 'start': 276.969, 'duration': 2.001}, {'end': 280.45, 'text': 'This is my training dataset.', 'start': 279.03, 'duration': 1.42}, {'end': 282.091, 'text': 'I have created a best fit line.', 'start': 280.81, 'duration': 1.281}, {'end': 287.223, 'text': 'But suppose my new points or new test data is here, here, here.', 'start': 283.319, 'duration': 3.904}], 'summary': 'Explains overfitting and underfitting, emphasizing high error for training and test datasets.', 'duration': 29.459, 'max_score': 257.764, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s257764.jpg'}], 'start': 171.723, 'title': 'Overfitting and underfitting in machine learning', 'summary': 'Explains the concept of overfitting and underfitting in machine learning, emphasizing the importance of creating a generalized model to avoid low bias for the training data set and higher error for the test data set.', 'chapters': [{'end': 294.969, 'start': 171.723, 'title': 'Overfitting and underfitting in machine learning', 'summary': 'Explains the concept of overfitting and underfitting in machine learning, emphasizing the importance of creating a generalized model to avoid low bias for the training data set and higher error for the test data set.', 'duration': 123.246, 'highlights': ['Overfitting occurs when the model has low bias or less error for the training data set, but results in higher error for the test data set, leading to a significant difference in predictions. Overfitting results in low bias or less error for the training data set, but higher error for the test data set, causing a significant discrepancy in predictions.', 'Underfitting refers to a scenario where both the training and test datasets yield high error, indicating a lack of model generalization and suitability for the data. Underfitting occurs when both the training and test datasets yield high error, indicating a lack of model generalization and suitability for the data.', 'Emphasizes the importance of creating a generalized model to ensure low bias for the training data set and avoid higher error for the test data set. The chapter stresses the significance of creating a generalized model to maintain low bias for the training data set and prevent higher error for the test data set.']}], 'duration': 123.246, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s171723.jpg', 'highlights': ['Overfitting results in low bias for the training data set but higher error for the test data set, causing significant discrepancies in predictions.', 'Underfitting occurs when both the training and test datasets yield high error, indicating a lack of model generalization and suitability for the data.', 'The chapter stresses the significance of creating a generalized model to maintain low bias for the training data set and prevent higher error for the test data set.']}, {'end': 596.417, 'segs': [{'end': 343.664, 'src': 'embed', 'start': 295.709, 'weight': 0, 'content': [{'end': 298.872, 'text': 'And if I go and see this particular difference, this is huge.', 'start': 295.709, 'duration': 3.163}, {'end': 301.794, 'text': 'So this is basically leading to an overfitting condition.', 'start': 299.212, 'duration': 2.582}, {'end': 311.082, 'text': 'Now, we can use ridge and lasso regression so that we make this overfitting condition, which is basically high variance.', 'start': 302.555, 'duration': 8.527}, {'end': 315.971, 'text': 'We can convert this high variance into low variance.', 'start': 312.329, 'duration': 3.642}, {'end': 319.473, 'text': "Now we'll try to understand.", 'start': 318.112, 'duration': 1.361}, {'end': 326.556, 'text': 'Always remember guys, a generalized model or a good model should always have low bias and low variance.', 'start': 319.613, 'duration': 6.943}, {'end': 333.219, 'text': 'Okay So that is as much as low, you know, as much as low for both low bias and low variance.', 'start': 327.537, 'duration': 5.682}, {'end': 337.982, 'text': 'It should have this kind of properties that particular model is basically called as a generalized model.', 'start': 333.32, 'duration': 4.662}, {'end': 343.664, 'text': 'So let us go and understand how ridge and lasso regression solves this particular problem.', 'start': 338.442, 'duration': 5.222}], 'summary': 'Using ridge and lasso regression to reduce overfitting, achieving low bias and low variance in a generalized model.', 'duration': 47.955, 'max_score': 295.709, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s295709.jpg'}, {'end': 436.127, 'src': 'embed', 'start': 407.037, 'weight': 3, 'content': [{'end': 415.604, 'text': 'right, and you know that in linear regression we try to reduce this error, but in ridge regression what we do is that we add two more parameters.', 'start': 407.037, 'duration': 8.567}, {'end': 417.366, 'text': 'we add one more parameter.', 'start': 415.604, 'duration': 1.762}, {'end': 426.153, 'text': 'so this this whole square difference plus lambda multiplied by slope, whole square.', 'start': 417.366, 'duration': 8.787}, {'end': 432.985, 'text': 'Now we will try to reduce this whole operation in case of ridge regression.', 'start': 427.742, 'duration': 5.243}, {'end': 436.127, 'text': "I'll make you understand what exactly this is.", 'start': 433.946, 'duration': 2.181}], 'summary': 'Ridge regression adds two parameters to reduce error in linear regression.', 'duration': 29.09, 'max_score': 407.037, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s407037.jpg'}, {'end': 515.663, 'src': 'embed', 'start': 487.855, 'weight': 4, 'content': [{'end': 491.976, 'text': 'So this scenario is something called a steep slope.', 'start': 487.855, 'duration': 4.121}, {'end': 501.599, 'text': 'So understand here in this particular case if we have a steep slope always remember that it will lead to an overfitting case.', 'start': 492.997, 'duration': 8.602}, {'end': 502.84, 'text': 'Why? I will tell you just now.', 'start': 501.679, 'duration': 1.161}, {'end': 504.46, 'text': 'I will just try to explain this.', 'start': 503.16, 'duration': 1.3}, {'end': 511.422, 'text': 'Now you can see that with a unit increase in the years of experience there is a high increase in salary.', 'start': 505.24, 'duration': 6.182}, {'end': 515.663, 'text': 'Now what will happen in this particular case?', 'start': 513.923, 'duration': 1.74}], 'summary': 'Steep slope leads to overfitting, causing high salary increase with experience.', 'duration': 27.808, 'max_score': 487.855, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s487855.jpg'}, {'end': 566.003, 'src': 'embed', 'start': 538.047, 'weight': 5, 'content': [{'end': 541.768, 'text': 'Overfitting and we need to reduce this overfitting with the help of ridge regression.', 'start': 538.047, 'duration': 3.721}, {'end': 543.389, 'text': 'That is what we are trying to do over here.', 'start': 541.888, 'duration': 1.501}, {'end': 548.15, 'text': 'Okay So understand that this particular value is 0 plus.', 'start': 543.789, 'duration': 4.361}, {'end': 557.133, 'text': 'Okay Now lambda consider that this is this lambda this lambda value I will try to take I will try to assign one value as 1.', 'start': 548.45, 'duration': 8.683}, {'end': 566.003, 'text': "I'll tell you what all values we can assign lambda as but usually from 0 to any positive number that particular value can be assigned to lambda.", 'start': 557.133, 'duration': 8.87}], 'summary': 'Reducing overfitting with ridge regression by adjusting lambda values.', 'duration': 27.956, 'max_score': 538.047, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s538047.jpg'}], 'start': 295.709, 'title': 'Ridge and lasso regression', 'summary': 'Discusses the importance of low bias and low variance in a generalized model, and how ridge and lasso regression help in converting high variance to low variance, addressing overfitting conditions. additionally, it explains the concept of ridge regression and its role in reducing overfitting by adding a regularization parameter to the cost function, emphasizing the significance of the slope and the impact of overfitting on the best fit line.', 'chapters': [{'end': 383.163, 'start': 295.709, 'title': 'Understanding ridge and lasso regression', 'summary': 'Discusses the importance of low bias and low variance in a generalized model, and how ridge and lasso regression help in converting high variance to low variance, addressing overfitting conditions.', 'duration': 87.454, 'highlights': ['Ridge and lasso regression are used to convert overfitting conditions, which result in high variance, to low variance, thus contributing to a generalized model with low bias and low variance.', 'A generalized model should always have low bias and low variance, which is crucial for the model to be considered as a generalized model.', 'In linear regression, the cost function is aimed to be minimized, and in the discussed case, a huge error led to an overfitting condition, emphasizing the need for methods like ridge and lasso regression.', 'Understanding how ridge regression works is crucial in addressing overfitting conditions and converting high variance to low variance for a generalized model.']}, {'end': 596.417, 'start': 384.024, 'title': 'Understanding ridge regression', 'summary': 'Explains the concept of ridge regression and its role in reducing overfitting by adding a regularization parameter to the cost function, emphasizing the significance of the slope and the impact of overfitting on the best fit line.', 'duration': 212.393, 'highlights': ['Ridge regression involves adding a regularization parameter to the cost function, which includes a term for the slope squared, to reduce overfitting. Ridge regression introduces a regularization parameter to the cost function, incorporating a term for the slope squared to mitigate overfitting.', 'The significance of a steep slope in linear regression is explained, emphasizing its relation to overfitting and the potential impact on the best fit line passing through all points. The significance of a steep slope in linear regression is highlighted, illustrating its potential to lead to overfitting and the best fit line passing through all points.', 'Explanation of the impact of the regularization parameter (lambda) on the slope and its role in reducing overfitting in ridge regression. Insight into the impact of the regularization parameter (lambda) on the slope and its role in mitigating overfitting in ridge regression.']}], 'duration': 300.708, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s295709.jpg', 'highlights': ['Ridge and lasso regression convert overfitting conditions to low variance, contributing to a generalized model with low bias and variance.', 'A generalized model should always have low bias and low variance, crucial for it to be considered as a generalized model.', 'Understanding how ridge regression works is crucial in addressing overfitting conditions and converting high variance to low variance for a generalized model.', 'Ridge regression involves adding a regularization parameter to the cost function, including a term for the slope squared, to reduce overfitting.', 'The significance of a steep slope in linear regression is explained, emphasizing its relation to overfitting and the potential impact on the best fit line passing through all points.', 'Explanation of the impact of the regularization parameter (lambda) on the slope and its role in reducing overfitting in ridge regression.']}, {'end': 1020.532, 'segs': [{'end': 621.614, 'src': 'embed', 'start': 596.417, 'weight': 0, 'content': [{'end': 601.683, 'text': 'Now, in this particular case, if I do the calculation, it will be nothing but 0 plus 169, which is nothing but 1.69..', 'start': 596.417, 'duration': 5.266}, {'end': 604.867, 'text': 'Okay Now, remember, guys, this is my.', 'start': 601.683, 'duration': 3.184}, {'end': 606.743, 'text': 'sum of residual.', 'start': 605.962, 'duration': 0.781}, {'end': 610.606, 'text': 'this this whole value, and i need to reduce this value.', 'start': 606.743, 'duration': 3.863}, {'end': 613.228, 'text': 'currently it is 1.69.', 'start': 610.606, 'duration': 2.622}, {'end': 614.349, 'text': 'okay, 1.69.', 'start': 613.228, 'duration': 1.121}, {'end': 618.412, 'text': 'now, in in my linear regression, this cost function when it was zero.', 'start': 614.349, 'duration': 4.063}, {'end': 621.614, 'text': "i used to stop there, but now i'm not stopping over here.", 'start': 618.412, 'duration': 3.202}], 'summary': 'Linear regression cost function reduced from 1.69 to 0', 'duration': 25.197, 'max_score': 596.417, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s596417.jpg'}, {'end': 773.012, 'src': 'embed', 'start': 741.203, 'weight': 2, 'content': [{'end': 745.084, 'text': 'We are penalizing features which has higher slopes.', 'start': 741.203, 'duration': 3.881}, {'end': 751.865, 'text': 'What does this mean? Suppose this is my equation y is equal to mx plus c.', 'start': 746.384, 'duration': 5.481}, {'end': 752.586, 'text': 'M is my slope.', 'start': 751.865, 'duration': 0.721}, {'end': 762.128, 'text': 'Now if this slope value is huge, we try to penalize with the help of ridge regression by using this small formula.', 'start': 753.906, 'duration': 8.222}, {'end': 764.161, 'text': 'how we are penalizing.', 'start': 763.1, 'duration': 1.061}, {'end': 767.205, 'text': 'we are using this lambda along with the cost function.', 'start': 764.161, 'duration': 3.044}, {'end': 773.012, 'text': 'we are using these values and every time you will be seeing that our values are actually increasing.', 'start': 767.205, 'duration': 5.807}], 'summary': 'Penalizing features with higher slopes using ridge regression to control slope values with lambda.', 'duration': 31.809, 'max_score': 741.203, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s741203.jpg'}, {'end': 975.946, 'src': 'embed', 'start': 947.684, 'weight': 1, 'content': [{'end': 952.105, 'text': 'I know it looks little bit complicated but our main aim is just to reduce the overfitting.', 'start': 947.684, 'duration': 4.421}, {'end': 954.126, 'text': 'How we are doing reducing the overfitting?', 'start': 952.466, 'duration': 1.66}, {'end': 956.247, 'text': 'This is my sum of residual square right?', 'start': 954.146, 'duration': 2.101}, {'end': 960.249, 'text': 'During linear regression, I have to just minimize this function.', 'start': 957.147, 'duration': 3.102}, {'end': 966.231, 'text': 'But in case of ridge regression, we have to add lambda multiplied by slope square.', 'start': 960.809, 'duration': 5.422}, {'end': 971.233, 'text': 'And lambda value will always be greater than 0 to any positive value.', 'start': 966.751, 'duration': 4.482}, {'end': 975.946, 'text': 'any positive value.', 'start': 974.306, 'duration': 1.64}], 'summary': 'Goal: reduce overfitting in linear and ridge regression by adding lambda multiplied by the slope square.', 'duration': 28.262, 'max_score': 947.684, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s947684.jpg'}], 'start': 596.417, 'title': 'Optimizing cost and ridge regression in linear regression', 'summary': "Explains the iterative process of optimizing the cost function in linear regression to minimize the residual sum, and discusses ridge regression's use in penalizing higher slopes to reduce errors and overfitting, with practical examples and quantifiable values.", 'chapters': [{'end': 716.204, 'start': 596.417, 'title': 'Optimizing cost function in linear regression', 'summary': 'Explains the process of optimizing the cost function in linear regression by iteratively adjusting the best fit line to minimize the residual sum, with a detailed example using quantifiable values.', 'duration': 119.787, 'highlights': ['The cost function is reduced from 1.69 to 1.39 by adjusting the slope of the best fit line, resulting in a decrease in the residual sum.', 'The process involves iteratively finding a line that reduces the cost function, indicating a smaller difference between predicted and actual values.', 'The example demonstrates the decrease in cost function from 1.69 to 1.39 by adjusting the slope to achieve a smaller residual sum.']}, {'end': 817.39, 'start': 717.245, 'title': 'Ridge regression and penalizing higher slopes', 'summary': 'Discusses the concept of penalizing higher slopes using ridge regression to reduce errors, by adding a penalty term to the cost function, and using lambda to make the best fit line less steeper.', 'duration': 100.145, 'highlights': ['Ridge regression penalizes features with higher slopes by adding a penalty term to the cost function, aiming to reduce errors and make the best fit line less steeper.', 'The penalty term in ridge regression, represented by lambda, is used to penalize higher slope values and reduce the steepness of the best fit line.', 'In ridge regression, the penalty term is added to the cost function, resulting in the values being increased to make the best fit line less steeper.']}, {'end': 1020.532, 'start': 817.47, 'title': 'Ridge regression for overfitting reduction', 'summary': 'Discusses the use of ridge regression to reduce overfitting by penalizing steeper slopes, leading to reduced variance and higher bias, while aiming to minimize the sum of residual square function by adding lambda multiplied by slope square, resulting in slopes close to zero.', 'duration': 203.062, 'highlights': ['Ridge regression reduces overfitting by penalizing steeper slopes with lambda multiplied by slope square, leading to reduced variance and higher bias for the training dataset, with the aim of minimizing the sum of residual square function.', 'As lambda values increase, the slopes get very close to zero, effectively minimizing the sum of residual square function and reducing overfitting.', 'Ridge regression aims to minimize the sum of residual square function by adding lambda multiplied by slope square, with lambda always being greater than 0, leading to slopes very close to zero, thereby reducing overfitting.']}], 'duration': 424.115, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s596417.jpg', 'highlights': ['The cost function is reduced from 1.69 to 1.39 by adjusting the slope of the best fit line, resulting in a decrease in the residual sum.', 'Ridge regression reduces overfitting by penalizing steeper slopes with lambda multiplied by slope square, leading to reduced variance and higher bias for the training dataset, with the aim of minimizing the sum of residual square function.', 'Ridge regression penalizes features with higher slopes by adding a penalty term to the cost function, aiming to reduce errors and make the best fit line less steeper.']}, {'end': 1216.044, 'segs': [{'end': 1058.267, 'src': 'embed', 'start': 1020.992, 'weight': 0, 'content': [{'end': 1024.835, 'text': "Now for lasso regression, I'll just make a small equation change.", 'start': 1020.992, 'duration': 3.843}, {'end': 1031.279, 'text': 'In this, instead of writing slope square, this will be magnitude of slope.', 'start': 1025.375, 'duration': 5.904}, {'end': 1033.367, 'text': 'magnitude of slope.', 'start': 1032.327, 'duration': 1.04}, {'end': 1035.69, 'text': 'Now why this is basically used?', 'start': 1034.147, 'duration': 1.543}, {'end': 1046.278, 'text': 'Lasso regression just not only helps in fitting or in overcoming this overfitting scenario, but also it helps us to do feature selection.', 'start': 1036.69, 'duration': 9.588}, {'end': 1058.267, 'text': 'Now how does it helps us to do feature selection? Suppose I have multiple features like M3X3, M4X4 and this is my C1.', 'start': 1047.319, 'duration': 10.948}], 'summary': 'Lasso regression reduces overfitting and aids feature selection.', 'duration': 37.275, 'max_score': 1020.992, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s1020992.jpg'}, {'end': 1123.5, 'src': 'heatmap', 'start': 1089.774, 'weight': 0.803, 'content': [{'end': 1098.876, 'text': 'I told you that whenever I am applying this slope square right, in case of ridge regression, this was finally getting somewhere near to zero.', 'start': 1089.774, 'duration': 9.102}, {'end': 1099.856, 'text': 'right. it was not.', 'start': 1098.876, 'duration': 0.98}, {'end': 1100.456, 'text': 'it was not.', 'start': 1099.856, 'duration': 0.6}, {'end': 1103.116, 'text': 'this whole slope was not just a straight line.', 'start': 1100.456, 'duration': 2.66}, {'end': 1105.517, 'text': 'it was moving towards zero.', 'start': 1103.116, 'duration': 2.401}, {'end': 1110.018, 'text': 'I will not say exactly zero, but in this case it will move towards zero.', 'start': 1105.517, 'duration': 4.501}, {'end': 1115.632, 'text': 'Okay so wherever the slope value is very, very less, those features will be removed.', 'start': 1111.107, 'duration': 4.525}, {'end': 1118.575, 'text': 'Okay, those features will be removed.', 'start': 1117.193, 'duration': 1.382}, {'end': 1123.5, 'text': 'That basically means these features are not important for predicting our best fit line.', 'start': 1118.875, 'duration': 4.625}], 'summary': 'In ridge regression, the slope square moves toward zero, removing unimportant features.', 'duration': 33.726, 'max_score': 1089.774, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s1089774.jpg'}, {'end': 1132.029, 'src': 'embed', 'start': 1099.856, 'weight': 1, 'content': [{'end': 1100.456, 'text': 'it was not.', 'start': 1099.856, 'duration': 0.6}, {'end': 1103.116, 'text': 'this whole slope was not just a straight line.', 'start': 1100.456, 'duration': 2.66}, {'end': 1105.517, 'text': 'it was moving towards zero.', 'start': 1103.116, 'duration': 2.401}, {'end': 1110.018, 'text': 'I will not say exactly zero, but in this case it will move towards zero.', 'start': 1105.517, 'duration': 4.501}, {'end': 1115.632, 'text': 'Okay so wherever the slope value is very, very less, those features will be removed.', 'start': 1111.107, 'duration': 4.525}, {'end': 1118.575, 'text': 'Okay, those features will be removed.', 'start': 1117.193, 'duration': 1.382}, {'end': 1123.5, 'text': 'That basically means these features are not important for predicting our best fit line.', 'start': 1118.875, 'duration': 4.625}, {'end': 1127.584, 'text': 'So this in turn also helps us to do the feature selection.', 'start': 1123.921, 'duration': 3.663}, {'end': 1132.029, 'text': "So finally we'll just be using the most important feature like X1 and X2.", 'start': 1128.105, 'duration': 3.924}], 'summary': 'The slope moved towards zero, leading to removal of unimportant features and selection of x1 and x2.', 'duration': 32.173, 'max_score': 1099.856, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s1099856.jpg'}, {'end': 1193.061, 'src': 'embed', 'start': 1168.206, 'weight': 2, 'content': [{'end': 1173.87, 'text': 'In short what will happen is that if I using magnitude of slope this sum of the slopes will get zero after some time.', 'start': 1168.206, 'duration': 5.664}, {'end': 1179.334, 'text': 'As we are going to select the best fit line for the other features this slopes will become zero.', 'start': 1174.57, 'duration': 4.764}, {'end': 1183.696, 'text': 'But in case of ridge regression, all the slopes will not get zero.', 'start': 1180.074, 'duration': 3.622}, {'end': 1184.456, 'text': 'It will shrink.', 'start': 1183.756, 'duration': 0.7}, {'end': 1187.178, 'text': 'It will shrink but it will never reach zero.', 'start': 1184.977, 'duration': 2.201}, {'end': 1193.061, 'text': 'So that is proved mathematically and this is the basic difference ridge and lasso regression.', 'start': 1188.298, 'duration': 4.763}], 'summary': "In ridge regression, slopes shrink but don't reach zero, unlike lasso regression.", 'duration': 24.855, 'max_score': 1168.206, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s1168206.jpg'}], 'start': 1020.992, 'title': 'Regression and feature selection', 'summary': "Covers lasso regression, emphasizing its role in feature selection to overcome overfitting and discusses the importance of magnitude of slope in ridge regression for selecting features based on slope's value, while highlighting the differences between ridge and lasso regression.", 'chapters': [{'end': 1071.549, 'start': 1020.992, 'title': 'Lasso regression and feature selection', 'summary': 'Explains the concept of lasso regression and its importance in feature selection, emphasizing its role in overcoming overfitting and the use of magnitude of slope instead of slope square in the equation.', 'duration': 50.557, 'highlights': ['Lasso regression not only helps in fitting or in overcoming overfitting scenario, but also it helps us to do feature selection.', 'In lasso regression, instead of writing slope square, magnitude of slope is used, which assists in feature selection and overcoming overfitting.']}, {'end': 1146.4, 'start': 1071.589, 'title': 'Magnitude of slope and feature selection', 'summary': "Discusses the importance of the magnitude of slope in ridge regression, and how it helps in feature selection by removing less important features based on the slope's value moving towards zero, ultimately leaving only the most important features like x1 and x2.", 'duration': 74.811, 'highlights': ["The magnitude of slope in ridge regression is important as it determines the removal of less important features based on the slope's value moving towards zero, leaving only the most important features like X1 and X2.", 'The slope value being very less indicates the removal of features, thereby aiding in feature selection.']}, {'end': 1216.044, 'start': 1146.4, 'title': 'Difference between ridge and lasso regression', 'summary': 'Explains the difference between ridge and lasso regression, stating that lasso regression can lead some slope values to become zero, unlike ridge regression where the slope values shrink but never reach zero. the speaker also mentions the plan to explain the practical implementation in upcoming videos.', 'duration': 69.644, 'highlights': ['The difference between ridge and lasso regression is that in lasso regression, some slope values can become zero, while in ridge regression, the slope values shrink but never reach zero.', 'The speaker plans to explain the practical implementation of ridge and lasso regression in upcoming videos.']}], 'duration': 195.052, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/9lRv01HDU0s/pics/9lRv01HDU0s1020992.jpg', 'highlights': ['Lasso regression assists in feature selection and overcoming overfitting.', 'Magnitude of slope in ridge regression determines removal of less important features.', 'Difference: lasso regression can have zero slope values, ridge regression never reaches zero.']}], 'highlights': ['Understanding linear regression, bias, and variance is emphasized before delving into ridge and lasso regression.', 'Ridge and lasso regression convert overfitting conditions to low variance, contributing to a generalized model with low bias and variance.', 'Ridge regression involves adding a regularization parameter to the cost function, including a term for the slope squared, to reduce overfitting.', 'Lasso regression assists in feature selection and overcoming overfitting.', 'Overfitting results in low bias for the training data set but higher error for the test data set, causing significant discrepancies in predictions.', 'The chapter explains ridge and lasso regression as regularization techniques to reduce cost function.', 'A generalized model should always have low bias and low variance, crucial for it to be considered as a generalized model.', 'The cost function is reduced from 1.69 to 1.39 by adjusting the slope of the best fit line, resulting in a decrease in the residual sum.']}