title
Ridge, Lasso and Elastic-Net Regression in R

description
The code in this video can be found on the StatQuest GitHub: https://github.com/StatQuest/ridge_lasso_elastic_net_demo/blob/master/ridge_lass_elastic_net_demo.R This video is assumes you already know about Ridge, Lasso and Elastic-Net Regression, if not, here are the links to the Quests... https://youtu.be/1dKRdX9bfIo https://youtu.be/NGf0voTMlcs https://youtu.be/Q81RR3yKn30 For a complete index of all the StatQuest videos, check out: https://statquest.org/video-index/ If you'd like to support StatQuest, please consider... Buying The StatQuest Illustrated Guide to Machine Learning!!! PDF - https://statquest.gumroad.com/l/wvtmc Paperback - https://www.amazon.com/dp/B09ZCKR4H6 Kindle eBook - https://www.amazon.com/dp/B09ZG79HXC Patreon: https://www.patreon.com/statquest ...or... YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join ...a cool StatQuest t-shirt or sweatshirt: https://shop.spreadshirt.com/statquest-with-josh-starmer/ ...buying one or two of my songs (or go large and get a whole album!) https://joshuastarmer.bandcamp.com/ ...or just donating to StatQuest! https://www.paypal.me/statquest Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter: https://twitter.com/joshuastarmer #statquest #regularization

detail
{'title': 'Ridge, Lasso and Elastic-Net Regression in R', 'heatmap': [{'end': 314.589, 'start': 279.522, 'weight': 0.76}, {'end': 501.45, 'start': 448.015, 'weight': 0.769}, {'end': 643.121, 'start': 502.27, 'weight': 0.713}, {'end': 676.818, 'start': 653.249, 'weight': 0.706}], 'summary': 'Covers ridge, lasso, and elastic net regression in r using the glmnet library, including the impact of alpha and lambda parameters, creating a dataset with 1,000 samples and 5,000 parameters, data division, prediction using cross-validation, and comparing regression performance, with lasso regression producing the smallest error.', 'chapters': [{'end': 347.575, 'segs': [{'end': 65.63, 'src': 'embed', 'start': 39.855, 'weight': 0, 'content': [{'end': 46.08, 'text': 'To do Ridge, Lasso, and Elastic Net Regression in R, we will use the glmnet library.', 'start': 39.855, 'duration': 6.225}, {'end': 57.568, 'text': 'The GLM part of GLMnet stands for generalized linear models, which means that this tool can be applied to linear regression and logistic regression,', 'start': 47.366, 'duration': 10.202}, {'end': 58.928, 'text': 'as well as a few other models.', 'start': 57.568, 'duration': 1.36}, {'end': 63.329, 'text': 'The net part of GLMnet is from ElasticNet.', 'start': 60.129, 'duration': 3.2}, {'end': 65.63, 'text': "At least I'm pretty sure that's where it's from.", 'start': 63.929, 'duration': 1.701}], 'summary': 'Using glmnet library in r for ridge, lasso, and elastic net regression, applicable to linear and logistic regression.', 'duration': 25.775, 'max_score': 39.855, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI39855.jpg'}, {'end': 135.252, 'src': 'embed', 'start': 96.837, 'weight': 1, 'content': [{'end': 105.042, 'text': 'Instead of two different lambdas, glmnet has a single lambda and another parameter called alpha.', 'start': 96.837, 'duration': 8.205}, {'end': 110.598, 'text': 'Alpha can be any value from 0 to 1.', 'start': 106.596, 'duration': 4.002}, {'end': 118.082, 'text': 'When alpha equals 0, then the whole lasso penalty goes to 0 and goes away.', 'start': 110.598, 'duration': 7.484}, {'end': 122.425, 'text': "And we're left with just the ridge regression penalty.", 'start': 119.983, 'duration': 2.442}, {'end': 126.247, 'text': 'And the whole thing reduces to ridge regression.', 'start': 124.006, 'duration': 2.241}, {'end': 135.252, 'text': 'When alpha equals 1, then the whole ridge penalty goes to 0 and goes away.', 'start': 128.388, 'duration': 6.864}], 'summary': 'Glmnet uses a single lambda and alpha parameter, ranging from 0 to 1, impacting lasso and ridge regression penalties.', 'duration': 38.415, 'max_score': 96.837, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI96837.jpg'}, {'end': 190.136, 'src': 'embed', 'start': 164.816, 'weight': 2, 'content': [{'end': 172.763, 'text': "then the whole penalty goes away and we're just doing standard least squares for linear regression or maximum likelihood for logistic regression.", 'start': 164.816, 'duration': 7.947}, {'end': 181.149, 'text': 'And when lambda is greater than zero, then the elastic net penalty kicks in and we start shrinking parameter estimates.', 'start': 174.524, 'duration': 6.625}, {'end': 190.136, 'text': 'Thus, when we use the glmnet package to do elastic net regression, we will test different values for lambda and alpha.', 'start': 182.73, 'duration': 7.406}], 'summary': 'Elastic net regression uses glmnet package to test lambda and alpha values.', 'duration': 25.32, 'max_score': 164.816, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI164816.jpg'}, {'end': 314.589, 'src': 'heatmap', 'start': 225.295, 'weight': 3, 'content': [{'end': 231.038, 'text': "Now let's make up a data set that we can use to test out ridge, lasso, and elastic net regression.", 'start': 225.295, 'duration': 5.743}, {'end': 240.784, 'text': 'The made-up data set will have n equals 1,000 samples and p equals 5,000 parameters to estimate.', 'start': 232.639, 'duration': 8.145}, {'end': 246.808, 'text': 'However, only 15 of those parameters will help us predict the outcome.', 'start': 242.665, 'duration': 4.143}, {'end': 253.402, 'text': 'The remaining 4,985 parameters will just be random noise.', 'start': 248.66, 'duration': 4.742}, {'end': 259.524, 'text': 'Now we create a matrix called x that is full of randomly generated data.', 'start': 255.223, 'duration': 4.301}, {'end': 268.888, 'text': 'The matrix has 1,000 rows, since n equals 1,000, and 5,000 columns, since p equals 5,000.', 'start': 261.365, 'duration': 7.523}, {'end': 279.522, 'text': 'And the values in the matrix come from a standard normal distribution, with mean equals 0 and standard deviation equals 1.', 'start': 268.888, 'duration': 10.634}, {'end': 289.37, 'text': "and we'll need n times p or 1000 times 5000 equals 5 million values since our matrix has n rows and p columns.", 'start': 279.522, 'duration': 9.848}, {'end': 298.757, 'text': 'Now we create a vector of values called y that we will try to predict with the data in x.', 'start': 291.331, 'duration': 7.426}, {'end': 307.744, 'text': 'This call to apply will return a vector of 1000 values that are the sums of the first 15 columns in x since x has 1000 rows.', 'start': 298.757, 'duration': 8.987}, {'end': 314.589, 'text': 'This is what isolates columns 1 through 15 from X,', 'start': 310.285, 'duration': 4.304}], 'summary': 'A data set with 1,000 samples and 5,000 parameters will be used for ridge, lasso, and elastic net regression, with only 15 parameters helping to predict the outcome.', 'duration': 64.075, 'max_score': 225.295, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI225295.jpg'}, {'end': 358.287, 'src': 'embed', 'start': 322.477, 'weight': 5, 'content': [{'end': 325.82, 'text': 'And SUM is the function we want to apply to each row.', 'start': 322.477, 'duration': 3.343}, {'end': 334.549, 'text': 'To summarize, this call to apply will return a vector of values that depend on the first 15 columns in X.', 'start': 327.482, 'duration': 7.067}, {'end': 342.21, 'text': 'Once we have that vector of sums, we add a little bit of noise using the rnorm function,', 'start': 336.764, 'duration': 5.446}, {'end': 347.575, 'text': 'which in this case returns 1000 random values from a standard normal distribution.', 'start': 342.21, 'duration': 5.365}, {'end': 358.287, 'text': 'So this whole thing creates a vector called y that is dependent on the first 15 columns in x, plus a little noise to make things interesting.', 'start': 349.498, 'duration': 8.789}], 'summary': "Applying sum function to rows in x's first 15 columns, adding noise with rnorm to create y vector.", 'duration': 35.81, 'max_score': 322.477, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI322477.jpg'}], 'start': 1.138, 'title': 'Regression in r using glmnet', 'summary': 'Delves into ridge, lasso, and elastic net regression in r using the glmnet library, explaining the impact of alpha and lambda parameters on penalties and parameter estimates. it also covers creating a data set with 1,000 samples and 5,000 parameters, with only 15 being useful for prediction, for testing regression techniques.', 'chapters': [{'end': 223.594, 'start': 1.138, 'title': 'Elastic net regression in r', 'summary': 'Explores ridge, lasso, and elastic net regression in r using the glmnet library, explaining how alpha and lambda parameters control the penalties and the effect on parameter estimates, promoting a better understanding of glmnet workings.', 'duration': 222.456, 'highlights': ['The glmnet library in R can be applied to linear regression and logistic regression, and it combines the lasso regression penalty with the ridge regression penalty to provide a better job shrinking correlated variables. The glmnet library in R can be applied to linear regression and logistic regression, and it combines the lasso regression penalty with the ridge regression penalty to provide a better job shrinking correlated variables than either lasso or ridge does on their own.', 'The alpha parameter in glmnet can be any value from 0 to 1 and controls the mixture of the two penalties, where alpha=0 results in ridge regression, alpha=1 results in lasso regression, and 00 results in the elastic net penalty kicking in and shrinking parameter estimates. Lambda in glmnet controls how much of the penalty to apply to the regression, where lambda=0 results in no penalty and standard least squares for linear regression or maximum likelihood for logistic regression, and lambda>0 results in the elastic net penalty kicking in and shrinking parameter estimates.']}, {'end': 347.575, 'start': 225.295, 'title': 'Data set for regression testing', 'summary': 'Discusses creating a data set with 1,000 samples and 5,000 parameters, of which only 15 are useful for predicting the outcome, using standard normal distribution and noise, for testing ridge, lasso, and elastic net regression.', 'duration': 122.28, 'highlights': ['Creating a data set with 1,000 samples and 5,000 parameters, with only 15 parameters useful for prediction. Highlights the key details of the dataset, emphasizing the small number of useful parameters among a large set.', 'Using a standard normal distribution to generate values for the matrix x. Describes the method of generating data for the matrix x, indicating the distribution and characteristics of the values.', 'Adding noise to the vector of sums using the rnorm function. Highlights the introduction of random noise to the vector of sums, indicating the method and source of the noise.']}], 'duration': 346.437, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI1138.jpg', 'highlights': ['The glmnet library in R combines lasso and ridge regression penalties for better shrinking of correlated variables.', 'The alpha parameter in glmnet controls the mixture of penalties, with alpha=0 resulting in ridge regression, alpha=1 in lasso regression, and 00 in the elastic net penalty kicking in.', 'Creating a data set with 1,000 samples and 5,000 parameters, with only 15 being useful for prediction.', 'Using a standard normal distribution to generate values for the matrix x.', 'Adding noise to the vector of sums using the rnorm function.']}, {'end': 522.336, 'segs': [{'end': 403.429, 'src': 'embed', 'start': 377.494, 'weight': 1, 'content': [{'end': 385.298, 'text': 'So we make a vector of indexes called train underscore rows that contains the row numbers of the rows that will be in the training set.', 'start': 377.494, 'duration': 7.804}, {'end': 392.964, 'text': 'The sample function randomly selects numbers between 1 and n, the number of rows in our data set.', 'start': 387.042, 'duration': 5.922}, {'end': 398.407, 'text': 'And it will select 0.66 times n row numbers.', 'start': 394.525, 'duration': 3.882}, {'end': 403.429, 'text': 'In other words, two thirds of the data will be in the training set.', 'start': 399.827, 'duration': 3.602}], 'summary': 'Randomly select 66% of data for training set', 'duration': 25.935, 'max_score': 377.494, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI377494.jpg'}, {'end': 522.336, 'src': 'heatmap', 'start': 448.015, 'weight': 0, 'content': [{'end': 456.979, 'text': "We'll apply ridge, lasso, and elastic net regression separately to these datasets so that we can see how it's done and see which method works best.", 'start': 448.015, 'duration': 8.964}, {'end': 459.76, 'text': "We'll start with ridge regression.", 'start': 458.12, 'duration': 1.64}, {'end': 464.763, 'text': 'The first thing we need to do is fit a model to the training data.', 'start': 461.461, 'duration': 3.302}, {'end': 469.505, 'text': 'We do this with the cv.glmNet function.', 'start': 466.103, 'duration': 3.402}, {'end': 476.808, 'text': 'The cv part means we want to use cross-validation to obtain the optimal values for lambda.', 'start': 471.265, 'duration': 5.543}, {'end': 483.384, 'text': 'By default, cv.glmnet uses tenfold cross-validation.', 'start': 478.322, 'duration': 5.062}, {'end': 487.625, 'text': 'The first two parameters specify the training sets.', 'start': 485.084, 'duration': 2.541}, {'end': 492.047, 'text': 'In this case, we want to use xtrain to predict ytrain.', 'start': 488.285, 'duration': 3.762}, {'end': 501.45, 'text': 'Unlike the lm or glm functions, cv.glmnet does not accept formula notation.', 'start': 494.728, 'duration': 6.722}, {'end': 504.431, 'text': 'x and y must be passed in separately.', 'start': 502.27, 'duration': 2.161}, {'end': 510.332, 'text': 'is how the cross-validation will be evaluated.', 'start': 507.731, 'duration': 2.601}, {'end': 515.594, 'text': 'And it is set to MSE, which stands for mean squared error.', 'start': 511.852, 'duration': 3.742}, {'end': 522.336, 'text': 'Mean squared error is just the sum of the squared residuals divided by the sample size.', 'start': 517.294, 'duration': 5.042}], 'summary': 'Applying ridge, lasso, and elastic net regression separately to datasets with tenfold cross-validation for optimal lambda values.', 'duration': 27.608, 'max_score': 448.015, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI448015.jpg'}], 'start': 349.498, 'title': 'Data division and regression prediction', 'summary': 'Explains the process of dividing data into training and testing sets, and applying ridge, lasso, and elastic net regression to predict values in y, with emphasis on using cross-validation with tenfold cross-validation and evaluating with mean squared error.', 'chapters': [{'end': 522.336, 'start': 349.498, 'title': 'Data division and regression prediction', 'summary': 'Explains the process of dividing data into training and testing sets, and applying ridge, lasso, and elastic net regression to predict values in y, with emphasis on using cross-validation with tenfold cross-validation and evaluating with mean squared error.', 'duration': 172.838, 'highlights': ['The training set comprises approximately two-thirds of the data, with the remaining third allocated to the testing set. Two-thirds of the data is allocated to the training set, while the remaining data is used for testing.', 'The cv.glmNet function is used to fit a model to the training data, employing tenfold cross-validation to obtain the optimal values for lambda and evaluating with mean squared error. The cv.glmNet function utilizes tenfold cross-validation to find optimal lambda values and evaluates using mean squared error.']}], 'duration': 172.838, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI349498.jpg', 'highlights': ['The cv.glmNet function utilizes tenfold cross-validation to find optimal lambda values and evaluates using mean squared error.', 'Two-thirds of the data is allocated to the training set, while the remaining data is used for testing.']}, {'end': 821.141, 'segs': [{'end': 581.058, 'src': 'embed', 'start': 551.678, 'weight': 0, 'content': [{'end': 558.905, 'text': 'Altogether. this call to CV.GLMNet will fit a linear regression with a ridge regression penalty,', 'start': 551.678, 'duration': 7.227}, {'end': 562.889, 'text': 'using tenfold cross-validation to find optimal values for lambda.', 'start': 558.905, 'duration': 3.984}, {'end': 570.569, 'text': 'and the fit model, along with optimal values for lambda, is saved as alpha zero dot fit,', 'start': 564.383, 'duration': 6.186}, {'end': 574.292, 'text': 'which will help us remember that we set alpha to zero for ridge regression.', 'start': 570.569, 'duration': 3.723}, {'end': 581.058, 'text': 'Now we will use the predict function to apply alpha zero dot fit to the testing data.', 'start': 575.993, 'duration': 5.065}], 'summary': 'Uses cv.glmnet to fit ridge regression with tenfold cross-validation and applies the model to testing data.', 'duration': 29.38, 'max_score': 551.678, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI551678.jpg'}, {'end': 676.818, 'src': 'heatmap', 'start': 653.249, 'weight': 0.706, 'content': [{'end': 665.116, 'text': "What's going on? Since we will compare ridge to lasso and elastic net regression, we will use lambda.1se for all three cases to be consistent.", 'start': 653.249, 'duration': 11.867}, {'end': 671.58, 'text': 'Lastly, we set new x to the testing data set x.test.', 'start': 666.957, 'duration': 4.623}, {'end': 676.818, 'text': 'And we save the predicted values as alpha zero dot predicted.', 'start': 673.137, 'duration': 3.681}], 'summary': 'Comparing ridge, lasso, and elastic net regressions using lambda.1se, setting new x to testing data, and saving predicted values as alpha zero dot predicted.', 'duration': 23.569, 'max_score': 653.249, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI653249.jpg'}, {'end': 758.395, 'src': 'embed', 'start': 730.908, 'weight': 1, 'content': [{'end': 738.37, 'text': 'only this time we pass an alpha1.fit and save the results as alpha1.predicted.', 'start': 730.908, 'duration': 7.462}, {'end': 748.693, 'text': 'Then we calculate the mean squared error, and we get 1.19 something something.', 'start': 740.31, 'duration': 8.383}, {'end': 758.395, 'text': '1.19 something something is way smaller than 14.47 something something, so lasso regression is much better with this data than ridge regression.', 'start': 748.713, 'duration': 9.682}], 'summary': 'Lasso regression yields 1.19 mse, outperforming ridge regression (14.47 mse)', 'duration': 27.487, 'max_score': 730.908, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI730908.jpg'}, {'end': 821.141, 'src': 'embed', 'start': 792.794, 'weight': 2, 'content': [{'end': 799.581, 'text': 'Then we call the predict function with alpha 0.5.fit and calculate the mean squared error.', 'start': 792.794, 'duration': 6.787}, {'end': 803.185, 'text': 'And we get 1.24 something something.', 'start': 801.083, 'duration': 2.102}, {'end': 810.456, 'text': 'This is slightly larger than the 1.19 something something we got with lasso regression.', 'start': 805.213, 'duration': 5.243}, {'end': 813.677, 'text': 'So, so far, lasso wins.', 'start': 810.896, 'duration': 2.781}, {'end': 821.141, 'text': 'Yee-haw! But to really know if lasso wins, we need to try a lot of different values for alpha.', 'start': 814.258, 'duration': 6.883}], 'summary': 'Using alpha 0.5, the mean squared error is 1.24, slightly larger than 1.19 from lasso regression. further testing needed.', 'duration': 28.347, 'max_score': 792.794, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI792794.jpg'}], 'start': 523.957, 'title': 'Regression in glmnet', 'summary': 'Explores elastic net regression for logistic regression, using alpha zero for ridge regression, and applying tenfold cross-validation to determine optimal lambda values for linear regression. it also compares the performance of ridge, lasso, and elastic net regression, with lasso regression producing the smallest error of 1.19, followed by elastic net regression with an error of 1.24, and ridge regression resulting in the largest error of 14.47.', 'chapters': [{'end': 574.292, 'start': 523.957, 'title': 'Elastic net regression in glmnet', 'summary': 'Discusses the application of elastic net regression to logistic regression, setting alpha to zero for ridge regression, and using tenfold cross-validation to find optimal values for lambda in glmnet for linear regression.', 'duration': 50.335, 'highlights': ['GLMNet is used to fit a linear regression with a ridge regression penalty, using tenfold cross-validation to find optimal values for lambda, saving the fit model as alpha zero dot fit. (Quantifiable: tenfold cross-validation)', 'Setting alpha to zero indicates the use of ridge regression. (Quantifiable: alpha value)', 'The choice of family as Gaussian signifies the use of linear regression. (Quantifiable: family type)']}, {'end': 821.141, 'start': 575.993, 'title': 'Predicting with ridge, lasso, and elastic net regression', 'summary': 'Outlines the process of applying alpha zero dot fit, alpha1.fit, and alpha 0.5.fit to the testing data, calculating mean squared errors, and comparing the performance of ridge, lasso, and elastic net regression, with lasso regression yielding the smallest error of 1.19, followed by elastic net regression with an error of 1.24, and ridge regression resulting in the largest error of 14.47.', 'duration': 245.148, 'highlights': ['Lasso regression yielded a mean squared error of 1.19, indicating superior performance compared to ridge regression which resulted in a mean squared error of 14.47. The mean squared error for lasso regression was 1.19, significantly smaller than the 14.47 mean squared error for ridge regression.', "Elastic net regression produced a mean squared error of 1.24, slightly larger than the 1.19 error from lasso regression. Elastic net regression resulted in a mean squared error of 1.24, slightly larger than the 1.19 error from lasso regression, indicating lasso's better performance.", 'Ridge regression led to the largest mean squared error of 14.47. Ridge regression yielded the largest mean squared error of 14.47, indicating its inferior performance compared to lasso and elastic net regression.']}], 'duration': 297.184, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI523957.jpg', 'highlights': ['GLMNet fits linear regression with ridge regression penalty using tenfold cross-validation to find optimal lambda values.', 'Lasso regression yields a mean squared error of 1.19, outperforming ridge regression with a mean squared error of 14.47.', "Elastic net regression results in a mean squared error of 1.24, slightly larger than lasso regression's error of 1.19."]}, {'end': 1063.216, 'segs': [{'end': 848.945, 'src': 'embed', 'start': 822.742, 'weight': 1, 'content': [{'end': 831.347, 'text': "To try a bunch of values for alpha, we'll start by making an empty list called list of fits that will store a bunch of elastic net regression fits.", 'start': 822.742, 'duration': 8.605}, {'end': 836.374, 'text': 'Then we use a for loop to try different values for alpha.', 'start': 833.131, 'duration': 3.243}, {'end': 843.72, 'text': 'In this for loop, i will be integer values from 0 to 10.', 'start': 838.015, 'duration': 5.705}, {'end': 848.945, 'text': 'First, we paste together a name for the elastic net fit that we are going to create.', 'start': 843.72, 'duration': 5.225}], 'summary': 'Using a for loop to test alpha values from 0 to 10 in elastic net regression.', 'duration': 26.203, 'max_score': 822.742, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI822742.jpg'}, {'end': 971.232, 'src': 'embed', 'start': 919.667, 'weight': 2, 'content': [{'end': 926.412, 'text': "We'll start by creating an empty data frame called results that will store the mean squared errors and a few other things.", 'start': 919.667, 'duration': 6.745}, {'end': 934.497, 'text': "Then we'll use another for loop to predict values using the testing data set to calculate the mean squared errors.", 'start': 928.253, 'duration': 6.244}, {'end': 941.141, 'text': 'Just like before, the for loop goes from 0 to 10.', 'start': 935.938, 'duration': 5.203}, {'end': 947.846, 'text': "And just like before, we'll create a variable called fit.name that contains the name of the elastic net regression fit.", 'start': 941.141, 'duration': 6.705}, {'end': 956.141, 'text': "However, this time we'll use the predict function to predict values with x.test, the testing data.", 'start': 949.636, 'duration': 6.505}, {'end': 971.232, 'text': 'We use the list of fits list and fit.name to pass a specific fit to the predict function, and to pass the lambda.1se value for s.', 'start': 957.822, 'duration': 13.41}], 'summary': 'Creating a data frame to store mean squared errors and predicting values using elastic net regression fit.', 'duration': 51.565, 'max_score': 919.667, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI919667.jpg'}, {'end': 1054.229, 'src': 'embed', 'start': 1002.776, 'weight': 0, 'content': [{'end': 1008.799, 'text': 'The first column has the values for alpha, ranging from 0 to 1.', 'start': 1002.776, 'duration': 6.023}, {'end': 1011.34, 'text': 'The second column has the mean squared errors.', 'start': 1008.799, 'duration': 2.541}, {'end': 1016.643, 'text': 'Note, these are slightly different from what we got before,', 'start': 1013.061, 'duration': 3.582}, {'end': 1022.565, 'text': 'because the parameter values prior to regularization and optimization are randomly initialized.', 'start': 1016.643, 'duration': 5.922}, {'end': 1029.709, 'text': 'Thus, this is another good reason to use set.seed to ensure consistent results.', 'start': 1023.946, 'duration': 5.763}, {'end': 1034.171, 'text': 'And in the last column, we have the name of the fit.', 'start': 1031.448, 'duration': 2.723}, {'end': 1042.141, 'text': 'The fit, where alpha equals 1, is still the best, so lasso regression is the best method to use with this data.', 'start': 1035.553, 'duration': 6.588}, {'end': 1050.547, 'text': "Triple bam! Hooray! We've made it to the end of another exciting stat quest.", 'start': 1043.462, 'duration': 7.085}, {'end': 1054.229, 'text': 'If you like this stat quest and want to see more, please subscribe.', 'start': 1051.107, 'duration': 3.122}], 'summary': 'Lasso regression with alpha=1 is the best method. exciting stat quest.', 'duration': 51.453, 'max_score': 1002.776, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI1002776.jpg'}], 'start': 822.742, 'title': 'Elastic net regression', 'summary': 'Explains the process of fitting elastic net regression with different alpha values, iterating through integers 0 to 10, and determining lasso regression as the best method for prediction.', 'chapters': [{'end': 941.141, 'start': 822.742, 'title': 'Elastic net regression fits', 'summary': 'Explains the process of trying different values for alpha in elastic net regression, creating fits, and calculating mean squared errors, using a for loop to iterate through integer values from 0 to 10.', 'duration': 118.399, 'highlights': ['Creating a list of elastic net regression fits and iterating through different values for alpha using a for loop. The process involves creating an empty list to store elastic net regression fits and using a for loop to iterate through different values for alpha from 0 to 10.', 'Naming the elastic net fit based on the value of alpha and creating the fit using the cv.glmnet function. The naming convention for the elastic net fit is based on the value of alpha, and the fit is created using the cv.glmnet function with varying alpha values.', 'Storing each fit in the list and calculating mean squared errors for each fit using the testing dataset. Each fit is stored in the list, and mean squared errors are calculated for each fit using the testing dataset.']}, {'end': 1063.216, 'start': 941.141, 'title': 'Elastic net regression prediction process', 'summary': 'Details the process of using elastic net regression to predict values, calculate mean squared errors, and determine the best fit, with the conclusion that lasso regression is the best method to use with the data.', 'duration': 122.075, 'highlights': ['The fit with alpha equals 1 is the best, indicating that lasso regression is the best method to use with this data, as it yields the lowest mean squared error.', 'The process involves creating a variable fit.name containing the name of the elastic net regression fit, using the predict function to predict values with x.test, and storing the predicted values in a variable called predicted.', 'The mean squared errors are calculated for the fit, and the values for alpha, the mean squared error, and the name of the fit are stored in a temporary data frame called temp, which is then appended to the results data frame using the rbind function.', 'The results display the values for alpha, mean squared errors, and the name of the fit, with the conclusion that lasso regression is the best method to use with this data.', 'The importance of using set.seed to ensure consistent results due to the random initialization of parameter values prior to regularization and optimization is highlighted.']}], 'duration': 240.474, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/ctmNq7FgbvI/pics/ctmNq7FgbvI822742.jpg', 'highlights': ['The fit with alpha equals 1 is the best, indicating that lasso regression is the best method to use with this data, as it yields the lowest mean squared error.', 'Creating a list of elastic net regression fits and iterating through different values for alpha using a for loop.', 'Storing each fit in the list and calculating mean squared errors for each fit using the testing dataset.', 'The process involves creating a variable fit.name containing the name of the elastic net regression fit, using the predict function to predict values with x.test, and storing the predicted values in a variable called predicted.', 'The importance of using set.seed to ensure consistent results due to the random initialization of parameter values prior to regularization and optimization is highlighted.']}], 'highlights': ['The glmnet library in R combines lasso and ridge regression penalties for better shrinking of correlated variables.', 'The alpha parameter in glmnet controls the mixture of penalties, with alpha=0 resulting in ridge regression, alpha=1 in lasso regression, and 00 in the elastic net penalty kicking in.', 'Creating a data set with 1,000 samples and 5,000 parameters, with only 15 being useful for prediction.', 'Using a standard normal distribution to generate values for the matrix x.', 'Adding noise to the vector of sums using the rnorm function.', 'The cv.glmNet function utilizes tenfold cross-validation to find optimal lambda values and evaluates using mean squared error.', 'Two-thirds of the data is allocated to the training set, while the remaining data is used for testing.', 'GLMNet fits linear regression with ridge regression penalty using tenfold cross-validation to find optimal lambda values.', 'Lasso regression yields a mean squared error of 1.19, outperforming ridge regression with a mean squared error of 14.47.', "Elastic net regression results in a mean squared error of 1.24, slightly larger than lasso regression's error of 1.19.", 'The fit with alpha equals 1 is the best, indicating that lasso regression is the best method to use with this data, as it yields the lowest mean squared error.', 'Creating a list of elastic net regression fits and iterating through different values for alpha using a for loop.', 'Storing each fit in the list and calculating mean squared errors for each fit using the testing dataset.', 'The process involves creating a variable fit.name containing the name of the elastic net regression fit, using the predict function to predict values with x.test, and storing the predicted values in a variable called predicted.', 'The importance of using set.seed to ensure consistent results due to the random initialization of parameter values prior to regularization and optimization is highlighted.']}