title
Machine Learning Tutorial Python - 19: Principal Component Analysis (PCA) with Python Code

description
PCA or principal component analysis is a dimensionality reduction technique that can help us reduce dimensions of dataset that we use in machine learning for training. It helps with famous dimensionality curse problem. In this video we will understand what PCA is all about, write python code for handwritten digits dataset classification and then use PCA to train the same model using PCA. Code: https://github.com/codebasics/py/blob/master/ML/18_PCA/PCA_tutorial_digits.ipynb Exercise: https://github.com/codebasics/py/blob/master/ML/18_PCA/pca_exercise.md ⭐️ Timestamps ⭐️ 00:00 Theory 09:12 Coding 23:04 Exercise Do you want to learn technology from me? Check https://codebasics.io/?utm_source=description&utm_medium=yt&utm_campaign=description&utm_id=description for my affordable video courses. 🌎 My Website For Video Courses: https://codebasics.io/?utm_source=description&utm_medium=yt&utm_campaign=description&utm_id=description Need help building software or data analytics and AI solutions? My company https://www.atliq.com/ can help. Click on the Contact button on that website. 🎥 Codebasics Hindi channel: https://www.youtube.com/channel/UCTmFBhuhMibVoSfYom1uXEg #️⃣ Social Media #️⃣ 🔗 Discord: https://discord.gg/r42Kbuk 📸 Dhaval's Personal Instagram: https://www.instagram.com/dhavalsays/ 📸 Instagram: https://www.instagram.com/codebasicshub/ 🔊 Facebook: https://www.facebook.com/codebasicshub 📱 Twitter: https://twitter.com/codebasicshub 📝 Linkedin (Personal): https://www.linkedin.com/in/dhavalsays/ 📝 Linkedin (Codebasics): https://www.linkedin.com/company/codebasics/ ❗❗ DISCLAIMER: All opinions expressed in this video are of my own and not that of my employers'.

detail
{'title': 'Machine Learning Tutorial Python - 19: Principal Component Analysis (PCA) with Python Code', 'heatmap': [{'end': 265.573, 'start': 226.624, 'weight': 0.798}, {'end': 654.7, 'start': 590.829, 'weight': 0.768}, {'end': 1064.024, 'start': 1013.093, 'weight': 1}, {'end': 1088.473, 'start': 1068.426, 'weight': 0.828}, {'end': 1149.61, 'start': 1108.248, 'weight': 0.87}], 'summary': 'The tutorial covers principal component analysis (pca) in machine learning for dimensionality reduction, with examples of reducing features from 64 to 2 for enhanced visualization and decision-making. it explains how pca reduces dimensionality from 64 to 6, aids in faster computation, and addresses the dimensionality curse problem in machine learning. additionally, it discusses dataset classification, feature scaling, logistic regression training, and the impact of reducing features using pca on accuracy.', 'chapters': [{'end': 296.292, 'segs': [{'end': 48.352, 'src': 'embed', 'start': 0.809, 'weight': 0, 'content': [{'end': 5.793, 'text': 'Principal component analysis is a technique used in machine learning to reduce dimension.', 'start': 0.809, 'duration': 4.984}, {'end': 8.614, 'text': 'In this video we are going to look at what it is.', 'start': 6.353, 'duration': 2.261}, {'end': 13.297, 'text': "we'll write some Python code and in the end there will be an interesting exercise for you to work on.", 'start': 8.614, 'duration': 4.683}, {'end': 18.801, 'text': "Let's say you're working on building a machine learning model to predict property prices.", 'start': 14.078, 'duration': 4.723}, {'end': 29.7, 'text': 'Here, all the columns on the left hand side are the features or the attributes of the property that decides the price.', 'start': 20.031, 'duration': 9.669}, {'end': 32.963, 'text': 'So the column in green is actually your target variable.', 'start': 29.72, 'duration': 3.243}, {'end': 44.051, 'text': 'You probably already know that the price of the home is mainly dependent on area, which town it is in, plot, et cetera.', 'start': 34.588, 'duration': 9.463}, {'end': 48.352, 'text': 'It depends on how many bathrooms you have in a property, but not as much.', 'start': 44.751, 'duration': 3.601}], 'summary': 'Pca reduces dimension in machine learning. python code example provided.', 'duration': 47.543, 'max_score': 0.809, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc809.jpg'}, {'end': 126.734, 'src': 'embed', 'start': 98.717, 'weight': 3, 'content': [{'end': 104.521, 'text': 'And you need to do something to identify the features which are very important.', 'start': 98.717, 'duration': 5.804}, {'end': 112.466, 'text': "You're working on, let's say, handwritten digit classification, where you know you have digits which are written by hand,", 'start': 105.762, 'duration': 6.704}, {'end': 117.068, 'text': 'and then you try to classify as one of the numbers from 0 to 9..', 'start': 112.466, 'duration': 4.602}, {'end': 120.09, 'text': 'Here, this image is represented as pixel.', 'start': 117.068, 'duration': 3.022}, {'end': 126.734, 'text': "Let's say this is a grid of eight by eight and every number presents the color.", 'start': 120.51, 'duration': 6.224}], 'summary': 'Identify important features for handwritten digit classification, with images represented as 8x8 grids of pixels.', 'duration': 28.017, 'max_score': 98.717, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc98717.jpg'}, {'end': 265.573, 'src': 'heatmap', 'start': 214.343, 'weight': 5, 'content': [{'end': 225.144, 'text': 'a great Data visualization helps you a lot in terms of your final decision making with regards to what kind of model you want to build.', 'start': 214.343, 'duration': 10.801}, {'end': 236.327, 'text': 'PCA is a process of figuring out the most important features or the principal component that has a maximum impact on a target variable.', 'start': 226.624, 'duration': 9.703}, {'end': 244.89, 'text': 'PCA will create, in fact, the new feature called principal component, you know, PC1, PC2, and so on.', 'start': 237.207, 'duration': 7.683}, {'end': 251.196, 'text': "So again, going back to the digits example, let's say I have 64 features.", 'start': 245.99, 'duration': 5.206}, {'end': 255.541, 'text': "Out of that, I'm plotting only two features, corner pixel and central pixel.", 'start': 251.597, 'duration': 3.944}, {'end': 259.546, 'text': 'Now here, these cluster represent different digits.', 'start': 256.603, 'duration': 2.943}, {'end': 265.573, 'text': 'You immediately notice that the corner pixel is not playing an important role.', 'start': 260.527, 'duration': 5.046}], 'summary': 'Data visualization aids decision-making by identifying important features for model building, e.g., pca creates principal components with maximum impact on target variable.', 'duration': 30.547, 'max_score': 214.343, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc214343.jpg'}], 'start': 0.809, 'title': 'Principal component analysis and feature identification', 'summary': 'Covers the use of principal component analysis in machine learning for dimensionality reduction and its application in predicting property prices. it also discusses the identification of important features in handwritten digit classification using an 8x8 pixel grid, emphasizing the significance of certain pixels in determining the digit. additionally, the benefits of feature reduction using pca are highlighted, including faster training, easier visualization, and improved decision-making, with a specific example of reducing 64 features to 2 for enhanced visualization and decision-making.', 'chapters': [{'end': 97.457, 'start': 0.809, 'title': 'Understanding principal component analysis', 'summary': 'Introduces principal component analysis, explaining its application in machine learning to reduce dimensionality and demonstrates its relevance in predicting property prices through a real-life example, emphasizing the importance of certain features over others.', 'duration': 96.648, 'highlights': ['Principal component analysis is used in machine learning to reduce dimensionality, which is crucial when dealing with a large number of features in real-life problems.', 'The example of predicting property prices highlights the significance of certain features such as area and plot in comparison to others like the number of bathrooms or trees nearby, showcasing the practical relevance of principal component analysis.', 'The practical demonstration of writing Python code and the provision of an exercise enhance the engagement and learning experience for the audience.']}, {'end': 158.437, 'start': 98.717, 'title': 'Identifying important features in handwritten digit classification', 'summary': 'Discusses the process of identifying important features in handwritten digit classification, where an image is represented as a grid of 8x8 pixels, with each pixel representing a color value ranging from 0 to 16, resulting in a total of 64 features; the discussion emphasizes the importance of identifying the significant pixels in determining the digit.', 'duration': 59.72, 'highlights': ['The image in handwritten digit classification is represented as a grid of 8x8 pixels, with each pixel representing a color value ranging from 0 to 16, resulting in a total of 64 features.', 'It is essential to identify the significant pixels that play a role in determining the digit, as not all pixels contribute to the classification process.']}, {'end': 296.292, 'start': 159.477, 'title': 'Pca for feature reduction', 'summary': 'Discusses the benefits of reducing features using pca, including faster training and inference, easier data visualization, and improved decision-making with regards to model building, with a specific example of reducing 64 features to 2 features for improved visualization and decision-making.', 'duration': 136.815, 'highlights': ['PCA helps in reducing the number of features, making machine learning training and inference faster and more lightweight, while also aiding in easier data visualization and final decision-making.', "Reducing 64 features to 2 features through PCA enables improved data visualization and understanding of feature importance, as demonstrated by the example of corner and central pixels' impact on digit representation.", 'PCA identifies the most important features or principal components with maximum impact on the target variable, such as determining that the central pixel has the maximum variance in a specific example.', 'The process of reducing dimensions from two to one through PCA simplifies visualization and understanding of data, as demonstrated by the example of reducing the corner pixel for improved one-dimensional representation.']}], 'duration': 295.483, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc809.jpg', 'highlights': ['PCA reduces dimensionality in machine learning for real-life problems.', 'Significant features like area and plot are crucial in predicting property prices.', 'Engagement and learning experience are enhanced through practical Python code and exercises.', 'Handwritten digit classification uses an 8x8 pixel grid with 64 features.', 'Identifying significant pixels is essential for accurate digit classification.', 'PCA accelerates machine learning training and inference, aiding in visualization and decision-making.', 'Reducing 64 features to 2 through PCA enhances data visualization and feature understanding.', 'PCA identifies principal components with maximum impact on the target variable.', 'Reducing dimensions through PCA simplifies visualization and understanding of data.']}, {'end': 746.041, 'segs': [{'end': 361.761, 'src': 'embed', 'start': 326.207, 'weight': 2, 'content': [{'end': 330.608, 'text': 'you know, and these are called principal component.', 'start': 326.207, 'duration': 4.401}, {'end': 336.711, 'text': 'so here pc1 is this axis which covers the most of the variance.', 'start': 330.608, 'duration': 6.103}, {'end': 340.992, 'text': 'pc2 is the axis that covers the second most variance.', 'start': 336.711, 'duration': 4.281}, {'end': 347.19, 'text': 'So when you apply PCA, you get a chart like this', 'start': 343.587, 'duration': 3.603}, {'end': 351.753, 'text': 'And I know I have done only this graph for two dimension.', 'start': 347.39, 'duration': 4.363}, {'end': 361.761, 'text': "But really, if you have 100 dimension and if you apply PCA, you can figure out, let's say, 10 most important principal components.", 'start': 352.254, 'duration': 9.507}], 'summary': 'Pca identifies principal components, with 10 important components for 100 dimensions.', 'duration': 35.554, 'max_score': 326.207, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc326207.jpg'}, {'end': 540.672, 'src': 'embed', 'start': 461.593, 'weight': 0, 'content': [{'end': 467.496, 'text': "is you need to scale the features, because if you don't scale it, it's not gonna work.", 'start': 461.593, 'duration': 5.903}, {'end': 474.7, 'text': 'okay, for example, here is a chart, and if this graph is in millions and this graph is in a normal value,', 'start': 467.496, 'duration': 7.204}, {'end': 480.303, 'text': 'you know the graph might become skewed and PC and the PCA will not work.', 'start': 474.7, 'duration': 5.603}, {'end': 484.509, 'text': 'as per your expectation, Accuracy might drop.', 'start': 480.303, 'duration': 4.206}, {'end': 486.891, 'text': "so it's a trade off when you're trying to reduce.", 'start': 484.509, 'duration': 2.382}, {'end': 496.437, 'text': "Let's say you have 100 features and if all 100 features are important and they're trying to contribute to end target variable.", 'start': 487.671, 'duration': 8.766}, {'end': 503.121, 'text': "If you reduce from 100 to 5, you're going to lose a lot of information, so we'll see all of that in the coding session.", 'start': 497.538, 'duration': 5.583}, {'end': 511.501, 'text': 'But just to summarize, PC is called a dimensionality reduction technique as it helps us reduce dimension.', 'start': 504.195, 'duration': 7.306}, {'end': 515.784, 'text': 'We already saw here in this case, we had 64 dimension.', 'start': 512.062, 'duration': 3.722}, {'end': 519.587, 'text': 'This helped us to get to six dimension.', 'start': 516.466, 'duration': 3.121}, {'end': 524.673, 'text': 'And when you have less columns, your computation becomes much faster.', 'start': 520.009, 'duration': 4.664}, {'end': 531.11, 'text': 'And it also helps you to address the dimensionality curse problem.', 'start': 526.989, 'duration': 4.121}, {'end': 535.751, 'text': 'In machine learning, dimensionality curse is a big problem.', 'start': 531.51, 'duration': 4.241}, {'end': 540.672, 'text': 'So many data sets having so many columns, you know, so many dimensions.', 'start': 536.111, 'duration': 4.561}], 'summary': 'Scaling features is crucial for pca to reduce dimensions and improve computation speed in ml.', 'duration': 79.079, 'max_score': 461.593, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc461593.jpg'}, {'end': 654.7, 'src': 'heatmap', 'start': 590.829, 'weight': 0.768, 'content': [{'end': 595.975, 'text': "right?. Now let's look at dataset.data.shape.", 'start': 590.829, 'duration': 5.146}, {'end': 603.814, 'text': 'So there are almost 1700 or 1800 samples.', 'start': 598.772, 'duration': 5.042}, {'end': 606.956, 'text': 'Every sample has 64 columns.', 'start': 604.354, 'duration': 2.602}, {'end': 612.898, 'text': "So if you look at the first sample, it's a flat one dimensional array of 64 pixels.", 'start': 606.976, 'duration': 5.922}, {'end': 622.32, 'text': 'and if you want to visualize this data using matplotlib library, you need to convert it into two dimensional array.', 'start': 614.879, 'duration': 7.441}, {'end': 629.382, 'text': 'and the way you do that is by calling reshape function, because this is numpy array and when you do eight by eight,', 'start': 622.32, 'duration': 7.062}, {'end': 636.503, 'text': 'it just converts one dimensional array into two dimensional array of pixels.', 'start': 629.382, 'duration': 7.121}, {'end': 646.097, 'text': 'now you can import matplotlib and use the plot.', 'start': 636.503, 'duration': 9.594}, {'end': 654.7, 'text': "you know, i'm gonna just plot a gray image, and when you do plot dot match show like metric show,", 'start': 646.097, 'duration': 8.603}], 'summary': 'Dataset contains 1700-1800 samples, each with 64 columns. data can be visualized using matplotlib library by reshaping into a two-dimensional array.', 'duration': 63.871, 'max_score': 590.829, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc590829.jpg'}], 'start': 297.372, 'title': 'Understanding pca for dimensionality reduction', 'summary': 'Explains how principal component analysis (pca) reduces dimensionality from 64 to 6, aiding in faster computation and addressing the dimensionality curse problem in machine learning.', 'chapters': [{'end': 746.041, 'start': 297.372, 'title': 'Understanding pca for dimensionality reduction', 'summary': 'Explains how principal component analysis (pca) helps reduce dimensionality by extracting the most important components, such as in the case of the handwritten digits dataset, where it reduced 64 dimensions to 6, aiding in faster computation and addressing the dimensionality curse problem in machine learning.', 'duration': 448.669, 'highlights': ['PCA helps in reducing dimensionality by extracting the most important components, as demonstrated in the case of the handwritten digits dataset where it reduced 64 dimensions to 6, aiding in faster computation and addressing the dimensionality curse problem in machine learning.', 'PCA allows for the extraction of the most important principal components, even in cases with 100 features, aiding in information extraction and variance coverage.', 'When applying PCA to the handwritten digits dataset, it was able to reduce the dimension from 64 to 6, thus aiding in faster computation and addressing the dimensionality curse in machine learning.', 'The importance of scaling features before using PCA is emphasized, as unscaled features can lead to skewed graphs and potential inefficacy of PCA, impacting accuracy and trade-offs in information loss.', 'PCA is highlighted as a dimensionality reduction technique that helps in reducing the complexity of models, addressing the dimensionality curse problem in machine learning, and enabling easier visualization of data.']}], 'duration': 448.669, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc297372.jpg', 'highlights': ['PCA reduces dimensionality from 64 to 6, aiding in faster computation and addressing dimensionality curse in ML.', 'PCA extracts most important components, reducing 64 dimensions to 6, aiding in faster computation and addressing dimensionality curse in ML.', 'PCA allows extraction of important principal components, aiding in information extraction and variance coverage.', 'PCA reduces dimension from 64 to 6, aiding in faster computation and addressing dimensionality curse in ML.', 'Importance of scaling features before using PCA is emphasized to avoid skewed graphs and inefficacy, impacting accuracy and trade-offs in information loss.', 'PCA is a dimensionality reduction technique that reduces model complexity, addresses dimensionality curse in ML, and enables easier data visualization.']}, {'end': 947.654, 'segs': [{'end': 819.908, 'src': 'embed', 'start': 746.041, 'weight': 0, 'content': [{'end': 751.963, 'text': 'so this data set dot data is your feature and data set dot target is your class.', 'start': 746.041, 'duration': 5.922}, {'end': 758.624, 'text': 'basically, we are classifying this in 10 classes 0 to 9..', 'start': 751.963, 'duration': 6.661}, {'end': 767.146, 'text': 'okay, let me create a data frame now, so i can do pd dot data frame and just supply data.', 'start': 758.624, 'duration': 8.522}, {'end': 770.247, 'text': 'set dot data here and that will create a data frame for you.', 'start': 767.146, 'duration': 3.101}, {'end': 777.142, 'text': 'But the column headers are, you know, randomly assigned.', 'start': 772, 'duration': 5.142}, {'end': 780.764, 'text': "So I'm going to do columns.", 'start': 777.162, 'duration': 3.602}, {'end': 784.445, 'text': 'You can say dataset dot feature names.', 'start': 782.024, 'duration': 2.421}, {'end': 789.247, 'text': 'And the feature names are all the pixels, you know.', 'start': 787.286, 'duration': 1.961}, {'end': 794.229, 'text': 'Like zero throw, zero pixel, zero throw, first pixel and so on.', 'start': 790.727, 'duration': 3.502}, {'end': 799.371, 'text': 'And I will just save that into a data frame object.', 'start': 795.289, 'duration': 4.082}, {'end': 802.979, 'text': 'OK, my DataFrame object is ready.', 'start': 801.098, 'duration': 1.881}, {'end': 816.126, 'text': 'Now I mean, I know from this data set the values are between 0 to 16, but if you want to confirm you can do df.describe.', 'start': 804.14, 'duration': 11.986}, {'end': 819.908, 'text': 'And for each of the columns.', 'start': 818.467, 'duration': 1.441}], 'summary': 'Creating a data frame with 10 classes 0 to 9 and pixel values between 0 to 16.', 'duration': 73.867, 'max_score': 746.041, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc746041.jpg'}, {'end': 884.801, 'src': 'embed', 'start': 854.744, 'weight': 3, 'content': [{'end': 856.584, 'text': 'you know the digit which tells you zero to nine.', 'start': 854.744, 'duration': 1.84}, {'end': 865.885, 'text': "Now I would like to scale these features before building my machine learning model, so I'm just going to use the standard scalar.", 'start': 858.078, 'duration': 7.807}, {'end': 869.809, 'text': 'You know, scaling is a common thing that I do before training my model.', 'start': 865.925, 'duration': 3.884}, {'end': 876.198, 'text': 'and xclr provides very convenient api to do that.', 'start': 871.676, 'duration': 4.522}, {'end': 884.801, 'text': 'when you do scalar.fit transform, it will just scale that x thing and you can store this into x scale.', 'start': 876.198, 'duration': 8.603}], 'summary': 'Scaling features from zero to nine using standard scalar for machine learning model training.', 'duration': 30.057, 'max_score': 854.744, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc854744.jpg'}], 'start': 746.041, 'title': 'Dataset classification and feature scaling', 'summary': "Covers dataset classification into 10 classes, data frame creation with feature names, and confirmation of pixel values ranging from 0 to 16. it also explains scaling features using standard scalar before training a machine learning model, highlighting xclr's api and a 20 percent test size for reproducibility.", 'chapters': [{'end': 854.744, 'start': 746.041, 'title': 'Data set classification and dataframe creation', 'summary': 'Covers the classification of a dataset into 10 classes 0 to 9, creation of a data frame with feature names as column headers, and confirmation of pixel values ranging between 0 to 16, leading to the creation of x and y data sets.', 'duration': 108.703, 'highlights': ['A dataset is classified into 10 classes, from 0 to 9, for the feature data set dot data and data set dot target for the class.', 'Creation of a data frame is demonstrated using pd dot data frame and supplying data set dot data, with column headers assigned as dataset dot feature names and saving it into a data frame object.', 'Confirmation of pixel values ranging between 0 to 16 is done using df.describe, revealing the minimum and maximum values for most pixels, which are between 0 and 16.', 'The creation of x and y data sets is mentioned, where x represents the data frame and y represents the target classes, based on the dataset provided.']}, {'end': 947.654, 'start': 854.744, 'title': 'Scaling features for machine learning', 'summary': "Explains the process of scaling features using standard scalar before training a machine learning model, emphasizing the convenience of xclr's api and the use of train test split with a 20 percent test size for reproducibility.", 'duration': 92.91, 'highlights': ["The process of scaling features using standard scalar before training a machine learning model is highlighted, emphasizing the convenience of xclr's API and the use of train test split with a 20 percent test size for reproducibility.", 'Standard scalar is used for scaling features, with the values being scaled to stand between -1 and 1, and the availability of the option to use min max scalar is mentioned.', "The convenience of xclr's API for scaling features is emphasized, indicating that scalar.fit transform can be used to scale the features and store them into x scale for further use."]}], 'duration': 201.613, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc746041.jpg', 'highlights': ['A dataset is classified into 10 classes, from 0 to 9, for the feature data set dot data and data set dot target for the class.', 'Creation of a data frame is demonstrated using pd dot data frame and supplying data set dot data, with column headers assigned as dataset dot feature names and saving it into a data frame object.', 'Confirmation of pixel values ranging between 0 to 16 is done using df.describe, revealing the minimum and maximum values for most pixels, which are between 0 and 16.', "The process of scaling features using standard scalar before training a machine learning model is highlighted, emphasizing the convenience of xclr's API and the use of train test split with a 20 percent test size for reproducibility."]}, {'end': 1268.189, 'segs': [{'end': 1012.713, 'src': 'embed', 'start': 973.984, 'weight': 0, 'content': [{'end': 975.605, 'text': 'Again, pretty standard.', 'start': 973.984, 'duration': 1.621}, {'end': 979.865, 'text': 'and model.score.', 'start': 978.763, 'duration': 1.102}, {'end': 984.013, 'text': 'So I do model.score to measure the accuracy.', 'start': 980.346, 'duration': 3.667}, {'end': 990.605, 'text': '97%, see this is pretty amazing.', 'start': 988.341, 'duration': 2.264}, {'end': 997.365, 'text': 'I get 97% accuracy by training on all these features.', 'start': 992.222, 'duration': 5.143}, {'end': 999.786, 'text': "But now let's do PCA.", 'start': 997.925, 'duration': 1.861}, {'end': 1005.129, 'text': 'So to do PCA, you can import PCA from sklearn decomposition.', 'start': 999.966, 'duration': 5.163}, {'end': 1009.491, 'text': 'And then I will create a PCA variable.', 'start': 1006.089, 'duration': 3.402}, {'end': 1010.832, 'text': 'So let me increase my font size.', 'start': 1009.511, 'duration': 1.321}, {'end': 1012.713, 'text': "It's very simple, PCA.", 'start': 1011.412, 'duration': 1.301}], 'summary': 'Model achieves 97% accuracy before pca implementation.', 'duration': 38.729, 'max_score': 973.984, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc973984.jpg'}, {'end': 1064.024, 'src': 'heatmap', 'start': 1013.093, 'weight': 1, 'content': [{'end': 1015.014, 'text': 'And here you supply two type of arguments.', 'start': 1013.093, 'duration': 1.921}, {'end': 1018.255, 'text': 'You can either say how many components you want.', 'start': 1015.114, 'duration': 3.141}, {'end': 1020.256, 'text': 'You can say two, five, trial and error.', 'start': 1018.616, 'duration': 1.64}, {'end': 1020.977, 'text': 'You can figure it out.', 'start': 1020.297, 'duration': 0.68}, {'end': 1026.146, 'text': "But what I'm going to do is I'm going to supply 0.95.", 'start': 1021.437, 'duration': 4.709}, {'end': 1029.694, 'text': 'What this will tell PCA is that retain 95% of useful features.', 'start': 1026.146, 'duration': 3.548}, {'end': 1036.414, 'text': 'And then create new dimensions.', 'start': 1034.032, 'duration': 2.382}, {'end': 1040.694, 'text': "You're not explicitly saying I want only two dimensional as an output.", 'start': 1037.034, 'duration': 3.66}, {'end': 1051.958, 'text': "You're saying give me, retain 95% of information, retain 95% variation, and then whatever features you come up with, I'm happy with that.", 'start': 1041.115, 'duration': 10.843}, {'end': 1057.4, 'text': 'And then pca.fit transform.', 'start': 1053.499, 'duration': 3.901}, {'end': 1061.543, 'text': 'The API is quite simple.', 'start': 1059.662, 'duration': 1.881}, {'end': 1064.024, 'text': 'You supply your data frame here.', 'start': 1062.603, 'duration': 1.421}], 'summary': 'Pca can retain 95% useful features, create new dimensions based on that, and transform the data.', 'duration': 50.931, 'max_score': 1013.093, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc1013093.jpg'}, {'end': 1092.015, 'src': 'heatmap', 'start': 1068.426, 'weight': 0.828, 'content': [{'end': 1075.569, 'text': "And when you do xpca.shape, see, now you've got only 29 columns.", 'start': 1068.426, 'duration': 7.143}, {'end': 1082.588, 'text': 'If you look at your original data frame, you had 64 columns.', 'start': 1076.61, 'duration': 5.978}, {'end': 1088.473, 'text': 'So 64 columns to 29, it probably got rid of unnecessary things such as this guy, for example.', 'start': 1082.628, 'duration': 5.845}, {'end': 1090.414, 'text': 'This is 0th pixel.', 'start': 1088.493, 'duration': 1.921}, {'end': 1092.015, 'text': 'All values are 0.', 'start': 1091.055, 'duration': 0.96}], 'summary': 'After applying xpca.shape, the columns reduced from 64 to 29, eliminating unnecessary features.', 'duration': 23.589, 'max_score': 1068.426, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc1068426.jpg'}, {'end': 1149.61, 'src': 'heatmap', 'start': 1108.248, 'weight': 0.87, 'content': [{'end': 1132.052, 'text': 'Okay, So if you do X PCA, you see it actually calculated new columns for you, and if you do PCA variable dot, explained variance ratio,', 'start': 1108.248, 'duration': 23.804}, {'end': 1143.964, 'text': 'It will tell you that my first column is capturing, you know, 14% of variation or 14% of useful information from my dataset.', 'start': 1135.174, 'duration': 8.79}, {'end': 1149.61, 'text': 'PC2 second component is capturing 13% of information and so on.', 'start': 1144.705, 'duration': 4.905}], 'summary': 'Pca analysis resulted in capturing 14% of variation in first component, 13% in second component, and so on.', 'duration': 41.362, 'max_score': 1108.248, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc1108248.jpg'}, {'end': 1183.09, 'src': 'embed', 'start': 1153.729, 'weight': 2, 'content': [{'end': 1156.551, 'text': 'you print, explain variation ratio.', 'start': 1153.729, 'duration': 2.822}, {'end': 1159.033, 'text': 'there is another thing called n components.', 'start': 1156.551, 'duration': 2.482}, {'end': 1162.976, 'text': 'it will tell you how many components you got, which is basically your columns.', 'start': 1159.033, 'duration': 3.943}, {'end': 1164.637, 'text': 'you know 29.', 'start': 1162.976, 'duration': 1.661}, {'end': 1170.581, 'text': 'so here pca helped me reduce my columns from 64 to 29.', 'start': 1164.637, 'duration': 5.944}, {'end': 1174.164, 'text': 'all these 29 columns are computed column.', 'start': 1170.581, 'duration': 3.583}, {'end': 1175.185, 'text': 'they are the new feature.', 'start': 1174.164, 'duration': 1.021}, {'end': 1183.09, 'text': 'but good news is you can now use this uh, new data frame to train your model.', 'start': 1175.945, 'duration': 7.145}], 'summary': 'Pca reduced columns from 64 to 29, allowing use of new data frame for model training.', 'duration': 29.361, 'max_score': 1153.729, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc1153729.jpg'}, {'end': 1268.189, 'src': 'embed', 'start': 1229.497, 'weight': 4, 'content': [{'end': 1231.919, 'text': 'And then model dot score.', 'start': 1229.497, 'duration': 2.422}, {'end': 1241.448, 'text': 'and y is say, i got pretty good accuracy, 97.', 'start': 1235.643, 'duration': 5.805}, {'end': 1249.134, 'text': "but let's say this failed to converge because total number of iterations limit reached.", 'start': 1241.448, 'duration': 7.686}, {'end': 1251.636, 'text': 'so i need to supply some maximum iteration.', 'start': 1249.134, 'duration': 2.502}, {'end': 1256.44, 'text': 'so i will say max iter.', 'start': 1251.636, 'duration': 4.804}, {'end': 1265.067, 'text': 'so i need to increase my iteration basically to uh, so that the gradient descent can converge.', 'start': 1256.44, 'duration': 8.627}, {'end': 1268.189, 'text': 'see almost same accuracy.', 'start': 1266.647, 'duration': 1.542}], 'summary': 'Model achieved 97% accuracy but failed to converge due to iteration limit. increasing iterations improved convergence and maintained high accuracy.', 'duration': 38.692, 'max_score': 1229.497, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc1229497.jpg'}], 'start': 947.654, 'title': 'Machine learning: logistic regression, pca, and dimensionality reduction', 'summary': 'Covers training a model using logistic regression with 97% accuracy, implementing pca to reduce features from 64 to 29, capturing varying percentages of useful information, and addressing convergence issues by increasing the maximum iteration for gradient descent.', 'chapters': [{'end': 1104.725, 'start': 947.654, 'title': 'Machine learning: logistic regression and pca', 'summary': 'Covers training a model using logistic regression, achieving 97% accuracy, and implementing pca to reduce the number of features from 64 to 29.', 'duration': 157.071, 'highlights': ['Achieving 97% accuracy by training on all features using logistic regression. The model achieved 97% accuracy when trained on all the features using logistic regression.', 'Implementing PCA to reduce the number of features from 64 to 29. PCA was used to reduce the original 64 columns to 29, effectively eliminating unnecessary features and retaining 95% of useful information.']}, {'end': 1268.189, 'start': 1105.105, 'title': 'Utilizing pca for dimensionality reduction and model training', 'summary': 'Discusses using pca to reduce the number of columns from 64 to 29, with the new columns capturing varying percentages of useful information, allowing for successful logistic regression with a high accuracy of 97% and addressing convergence issues by increasing the maximum iteration for gradient descent.', 'duration': 163.084, 'highlights': ['Using PCA reduced the columns from 64 to 29, capturing varying percentages of useful information. PCA helped reduce columns from 64 to 29, with the new columns capturing 14% to 13% of useful information, enabling dimensionality reduction.', 'Successfully trained logistic regression model on PCA-transformed data with high accuracy of 97%. The logistic regression model trained on PCA-transformed data achieved an accuracy of 97%, demonstrating the effectiveness of the dimensionality-reduced features in model training.', 'Addressing convergence issues by increasing the maximum iteration for gradient descent. To address convergence issues, the maximum iteration for gradient descent was increased, resulting in successful convergence and a similar high accuracy of the logistic regression model.']}], 'duration': 320.535, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc947654.jpg', 'highlights': ['Achieving 97% accuracy by training on all features using logistic regression.', 'Successfully trained logistic regression model on PCA-transformed data with high accuracy of 97%.', 'Implementing PCA to reduce the number of features from 64 to 29.', 'Using PCA reduced the columns from 64 to 29, capturing varying percentages of useful information.', 'Addressing convergence issues by increasing the maximum iteration for gradient descent.']}, {'end': 1447.542, 'segs': [{'end': 1358.066, 'src': 'embed', 'start': 1303.251, 'weight': 0, 'content': [{'end': 1307.813, 'text': "so i'm going to do pca transform with my x, x pca dot shape.", 'start': 1303.251, 'duration': 4.562}, {'end': 1310.86, 'text': 'Now we are telling.', 'start': 1309.159, 'duration': 1.701}, {'end': 1314.861, 'text': 'That we want two components.', 'start': 1312.26, 'duration': 2.601}, {'end': 1323.883, 'text': 'OK, so it it will identify the two most important features, but here see let me see.', 'start': 1315.981, 'duration': 7.902}, {'end': 1326.424, 'text': 'So see every data point is just two features.', 'start': 1324.643, 'duration': 1.781}, {'end': 1333.566, 'text': 'Now did it capture enough information? We are not sure when you print explain variation variance ratio.', 'start': 1326.944, 'duration': 6.622}, {'end': 1339.653, 'text': 'you are capturing 14% and 30% around 27, 28% information.', 'start': 1334.83, 'duration': 4.823}, {'end': 1345.758, 'text': 'So naturally, when you use this data set to train your model, the accuracy is going to be low.', 'start': 1339.834, 'duration': 5.924}, {'end': 1352.122, 'text': "So I'm just going to like same code, but I'm using the new PC and the new data frame.", 'start': 1345.938, 'duration': 6.184}, {'end': 1358.066, 'text': 'And when you train the model, you see the accuracy is reduced to 60, 60.83.', 'start': 1352.542, 'duration': 5.524}], 'summary': 'Pca transformed data with 2 components, capturing 14-30% information, reducing model accuracy to 60.83%', 'duration': 54.815, 'max_score': 1303.251, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc1303251.jpg'}, {'end': 1433.05, 'src': 'embed', 'start': 1409.236, 'weight': 2, 'content': [{'end': 1415.679, 'text': 'you need to predict if the person has heart disease or not and you are going to use PCA.', 'start': 1409.236, 'duration': 6.443}, {'end': 1424.824, 'text': 'so work on all these six aspects of the exercise and let me know how it goes in the comment post below your solution, your github link, whatever,', 'start': 1415.679, 'duration': 9.145}, {'end': 1428.246, 'text': 'and if you have any question, post in a comment below.', 'start': 1424.824, 'duration': 3.422}, {'end': 1433.05, 'text': 'and if you like this video, share it with your friends, you know, so that they can learn.', 'start': 1428.246, 'duration': 4.804}], 'summary': 'Predict heart disease using pca. share if you like the video.', 'duration': 23.814, 'max_score': 1409.236, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc1409236.jpg'}], 'start': 1268.189, 'title': 'Using pca to reduce dimensions', 'summary': 'Discusses the impact of reducing features using pca, demonstrating a decrease in accuracy from 97.22% to 60.83% and highlights the importance of exercises, such as a heart disease prediction exercise using pca.', 'chapters': [{'end': 1447.542, 'start': 1268.189, 'title': 'Using pca to reduce dimensions', 'summary': 'Discusses using pca to reduce dimensions, showcasing how reducing features can impact accuracy, with an example showing a 97.22% accuracy dropping to 60.83%, and emphasizes the importance of exercises, specifically a heart disease prediction exercise using pca.', 'duration': 179.353, 'highlights': ['The accuracy drops from 97.22% to 60.83% when using a dataset reduced to two dimensions, demonstrating the trade-off between dimensionality reduction and model accuracy.', 'The tutorial encourages viewers to work on a heart disease prediction exercise using PCA, with a provided dataset and requests for their solutions and questions, emphasizing the practical application of the discussed concepts.', 'When explicitly setting the number of components to two in PCA, it captures around 14%, 30%, and 27-28% of the information from the original dataset, highlighting the impact of dimensionality reduction on information retention.']}], 'duration': 179.353, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/8klqIM9UvAc/pics/8klqIM9UvAc1268189.jpg', 'highlights': ['The accuracy drops from 97.22% to 60.83% when using a dataset reduced to two dimensions, demonstrating the trade-off between dimensionality reduction and model accuracy.', 'When explicitly setting the number of components to two in PCA, it captures around 14%, 30%, and 27-28% of the information from the original dataset, highlighting the impact of dimensionality reduction on information retention.', 'The tutorial encourages viewers to work on a heart disease prediction exercise using PCA, with a provided dataset and requests for their solutions and questions, emphasizing the practical application of the discussed concepts.']}], 'highlights': ['PCA reduces dimensionality from 64 to 6, aiding in faster computation and addressing dimensionality curse in ML.', 'PCA reduces dimensionality in machine learning for real-life problems.', 'PCA accelerates machine learning training and inference, aiding in visualization and decision-making.', 'Significant features like area and plot are crucial in predicting property prices.', 'Achieving 97% accuracy by training on all features using logistic regression.', 'The accuracy drops from 97.22% to 60.83% when using a dataset reduced to two dimensions, demonstrating the trade-off between dimensionality reduction and model accuracy.', 'The tutorial encourages viewers to work on a heart disease prediction exercise using PCA, with a provided dataset and requests for their solutions and questions, emphasizing the practical application of the discussed concepts.']}