title
Tutorial I
description
This is the first Tutorial of the Introduction to Machine Learning MOOC on NPTEL. It summarizes the content covered in the first week of the course and teaches how to solve problems in exam.
Topics covered are:
1. Types of learning: Supervised v.s. Unsupervised
2. Types of features: Categorical v.s. Continuous
3. Types of supervised learning: Classification v.s. Regression
4. Bias Variance Tradeoff
5. Generalization of performance of a learning algorithm
Tutor: Anirban Santara, Google PhD Fellow, Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur. Homepage: http://santara.github.io
Course homepage: https://onlinecourses.nptel.ac.in/noc16_cs18/course
detail
{'title': 'Tutorial I', 'heatmap': [{'end': 1702.175, 'start': 1614.139, 'weight': 0.718}, {'end': 1780.338, 'start': 1736.245, 'weight': 0.714}], 'summary': 'The tutorial covers machine learning fundamentals, supervised vs unsupervised learning, types of features and learning algorithms, bias in learning algorithms, bias and variance in learning, and machine learning generalization, addressing practical applications, examples, and performance analysis in machine learning.', 'chapters': [{'end': 384.072, 'segs': [{'end': 49.319, 'src': 'embed', 'start': 18.376, 'weight': 0, 'content': [{'end': 21.479, 'text': 'Hello friends, welcome to the first tutorial session of this course.', 'start': 18.376, 'duration': 3.103}, {'end': 26.744, 'text': 'I am Anirban Santara, I am doing my PhD in machine learning and I am a teaching assistant of this course.', 'start': 22.039, 'duration': 4.705}, {'end': 36.993, 'text': 'And in these tutorial sessions, which we will hold one every week, we will discuss the topics that have been covered in the particular week.', 'start': 27.344, 'duration': 9.649}, {'end': 40.715, 'text': 'and do a quick summary of all the topics that have been covered in this week.', 'start': 37.513, 'duration': 3.202}, {'end': 49.319, 'text': 'And then we will learn how to solve questions and these questions will be the ones that you would expect in the assignments and in the final exams.', 'start': 41.155, 'duration': 8.164}], 'summary': 'Anirban santara, a phd in machine learning, will hold weekly tutorial sessions to discuss and summarize covered topics, and solve questions relevant to assignments and final exams.', 'duration': 30.943, 'max_score': 18.376, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs18376.jpg'}, {'end': 126.749, 'src': 'embed', 'start': 85.951, 'weight': 2, 'content': [{'end': 95.056, 'text': 'categorical versus continuous features.', 'start': 85.951, 'duration': 9.105}, {'end': 108.509, 'text': 'different kinds of supervised learning problems regression versus classification.', 'start': 99.87, 'duration': 8.639}, {'end': 118.644, 'text': 'Then we will go ahead to bias-variance tradeoff, bias versus variance.', 'start': 112.18, 'duration': 6.464}, {'end': 126.749, 'text': 'And we will study how bias and variance they vary with different parameters of the learning algorithm.', 'start': 122.086, 'duration': 4.663}], 'summary': 'Comparison of categorical and continuous features, supervised learning problems, and bias-variance tradeoff in machine learning.', 'duration': 40.798, 'max_score': 85.951, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs85951.jpg'}, {'end': 207.5, 'src': 'embed', 'start': 175.868, 'weight': 1, 'content': [{'end': 182.436, 'text': 'So, the supervised and unsupervised learnings are two major broad categories of machine learning algorithms.', 'start': 175.868, 'duration': 6.568}, {'end': 193.188, 'text': 'The only difference, so whenever you have been given a question and asked whether and a particular learning algorithm has been described.', 'start': 183.116, 'duration': 10.072}, {'end': 194.95, 'text': 'So you have been asked.', 'start': 193.388, 'duration': 1.562}, {'end': 207.5, 'text': 'So there is a big paragraph describing a particular scenario and you have to identify what kind of learning algorithm is going on in this particular scenario.', 'start': 196.572, 'duration': 10.928}], 'summary': 'Supervised and unsupervised learnings are two major categories of machine learning algorithms.', 'duration': 31.632, 'max_score': 175.868, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs175868.jpg'}, {'end': 315.456, 'src': 'embed', 'start': 230.676, 'weight': 3, 'content': [{'end': 236.24, 'text': 'So, let me first supervised learning.', 'start': 230.676, 'duration': 5.564}, {'end': 250.426, 'text': 'So, in supervised learning the training will always come as pairs.', 'start': 240.064, 'duration': 10.362}, {'end': 260.61, 'text': 'So, these pairs will be the first term will always be the input and then you will have the target value.', 'start': 251.967, 'duration': 8.643}, {'end': 276.372, 'text': 'So, this is the true value and this is the description of the present case.', 'start': 260.91, 'duration': 15.462}, {'end': 289.416, 'text': 'So, it could be like an image and the true value may be whether it is happy or sad.', 'start': 277.913, 'duration': 11.503}, {'end': 294.638, 'text': 'So, it is already given that this particular image describes a happy face.', 'start': 289.896, 'duration': 4.742}, {'end': 302.01, 'text': 'And so, Now, the input examples will always come as pairs of input and target values.', 'start': 295.018, 'duration': 6.992}, {'end': 309.373, 'text': 'So if the target values are given and if you have been asked to predict the target values after looking at the input,', 'start': 302.41, 'duration': 6.963}, {'end': 311.474, 'text': 'then it is a supervised learning problem.', 'start': 309.373, 'duration': 2.101}, {'end': 315.456, 'text': 'Whereas, in unsupervised learning you do not have targets.', 'start': 311.854, 'duration': 3.602}], 'summary': 'Supervised learning uses input-target pairs for prediction, while unsupervised learning lacks targets.', 'duration': 84.78, 'max_score': 230.676, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs230676.jpg'}], 'start': 18.376, 'title': 'Machine learning fundamentals', 'summary': 'Covers an introductory session exploring supervised and unsupervised learning, features, regression versus classification, bias-variance tradeoff, and generalization performance analysis in machine learning. it also explains the difference between supervised and unsupervised learning involving training with input-target pairs and finding patterns or information from input data without target values.', 'chapters': [{'end': 230.396, 'start': 18.376, 'title': 'Intro to machine learning', 'summary': 'Covers the first tutorial session discussing supervised and unsupervised learning, features, regression versus classification, bias-variance tradeoff, and generalization performance analysis in machine learning.', 'duration': 212.02, 'highlights': ['The tutorial session covers topics like supervised versus unsupervised learning, different kinds of features, regression versus classification, bias-variance tradeoff, and generalization performance analysis of a learning algorithm.', 'Anirban Santara, a PhD candidate in machine learning, is conducting the tutorial sessions weekly to discuss the covered topics and solve questions related to assignments and final exams.', 'Supervised and unsupervised learnings are two major broad categories of machine learning algorithms, with a focus on the type of training examples presented.', 'The session emphasizes how bias and variance vary with different parameters of the learning algorithm and how to control and analyze the generalization performance of a learning algorithm.']}, {'end': 384.072, 'start': 230.676, 'title': 'Supervised vs unsupervised learning', 'summary': 'Explains the difference between supervised and unsupervised learning, where supervised learning involves training with input-target pairs to predict target values, while unsupervised learning involves finding patterns or information from input data without target values.', 'duration': 153.396, 'highlights': ['In supervised learning, training involves input-target pairs to predict target values, such as predicting whether an image is happy or sad.', 'Unsupervised learning involves finding patterns or information from input data without target values, like identifying regular patterns or expressions from given faces.']}], 'duration': 365.696, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs18376.jpg', 'highlights': ['Anirban Santara, a PhD candidate in machine learning, conducts weekly tutorial sessions covering supervised versus unsupervised learning, features, regression versus classification, bias-variance tradeoff, and generalization performance analysis.', 'Supervised and unsupervised learnings are two major broad categories of machine learning algorithms, focusing on the type of training examples presented.', 'The session emphasizes how bias and variance vary with different parameters of the learning algorithm and how to control and analyze the generalization performance of a learning algorithm.', 'In supervised learning, training involves input-target pairs to predict target values, such as predicting whether an image is happy or sad.', 'Unsupervised learning involves finding patterns or information from input data without target values, like identifying regular patterns or expressions from given faces.']}, {'end': 1207.798, 'segs': [{'end': 492.391, 'src': 'embed', 'start': 435.406, 'weight': 0, 'content': [{'end': 451.531, 'text': 'So, this is again supervised, unsupervised, supervised versus unsupervised learning and let us have this kind of a question.', 'start': 435.406, 'duration': 16.125}, {'end': 476.497, 'text': 'So, suppose I have been given 1 million images with their ground truth labels.', 'start': 452.691, 'duration': 23.806}, {'end': 486.705, 'text': 'So, I have a huge collection of images and I know from the data that is given to us.', 'start': 478.777, 'duration': 7.928}, {'end': 492.391, 'text': 'So, the data consist of these image and its corresponding label pairs.', 'start': 487.406, 'duration': 4.985}], 'summary': 'Supervised vs unsupervised learning with 1 million labeled images', 'duration': 56.985, 'max_score': 435.406, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs435406.jpg'}, {'end': 982.197, 'src': 'embed', 'start': 955.351, 'weight': 1, 'content': [{'end': 963.774, 'text': 'So, over here as we had decided and as we had discussed before that in unsupervised learning, we do not have target values.', 'start': 955.351, 'duration': 8.423}, {'end': 967.175, 'text': 'So, we are not using the class labels over here.', 'start': 964.914, 'duration': 2.261}, {'end': 977.458, 'text': 'We are just looking at the images and trying to find groups or clusters of images which look similar to each other.', 'start': 968.015, 'duration': 9.443}, {'end': 982.197, 'text': 'And, the second step is supervised.', 'start': 979.163, 'duration': 3.034}], 'summary': 'Unsupervised learning: no target values, finding similar image clusters.', 'duration': 26.846, 'max_score': 955.351, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs955351.jpg'}, {'end': 1114.961, 'src': 'embed', 'start': 1060.673, 'weight': 2, 'content': [{'end': 1085.655, 'text': 'So, categorical features are features that have So, these features they appear in a finite number of categories or classes.', 'start': 1060.673, 'duration': 24.982}, {'end': 1096.925, 'text': 'So, for example, gender of a person, then age group.', 'start': 1088.017, 'duration': 8.908}, {'end': 1105.65, 'text': 'of a person like teenagers or children, that is, those who are below, say 13.', 'start': 1098.18, 'duration': 7.47}, {'end': 1107.212, 'text': 'they are children.', 'start': 1105.65, 'duration': 1.562}, {'end': 1108.173, 'text': '13 to 19.', 'start': 1107.212, 'duration': 0.961}, {'end': 1114.961, 'text': 'we are teenagers and then ahead, then we have, like young people, till 35..', 'start': 1108.173, 'duration': 6.788}], 'summary': 'Categorical features have finite categories, e.g. gender and age groups.', 'duration': 54.288, 'max_score': 1060.673, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1060673.jpg'}], 'start': 384.432, 'title': 'Supervised vs unsupervised learning', 'summary': 'Compares supervised and unsupervised learning, emphasizing practical applications, and provides examples of clustering algorithms and image labeling with input data. it also explores image clustering for classifier training, examining the use of categorical versus continuous features.', 'chapters': [{'end': 533.127, 'start': 384.432, 'title': 'Supervised vs unsupervised learning', 'summary': 'Discusses the concepts of supervised and unsupervised learning, highlighting the differences and practical applications, using examples of clustering algorithms and image labeling with input data.', 'duration': 148.695, 'highlights': ['Clustering algorithm identifies groups based on the visual appearance of examples, such as grouping faces together, without the use of target data.', 'In a supervised learning scenario, a large dataset of images with corresponding labels is used, where the input data appears as input label pairs.', 'The chapter emphasizes the practical application of machine learning tasks and the general practice in machine learning when dealing with such scenarios.']}, {'end': 1207.798, 'start': 533.987, 'title': 'Image clustering for classifier training', 'summary': 'Discusses clustering images into similar groups as an unsupervised learning problem and training one classifier per cluster as a supervised learning problem, with a focus on categorical versus continuous features.', 'duration': 673.811, 'highlights': ['The chapter discusses clustering images into similar groups as an unsupervised learning problem and training one classifier per cluster as a supervised learning problem This approach involves using unsupervised learning to cluster images into similar groups and then training a separate classifier for each cluster, making it a two-step process involving both unsupervised and supervised learning.', 'The chapter explains the concept of categorical features and provides examples such as gender and age groups Categorical features, such as gender and age groups, are described as having a finite number of categories, providing a clear understanding of the concept and its relevance in data analysis.']}], 'duration': 823.366, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs384432.jpg', 'highlights': ['In a supervised learning scenario, a large dataset of images with corresponding labels is used, where the input data appears as input label pairs.', 'The chapter discusses clustering images into similar groups as an unsupervised learning problem and training one classifier per cluster as a supervised learning problem.', 'The chapter explains the concept of categorical features and provides examples such as gender and age groups.']}, {'end': 1611.521, 'segs': [{'end': 1280.627, 'src': 'embed', 'start': 1207.818, 'weight': 0, 'content': [{'end': 1213.383, 'text': 'May be there is some other features that would further like you can carry on the diagnosis using them.', 'start': 1207.818, 'duration': 5.565}, {'end': 1216.324, 'text': 'So this kind of feature, like presence or absence of something.', 'start': 1213.763, 'duration': 2.561}, {'end': 1221.785, 'text': 'this is also categorical features, because it can take just two values presence or absence, yes or no.', 'start': 1216.324, 'duration': 5.461}, {'end': 1239.647, 'text': 'So these are all categorical features, whereas continuous features are those which can take an infinite number of values,', 'start': 1222.105, 'duration': 17.542}, {'end': 1249.035, 'text': 'theoretically infinite number of values.', 'start': 1239.647, 'duration': 9.388}, {'end': 1265.484, 'text': 'For example, if Age of a person or something height theoretically they can take an infinite number of values weight of a person,', 'start': 1250.576, 'duration': 14.908}, {'end': 1266.964, 'text': 'then price of something.', 'start': 1265.484, 'duration': 1.48}, {'end': 1271.266, 'text': 'So, these are continuous features.', 'start': 1269.625, 'duration': 1.641}, {'end': 1280.627, 'text': 'So it is very easy to understand that which one is a categorical feature it has to take some, an infinite number of values,', 'start': 1272.905, 'duration': 7.722}], 'summary': 'Transcript discusses categorical and continuous features for diagnosis with quantifiable data.', 'duration': 72.809, 'max_score': 1207.818, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1207818.jpg'}, {'end': 1436.138, 'src': 'embed', 'start': 1401.385, 'weight': 3, 'content': [{'end': 1407.808, 'text': 'kind of supervised learning problem is regression of classification, then you have to just check what is the nature of the output variable,', 'start': 1401.385, 'duration': 6.423}, {'end': 1409.049, 'text': 'whether it is discrete or continuous.', 'start': 1407.808, 'duration': 1.241}, {'end': 1419.555, 'text': 'So, in regression problem the output variable or target variable may be is categorical.', 'start': 1409.069, 'duration': 10.486}, {'end': 1436.138, 'text': 'or discrete, where in classification, I am sorry.', 'start': 1423.451, 'duration': 12.687}], 'summary': 'Supervised learning involves regression or classification based on nature of output variable.', 'duration': 34.753, 'max_score': 1401.385, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1401385.jpg'}], 'start': 1207.818, 'title': 'Types of features and learning algorithms', 'summary': 'Covers the distinction between categorical and continuous features, with categorical features being able to take only two values and continuous features being able to take an infinite number of values. it also introduces the types of supervised learning algorithm, providing an overview of distinguishing between categorical and continuous features, understanding the types of supervised learning (regression and classification), and examples of each, emphasizing the nature of the output variable.', 'chapters': [{'end': 1301.099, 'start': 1207.818, 'title': 'Types of features and learning algorithms', 'summary': 'Covers the distinction between categorical and continuous features, with categorical features being able to take only two values and continuous features being able to take an infinite number of values. it also introduces the types of supervised learning algorithm.', 'duration': 93.281, 'highlights': ['Categorical features can take just two values: presence or absence, such as yes or no. This highlights the definition of categorical features and provides examples of their possible values.', 'Continuous features, such as age, height, weight, and price, can theoretically take an infinite number of values. This details the nature of continuous features and provides examples of such features.', 'Introduces the types of supervised learning algorithm. This highlights the introduction of the types of supervised learning algorithms, setting the stage for further discussion.']}, {'end': 1611.521, 'start': 1301.099, 'title': 'Identifying categorical and continuous features', 'summary': 'Provides an overview of distinguishing between categorical and continuous features, understanding the types of supervised learning (regression and classification), and examples of each, emphasizing the nature of the output variable.', 'duration': 310.422, 'highlights': ['Understanding the distinction between categorical and continuous features is essential in data analysis, where categorical features have a fixed number of values and continuous features can take infinite values theoretically. This distinction is crucial as it impacts the analysis and modeling of the data, influencing the selection of appropriate techniques and algorithms for processing.', 'Supervised learning encompasses regression and classification, with regression focusing on output variables that are continuous, while classification deals with output variables that are discrete. This distinction is important as it determines the type of problem and the nature of the target variable, guiding the choice of predictive modeling techniques and evaluation metrics.', 'Classification problems involve predicting discrete values, such as animal species or medical diagnoses, while regression problems entail predicting continuous values, like car prices or personal attributes. Recognizing the nature of the output variable is crucial in determining the appropriate approach for addressing the problem and selecting the suitable algorithms for prediction.']}], 'duration': 403.703, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1207818.jpg', 'highlights': ['Categorical features can take just two values: presence or absence, such as yes or no. This highlights the definition of categorical features and provides examples of their possible values.', 'Continuous features, such as age, height, weight, and price, can theoretically take an infinite number of values. This details the nature of continuous features and provides examples of such features.', 'Understanding the distinction between categorical and continuous features is essential in data analysis, where categorical features have a fixed number of values and continuous features can take infinite values theoretically. This distinction is crucial as it impacts the analysis and modeling of the data, influencing the selection of appropriate techniques and algorithms for processing.', 'Supervised learning encompasses regression and classification, with regression focusing on output variables that are continuous, while classification deals with output variables that are discrete. This distinction is important as it determines the type of problem and the nature of the target variable, guiding the choice of predictive modeling techniques and evaluation metrics.']}, {'end': 1805.442, 'segs': [{'end': 1702.175, 'src': 'heatmap', 'start': 1614.139, 'weight': 0.718, 'content': [{'end': 1627.003, 'text': 'So, these all are continuous variables and prediction of these continuous variables from certain attributes of the person is a regression problem.', 'start': 1614.139, 'duration': 12.864}, {'end': 1639.406, 'text': 'So, let us move on to the next topic which is bias variance versus variance.', 'start': 1627.563, 'duration': 11.843}, {'end': 1657.744, 'text': 'What is the bias of a learning algorithm? The bias of a learning algorithm is a set of erroneous assumptions in the learning algorithm.', 'start': 1640.026, 'duration': 17.718}, {'end': 1670.757, 'text': 'And so, this particular thing bias, it is due to the learning algorithm.', 'start': 1664.893, 'duration': 5.864}, {'end': 1673.679, 'text': 'So, it is not due to the training examples that have been given to you.', 'start': 1671.078, 'duration': 2.601}, {'end': 1681.245, 'text': 'So, there are certain assumptions within the learning algorithm, which keeps your model limited in its capacity to learn.', 'start': 1674.1, 'duration': 7.145}, {'end': 1683.306, 'text': 'So, that is what is bias.', 'start': 1681.525, 'duration': 1.781}, {'end': 1688.29, 'text': 'And due to bias, you can miss out important relations between inputs and outputs.', 'start': 1683.687, 'duration': 4.603}, {'end': 1695.513, 'text': 'So you can seek, you can observe that there is a very valid input output relationship, but your learning algorithm cannot capture it,', 'start': 1688.57, 'duration': 6.943}, {'end': 1702.175, 'text': 'just because it is not equipped enough, it is not capable enough to learn that particular thing,', 'start': 1695.513, 'duration': 6.662}], 'summary': "Regression problem involves predicting continuous variables. bias limits model's learning capacity.", 'duration': 88.036, 'max_score': 1614.139, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1614139.jpg'}, {'end': 1695.513, 'src': 'embed', 'start': 1664.893, 'weight': 0, 'content': [{'end': 1670.757, 'text': 'And so, this particular thing bias, it is due to the learning algorithm.', 'start': 1664.893, 'duration': 5.864}, {'end': 1673.679, 'text': 'So, it is not due to the training examples that have been given to you.', 'start': 1671.078, 'duration': 2.601}, {'end': 1681.245, 'text': 'So, there are certain assumptions within the learning algorithm, which keeps your model limited in its capacity to learn.', 'start': 1674.1, 'duration': 7.145}, {'end': 1683.306, 'text': 'So, that is what is bias.', 'start': 1681.525, 'duration': 1.781}, {'end': 1688.29, 'text': 'And due to bias, you can miss out important relations between inputs and outputs.', 'start': 1683.687, 'duration': 4.603}, {'end': 1695.513, 'text': 'So you can seek, you can observe that there is a very valid input output relationship, but your learning algorithm cannot capture it,', 'start': 1688.57, 'duration': 6.943}], 'summary': "Bias in learning algorithm limits model's capacity to capture valid input-output relationships.", 'duration': 30.62, 'max_score': 1664.893, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1664893.jpg'}, {'end': 1787.863, 'src': 'heatmap', 'start': 1736.245, 'weight': 1, 'content': [{'end': 1762.369, 'text': 'So, the model you are learning towards noise rather than important output are important features of the relationship between input and output.', 'start': 1736.245, 'duration': 26.124}, {'end': 1773.313, 'text': 'So, variance happens when your learning algorithm it tries to concentrate on unwanted like or trivial or noisy variations in the data.', 'start': 1763.869, 'duration': 9.444}, {'end': 1780.338, 'text': 'Rather than the important aspects of the data that needs to be captured and happens when you have too many features,', 'start': 1774.473, 'duration': 5.865}, {'end': 1787.863, 'text': 'or you have too many parameters in your learning algorithm in your model, or you do not have much training data.', 'start': 1780.338, 'duration': 7.525}], 'summary': 'Focusing on noise rather than important features leads to variance, caused by excessive features or parameters and insufficient training data.', 'duration': 83.708, 'max_score': 1736.245, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1736245.jpg'}], 'start': 1614.139, 'title': 'Bias in learning algorithms', 'summary': 'Discusses bias in learning algorithms, emphasizing its impact on learning input-output relationships, and sensitivity towards noise and features.', 'chapters': [{'end': 1704.115, 'start': 1614.139, 'title': 'Understanding bias in learning algorithms', 'summary': "Discusses the concept of bias in learning algorithms, highlighting how it results from erroneous assumptions and limits the model's capacity to learn important input-output relationships.", 'duration': 89.976, 'highlights': ["The bias of a learning algorithm is a set of erroneous assumptions that limits the model's capacity to learn important input-output relationships.", 'Bias is due to the assumptions within the learning algorithm and not the training examples, leading to a limited capacity to capture valid input-output relationships.', "Learning algorithm's inability to capture important input-output relationships is a result of its limited capability, causing it to miss out on crucial input-output relations."]}, {'end': 1805.442, 'start': 1704.155, 'title': 'Bias vs variance in learning algorithms', 'summary': 'Discusses the bias and variance of learning algorithms, highlighting the impact of sensitivity towards noise, the concentration on noisy variations, and the influence of features and training data.', 'duration': 101.287, 'highlights': ['The variance of a learning algorithm is due to its sensitivity towards noise, leading it to focus on trivial or noisy variations rather than important aspects of the data.', 'Variance occurs when there are too many features or parameters in the learning algorithm, or when there is insufficient training data, inhibiting the capture of important data aspects.', 'Bias in a learning algorithm is reflected in its tendency to learn towards noise rather than important output features, indicating a bias in the relationship between input and output.']}], 'duration': 191.303, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1614139.jpg', 'highlights': ['The bias of a learning algorithm limits its capacity to learn important input-output relationships.', 'Variance occurs when there are too many features or parameters in the learning algorithm, inhibiting the capture of important data aspects.', 'Bias in a learning algorithm is reflected in its tendency to learn towards noise rather than important output features.']}, {'end': 2048.489, 'segs': [{'end': 1885.459, 'src': 'embed', 'start': 1806.364, 'weight': 0, 'content': [{'end': 1824.26, 'text': 'So, let us learn, let us try to predict how bias and variance varies with if you vary different aspects of our learning algorithm.', 'start': 1806.364, 'duration': 17.896}, {'end': 1830.403, 'text': 'And we will study the variation of bias and variance with three things.', 'start': 1824.4, 'duration': 6.003}, {'end': 1847.5, 'text': 'So, number of features, number of parameters of your model and number of training examples.', 'start': 1831.824, 'duration': 15.676}, {'end': 1865.653, 'text': 'So when you increase the number of features of your learning algorithm, then the bias decreases, because your learning algorithm,', 'start': 1852.964, 'duration': 12.689}, {'end': 1873.016, 'text': 'it looks at more and more features of the data and becomes more and more sophisticated.', 'start': 1865.653, 'duration': 7.363}, {'end': 1876.358, 'text': 'So, the bias decreases, whereas the variance increases.', 'start': 1873.116, 'duration': 3.242}, {'end': 1885.459, 'text': 'because the more the number of input features it looks at, the more is the amount of noise that it gets exposed to.', 'start': 1877.757, 'duration': 7.702}], 'summary': 'Bias decreases with more features, while variance increases.', 'duration': 79.095, 'max_score': 1806.364, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1806364.jpg'}, {'end': 1968.797, 'src': 'embed', 'start': 1907.809, 'weight': 2, 'content': [{'end': 1915.659, 'text': 'the model becomes more and more sophisticated and its capacity learning capacity increases, whereas the variance again increases.', 'start': 1907.809, 'duration': 7.85}, {'end': 1924.343, 'text': 'Because, it can use the parameters, the more parameters it got to model noise more efficiently.', 'start': 1917, 'duration': 7.343}, {'end': 1931.245, 'text': 'Whereas, with the number of training examples, bias remains constant, remains the same.', 'start': 1925.283, 'duration': 5.962}, {'end': 1942.981, 'text': 'because bias is not a property that is directly related to the number of training examples that has been presented to the learning algorithm.', 'start': 1933.792, 'duration': 9.189}, {'end': 1947.264, 'text': 'rather, it is a limitation of the model of the learning algorithm.', 'start': 1942.981, 'duration': 4.283}, {'end': 1952.629, 'text': 'So your learning algorithm, your model, is not capable enough to learn important features in the data,', 'start': 1947.585, 'duration': 5.044}, {'end': 1959.134, 'text': 'and that is why bias comes in and it is not a problem which comes directly from the training data.', 'start': 1952.629, 'duration': 6.505}, {'end': 1963.755, 'text': 'So, if you increase the number of training examples and your model remains equally incapable as before.', 'start': 1959.354, 'duration': 4.401}, {'end': 1968.797, 'text': 'So, the bias persists and you cannot do anything, whereas the variance goes down it decreases.', 'start': 1963.995, 'duration': 4.802}], 'summary': "As the model's learning capacity increases, variance increases, while bias remains constant with the number of training examples.", 'duration': 60.988, 'max_score': 1907.809, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1907809.jpg'}, {'end': 2048.489, 'src': 'embed', 'start': 2024.924, 'weight': 5, 'content': [{'end': 2032.808, 'text': 'And the last thing that we are going to discuss today is generalization of performance.', 'start': 2024.924, 'duration': 7.884}, {'end': 2041.286, 'text': 'So what do you mean by generalization of the performance of a learning algorithm?', 'start': 2038.544, 'duration': 2.742}, {'end': 2048.489, 'text': 'The generalization means that how good the learning algorithm would perform when it is presented examples which it has not seen before.', 'start': 2041.526, 'duration': 6.963}], 'summary': 'Generalization of performance in learning algorithms is about how well they perform with unseen examples.', 'duration': 23.565, 'max_score': 2024.924, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs2024924.jpg'}], 'start': 1806.364, 'title': 'Bias and variance in learning', 'summary': 'Discusses the impact of varying number of features, parameters, and training examples on bias and variance in learning algorithms. increasing features and parameters decrease bias but increase variance, while the number of training examples has no impact on bias. it also emphasizes that bias is a limitation of the model and learning algorithm, not directly related to the number of training examples, while variance decreases as the model becomes more capable. the chapter touches on the generalization of performance in learning algorithms.', 'chapters': [{'end': 1931.245, 'start': 1806.364, 'title': 'Bias and variance in learning algorithms', 'summary': 'Discusses the impact of varying number of features, parameters, and training examples on bias and variance in learning algorithms, revealing that increasing features and parameters decrease bias but increase variance, while the number of training examples has no impact on bias.', 'duration': 124.881, 'highlights': ['When increasing the number of features, bias decreases while variance increases, as the algorithm becomes more sophisticated and exposed to more noise.', 'Similarly, increasing the number of parameters decreases bias as the learning capacity increases, but it also increases variance as it can model noise more efficiently.', 'The number of training examples has no impact on bias.']}, {'end': 2048.489, 'start': 1933.792, 'title': 'Bias and variance in learning', 'summary': 'Discusses the concepts of bias and variance in learning algorithms, emphasizing that bias is a limitation of the model and learning algorithm, not directly related to the number of training examples, while variance decreases as the model becomes more capable. it also touches on the generalization of performance in learning algorithms.', 'duration': 114.697, 'highlights': ['Bias is a limitation of the model and learning algorithm, not directly related to the number of training examples, while variance decreases as the model becomes more capable.', 'Generalization of performance in learning algorithms refers to how well the algorithm performs when presented with examples it has not seen before.', 'Increasing the number of training examples does not necessarily reduce bias if the model remains equally incapable, whereas variance decreases as the model becomes more sophisticated and capable.', 'Bias and variance of a learning algorithm will vary as different parameters of the learning algorithm are changed or modified.', 'The capacity of the model to learn important features in the data affects the presence of bias, while the capability of the model and learning algorithm affects the variance.']}], 'duration': 242.125, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs1806364.jpg', 'highlights': ['Increasing features decreases bias but increases variance.', 'Increasing parameters decreases bias but increases variance.', 'The number of training examples has no impact on bias.', 'Bias is a limitation of the model and learning algorithm.', 'Variance decreases as the model becomes more capable.', 'Generalization of performance refers to how well the algorithm performs with unseen examples.', 'Increasing training examples may not reduce bias if the model remains incapable.', 'Bias and variance vary as different parameters of the learning algorithm are changed.', 'The capacity of the model to learn important features affects bias.', 'The capability of the model and learning algorithm affects variance.']}, {'end': 2534.362, 'segs': [{'end': 2161.324, 'src': 'embed', 'start': 2111.381, 'weight': 0, 'content': [{'end': 2119.388, 'text': 'And it may mess things up because this particular set of symptoms which is really rare did not appear in the training example set.', 'start': 2111.381, 'duration': 8.007}, {'end': 2123.192, 'text': 'So, that is why it will fail in that particular scenario.', 'start': 2120.389, 'duration': 2.803}, {'end': 2134.702, 'text': 'So what is going to happen in the real world, when the system is really deployed, is that, you know, is what really matters,', 'start': 2123.532, 'duration': 11.17}, {'end': 2138.526, 'text': 'and that is why that is what we mean by generalization performance.', 'start': 2134.702, 'duration': 3.824}, {'end': 2143.19, 'text': 'So you have a certain training error.', 'start': 2138.946, 'duration': 4.244}, {'end': 2148.775, 'text': 'right, but, and that is say, we say, let us assume that that training error is really small.', 'start': 2143.19, 'duration': 5.585}, {'end': 2154.22, 'text': 'and so you are confident that, yeah, your system is performing really great on your training set.', 'start': 2148.775, 'duration': 5.445}, {'end': 2161.324, 'text': 'all right, but to get an idea about how good it would perform in the real world when it is really deployed,', 'start': 2154.22, 'duration': 7.104}], 'summary': 'Rare symptoms not in training set lead to failure in real-world scenario. generalization performance is crucial.', 'duration': 49.943, 'max_score': 2111.381, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs2111381.jpg'}, {'end': 2333.804, 'src': 'embed', 'start': 2311.678, 'weight': 3, 'content': [{'end': 2320.384, 'text': 'So, if you see that the validation error is quite high while the training error was low, you can say that the machine is not generalizing.', 'start': 2311.678, 'duration': 8.706}, {'end': 2321.485, 'text': 'my machine is not generalizing.', 'start': 2320.384, 'duration': 1.101}, {'end': 2324.107, 'text': 'So I need to check the learning algorithm.', 'start': 2321.505, 'duration': 2.602}, {'end': 2331.573, 'text': 'I must train it all over again, because when you are showing some data that it has not seen in the training examples, then it is messing things up.', 'start': 2324.107, 'duration': 7.466}, {'end': 2333.804, 'text': 'we go.', 'start': 2333.524, 'duration': 0.28}], 'summary': 'Validation error high, training error low: machine not generalizing. need to check learning algorithm and retrain.', 'duration': 22.126, 'max_score': 2311.678, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs2311678.jpg'}, {'end': 2400.381, 'src': 'embed', 'start': 2355.459, 'weight': 4, 'content': [{'end': 2366.243, 'text': 'we trained it on the training set and periodically evaluated on the validation set and finally we have a machine which has low training error as well as low validation error.', 'start': 2355.459, 'duration': 10.784}, {'end': 2371.826, 'text': 'So, I have a little bit of confidence that the machine has learnt the problem really well.', 'start': 2366.564, 'duration': 5.262}, {'end': 2382.431, 'text': 'having a small amount of training error, also it is generalizing quite well because it has a small amount of validation error as well.', 'start': 2374.827, 'duration': 7.604}, {'end': 2391.035, 'text': 'Now you must be careful to notice that this validation data validation set was not used for training the system,', 'start': 2382.611, 'duration': 8.424}, {'end': 2395.097, 'text': 'but it was used to evaluate the system while it was getting trained.', 'start': 2391.035, 'duration': 4.062}, {'end': 2400.381, 'text': 'And on the basis of the validation set results,', 'start': 2396.838, 'duration': 3.543}], 'summary': 'Machine has low training and validation error, indicating good learning and generalization.', 'duration': 44.922, 'max_score': 2355.459, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs2355459.jpg'}, {'end': 2506.973, 'src': 'embed', 'start': 2471.052, 'weight': 6, 'content': [{'end': 2476.095, 'text': 'or whether the test error or the validation error are measures of the true error.', 'start': 2471.052, 'duration': 5.043}, {'end': 2482.639, 'text': 'So the training error is not at all the measure of the true error, because it is the error that has been, you know,', 'start': 2476.135, 'duration': 6.504}, {'end': 2491.264, 'text': 'incurred by the system on a set of training examples which it has already seen during its training, and it was improving on that particular set,', 'start': 2482.639, 'duration': 8.625}, {'end': 2494.025, 'text': 'whereas the test data is completely unseen.', 'start': 2491.264, 'duration': 2.761}, {'end': 2506.973, 'text': 'So it is the measure of the true error when it will be deployed in the real world and the validation set it gives an estimate of the generalization performance while the system is being trained.', 'start': 2494.245, 'duration': 12.728}], 'summary': "Training error isn't a measure of true error; test data measures real-world deployment, and validation set estimates generalization performance during training.", 'duration': 35.921, 'max_score': 2471.052, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs2471052.jpg'}], 'start': 2048.53, 'title': 'Machine learning generalization', 'summary': 'Discusses the challenges of machine learning algorithm generalization performance in real-world scenarios, emphasizing potential failure due to rare or unseen data despite training on a large set of examples. it also explains controlling generalization performance by managing bias and variance, evaluating using training, validation, and test sets, and assessing true error for deployment.', 'chapters': [{'end': 2161.324, 'start': 2048.53, 'title': 'Generalization performance in machine learning', 'summary': "Discusses the challenge of a machine learning algorithm's generalization performance in real-world scenarios, highlighting the potential failure due to rare or unseen data, despite training on a large set of examples.", 'duration': 112.794, 'highlights': ['The challenge of generalization performance in machine learning is the potential failure when encountering rare or unseen data, despite being trained on a large set of examples. Discussion on the potential failure of a machine learning algorithm when encountering rare or unseen data in real-world scenarios.', "The uncertainty of a machine learning algorithm's performance in real-world deployment, particularly in scenarios involving rare types of cancer or symptoms not encountered during training. Highlighting the uncertainty of performance in real-world deployment, especially in scenarios involving rare types of cancer or symptoms.", 'The importance of generalization performance in machine learning, emphasizing the significance of real-world deployment scenarios over training set performance. Emphasizing the importance of generalization performance in real-world deployment scenarios over training set performance.']}, {'end': 2534.362, 'start': 2161.324, 'title': 'Generalization performance of learning algorithm', 'summary': 'Explains how to control the generalization performance of a learning algorithm by managing bias and variance, evaluating using training, validation, and test sets, and assessing the true error for deployment.', 'duration': 373.038, 'highlights': ['The validation error is quite high while the training error was low, indicating poor generalization, prompting the need to retrain the learning algorithm. Validation error being significantly higher than the training error indicates poor generalization, warranting retraining of the learning algorithm.', "The system is finally trained with low training and validation errors, instilling confidence in the machine's problem-solving aptitude. Achieving low training and validation errors instills confidence in the machine's ability to learn and generalize effectively.", "Using the test data, the system's performance is evaluated and the test error is reported, providing an estimate of the true error for deployment in the real world. Evaluation of the system's performance using test data provides an estimate of the true error for deployment in the real world.", 'The training error does not measure the true error as it pertains to the error on the training examples seen during training, while the test data represents the true error in real-world deployment. Training error reflects the error on examples seen during training, while test data represents the true error in real-world deployment.', "The validation set gives an estimate of the generalization performance during the system's training phase. The validation set provides an estimate of the system's generalization performance during training."]}], 'duration': 485.832, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/uFydF-g-AJs/pics/uFydF-g-AJs2048530.jpg', 'highlights': ['Discussion on the potential failure of a machine learning algorithm when encountering rare or unseen data in real-world scenarios.', 'Highlighting the uncertainty of performance in real-world deployment, especially in scenarios involving rare types of cancer or symptoms.', 'Emphasizing the importance of generalization performance in real-world deployment scenarios over training set performance.', 'Validation error being significantly higher than the training error indicates poor generalization, warranting retraining of the learning algorithm.', "Achieving low training and validation errors instills confidence in the machine's ability to learn and generalize effectively.", "Evaluation of the system's performance using test data provides an estimate of the true error for deployment in the real world.", 'Training error reflects the error on examples seen during training, while test data represents the true error in real-world deployment.', "The validation set provides an estimate of the system's generalization performance during training."]}], 'highlights': ['The tutorial covers machine learning fundamentals, supervised vs unsupervised learning, features and learning algorithms, bias in learning algorithms, bias and variance in learning, and machine learning generalization, addressing practical applications, examples, and performance analysis in machine learning.', 'Anirban Santara, a PhD candidate in machine learning, conducts weekly tutorial sessions covering supervised versus unsupervised learning, features, regression versus classification, bias-variance tradeoff, and generalization performance analysis.', 'Supervised and unsupervised learnings are two major broad categories of machine learning algorithms, focusing on the type of training examples presented.', 'The session emphasizes how bias and variance vary with different parameters of the learning algorithm and how to control and analyze the generalization performance of a learning algorithm.', 'In supervised learning, training involves input-target pairs to predict target values, such as predicting whether an image is happy or sad.', 'Unsupervised learning involves finding patterns or information from input data without target values, like identifying regular patterns or expressions from given faces.', 'In a supervised learning scenario, a large dataset of images with corresponding labels is used, where the input data appears as input label pairs.', 'The chapter discusses clustering images into similar groups as an unsupervised learning problem and training one classifier per cluster as a supervised learning problem.', 'The chapter explains the concept of categorical features and provides examples such as gender and age groups.', 'Categorical features can take just two values: presence or absence, such as yes or no. This highlights the definition of categorical features and provides examples of their possible values.', 'Continuous features, such as age, height, weight, and price, can theoretically take an infinite number of values. This details the nature of continuous features and provides examples of such features.', 'Understanding the distinction between categorical and continuous features is essential in data analysis, where categorical features have a fixed number of values and continuous features can take infinite values theoretically. This distinction is crucial as it impacts the analysis and modeling of the data, influencing the selection of appropriate techniques and algorithms for processing.', 'Supervised learning encompasses regression and classification, with regression focusing on output variables that are continuous, while classification deals with output variables that are discrete. This distinction is important as it determines the type of problem and the nature of the target variable, guiding the choice of predictive modeling techniques and evaluation metrics.', 'The bias of a learning algorithm limits its capacity to learn important input-output relationships.', 'Variance occurs when there are too many features or parameters in the learning algorithm, inhibiting the capture of important data aspects.', 'Bias in a learning algorithm is reflected in its tendency to learn towards noise rather than important output features.', 'Increasing features decreases bias but increases variance.', 'Increasing parameters decreases bias but increases variance.', 'The number of training examples has no impact on bias.', 'Bias is a limitation of the model and learning algorithm.', 'Variance decreases as the model becomes more capable.', 'Generalization of performance refers to how well the algorithm performs with unseen examples.', 'Increasing training examples may not reduce bias if the model remains incapable.', 'Bias and variance vary as different parameters of the learning algorithm are changed.', 'The capacity of the model to learn important features affects bias.', 'The capability of the model and learning algorithm affects variance.', 'Discussion on the potential failure of a machine learning algorithm when encountering rare or unseen data in real-world scenarios.', 'Highlighting the uncertainty of performance in real-world deployment, especially in scenarios involving rare types of cancer or symptoms.', 'Emphasizing the importance of generalization performance in real-world deployment scenarios over training set performance.', 'Validation error being significantly higher than the training error indicates poor generalization, warranting retraining of the learning algorithm.', "Achieving low training and validation errors instills confidence in the machine's ability to learn and generalize effectively.", "Evaluation of the system's performance using test data provides an estimate of the true error for deployment in the real world.", 'Training error reflects the error on examples seen during training, while test data represents the true error in real-world deployment.', "The validation set provides an estimate of the system's generalization performance during training."]}