title
Log Normal Distribution in Statistics

description
Here is the detailed discussion about the Log Normal Distribution. We will also discuss the basic difference between the Log Normal Distribution and Gaussian distribution. We will see various examples of Log Normal and Gaussian distribution. For more videos on ML or deep learning please check the below url NLP playlist: https://www.youtube.com/playlist… Deep Learning : https://goo.gl/iwek57 Statistics in ML :https://goo.gl/x7mkUH Feature Engineering:https://goo.gl/6wiaGt Data Preprocessing Techniques: https://goo.gl/YfC9Kc Machine learning: https://goo.gl/XhHdCd

detail
{'title': 'Log Normal Distribution in Statistics', 'heatmap': [{'end': 788.683, 'start': 663.502, 'weight': 0.901}], 'summary': "Covers gaussian distribution, empirical formula with percentages of distribution within one, two, and three standard deviations, log normal distribution's properties, real-world relevance, and distribution patterns in income and product comments, emphasizing the importance of understanding distribution for data scaling.", 'chapters': [{'end': 42.143, 'segs': [{'end': 47.184, 'src': 'embed', 'start': 22.017, 'weight': 0, 'content': [{'end': 29.399, 'text': 'empirical formula was something related to, Within one standard deviation, how much percentage of the total distribution falls.', 'start': 22.017, 'duration': 7.382}, {'end': 30.179, 'text': 'that is around 65 percent.', 'start': 29.399, 'duration': 0.78}, {'end': 33.541, 'text': 'sorry, this is 68%.', 'start': 31.36, 'duration': 2.181}, {'end': 42.143, 'text': 'in my second standard deviation it was around 95% and my third standard deviation it was around 99.7% of the whole distribution.', 'start': 33.541, 'duration': 8.602}, {'end': 47.184, 'text': "and we also saw that usually the Gaussian distribution usually follows this guy's bell curve.", 'start': 42.143, 'duration': 5.041}], 'summary': 'Empirical formula: 68% within 1 std dev, 95% within 2 std devs, 99.7% within 3 std devs.', 'duration': 25.167, 'max_score': 22.017, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ422017.jpg'}], 'start': 0.973, 'title': 'Gaussian distribution & empirical formula', 'summary': 'Covers the concept of gaussian distribution and the empirical formula, describing the percentages of distribution within one, two, and three standard deviations as approximately 68%, 95%, and 99.7%.', 'chapters': [{'end': 42.143, 'start': 0.973, 'title': 'Statistics: gaussian distribution & empirical formula', 'summary': 'Covers the concept of gaussian distribution and the empirical formula, where within one, two, and three standard deviations, approximately 68%, 95%, and 99.7% of the distribution falls, respectively.', 'duration': 41.17, 'highlights': ['The empirical formula states that within one standard deviation, approximately 68% of the total distribution falls.', 'Within two standard deviations, around 95% of the distribution falls.', 'Approximately 99.7% of the total distribution falls within three standard deviations.']}], 'duration': 41.17, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4973.jpg', 'highlights': ['Approximately 99.7% of the total distribution falls within three standard deviations.', 'Within two standard deviations, around 95% of the distribution falls.', 'The empirical formula states that within one standard deviation, approximately 68% of the total distribution falls.']}, {'end': 371.516, 'segs': [{'end': 260.752, 'src': 'embed', 'start': 229.515, 'weight': 1, 'content': [{'end': 232.237, 'text': 'okay, so these are various kind of distributions.', 'start': 229.515, 'duration': 2.722}, {'end': 238.524, 'text': 'usually, usually, data that is actually present follows Gaussian distribution.', 'start': 232.237, 'duration': 6.287}, {'end': 240.045, 'text': 'most of our data follows distribution.', 'start': 238.524, 'duration': 1.521}, {'end': 244.546, 'text': 'There is a huge chunk of data that also follows log normal distribution.', 'start': 240.065, 'duration': 4.481}, {'end': 253.029, 'text': 'Now, if I take an example of height, height of the people in this world, I can say that this follows a Gaussian distribution,', 'start': 244.986, 'duration': 8.043}, {'end': 260.752, 'text': 'because this is the information, because this type of problems or use cases that have come in machine learning.', 'start': 253.029, 'duration': 7.723}], 'summary': 'Data follows gaussian distribution, with some also following log normal distribution, e.g. heights in the world.', 'duration': 31.237, 'max_score': 229.515, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4229515.jpg'}, {'end': 376.518, 'src': 'embed', 'start': 346.384, 'weight': 0, 'content': [{'end': 349.265, 'text': 'just you try to find out it usually follows this belka.', 'start': 346.384, 'duration': 2.881}, {'end': 351.066, 'text': 'one good example is the irish data set.', 'start': 349.265, 'duration': 1.801}, {'end': 359.55, 'text': 'if you, if you know about irish data set, in irish data set there are parameters like petal length, features like petal length.', 'start': 351.886, 'duration': 7.664}, {'end': 365.573, 'text': 'if you try to plot petal length in this form, usually this kind of gaussian distribution is formed.', 'start': 359.55, 'duration': 6.023}, {'end': 371.516, 'text': 'okay, and this kind of distribution usually forms in such examples.', 'start': 365.573, 'duration': 5.943}, {'end': 376.518, 'text': 'let us see one example is income of the people, income of the people.', 'start': 371.516, 'duration': 5.002}], 'summary': 'In the irish data set, petal length exhibits a gaussian distribution, and a similar pattern emerges in income distribution.', 'duration': 30.134, 'max_score': 346.384, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4346384.jpg'}], 'start': 42.143, 'title': 'Gaussian & log normal distribution', 'summary': 'Covers the properties of gaussian distribution, including its bell curve, mean, and symmetrical nature, with 50% of data on each side. additionally, it explains log normal distribution, its characteristics, and real-world relevance, with examples of height and petal length and their impact on machine learning use cases.', 'chapters': [{'end': 91.722, 'start': 42.143, 'title': 'Gaussian & log normal distribution', 'summary': 'Explains the properties of a gaussian distribution, showcasing its bell curve, mean, and symmetrical nature, with 50% of data on each side, followed by an introduction to log normal distribution.', 'duration': 49.579, 'highlights': ['The bell curve of Gaussian distribution is symmetrical with the mean at the middle point, representing 50% of the data on each side.', 'Introduction to log normal distribution is the next topic to be discussed.']}, {'end': 371.516, 'start': 92.123, 'title': 'Log normal distribution', 'summary': 'Explains log normal distribution, its characteristics, and examples, emphasizing the relevance of different distributions in real-world data analysis, such as gaussian distribution and log normal distribution, with examples of height and petal length, and their impact on machine learning use cases.', 'duration': 279.393, 'highlights': ['Log normal distribution is explained as a random variable belonging to a log normal distribution if log of x belongs to a Gaussian distribution with some value of mean and sigma. The concept of log normal distribution is defined as a random variable belonging to a log normal distribution if log of x is normally distributed with a Gaussian distribution having specific values for mean and sigma.', 'Real-world data, like the height of people, generally follows a Gaussian distribution, while examples like the petal length in the Irish data set exhibit Gaussian distribution. Real-world data, such as the height of people and petal length in the Irish data set, typically follows a Gaussian distribution, demonstrating the relevance of different distributions in real-world data analysis.', 'The chapter emphasizes the relevance of different distributions in real-world data analysis, such as Gaussian distribution and log normal distribution, with examples of height and petal length, and their impact on machine learning use cases. The chapter underscores the significance of various distributions in real-world data analysis, including Gaussian distribution and log normal distribution, and their implications for machine learning applications, with specific examples like height and petal length.']}], 'duration': 329.373, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ442143.jpg', 'highlights': ['Real-world data, such as the height of people and petal length in the Irish data set, typically follows a Gaussian distribution, demonstrating the relevance of different distributions in real-world data analysis.', 'The chapter emphasizes the relevance of different distributions in real-world data analysis, such as Gaussian distribution and log normal distribution, with examples of height and petal length, and their impact on machine learning use cases.']}, {'end': 947.411, 'segs': [{'end': 401.829, 'src': 'embed', 'start': 371.516, 'weight': 0, 'content': [{'end': 376.518, 'text': 'let us see one example is income of the people, income of the people.', 'start': 371.516, 'duration': 5.002}, {'end': 385.42, 'text': "Now, when I'm saying income of the people, now you see that in the left-hand side you can see that income of the people, richer people.", 'start': 377.455, 'duration': 7.965}, {'end': 390.342, 'text': 'there are very less number of richer people, if I just consider this axis as my income.', 'start': 385.42, 'duration': 4.922}, {'end': 395.926, 'text': 'As the income increases, there are less number of people who are earning that much amount of money.', 'start': 391.763, 'duration': 4.163}, {'end': 401.829, 'text': 'Whereas in this mean case, there are many people who will be earning this many amount of money.', 'start': 396.646, 'duration': 5.183}], 'summary': 'Income distribution shows fewer richer people and more people earning a mean amount.', 'duration': 30.313, 'max_score': 371.516, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4371516.jpg'}, {'end': 544.157, 'src': 'embed', 'start': 517.263, 'weight': 1, 'content': [{'end': 524.446, 'text': 'similarly, there will be very, very, very smaller comment comment length as we go ahead with different, different, you know, different,', 'start': 517.263, 'duration': 7.183}, {'end': 529.109, 'text': 'different comments that we see, larger description of comments will be very, very less.', 'start': 524.446, 'duration': 4.663}, {'end': 533.611, 'text': 'larger description of comments will be very, very nice.', 'start': 529.109, 'duration': 4.502}, {'end': 536.094, 'text': 'larger description of comments will be very, very so.', 'start': 533.611, 'duration': 2.483}, {'end': 544.157, 'text': 'one more example is very, very nicely like, and most of the most of the companies, like amazon, right and when, whenever we are working, uh,', 'start': 536.094, 'duration': 8.063}], 'summary': 'Comments will be very small, less detailed, and mostly positive.', 'duration': 26.894, 'max_score': 517.263, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4517263.jpg'}, {'end': 788.683, 'src': 'heatmap', 'start': 663.502, 'weight': 0.901, 'content': [{'end': 666.024, 'text': 'Okay Suppose it follows a Gaussian distribution.', 'start': 663.502, 'duration': 2.522}, {'end': 668.853, 'text': 'suppose it follows the Gaussian distribution.', 'start': 666.892, 'duration': 1.961}, {'end': 675.377, 'text': 'what I can do is that I can convert this Gaussian distribution to something called a standard normal distribution,', 'start': 668.853, 'duration': 6.524}, {'end': 678.558, 'text': 'where my mean will be 0 and standard deviation will be 1.', 'start': 675.377, 'duration': 3.181}, {'end': 683.301, 'text': 'right then, when I convert this, this will be standard scalar.', 'start': 678.558, 'duration': 4.743}, {'end': 684.642, 'text': 'we are scaling down.', 'start': 683.301, 'duration': 1.341}, {'end': 686.262, 'text': 'we are scaling down.', 'start': 684.642, 'duration': 1.62}, {'end': 693.231, 'text': 'we are scaling down to standard normal deviation, with some mean is equal to zero and standard deviation is equal to one.', 'start': 686.262, 'duration': 6.969}, {'end': 695.473, 'text': 'so the scale will be same for r d.', 'start': 693.231, 'duration': 2.242}, {'end': 699.316, 'text': "now, suppose, if i am having marketing, i'll just clear this.", 'start': 695.473, 'duration': 3.843}, {'end': 707.183, 'text': "suppose, if i'm having marketing and this marketing values follow a log normal distribution.", 'start': 699.316, 'duration': 7.867}, {'end': 711.087, 'text': 'now, if i want to convert this into standard normal distribution, how can i do it?', 'start': 707.183, 'duration': 3.904}, {'end': 714.71, 'text': 'i will just directly find the log of these numbers.', 'start': 711.087, 'duration': 3.623}, {'end': 722.091, 'text': "i'll find the log of all these numbers right, Because when I'm finding the log of these numbers, this becomes log normal distribution.", 'start': 714.71, 'duration': 7.381}, {'end': 732.379, 'text': 'And then I know that if I know that this is log normal distributed, once I find out the log of each and every values in this marketing,', 'start': 722.111, 'duration': 10.268}, {'end': 734.921, 'text': 'I know that this will follow a Gaussian distribution.', 'start': 732.379, 'duration': 2.542}, {'end': 741.487, 'text': 'Because that is the property of log normal distribution, right? With some mean value and sigma value.', 'start': 735.742, 'duration': 5.745}, {'end': 747.564, 'text': 'and then, if i have this gaussian distribution, i can definitely convert this into standard normal distribution.', 'start': 742.06, 'duration': 5.504}, {'end': 753.308, 'text': 'i just have to apply a formula that is, x of i minus mu, divided by standard deviation, if i convert this.', 'start': 747.564, 'duration': 5.744}, {'end': 756.65, 'text': 'i hope you have discussed about standard normal distribution of this class.', 'start': 753.308, 'duration': 3.342}, {'end': 761.313, 'text': 'but just understand, if i want to convert this gaussian distribution to standard normal distribution,', 'start': 756.65, 'duration': 4.663}, {'end': 764.876, 'text': "i'll just apply this formula x of i minus mu is equal to standard deviation.", 'start': 761.313, 'duration': 3.563}, {'end': 770.207, 'text': "and finally i'll be also able to, you know, apply standard scalar, the standard scale.", 'start': 764.876, 'duration': 5.331}, {'end': 774.771, 'text': 'when i apply this will this marketing value will also become the same?', 'start': 770.207, 'duration': 4.564}, {'end': 783.338, 'text': 'it will be actually in the same scale as that of the rnd spend now, when i give my model.', 'start': 774.771, 'duration': 8.567}, {'end': 788.683, 'text': 'if, if i give my model this kind of data which are exactly in the same scale, i will be getting higher accuracy.', 'start': 783.338, 'duration': 5.345}], 'summary': 'Converting different distributions to standard normal improves accuracy.', 'duration': 125.181, 'max_score': 663.502, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4663502.jpg'}, {'end': 878.49, 'src': 'embed', 'start': 851.506, 'weight': 2, 'content': [{'end': 855.689, 'text': 'Right? So in order to do that, what I did, I found out the log of each and every value.', 'start': 851.506, 'duration': 4.183}, {'end': 858.252, 'text': 'Log of each and every value.', 'start': 857.031, 'duration': 1.221}, {'end': 859.793, 'text': 'Because I know the log.', 'start': 858.272, 'duration': 1.521}, {'end': 868.54, 'text': 'since it is log normal distribution, the log of this value will follow a Gaussian distribution with some value of mean and sigma.', 'start': 859.793, 'duration': 8.747}, {'end': 871.147, 'text': 'mean and sigma can be calculated.', 'start': 869.386, 'duration': 1.761}, {'end': 878.49, 'text': 'if, once i get all my log values of these values, once i get this, i will be knowing that this will follow gaussian distribution,', 'start': 871.147, 'duration': 7.343}], 'summary': 'By taking the log of each value, we can analyze a log normal distribution to determine its mean and sigma.', 'duration': 26.984, 'max_score': 851.506, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4851506.jpg'}, {'end': 939.147, 'src': 'embed', 'start': 893.894, 'weight': 3, 'content': [{'end': 907.104, 'text': "i'll be able to get my standard normal distribution and all the values will be scaled down to the same scale as that of an r d and finally i'll be able to give it to my model and then my model accuracy will be increased,", 'start': 893.894, 'duration': 13.21}, {'end': 910.146, 'text': 'will be increased.', 'start': 907.104, 'duration': 3.042}, {'end': 912.448, 'text': 'this is the technique behind all these things.', 'start': 910.146, 'duration': 2.302}, {'end': 914.389, 'text': 'it is a wonderful technique, you know.', 'start': 912.448, 'duration': 1.941}, {'end': 921.095, 'text': "uh, why many of them don't don't know why we do standard scaling and when we should do standard scaling, when we should do normal log normal?", 'start': 914.389, 'duration': 6.706}, {'end': 925.06, 'text': 'uh, i mean log normalization.', 'start': 922.379, 'duration': 2.681}, {'end': 935.045, 'text': 'okay, and this whole process, this whole process is basically called that log normalization, log normalization.', 'start': 925.06, 'duration': 9.985}, {'end': 936.426, 'text': 'just go through these guys again.', 'start': 935.045, 'duration': 1.381}, {'end': 939.147, 'text': 'this is the wonderful concepts altogether.', 'start': 936.426, 'duration': 2.721}], 'summary': 'Standard scaling improves model accuracy. log normalization is a key technique.', 'duration': 45.253, 'max_score': 893.894, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4893894.jpg'}], 'start': 371.516, 'title': 'Income and product comment distribution', 'summary': 'Highlights distribution patterns in income and product comments, showing that as income increases, the number of people earning that amount decreases, and explains the importance of understanding distribution such as log normal and gaussian distribution for data scaling.', 'chapters': [{'end': 536.094, 'start': 371.516, 'title': 'Patterns in income and product comments', 'summary': 'Highlights the distribution patterns in income and product comments, showing that as income increases, the number of people earning that amount decreases, and that comments typically follow a pattern with very few longer descriptions and more medium-sized descriptions.', 'duration': 164.578, 'highlights': ['The distribution of income shows that as income increases, there are less number of people who are earning that much amount of money. The transcript discusses the distribution of income, indicating that as income increases, the number of people earning that amount decreases.', 'Product comments tend to follow a pattern with very few longer descriptions and more medium-sized descriptions. The transcript explains that product comments typically follow a pattern with very few longer descriptions and more medium-sized descriptions, indicating a specific distribution pattern.']}, {'end': 947.411, 'start': 536.094, 'title': 'Understanding distribution for data scaling', 'summary': 'Explains the importance of understanding distribution such as log normal and gaussian distribution for data scaling, which helps in converting data to standard normal distribution to improve model accuracy and performance.', 'duration': 411.317, 'highlights': ['Understanding the importance of distribution such as log normal and Gaussian distribution for data scaling The transcript emphasizes the significance of understanding distributions like log normal and Gaussian distribution for scaling data to improve model accuracy and performance.', 'Converting data to standard normal distribution for improved model accuracy The speaker explains the process of converting data to standard normal distribution using techniques such as finding the log of values and applying formulas to scale down values, leading to an increase in model accuracy.', 'The concept of log normalization and its impact on data scaling The transcript introduces the concept of log normalization and highlights its importance in the process of scaling data to improve model accuracy and performance.']}], 'duration': 575.895, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/sPzPEeJ4OQ4/pics/sPzPEeJ4OQ4371516.jpg', 'highlights': ['As income increases, the number of people earning that amount decreases.', 'Product comments typically follow a pattern with very few longer descriptions and more medium-sized descriptions.', 'Understanding distributions like log normal and Gaussian distribution is significant for scaling data.', 'Converting data to standard normal distribution improves model accuracy.', 'Log normalization is important in scaling data to improve model accuracy.']}], 'highlights': ['Approximately 99.7% of the total distribution falls within three standard deviations.', 'Within two standard deviations, around 95% of the distribution falls.', 'The empirical formula states that within one standard deviation, approximately 68% of the total distribution falls.', 'Real-world data, such as the height of people and petal length in the Irish data set, typically follows a Gaussian distribution, demonstrating the relevance of different distributions in real-world data analysis.', 'The chapter emphasizes the relevance of different distributions in real-world data analysis, such as Gaussian distribution and log normal distribution, with examples of height and petal length, and their impact on machine learning use cases.', 'As income increases, the number of people earning that amount decreases.', 'Product comments typically follow a pattern with very few longer descriptions and more medium-sized descriptions.', 'Understanding distributions like log normal and Gaussian distribution is significant for scaling data.', 'Converting data to standard normal distribution improves model accuracy.', 'Log normalization is important in scaling data to improve model accuracy.']}