title
What is Recurrent Neural Network (RNN)? Deep Learning Tutorial 33 (Tensorflow, Keras & Python)

description
RNN or Recurrent Neural Network are also known as sequence models that are used mainly in the field of natural language processing as well as some other areas such as speech to text translation, video activity monitoring, etc. In this video we will understand the intuition behind RNN and see how RNN's work. Deep learning playlist: https://www.youtube.com/playlist?list=PLeo1K3hjS3uu7CxAacxVndI4bE_o3BDtO Machine learning playlist: https://www.youtube.com/playlist?list=PLeo1K3hjS3uvCeTYTeyfe0-rN5r8zn9rw   #recurrentneuralnetwork #rnn #rnndeeplearning #whatisrnn #deeplearningtutorial #rnnneuralnetwork #rnntutorial Do you want to learn technology from me? Check https://codebasics.io/?utm_source=description&utm_medium=yt&utm_campaign=description&utm_id=description for my affordable video courses. 🌎 My Website For Video Courses: https://codebasics.io/?utm_source=description&utm_medium=yt&utm_campaign=description&utm_id=description Need help building software or data analytics and AI solutions? My company https://www.atliq.com/ can help. Click on the Contact button on that website. #️⃣ Social Media #️⃣ 🔗 Discord: https://discord.gg/r42Kbuk 📸 Dhaval's Personal Instagram: https://www.instagram.com/dhavalsays/ 📸 Instagram: https://www.instagram.com/codebasicshub/ 🔊 Facebook: https://www.facebook.com/codebasicshub 📱 Twitter: https://twitter.com/codebasicshub 📝 Linkedin: https://www.linkedin.com/company/codebasics/ ❗❗ DISCLAIMER: All opinions expressed in this video are of my own and not that of my employers'.

detail
{'title': 'What is Recurrent Neural Network (RNN)? Deep Learning Tutorial 33 (Tensorflow, Keras & Python)', 'heatmap': [{'end': 183.097, 'start': 172.052, 'weight': 0.705}, {'end': 222.935, 'start': 206.365, 'weight': 0.745}, {'end': 539.522, 'start': 458.068, 'weight': 0.868}, {'end': 798.413, 'start': 718.485, 'weight': 0.725}, {'value': 0.8030879226345671, 'end_time': 798.413, 'start_time': 735.217}, {'end': 840.731, 'start': 805.647, 'weight': 0.775}], 'summary': "Covers recurrent neural networks (rnn) in nlp, including applications like gmail's auto-complete using rnn, challenges in neural network translation, and the need for specialized networks like rnn for accurate language translation and sequence problems.", 'chapters': [{'end': 75.231, 'segs': [{'end': 47.359, 'src': 'embed', 'start': 0.169, 'weight': 0, 'content': [{'end': 0.849, 'text': 'so far.', 'start': 0.169, 'duration': 0.68}, {'end': 6.692, 'text': 'in our deep learning tutorial series we looked at artificial neural network and convolutional neural network,', 'start': 0.849, 'duration': 5.843}, {'end': 8.873, 'text': 'which is mainly used for image processing.', 'start': 6.692, 'duration': 2.181}, {'end': 15.536, 'text': 'in this video we will talk about recurrent neural network, which is used mainly for natural language processing tasks.', 'start': 8.873, 'duration': 6.663}, {'end': 20.258, 'text': 'so if you think about deep learning overall, CNNs are mainly for images.', 'start': 15.536, 'duration': 4.722}, {'end': 22.479, 'text': 'RNNs are mainly for NLP.', 'start': 20.258, 'duration': 2.221}, {'end': 24.08, 'text': 'there are other use cases as well.', 'start': 22.479, 'duration': 1.601}, {'end': 34.208, 'text': "So we'll understand how recurrent neural network works and we'll look at different applications of RNN in the field of NLP as well as some other domains.", 'start': 24.58, 'duration': 9.628}, {'end': 40.133, 'text': 'We will be looking at some real life use cases where sequence models are useful.', 'start': 35.329, 'duration': 4.804}, {'end': 43.796, 'text': 'You must have used Google Mail, Gmail.', 'start': 40.754, 'duration': 3.042}, {'end': 47.359, 'text': 'Here when you type in a sentence it will auto complete it.', 'start': 44.177, 'duration': 3.182}], 'summary': 'Introducing recurrent neural network for nlp with real-life applications and use cases in gmail.', 'duration': 47.19, 'max_score': 0.169, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I169.jpg'}], 'start': 0.169, 'title': 'Recurrent neural network for nlp', 'summary': "Explores recurrent neural networks (rnn) and its applications in natural language processing (nlp), highlighting a real-life example of gmail's auto-complete feature using rnn for efficient user time-saving.", 'chapters': [{'end': 75.231, 'start': 0.169, 'title': 'Recurrent neural network for nlp', 'summary': "Delves into recurrent neural networks (rnn) and its applications in natural language processing (nlp), with a real-life example of gmail's auto-complete feature using rnn to save user time.", 'duration': 75.062, 'highlights': ['RNNs are mainly used for NLP, while CNNs are primarily utilized for image processing. Highlights the main purpose of RNNs and CNNs, providing a clear differentiation between their applications.', "Google's Gmail utilizes RNN to provide an auto-complete feature, saving time for users by predicting and completing their sentences. Illustrates a real-life application of RNN in Gmail, showcasing its practical usage and user benefits.", 'The video will cover how RNNs work and explore various applications of RNN in NLP and other domains, including sequence models. Outlines the upcoming content of the video, setting expectations for the audience regarding the coverage of RNN functionality and applications.']}], 'duration': 75.062, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I169.jpg', 'highlights': ["Google's Gmail utilizes RNN to provide an auto-complete feature, saving time for users by predicting and completing their sentences. Illustrates a real-life application of RNN in Gmail, showcasing its practical usage and user benefits.", 'The video will cover how RNNs work and explore various applications of RNN in NLP and other domains, including sequence models. Outlines the upcoming content of the video, setting expectations for the audience regarding the coverage of RNN functionality and applications.', 'RNNs are mainly used for NLP, while CNNs are primarily utilized for image processing. Highlights the main purpose of RNNs and CNNs, providing a clear differentiation between their applications.']}, {'end': 269.531, 'segs': [{'end': 105.278, 'src': 'embed', 'start': 76.051, 'weight': 0, 'content': [{'end': 77.312, 'text': 'Another use case is.', 'start': 76.051, 'duration': 1.261}, {'end': 86.714, 'text': 'translation. you must have used google translate, where you can translate sentence from one to another language easily.', 'start': 77.972, 'duration': 8.742}, {'end': 95.176, 'text': 'third use case is named entity recognition, where in the x, you know you give neural network a statement and in the y,', 'start': 86.714, 'duration': 8.462}, {'end': 99.837, 'text': 'neural network will tell you the person, name, the company and time.', 'start': 95.176, 'duration': 4.661}, {'end': 105.278, 'text': "for rudolph smith must be a millionaire with tesla's prices skyrocketing.", 'start': 99.837, 'duration': 5.441}], 'summary': 'Google translate enables easy language translation; named entity recognition identifies person, company, and time.', 'duration': 29.227, 'max_score': 76.051, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I76051.jpg'}, {'end': 183.097, 'src': 'heatmap', 'start': 126.674, 'weight': 1, 'content': [{'end': 134.089, 'text': "Now you would think, why can't we use a simple neural network to solve this problem?", 'start': 126.674, 'duration': 7.415}, {'end': 135.79, 'text': 'see all these problems.', 'start': 134.089, 'duration': 1.701}, {'end': 143.417, 'text': 'they are called sequence modeling problem, because the sequence is important when it comes to human language.', 'start': 135.79, 'duration': 7.627}, {'end': 144.558, 'text': 'sequence is very important.', 'start': 143.417, 'duration': 1.141}, {'end': 151.163, 'text': "for example, when you say how are you versus you are, how doesn't make sense, right.", 'start': 144.558, 'duration': 6.605}, {'end': 157.309, 'text': "so the sequence is important here and you would think, Why don't we use simple neural network for that?", 'start': 151.163, 'duration': 6.146}, {'end': 158.29, 'text': "Well, let's try it.", 'start': 157.369, 'duration': 0.921}, {'end': 164.116, 'text': 'So for language translation, how about we build this kind of neural network?', 'start': 159.451, 'duration': 4.665}, {'end': 169.661, 'text': 'We know where input is the English statement and the output could be Hindi statement.', 'start': 164.816, 'duration': 4.845}, {'end': 176.694, 'text': 'Once I build this network, what if my sentence size changes?', 'start': 172.052, 'duration': 4.642}, {'end': 183.097, 'text': 'So I might be inputting different sentence size and with a fixed neural network, architecture is not going to work,', 'start': 177.014, 'duration': 6.083}], 'summary': 'Sequence modeling in language translation challenges simple neural networks due to the importance of sequence in human language.', 'duration': 50.02, 'max_score': 126.674, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I126674.jpg'}, {'end': 236.369, 'src': 'heatmap', 'start': 206.365, 'weight': 0.745, 'content': [{'end': 212.267, 'text': 'so it will occupy four neuron, remaining 96, i will just say zero.', 'start': 206.365, 'duration': 5.902}, {'end': 214.528, 'text': 'or you know, blank statement.', 'start': 212.267, 'duration': 2.261}, {'end': 216.869, 'text': "that might work, but still it's not ideal.", 'start': 214.528, 'duration': 2.341}, {'end': 219.872, 'text': 'The second issue is too much computation.', 'start': 217.97, 'duration': 1.902}, {'end': 222.935, 'text': 'You all know neural networks work on numbers.', 'start': 220.853, 'duration': 2.082}, {'end': 224.136, 'text': "They don't work on string.", 'start': 223.055, 'duration': 1.081}, {'end': 228.621, 'text': 'So you have to convert your word into a vector.', 'start': 224.917, 'duration': 3.704}, {'end': 236.369, 'text': "So one of the ways of converting that into a vector is, let's say there are 25, 000 words in your vocabulary.", 'start': 229.362, 'duration': 7.007}], 'summary': 'Neural network may use 4 neurons, 25,000 words need vector conversion.', 'duration': 30.004, 'max_score': 206.365, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I206365.jpg'}, {'end': 256.118, 'src': 'embed', 'start': 223.055, 'weight': 2, 'content': [{'end': 224.136, 'text': "They don't work on string.", 'start': 223.055, 'duration': 1.081}, {'end': 228.621, 'text': 'So you have to convert your word into a vector.', 'start': 224.917, 'duration': 3.704}, {'end': 236.369, 'text': "So one of the ways of converting that into a vector is, let's say there are 25, 000 words in your vocabulary.", 'start': 229.362, 'duration': 7.007}, {'end': 245.153, 'text': "and you will do one hot encoding where you know how, let's say, is at 46, position r is, let's say, at second position.", 'start': 237.31, 'duration': 7.843}, {'end': 248.855, 'text': "u is, let's say, at 17 000 position.", 'start': 245.153, 'duration': 3.702}, {'end': 256.118, 'text': "so at that position you put one remaining position, you put zero and that's called one hot encoding.", 'start': 248.855, 'duration': 7.263}], 'summary': 'Words are converted into a 25,000-word vector using one hot encoding.', 'duration': 33.063, 'max_score': 223.055, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I223055.jpg'}], 'start': 76.051, 'title': 'Sequence models in language processing', 'summary': 'Explores the use cases of sequence models like translation, named entity recognition, and sentiment analysis. it also highlights the challenges of using simple neural networks and their application in language processing.', 'chapters': [{'end': 222.935, 'start': 76.051, 'title': 'Sequence models in language processing', 'summary': 'Explores the various use cases of sequence models, such as translation, named entity recognition, sentiment analysis, and the challenges of using simple neural networks for sequence modeling.', 'duration': 146.884, 'highlights': ['The use cases of sequence models include translation, named entity recognition, and sentiment analysis, demonstrating the effectiveness of sequence models in language processing.', "Sequence modeling problems are inherent in human language due to the importance of sequence, as seen in examples like 'how are you' versus 'you are', highlighting the significance of sequence models in language processing.", 'The challenges of using a simple neural network for language translation include the issue of fixed architecture not accommodating varying sentence sizes and the problem of deciding the size of neurons, illustrating the limitations of simple neural networks in sequence modeling.']}, {'end': 269.531, 'start': 223.055, 'title': 'Word to vector conversion', 'summary': 'Explains the process of converting words into vectors using one hot encoding, which involves converting a vocabulary of 25,000 words into binary vectors, leading to a significant increase in computation for input layer neurons.', 'duration': 46.476, 'highlights': ["Converting words into vectors using one hot encoding involves assigning a unique position to each word in a 25,000 word vocabulary, resulting in a binary vector with one at the word's position and zeros elsewhere.", 'One hot encoding for output also increases computation for each word, leading to a significant increase in the number of neurons required in the input layer.']}], 'duration': 193.48, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I76051.jpg', 'highlights': ['The use cases of sequence models include translation, named entity recognition, and sentiment analysis, demonstrating the effectiveness of sequence models in language processing.', 'Sequence modeling problems are inherent in human language due to the importance of sequence, highlighting the significance of sequence models in language processing.', "Converting words into vectors using one hot encoding involves assigning a unique position to each word in a 25,000 word vocabulary, resulting in a binary vector with one at the word's position and zeros elsewhere.", 'The challenges of using a simple neural network for language translation include the issue of fixed architecture not accommodating varying sentence sizes and the problem of deciding the size of neurons, illustrating the limitations of simple neural networks in sequence modeling.']}, {'end': 761.913, 'segs': [{'end': 353.451, 'src': 'embed', 'start': 329.312, 'weight': 2, 'content': [{'end': 336.82, 'text': "Here, the use of ANN or artificial neural network doesn't allow you to do that.", 'start': 329.312, 'duration': 7.508}, {'end': 343.368, 'text': 'Also, the most important part in all this discussion is the sequence.', 'start': 339.083, 'duration': 4.285}, {'end': 348.974, 'text': "See, when you have structured data, for example, you're trying to figure out if the transaction is fraud or not.", 'start': 343.928, 'duration': 5.046}, {'end': 353.451, 'text': "and let's say, your features are transaction amount,", 'start': 350.507, 'duration': 2.944}], 'summary': 'Ann not suitable for task, focus on structured data for fraud detection.', 'duration': 24.139, 'max_score': 329.312, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I329312.jpg'}, {'end': 437.144, 'src': 'embed', 'start': 404.829, 'weight': 1, 'content': [{'end': 411.912, 'text': 'Just to summarize, these are the three major problems with using ANN for sequence problems.', 'start': 404.829, 'duration': 7.083}, {'end': 418.355, 'text': "Let's once again talk about name entity recognition.", 'start': 414.253, 'duration': 4.102}, {'end': 422.218, 'text': "Let's say the world loves baby Yoda.", 'start': 419.537, 'duration': 2.681}, {'end': 424.559, 'text': 'I love my baby Grogu.', 'start': 422.798, 'duration': 1.761}, {'end': 432.002, 'text': 'I love Mandalorian series and we have got this nice baby Grogu at our home which actually talks with us.', 'start': 425.299, 'duration': 6.703}, {'end': 437.144, 'text': 'In this statement, the world and baby Yoda are person names.', 'start': 433.062, 'duration': 4.082}], 'summary': "Challenges of using ann for sequence problems; example of name entity recognition with 'baby yoda' and 'grogu'.", 'duration': 32.315, 'max_score': 404.829, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I404829.jpg'}, {'end': 539.522, 'src': 'heatmap', 'start': 458.068, 'weight': 0.868, 'content': [{'end': 465.11, 'text': 'so if the word is person name, you would mark it as one, and if it is not person name, you mark it as zero.', 'start': 458.068, 'duration': 7.042}, {'end': 469.69, 'text': "So let's see how RNN works here.", 'start': 466.926, 'duration': 2.764}, {'end': 471.992, 'text': 'RNN is also called recurrent neural network.', 'start': 470.13, 'duration': 1.862}, {'end': 475.096, 'text': 'So first of all, you have to convert double into some vector.', 'start': 472.473, 'duration': 2.623}, {'end': 477.399, 'text': "It doesn't matter how you convert it.", 'start': 475.116, 'duration': 2.283}, {'end': 480.824, 'text': 'You can take a vocabulary and use one hot encoding.', 'start': 477.439, 'duration': 3.385}, {'end': 483.367, 'text': 'And there are other ways of vectorizing a word.', 'start': 480.844, 'duration': 2.523}, {'end': 489.146, 'text': 'then you have a layer of neurons.', 'start': 484.622, 'duration': 4.524}, {'end': 490.967, 'text': 'so these are all individual neurons.', 'start': 489.146, 'duration': 1.821}, {'end': 494.61, 'text': "let's say, this is one layer, it's a hidden layer.", 'start': 490.967, 'duration': 3.643}, {'end': 497.332, 'text': 'you supply that and you get one output.', 'start': 494.61, 'duration': 2.722}, {'end': 504.238, 'text': 'okay, so each neuron, all you know, has a sigma function and activation function.', 'start': 497.332, 'duration': 6.906}, {'end': 509.762, 'text': 'so now, while processing the statement the world loves baby yoda, now i will process it word by word.', 'start': 504.238, 'duration': 5.524}, {'end': 512.341, 'text': 'So I supply double.', 'start': 510.981, 'duration': 1.36}, {'end': 515.322, 'text': 'get the output and then I go back again.', 'start': 512.341, 'duration': 2.981}, {'end': 527.186, 'text': 'Now I supply loves, convert it into vector, and the previous output which I got, which was y double, I now supply that as an input to this layer.', 'start': 516.203, 'duration': 10.983}, {'end': 533.457, 'text': 'So you see the input of the layer is not only the next word, but the previous output.', 'start': 528.393, 'duration': 5.064}, {'end': 539.522, 'text': 'Because the language makes sense, language needs to carry the context.', 'start': 534.478, 'duration': 5.044}], 'summary': 'Rnn processes words using vectors and previous output for context.', 'duration': 81.454, 'max_score': 458.068, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I458068.jpg'}, {'end': 504.238, 'src': 'embed', 'start': 470.13, 'weight': 0, 'content': [{'end': 471.992, 'text': 'RNN is also called recurrent neural network.', 'start': 470.13, 'duration': 1.862}, {'end': 475.096, 'text': 'So first of all, you have to convert double into some vector.', 'start': 472.473, 'duration': 2.623}, {'end': 477.399, 'text': "It doesn't matter how you convert it.", 'start': 475.116, 'duration': 2.283}, {'end': 480.824, 'text': 'You can take a vocabulary and use one hot encoding.', 'start': 477.439, 'duration': 3.385}, {'end': 483.367, 'text': 'And there are other ways of vectorizing a word.', 'start': 480.844, 'duration': 2.523}, {'end': 489.146, 'text': 'then you have a layer of neurons.', 'start': 484.622, 'duration': 4.524}, {'end': 490.967, 'text': 'so these are all individual neurons.', 'start': 489.146, 'duration': 1.821}, {'end': 494.61, 'text': "let's say, this is one layer, it's a hidden layer.", 'start': 490.967, 'duration': 3.643}, {'end': 497.332, 'text': 'you supply that and you get one output.', 'start': 494.61, 'duration': 2.722}, {'end': 504.238, 'text': 'okay, so each neuron, all you know, has a sigma function and activation function.', 'start': 497.332, 'duration': 6.906}], 'summary': 'Rnn uses a hidden layer of neurons with activation function for vectorizing words.', 'duration': 34.108, 'max_score': 470.13, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I470130.jpg'}], 'start': 271.357, 'title': 'Challenges in neural network language translation and using ann for sequence problems', 'summary': 'Discusses challenges in translating language for neural networks, covering parameter sharing, differentiation in translating similar english statements to a single hindi statement for training, and the limitations of using artificial neural networks for sequence problems such as name entity recognition (ner) and the architecture and training process of recurrent neural networks (rnn) for ner.', 'chapters': [{'end': 328.259, 'start': 271.357, 'title': 'Neural network language translation', 'summary': 'Discusses the challenges of translating language for neural networks, highlighting the issue of parameter sharing and differentiation in translating similar english statements to a single hindi statement for training, and how the neural network has to learn different sets of edges for variations in statement structure.', 'duration': 56.902, 'highlights': ["The neural network faces challenges in translating language, such as the need to adjust weights for different variations of similar statements, which can be illustrated by the example of translating 'on Sunday I ate golgappa' to 'I ate golgappa on Sunday' (Quantifiable data: example provided).", 'The differentiation in statement structure requires the neural network to learn different sets of edges, as highlighted by the need for the network to adjust weights for different variations of the same meaning (Quantifiable data: need for learning different sets of edges).', 'The issue of parameter sharing and differentiation in translating similar English statements to a single Hindi statement for training is highlighted, emphasizing the non-sharing of parameters and the need for different sets of edges to be learned (Quantifiable data: issue of parameter sharing and differentiation).']}, {'end': 761.913, 'start': 329.312, 'title': 'Challenges of using ann for sequence problems', 'summary': 'Discusses the challenges of using artificial neural networks (ann) for sequence problems, highlighting the importance of sequence in structured data analysis, the limitations of ann in handling sequence-based tasks like name entity recognition (ner), and the architecture and training process of recurrent neural networks (rnn) for ner.', 'duration': 432.601, 'highlights': ['The importance of sequence in structured data analysis is emphasized, demonstrating how altering the order of features in tasks like fraud detection does not affect the outcome, while in language translation, sequence alteration completely changes the meaning. When dealing with structured data, the sequence of input features, such as in fraud detection, does not affect the outcome, unlike in language translation where sequence alteration changes the meaning significantly.', 'The limitations of using ANN for sequence-based tasks like name entity recognition (NER) are outlined, showcasing the need for context and memory in language processing, which RNN architecture provides through its recurrent processing of words with a single hidden layer. ANN is unsuitable for sequence-based tasks like name entity recognition (NER) due to the need for context and memory in language processing, which is effectively provided by the recurrent processing architecture of RNN with a single hidden layer.', 'The architecture and training process of recurrent neural networks (RNN) for name entity recognition (NER) is explained, detailing how RNN processes words sequentially, carrying the context and memory from previous words to provide accurate NER output. The architecture and training process of RNN for NER involves sequential processing of words with a single hidden layer, enabling the network to carry context and memory from previous words, ultimately producing accurate NER output.']}], 'duration': 490.556, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I271357.jpg', 'highlights': ['The architecture and training process of recurrent neural networks (RNN) for name entity recognition (NER) is explained, detailing how RNN processes words sequentially, carrying the context and memory from previous words to provide accurate NER output.', 'The limitations of using ANN for sequence-based tasks like name entity recognition (NER) are outlined, showcasing the need for context and memory in language processing, which RNN architecture provides through its recurrent processing of words with a single hidden layer.', 'The importance of sequence in structured data analysis is emphasized, demonstrating how altering the order of features in tasks like fraud detection does not affect the outcome, while in language translation, sequence alteration completely changes the meaning.']}, {'end': 958.658, 'segs': [{'end': 792.131, 'src': 'embed', 'start': 762.213, 'weight': 0, 'content': [{'end': 765.515, 'text': 'Then I calculate y hat, which is predicted y.', 'start': 762.213, 'duration': 3.302}, {'end': 767.175, 'text': 'Then I compare with the real y.', 'start': 765.515, 'duration': 1.66}, {'end': 769.016, 'text': 'So real y here is 1 0 1 1.', 'start': 767.175, 'duration': 1.841}, {'end': 772.238, 'text': 'So I compare that with here.', 'start': 769.016, 'duration': 3.222}, {'end': 773.798, 'text': 'So 1 0 1 1.', 'start': 772.458, 'duration': 1.34}, {'end': 775.039, 'text': 'I compare that with y hat.', 'start': 773.798, 'duration': 1.241}, {'end': 777.26, 'text': 'And I find out the loss.', 'start': 775.939, 'duration': 1.321}, {'end': 782.368, 'text': 'Okay, and then I sum the loss.', 'start': 778.587, 'duration': 3.781}, {'end': 784.249, 'text': 'So that will be my total loss.', 'start': 782.848, 'duration': 1.401}, {'end': 792.131, 'text': 'You all know about gradient descent, right? So we compute the loss, then we back propagate the loss, and we adjust the weights.', 'start': 785.189, 'duration': 6.942}], 'summary': 'Calculating and comparing predicted vs real y, computing total loss, and adjusting weights through gradient descent.', 'duration': 29.918, 'max_score': 762.213, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I762213.jpg'}, {'end': 850.435, 'src': 'heatmap', 'start': 805.647, 'weight': 2, 'content': [{'end': 809.271, 'text': 'And then I do gradient descent to reduce the loss.', 'start': 805.647, 'duration': 3.624}, {'end': 812.494, 'text': 'So I keep on doing this for all my training samples.', 'start': 809.491, 'duration': 3.003}, {'end': 814.036, 'text': "Let's say I have 100 training samples.", 'start': 812.514, 'duration': 1.522}, {'end': 820.303, 'text': 'Passing all 100 training samples through this network will be one epoch.', 'start': 815.998, 'duration': 4.305}, {'end': 827.565, 'text': "We might do, let's say 20 epochs, and at the end of the 20 epochs, my loss might become very minimum.", 'start': 821.883, 'duration': 5.682}, {'end': 830.427, 'text': 'At that point, we can say my neural network is strained.', 'start': 827.826, 'duration': 2.601}, {'end': 834.848, 'text': "Let's take a look at language translation.", 'start': 831.807, 'duration': 3.041}, {'end': 840.731, 'text': 'So in language translation, what happens is you supply first word to your network.', 'start': 834.948, 'duration': 5.783}, {'end': 843.832, 'text': 'Then you get the output.', 'start': 841.931, 'duration': 1.901}, {'end': 845.893, 'text': 'Then again, same network.', 'start': 844.613, 'duration': 1.28}, {'end': 850.435, 'text': 'You supply second word and the output from previous step as an input.', 'start': 846.373, 'duration': 4.062}], 'summary': 'Using gradient descent, 100 samples are trained for 20 epochs to minimize loss in neural network. language translation involves sequential input and output.', 'duration': 44.788, 'max_score': 805.647, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I805647.jpg'}, {'end': 948.229, 'src': 'embed', 'start': 876.544, 'weight': 3, 'content': [{'end': 882.871, 'text': 'because after the statement I can push maybe one more word and that will just totally change my translation.', 'start': 876.544, 'duration': 6.327}, {'end': 889.598, 'text': "That's why for language translation you have to supply all the words and only then the network can translate for you.", 'start': 883.792, 'duration': 5.806}, {'end': 898.126, 'text': 'So the network will translate it like this and the first part is called encoder.', 'start': 891.482, 'duration': 6.644}, {'end': 899.948, 'text': 'The second part is called decoder.', 'start': 898.146, 'duration': 1.802}, {'end': 909.714, 'text': 'We will go more in depth into all this, but I want to quickly demonstrate how the neural network looks in the case of language translation.', 'start': 900.448, 'duration': 9.266}, {'end': 915.118, 'text': "Now this layer doesn't have to be just single layer.", 'start': 911.155, 'duration': 3.963}, {'end': 923.221, 'text': 'It can be a deep RNN as well, where the actual network might have multiple hidden layers.', 'start': 915.478, 'duration': 7.743}, {'end': 941.986, 'text': "okay, so i hope that clarifies the architecture behind RNN and you understand why you can't use simple neural network here and you have to use specialized neural network called RNN which can memorize for you which can remember previous state,", 'start': 923.221, 'duration': 18.765}, {'end': 944.787, 'text': 'because language is all about sequence.', 'start': 941.986, 'duration': 2.801}, {'end': 948.229, 'text': 'if you change the sequence, the meaning changes.', 'start': 945.507, 'duration': 2.722}], 'summary': 'Rnn architecture is crucial for language translation due to its ability to memorize and understand sequences.', 'duration': 71.685, 'max_score': 876.544, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I876544.jpg'}], 'start': 762.213, 'title': 'Neural network training and rnn for language translation', 'summary': 'Explains the process of training a neural network, involving the calculation of predicted y, comparison with real y, computation of loss, and the use of gradient descent for adjusting weights. it also discusses the concept of recurrent neural networks (rnn) for language translation, emphasizing the need for supplying all words for accurate translation and the necessity of using specialized neural networks like rnn due to the sequential nature of language.', 'chapters': [{'end': 820.303, 'start': 762.213, 'title': 'Neural network training process', 'summary': 'Explains the process of training a neural network, involving the calculation of predicted y, comparison with real y, computation of loss, and the use of gradient descent for adjusting weights, with an example of training samples and epochs.', 'duration': 58.09, 'highlights': ['The process involves calculating predicted y, comparing it with real y, and computing the loss, followed by summation to obtain the total loss.', 'Gradient descent is utilized to adjust the weights by back propagating the computed loss.', 'Training samples are passed through the network to complete one epoch, with the example citing 100 training samples as one epoch.']}, {'end': 958.658, 'start': 821.883, 'title': 'Understanding rnn for language translation', 'summary': 'Discusses the concept of recurrent neural networks (rnn) for language translation, emphasizing the need for supplying all words for accurate translation and the necessity of using specialized neural networks like rnn due to the sequential nature of language.', 'duration': 136.775, 'highlights': ['The need to supply all words for language translation to enable accurate network translation and the explanation of the encoder and decoder parts of the network.', 'The demonstration of the architecture behind RNN and the necessity of using specialized neural networks like RNN due to the sequential nature of language.', 'The explanation of how RNN can memorize and remember previous states, emphasizing the importance of sequence in language and the limitations of using simple neural networks for language translation.', 'The mention of using deep RNN with multiple hidden layers for language translation and the requirement for specialized neural networks like RNN to accurately translate language.']}], 'duration': 196.445, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Y2wfIKQyd1I/pics/Y2wfIKQyd1I762213.jpg', 'highlights': ['The process involves calculating predicted y, comparing it with real y, and computing the loss, followed by summation to obtain the total loss.', 'Gradient descent is utilized to adjust the weights by back propagating the computed loss.', 'Training samples are passed through the network to complete one epoch, with the example citing 100 training samples as one epoch.', 'The need to supply all words for language translation to enable accurate network translation and the explanation of the encoder and decoder parts of the network.', 'The demonstration of the architecture behind RNN and the necessity of using specialized neural networks like RNN due to the sequential nature of language.', 'The explanation of how RNN can memorize and remember previous states, emphasizing the importance of sequence in language and the limitations of using simple neural networks for language translation.', 'The mention of using deep RNN with multiple hidden layers for language translation and the requirement for specialized neural networks like RNN to accurately translate language.']}], 'highlights': ["Google's Gmail utilizes RNN to provide an auto-complete feature, saving time for users by predicting and completing their sentences. Illustrates a real-life application of RNN in Gmail, showcasing its practical usage and user benefits.", 'The video will cover how RNNs work and explore various applications of RNN in NLP and other domains, including sequence models. Outlines the upcoming content of the video, setting expectations for the audience regarding the coverage of RNN functionality and applications.', 'The use cases of sequence models include translation, named entity recognition, and sentiment analysis, demonstrating the effectiveness of sequence models in language processing.', 'The architecture and training process of recurrent neural networks (RNN) for name entity recognition (NER) is explained, detailing how RNN processes words sequentially, carrying the context and memory from previous words to provide accurate NER output.', 'The process involves calculating predicted y, comparing it with real y, and computing the loss, followed by summation to obtain the total loss.', 'Gradient descent is utilized to adjust the weights by back propagating the computed loss.', 'Training samples are passed through the network to complete one epoch, with the example citing 100 training samples as one epoch.', 'The need to supply all words for language translation to enable accurate network translation and the explanation of the encoder and decoder parts of the network.', 'The demonstration of the architecture behind RNN and the necessity of using specialized neural networks like RNN due to the sequential nature of language.', 'The explanation of how RNN can memorize and remember previous states, emphasizing the importance of sequence in language and the limitations of using simple neural networks for language translation.', 'The mention of using deep RNN with multiple hidden layers for language translation and the requirement for specialized neural networks like RNN to accurately translate language.', 'RNNs are mainly used for NLP, while CNNs are primarily utilized for image processing. Highlights the main purpose of RNNs and CNNs, providing a clear differentiation between their applications.']}