title
Tutorial 2- Fine Tuning Pretrained Model On Custom Dataset Using 🤗 Transformer

description
github: https://github.com/krishnaik06/Huggingfacetransformer In this tutorial, we will show you how to fine-tune a pretrained model from the Transformers library. In TensorFlow, models can be directly trained using Keras and the fit method. In PyTorch, there is no generic training loop so the 🤗 Transformers library provides an API with the class Trainer to let you fine-tune or train a model from scratch easily. --------------------------------------------------------------------------------------------------------------------------------------------------------------- ⭐ Kite is a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I've been using Kite for a few months and I love it! https://www.kite.com/get-kite/?utm_medium=referral&utm_source=youtube&utm_campaign=krishnaik&utm_content=description-only Subscribe my vlogging channel https://www.youtube.com/channel/UCjWY5hREA6FFYrthD0rZNIw Please donate if you want to support the channel through GPay UPID, Gpay: krishnaik06@okicici Telegram link: https://t.me/joinchat/N77M7xRvYUd403DgfE4TWw Please join as a member in my channel to get additional benefits like materials in Data Science, live streaming for Members and many more https://www.youtube.com/channel/UCNU_lfiiWBdtULKOw6X0Dig/join Connect with me here: Twitter: https://twitter.com/Krishnaik06 Facebook: https://www.facebook.com/krishnaik06 instagram: https://www.instagram.com/krishnaik06

detail
{'title': 'Tutorial 2- Fine Tuning Pretrained Model On Custom Dataset Using 🤗 Transformer', 'heatmap': [{'end': 60.283, 'start': 42.448, 'weight': 0.703}, {'end': 146.439, 'start': 126.545, 'weight': 0.766}, {'end': 213.189, 'start': 184.756, 'weight': 0.723}, {'end': 246.654, 'start': 236.342, 'weight': 0.817}, {'end': 370.453, 'start': 329.176, 'weight': 0.769}, {'end': 493.163, 'start': 463.103, 'weight': 0.863}, {'end': 561.047, 'start': 505.855, 'weight': 0.703}, {'end': 638.246, 'start': 593.555, 'weight': 0.827}], 'summary': 'The tutorial covers text classification with a spam collection dataset of 5572 rows and 2 columns, fine-tuning a distilbert model on a custom dataset, using transformers and tokenization in nlp, transforming encodings into data objects, and achieving 100% accuracy in 2-3 minutes for sentiment analysis using a pre-trained transformer in google collab pro.', 'chapters': [{'end': 235.862, 'segs': [{'end': 82.078, 'src': 'heatmap', 'start': 42.448, 'weight': 1, 'content': [{'end': 52.717, 'text': "I'm just going to show you an example wherein I'll be taking my own data sets and then I'll try to show you how we can use hugging phase transformers to Fine-tune a pre-trained model and this particular scenario.", 'start': 42.448, 'duration': 10.269}, {'end': 56.68, 'text': "the pre-trained model that I'm going to take is something called as distal Bert.", 'start': 52.717, 'duration': 3.963}, {'end': 60.283, 'text': 'Okay, so here you can see distal Bert on uncased.', 'start': 56.68, 'duration': 3.603}, {'end': 66.547, 'text': 'You can also use BERT and different different models, but I really wanted to show you one model and then we can also try other models.', 'start': 60.643, 'duration': 5.904}, {'end': 69.049, 'text': 'So to begin with, I have my data set.', 'start': 67.007, 'duration': 2.042}, {'end': 70.87, 'text': 'So this is my spam collection data set.', 'start': 69.089, 'duration': 1.781}, {'end': 74.212, 'text': "OK, first of all, I'm just going to upload the data set over here.", 'start': 71.37, 'duration': 2.842}, {'end': 77.094, 'text': 'So let me upload it quickly.', 'start': 75.113, 'duration': 1.981}, {'end': 82.078, 'text': "And remember, when you're uploading it, I'm doing it in Google Collab Pro guys.", 'start': 77.835, 'duration': 4.243}], 'summary': 'Using hugging face transformers to fine-tune pre-trained distilbert model on spam collection dataset in google colab pro', 'duration': 39.63, 'max_score': 42.448, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik42448.jpg'}, {'end': 147.119, 'src': 'heatmap', 'start': 126.545, 'weight': 0, 'content': [{'end': 132.87, 'text': "and let me tell you guys, once i take this df of message, i'm converting this into list and storing it in my x variable.", 'start': 126.545, 'duration': 6.325}, {'end': 134.772, 'text': 'so let me just execute it.', 'start': 132.87, 'duration': 1.902}, {'end': 136.453, 'text': 'and this is my y variable.', 'start': 134.772, 'duration': 1.681}, {'end': 138.114, 'text': 'okay, converting into a list.', 'start': 136.453, 'duration': 1.661}, {'end': 142.378, 'text': 'so if i go and see my y, so it is nothing, but it is having two categories like ham and spam.', 'start': 138.114, 'duration': 4.264}, {'end': 146.439, 'text': 'now, before i go ahead with this, so what i really need to do this?', 'start': 143.158, 'duration': 3.281}, {'end': 147.119, 'text': 'ham and spam.', 'start': 146.439, 'duration': 0.68}], 'summary': 'Converting message data into list, containing two categories: ham and spam.', 'duration': 45.985, 'max_score': 126.545, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik126545.jpg'}, {'end': 214.97, 'src': 'heatmap', 'start': 168.69, 'weight': 5, 'content': [{'end': 175.372, 'text': "Okay, because if I just execute something like this see this I'll just show you So let me do one thing.", 'start': 168.69, 'duration': 6.682}, {'end': 176.893, 'text': 'Let me just create a code.', 'start': 175.432, 'duration': 1.461}, {'end': 183.335, 'text': "So if I just execute like this, you'll be able to see I'll be getting one column with spam and okay.", 'start': 176.893, 'duration': 6.442}, {'end': 184.756, 'text': 'so zero, zero one.', 'start': 183.335, 'duration': 1.421}, {'end': 189.518, 'text': 'the one that basically means when it is zero it is ham, otherwise one it is spam.', 'start': 184.756, 'duration': 4.762}, {'end': 191.799, 'text': "okay, because i'm actually using like this.", 'start': 189.518, 'duration': 2.281}, {'end': 196.461, 'text': "so i'm going to take up this particular column and convert this entire things into list.", 'start': 191.799, 'duration': 4.662}, {'end': 198.082, 'text': 'that is the first thing over here.', 'start': 196.461, 'duration': 1.621}, {'end': 199.983, 'text': "so that is what i'm doing now.", 'start': 198.082, 'duration': 1.901}, {'end': 205.065, 'text': "if i go and see my y here, you'll be able to see that it will be a list.", 'start': 199.983, 'duration': 5.082}, {'end': 207.866, 'text': 'okay, list of zeros and ones, perfect.', 'start': 205.065, 'duration': 2.801}, {'end': 209.147, 'text': "now let's go to the next step.", 'start': 207.866, 'duration': 1.281}, {'end': 213.189, 'text': 'obviously this step is very much simple, not that difficult.', 'start': 209.147, 'duration': 4.042}, {'end': 214.97, 'text': 'doing a train test split.', 'start': 213.189, 'duration': 1.781}], 'summary': 'Converting a column into a list of zeros and ones, and performing a train test split.', 'duration': 46.28, 'max_score': 168.69, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik168690.jpg'}, {'end': 246.654, 'src': 'embed', 'start': 214.97, 'weight': 6, 'content': [{'end': 218.272, 'text': "okay. so here i'm just going to write it out train test split.", 'start': 214.97, 'duration': 3.302}, {'end': 224.535, 'text': "so this entire thing i'll be giving you guys, because many people were asking how we can fine tune a custom model.", 'start': 218.272, 'duration': 6.263}, {'end': 227.176, 'text': "so that is why i'm executing it.", 'start': 224.535, 'duration': 2.641}, {'end': 227.956, 'text': 'so train test split.', 'start': 227.176, 'duration': 0.78}, {'end': 229.437, 'text': "i've done it.", 'start': 227.956, 'duration': 1.481}, {'end': 233.779, 'text': 'if i want to go and check out my x train, i can definitely check it out Here.', 'start': 229.437, 'duration': 4.342}, {'end': 235.862, 'text': 'if I execute, this is my Xtrain entirely.', 'start': 233.779, 'duration': 2.083}, {'end': 237.544, 'text': 'So these are all my features.', 'start': 236.342, 'duration': 1.202}, {'end': 239.926, 'text': 'Now coming to the next step.', 'start': 238.985, 'duration': 0.941}, {'end': 246.654, 'text': 'We are going to install the transformers because we will be using a model of transformer itself like digital bird.', 'start': 240.567, 'duration': 6.087}], 'summary': 'Demonstration of train-test split and custom model fine-tuning using transformer like digital bird.', 'duration': 31.684, 'max_score': 214.97, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik214970.jpg'}], 'start': 13.888, 'title': 'Text classification and model fine-tuning', 'summary': 'Covers text classification using a spam collection dataset with 5572 rows and 2 columns, and fine-tuning a pre-trained distilbert model on a custom dataset for prediction.', 'chapters': [{'end': 60.283, 'start': 13.888, 'title': 'Fine-tuning pre-trained model with hugging phase transformer', 'summary': 'Discusses fine-tuning a pre-trained model using hugging face transformer, focusing on the example of fine-tuning a pre-trained distilbert model on a custom dataset for prediction.', 'duration': 46.395, 'highlights': ['The video demonstrates the process of fine-tuning a pre-trained model using Hugging Face Transformer, specifically showcasing the example of using a custom dataset to fine-tune a pre-trained DistilBERT model.', 'The presenter mentions the specific pre-trained model, DistilBERT, and its uncased version for the fine-tuning process.']}, {'end': 126.545, 'start': 60.643, 'title': 'Text classification with pandas and bert', 'summary': 'Demonstrates text classification using a spam collection data set, involving the upload and processing of a dataset in google collab, resulting in 5572 rows and 2 columns, and the conversion of features into independent and dependent categories.', 'duration': 65.902, 'highlights': ['The dataset consists of 5572 rows and 2 columns, showcasing the scale of the data and providing context for the text classification process.', 'The demonstration includes the upload and processing of the dataset in Google Collab, emphasizing the practical application of the text classification process.', 'The conversion of features into independent and dependent categories is showcased, highlighting the crucial step in preparing the data for text classification.']}, {'end': 235.862, 'start': 126.545, 'title': 'Converting ham and spam into zeros and ones', 'summary': "Involves converting the categories of 'ham' and 'spam' into zeros and ones, then performing a train-test split for a custom model.", 'duration': 109.317, 'highlights': ["Converting 'ham' and 'spam' into zeros and ones", 'Performing a train-test split', 'Storing messages in variables']}], 'duration': 221.974, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik13888.jpg', 'highlights': ['The dataset consists of 5572 rows and 2 columns, showcasing the scale of the data and providing context for the text classification process.', 'The video demonstrates the process of fine-tuning a pre-trained model using Hugging Face Transformer, specifically showcasing the example of using a custom dataset to fine-tune a pre-trained DistilBERT model.', 'The demonstration includes the upload and processing of the dataset in Google Collab, emphasizing the practical application of the text classification process.', 'The presenter mentions the specific pre-trained model, DistilBERT, and its uncased version for the fine-tuning process.', 'The conversion of features into independent and dependent categories is showcased, highlighting the crucial step in preparing the data for text classification.', "Converting 'ham' and 'spam' into zeros and ones", 'Performing a train-test split', 'Storing messages in variables']}, {'end': 428.671, 'segs': [{'end': 265.508, 'src': 'heatmap', 'start': 236.342, 'weight': 0.817, 'content': [{'end': 237.544, 'text': 'So these are all my features.', 'start': 236.342, 'duration': 1.202}, {'end': 239.926, 'text': 'Now coming to the next step.', 'start': 238.985, 'duration': 0.941}, {'end': 246.654, 'text': 'We are going to install the transformers because we will be using a model of transformer itself like digital bird.', 'start': 240.567, 'duration': 6.087}, {'end': 249.097, 'text': "So let's pip install transformers.", 'start': 246.994, 'duration': 2.103}, {'end': 252.741, 'text': 'So here you will be able to see the model will get downloaded.', 'start': 250.038, 'duration': 2.703}, {'end': 257.44, 'text': 'after the model gets downloaded.', 'start': 255.879, 'duration': 1.561}, {'end': 263.466, 'text': 'here you can actually, sorry, after the library gets downloaded, now you can basically call the transformer.', 'start': 257.44, 'duration': 6.026}, {'end': 265.508, 'text': 'now. see, guys, what are the steps.', 'start': 263.466, 'duration': 2.042}], 'summary': 'Installing transformers like digital bird model using pip for nlp tasks.', 'duration': 29.166, 'max_score': 236.342, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik236342.jpg'}, {'end': 370.453, 'src': 'heatmap', 'start': 304.844, 'weight': 0, 'content': [{'end': 307.566, 'text': "so in order to do that i'll go to my file over here.", 'start': 304.844, 'duration': 2.722}, {'end': 311.767, 'text': "so here you can see from transformers i'm going to import distil but tokenizer fast.", 'start': 307.566, 'duration': 4.201}, {'end': 314.669, 'text': 'so this is the tokenizer that will be used for here.', 'start': 311.767, 'duration': 2.902}, {'end': 321.272, 'text': "and then i'm saying tokenizer is equal to distil but tokenizer fast from pre-trained of distil but uncased.", 'start': 314.669, 'duration': 6.603}, {'end': 325.414, 'text': "okay. So similarly you'll be having different, different tokenizers all together.", 'start': 321.272, 'duration': 4.142}, {'end': 329.075, 'text': 'But for Distilbert, I would suggest take up this particular tokenizer.', 'start': 325.894, 'duration': 3.181}, {'end': 333.297, 'text': 'Again, you can explore the tokenizers in the documentation page of the Transformers.', 'start': 329.176, 'duration': 4.121}, {'end': 336.479, 'text': "Then I'm going to use this and try to execute it.", 'start': 334.118, 'duration': 2.361}, {'end': 344.003, 'text': 'So this is my tokenizer, which will be responsible for tokenizing or converting my sentences into some numerical values.', 'start': 337.299, 'duration': 6.704}, {'end': 348.284, 'text': "then here you can see i'm giving my tokenizer, i'm giving x underscore.", 'start': 345.083, 'duration': 3.201}, {'end': 351.426, 'text': 'train truncation is equal to true and padding is equal to true.', 'start': 348.284, 'duration': 3.142}, {'end': 353.006, 'text': 'these two are actually done.', 'start': 351.426, 'duration': 1.58}, {'end': 357.988, 'text': 'if i go and see over here, truncation and padding are actually usually done.', 'start': 353.006, 'duration': 4.982}, {'end': 362.31, 'text': 'see, padding is usually done to make all the sentences of the similar size.', 'start': 357.988, 'duration': 4.322}, {'end': 364.911, 'text': 'truncation is to remove the white space.', 'start': 362.31, 'duration': 2.601}, {'end': 366.712, 'text': 'okay, so all these things is done.', 'start': 364.911, 'duration': 1.801}, {'end': 370.453, 'text': 'so here you can see it is done for both x train and x test.', 'start': 366.712, 'duration': 3.741}], 'summary': 'Using distilbert tokenizer for tokenizing and converting sentences into numerical values, with truncation and padding applied to x_train and x_test.', 'duration': 61.868, 'max_score': 304.844, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik304844.jpg'}, {'end': 412.682, 'src': 'embed', 'start': 386.818, 'weight': 4, 'content': [{'end': 390.979, 'text': "so if i execute this, so here you can see i'm getting my input ids.", 'start': 386.818, 'duration': 4.161}, {'end': 395.401, 'text': 'this is my numerical values with respect to the sentences that i have actually given.', 'start': 390.979, 'duration': 4.422}, {'end': 400.976, 'text': 'and see here this is all padding, okay and by default, i think the size is 512..', 'start': 395.401, 'duration': 5.575}, {'end': 403.377, 'text': 'Okay, you can also give your own size.', 'start': 400.976, 'duration': 2.401}, {'end': 405.979, 'text': "It's not like you just have to give the same size.", 'start': 404.038, 'duration': 1.941}, {'end': 409.821, 'text': 'You can give your size with respect to anything that you want.', 'start': 406.919, 'duration': 2.902}, {'end': 412.682, 'text': 'Okay, probably you want to give 100.', 'start': 410.061, 'duration': 2.621}], 'summary': 'Demonstrating input ids and customizable size for numerical values in nlp model.', 'duration': 25.864, 'max_score': 386.818, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik386818.jpg'}], 'start': 236.342, 'title': 'Using transformers and tokenization in nlp', 'summary': "Covers the installation of the 'transformers' library, emphasizing the importance of using a specific tokenizer for a specific pre-trained model. it also explains tokenization, padding, and truncation in nlp, with a default input ids size of 512 and the flexibility to set a custom size.", 'chapters': [{'end': 344.003, 'start': 236.342, 'title': 'Using transformers for natural language processing', 'summary': "Covers the installation of the 'transformers' library, including steps to install a pre-trained model and a tokenizer, emphasizing the importance of using a specific tokenizer for a specific pre-trained model.", 'duration': 107.661, 'highlights': ['The importance of using a specific tokenizer for a specific pre-trained model is emphasized, with a recommendation to explore the tokenizers in the documentation page of the Transformers library.', "The process of installing and using a pre-trained model and tokenizer, such as 'distilbert', is explained, highlighting the use of the 'transformers' library for converting sentences into numerical values.", "The installation of the 'transformers' library and the specific tokenizer 'distilbert' is demonstrated, with a mention of importing 'distilbert tokenizer fast' and using it for tokenizing sentences."]}, {'end': 428.671, 'start': 345.083, 'title': 'Tokenization and padding in nlp', 'summary': 'Explains tokenization and padding in nlp for text data processing, including the purpose of padding to make sentences of similar size and truncation to remove white space, alongside the default size of 512 for input ids and the flexibility to set a custom size.', 'duration': 83.588, 'highlights': ['The chapter discusses tokenization and padding in NLP, highlighting the purpose of padding to make sentences of similar size and truncation to remove white space, with the default size of 512 for input ids.', 'The chapter explains the flexibility to set a custom size for padding, such as 100, instead of the default size of 512 for input ids.']}], 'duration': 192.329, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik236342.jpg', 'highlights': ["The process of installing and using a pre-trained model and tokenizer, such as 'distilbert', is explained, highlighting the use of the 'transformers' library for converting sentences into numerical values.", 'The importance of using a specific tokenizer for a specific pre-trained model is emphasized, with a recommendation to explore the tokenizers in the documentation page of the Transformers library.', 'The chapter discusses tokenization and padding in NLP, highlighting the purpose of padding to make sentences of similar size and truncation to remove white space, with the default size of 512 for input ids.', "The installation of the 'transformers' library and the specific tokenizer 'distilbert' is demonstrated, with a mention of importing 'distilbert tokenizer fast' and using it for tokenizing sentences.", 'The chapter explains the flexibility to set a custom size for padding, such as 100, instead of the default size of 512 for input ids.']}, {'end': 597.338, 'segs': [{'end': 458.699, 'src': 'embed', 'start': 428.831, 'weight': 0, 'content': [{'end': 431.573, 'text': 'We have found out the train encodings and test encodings.', 'start': 428.831, 'duration': 2.742}, {'end': 435.275, 'text': 'I have my input IDs completely with us.', 'start': 431.653, 'duration': 3.622}, {'end': 436.556, 'text': 'So this is perfect.', 'start': 435.716, 'duration': 0.84}, {'end': 439.598, 'text': "Now let's go and check out what is the next step.", 'start': 437.157, 'duration': 2.441}, {'end': 443.021, 'text': 'Now the next step is also very very much important.', 'start': 440.639, 'duration': 2.382}, {'end': 445.062, 'text': 'Let me note down in a notebook file.', 'start': 443.041, 'duration': 2.021}, {'end': 457.398, 'text': 'The next step is that we really need to convert these encodings, these encodings, into data objects.', 'start': 445.502, 'duration': 11.896}, {'end': 458.699, 'text': 'data set objects.', 'start': 457.398, 'duration': 1.301}], 'summary': 'Train and test encodings found, input ids complete. next step: converting encodings into data set objects.', 'duration': 29.868, 'max_score': 428.831, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik428831.jpg'}, {'end': 505.314, 'src': 'heatmap', 'start': 463.103, 'weight': 2, 'content': [{'end': 468.048, 'text': 'in case of, uh, when just let me write it down.', 'start': 463.103, 'duration': 4.945}, {'end': 475.035, 'text': "okay, in the case of what happens is that when i'm using transformers see over here when i'm using this transformer, i'm going to use tensorflow.", 'start': 468.048, 'duration': 6.987}, {'end': 478.817, 'text': 'okay, If you are using PyTorch, then it will be a different dataset object.', 'start': 475.035, 'duration': 3.782}, {'end': 483.819, 'text': 'In the case of TensorFlow, it will be tensors, you know? So we really need to create in the form of tensors.', 'start': 478.857, 'duration': 4.962}, {'end': 486.36, 'text': "So for doing that, here is what I'm doing.", 'start': 483.939, 'duration': 2.421}, {'end': 493.163, 'text': "tf.dataset from tensorslices, I'm giving the dictionaries, the training encoding in the form of dictionaries and my ytrain value.", 'start': 486.46, 'duration': 6.703}, {'end': 496.945, 'text': "Similarly, over here, I'm giving test encodings and ytest values.", 'start': 493.763, 'duration': 3.182}, {'end': 505.314, 'text': "If I execute it, here, now you'll be able to see what will be the format, okay? So this will take some time because I have a huge dataset.", 'start': 497.085, 'duration': 8.229}], 'summary': 'Using tensorflow for datasets, creating tensors from dictionaries, and working with a large dataset.', 'duration': 42.211, 'max_score': 463.103, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik463103.jpg'}, {'end': 561.047, 'src': 'heatmap', 'start': 505.855, 'weight': 0.703, 'content': [{'end': 511.464, 'text': "So I'll just make this one and probably just try to show you how my train underscore dataset looks like.", 'start': 505.855, 'duration': 5.609}, {'end': 517.582, 'text': "okay. so once i execute it, guys, here you'll be able to see it will be in the tensor slice.", 'start': 512.357, 'duration': 5.225}, {'end': 518.183, 'text': 'uh, shapes.', 'start': 517.582, 'duration': 0.601}, {'end': 519.684, 'text': 'the input ids will be there.', 'start': 518.183, 'duration': 1.501}, {'end': 521.486, 'text': 'attention mask will be there.', 'start': 519.684, 'duration': 1.802}, {'end': 529.253, 'text': 'uh, this input id is the attention mark, all all this particular information, because this is the format that the classification,', 'start': 521.486, 'duration': 7.767}, {'end': 532.276, 'text': 'sequence classification that we will be using, that is the distal bird.', 'start': 529.253, 'duration': 3.023}, {'end': 534.378, 'text': 'Now coming over here, guys, the next step.', 'start': 532.876, 'duration': 1.502}, {'end': 538.282, 'text': "over here you can see that from transformers I'm going to import this sequence classification.", 'start': 534.378, 'duration': 3.904}, {'end': 543.588, 'text': 'Remember the sequence classification is mostly for sentiment analysis not for question answer application.', 'start': 538.843, 'duration': 4.745}, {'end': 553.258, 'text': 'Any type of sentiment analysis that you really want to do you really need to import the sequence classification library.', 'start': 544.749, 'duration': 8.509}, {'end': 556.922, 'text': "So I'm using dfDigitalBird for sequence classification.", 'start': 554.019, 'duration': 2.903}, {'end': 561.047, 'text': 'tftrainer and tftraining arguments will be responsible for training my model.', 'start': 557.343, 'duration': 3.704}], 'summary': 'Demonstrating train dataset structure, using distilbert for sentiment analysis, and training model with tftrainer and tftraining arguments.', 'duration': 55.192, 'max_score': 505.855, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik505855.jpg'}, {'end': 588.671, 'src': 'embed', 'start': 561.448, 'weight': 1, 'content': [{'end': 565.953, 'text': "After my model is basically trained, I'll be using this tftrainer object to do the prediction.", 'start': 561.448, 'duration': 4.505}, {'end': 568.255, 'text': "So I'm going to use these arguments over here.", 'start': 566.493, 'duration': 1.762}, {'end': 571.058, 'text': 'My output directory will be results.', 'start': 569.016, 'duration': 2.042}, {'end': 573.259, 'text': 'Number of training epochs will be 2.', 'start': 571.318, 'duration': 1.941}, {'end': 575.861, 'text': 'Per device train batch size will be 8.', 'start': 573.259, 'duration': 2.602}, {'end': 578.323, 'text': 'Remember, try to make it as 8 and 16.', 'start': 575.861, 'duration': 2.462}, {'end': 580.405, 'text': "Then only you'll be able to run it in your Google Colab.", 'start': 578.323, 'duration': 2.082}, {'end': 585.028, 'text': 'Probably if memory out of error is coming, you try to reduce it to 4 and 8.', 'start': 580.545, 'duration': 4.483}, {'end': 586.67, 'text': 'Warm-up steps will be 500.', 'start': 585.028, 'duration': 1.642}, {'end': 588.671, 'text': 'And then all my logs will be stored over here.', 'start': 586.67, 'duration': 2.001}], 'summary': 'Using tftrainer for prediction with 2 training epochs, 8 train batch size, and 500 warm-up steps.', 'duration': 27.223, 'max_score': 561.448, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik561448.jpg'}], 'start': 428.831, 'title': 'Transforming encodings into data objects', 'summary': 'Explains the process of converting train and test encodings into dataset objects, using tensors for tensorflow and importing sequence classification for sentiment analysis, with specific training parameters provided.', 'chapters': [{'end': 597.338, 'start': 428.831, 'title': 'Transforming encodings into data objects', 'summary': 'Explains the process of converting train and test encodings into dataset objects, using tensors for tensorflow and importing sequence classification for sentiment analysis, with specific training parameters provided.', 'duration': 168.507, 'highlights': ['The next step involves converting the train and test encodings into dataset objects, using tensors for TensorFlow and importing sequence classification for sentiment analysis, with specific training parameters provided, such as number of training epochs, per device train batch size, warm-up steps, and logging steps.', 'The process involves creating dataset objects using TensorFlow, where the training encodings and ytrain values are converted into tensors, followed by a similar process for test encodings and ytest values.', 'The chapter emphasizes the use of specific training parameters, including the number of training epochs, per device train batch size, warm-up steps, and logging steps, to ensure efficient model training and prediction.', 'The use of specific training parameters, such as per device train batch size and logging steps, is highlighted as crucial for efficient model training and prediction, with recommendations to adjust parameters to avoid memory errors.']}], 'duration': 168.507, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik428831.jpg', 'highlights': ['The next step involves converting the train and test encodings into dataset objects, using tensors for TensorFlow and importing sequence classification for sentiment analysis, with specific training parameters provided, such as number of training epochs, per device train batch size, warm-up steps, and logging steps.', 'The chapter emphasizes the use of specific training parameters, including the number of training epochs, per device train batch size, warm-up steps, and logging steps, to ensure efficient model training and prediction.', 'The process involves creating dataset objects using TensorFlow, where the training encodings and ytrain values are converted into tensors, followed by a similar process for test encodings and ytest values.', 'The use of specific training parameters, such as per device train batch size and logging steps, is highlighted as crucial for efficient model training and prediction, with recommendations to adjust parameters to avoid memory errors.']}, {'end': 944.783, 'segs': [{'end': 630.639, 'src': 'embed', 'start': 598.931, 'weight': 3, 'content': [{'end': 605.834, 'text': "Now over here you will be able to see that I'm going to use this distal bird for sequence classification from pre-trained model distal bird, uncased,", 'start': 598.931, 'duration': 6.903}, {'end': 613.577, 'text': "okay?. And then I'm actually starting my training by the help of pftrainer, where I'm giving my model value over here, argument, training arguments,", 'start': 605.834, 'duration': 7.743}, {'end': 618.379, 'text': "whatever training arguments I've created over here, my train dataset, my trace dataset, and then I do trainer.ten.", 'start': 613.577, 'duration': 4.802}, {'end': 624.773, 'text': "Now, you may be thinking, Krish, how do you know all this thing? I'm just following this documentation, guys.", 'start': 619.628, 'duration': 5.145}, {'end': 627.476, 'text': "That's it, okay? To do my custom things.", 'start': 624.813, 'duration': 2.663}, {'end': 628.877, 'text': 'You can also do it in PyTorch.', 'start': 627.696, 'duration': 1.181}, {'end': 630.639, 'text': 'You can also do it in TensorFlow.', 'start': 628.917, 'duration': 1.722}], 'summary': 'Using distilbert for sequence classification with pftrainer in pytorch and tensorflow.', 'duration': 31.708, 'max_score': 598.931, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik598931.jpg'}, {'end': 752.522, 'src': 'embed', 'start': 684.403, 'weight': 1, 'content': [{'end': 686.024, 'text': 'So yes, let it go.', 'start': 684.403, 'duration': 1.621}, {'end': 690.187, 'text': 'These all warnings will be coming.', 'start': 686.044, 'duration': 4.143}, {'end': 691.368, 'text': "You don't have to worry about it.", 'start': 690.347, 'duration': 1.021}, {'end': 700.478, 'text': "So let's wait and let's wait till the training is done.", 'start': 698.161, 'duration': 2.317}, {'end': 715.402, 'text': 'Okay guys, so here you can see the training has been done.', 'start': 712.901, 'duration': 2.501}, {'end': 717.843, 'text': "Okay, I've just run it for two epochs.", 'start': 715.702, 'duration': 2.141}, {'end': 726.888, 'text': 'You can also run it for four to five epochs, but approximately took me somewhere around two to three minutes, probably in Google Collab Pro.', 'start': 717.883, 'duration': 9.005}, {'end': 733.691, 'text': 'Now if I go and execute training.evaluate, so here you can see my eval loss will be probably given some value.', 'start': 727.508, 'duration': 6.183}, {'end': 741.316, 'text': 'okay, so let me execute so it is somewhere on point zero two, okay, so it is very, very less.', 'start': 734.972, 'duration': 6.344}, {'end': 743.337, 'text': 'so it is almost near to zero.', 'start': 741.316, 'duration': 2.021}, {'end': 748.28, 'text': 'now, if i do the prediction in order to do the prediction, uh i can.', 'start': 743.337, 'duration': 4.943}, {'end': 749.68, 'text': "why i'm using this trainer.", 'start': 748.28, 'duration': 1.4}, {'end': 752.522, 'text': 'remember, this is my trainer right, which is actually trained.', 'start': 749.68, 'duration': 2.842}], 'summary': 'Trained the model for two epochs, achieving an evaluation loss of 0.02, near zero.', 'duration': 68.119, 'max_score': 684.403, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik684403.jpg'}, {'end': 902.918, 'src': 'embed', 'start': 868.954, 'weight': 0, 'content': [{'end': 870.396, 'text': 'This is very much simple, guys.', 'start': 868.954, 'duration': 1.442}, {'end': 871.537, 'text': 'You have got the accuracy score.', 'start': 870.436, 'duration': 1.101}, {'end': 872.998, 'text': 'You have got the prediction.', 'start': 871.557, 'duration': 1.441}, {'end': 874.539, 'text': 'So let me also write a comment over here.', 'start': 873.018, 'duration': 1.521}, {'end': 877.562, 'text': 'right. so this is very much simple.', 'start': 875.08, 'duration': 2.482}, {'end': 882.505, 'text': 'always remember, you need to know what all things we actually did.', 'start': 877.562, 'duration': 4.943}, {'end': 886.327, 'text': 'with respect to the encoding things, you know.', 'start': 882.505, 'duration': 3.822}, {'end': 889.569, 'text': 'so we took the tokenizer of this one distal bird tokenizer.', 'start': 886.327, 'duration': 3.242}, {'end': 894.392, 'text': 'fast. if you are using bert based uncased, then use bert tokenizer fast, okay.', 'start': 889.569, 'duration': 4.823}, {'end': 895.853, 'text': 'and then you have to use truncation.', 'start': 894.392, 'duration': 1.461}, {'end': 896.834, 'text': 'padding is equal to true.', 'start': 895.853, 'duration': 0.981}, {'end': 902.918, 'text': 'by default, the truncation and the padding is nothing but 512 characters and then convert this encoding into data set objects.', 'start': 896.834, 'duration': 6.084}], 'summary': 'Simple process with accuracy score, predictions, and tokenizer usage for data encoding.', 'duration': 33.964, 'max_score': 868.954, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik868954.jpg'}, {'end': 944.783, 'src': 'embed', 'start': 924.731, 'weight': 6, 'content': [{'end': 931.655, 'text': 'So you just try it by yourself and try to take your own data set and try to do fine tuning of this specific pre-trained model.', 'start': 924.731, 'duration': 6.924}, {'end': 938.819, 'text': 'This is just like a custom sentiment analysis by using a pre-trained model of transformers, hugging cage transformers.', 'start': 932.015, 'duration': 6.804}, {'end': 940.06, 'text': 'So I hope you like this particular video.', 'start': 938.839, 'duration': 1.221}, {'end': 942.121, 'text': "Please do subscribe to the channel if you haven't already subscribed.", 'start': 940.36, 'duration': 1.761}, {'end': 942.862, 'text': "I'll see you in the next video.", 'start': 942.141, 'duration': 0.721}, {'end': 943.442, 'text': 'Have a great day.', 'start': 942.882, 'duration': 0.56}, {'end': 944.062, 'text': 'Thank you and all.', 'start': 943.482, 'duration': 0.58}, {'end': 944.783, 'text': 'Bye-bye.', 'start': 944.263, 'duration': 0.52}], 'summary': 'Try fine-tuning a pre-trained model for custom sentiment analysis. subscribe for more content.', 'duration': 20.052, 'max_score': 924.731, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik924731.jpg'}], 'start': 598.931, 'title': 'Sequence classification and sentiment analysis', 'summary': "Demonstrates sequence classification using 'distal bird, uncased' pre-trained model with pftrainer for training and covers custom sentiment analysis model with pre-trained transformer achieving 100% accuracy in 2-3 minutes using google collab pro.", 'chapters': [{'end': 630.639, 'start': 598.931, 'title': 'Sequence classification with pre-trained model', 'summary': "Demonstrates the process of sequence classification using a pre-trained model 'distal bird, uncased' and mentions the use of pftrainer for training, with reference to following documentation and the ability to perform similar tasks in pytorch and tensorflow.", 'duration': 31.708, 'highlights': ["The process of sequence classification using a pre-trained model 'distal bird, uncased'", 'Mention of using pftrainer for training', 'Reference to following documentation for guidance', 'Ability to perform similar tasks in PyTorch and TensorFlow']}, {'end': 944.783, 'start': 631.119, 'title': 'Custom sentiment analysis with transformers', 'summary': 'Covers training a custom sentiment analysis model using a pre-trained transformer model, achieving 100% accuracy, and saving the model for future use, taking approximately 2-3 minutes in google collab pro.', 'duration': 313.664, 'highlights': ['The model achieved an eval loss of approximately 0.02, indicating high accuracy.', 'The model achieved a 100% accuracy score, showcasing its effectiveness in sentiment analysis.', 'The process of training the model took approximately 2-3 minutes in Google Collab Pro, making it a time-efficient solution.', 'The key steps involved using a pre-trained transformer model, such as tokenizer, encoding, and creating a confusion matrix for performance evaluation.']}], 'duration': 345.852, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/V1-Hm2rNkik/pics/V1-Hm2rNkik598931.jpg', 'highlights': ['Achieved 100% accuracy in sentiment analysis', 'Training model took 2-3 minutes in Google Collab Pro', 'Eval loss of approximately 0.02, indicating high accuracy', "Sequence classification using 'distal bird, uncased' pre-trained model", 'Using pftrainer for training', 'Ability to perform similar tasks in PyTorch and TensorFlow', 'Key steps involved using pre-trained transformer model']}], 'highlights': ['Achieved 100% accuracy in sentiment analysis', 'The dataset consists of 5572 rows and 2 columns, showcasing the scale of the data and providing context for the text classification process.', 'Training model took 2-3 minutes in Google Collab Pro', 'The video demonstrates the process of fine-tuning a pre-trained model using Hugging Face Transformer, specifically showcasing the example of using a custom dataset to fine-tune a pre-trained DistilBERT model.', "The process of installing and using a pre-trained model and tokenizer, such as 'distilbert', is explained, highlighting the use of the 'transformers' library for converting sentences into numerical values.", 'The next step involves converting the train and test encodings into dataset objects, using tensors for TensorFlow and importing sequence classification for sentiment analysis, with specific training parameters provided, such as number of training epochs, per device train batch size, warm-up steps, and logging steps.']}