title
5. Functions, Tools and Agents with LangChain | Andrew Ng | DeepLearning.ai - Full Course

description
The course comes from [https://learn.deeplearning.ai/functions-tools-agents-langchain/lesson/1/introduction](https://learn.deeplearning.ai/functions-tools-agents-langchain/lesson/1/introduction) created by Andrew Ng This YouTube video introduces the Functions, Tools and Agents course using LangChain. Language models (LLMs) have an amazing ability to interact with humans in natural language, but how to make LLMs interact with existing software infrastructure is a key issue. LangChain is an open source library designed to help developers bridge the gap between traditional software and LLMs. In addition, the course emphasizes new function call capabilities that enable LLMs to decide when to call functions of other programs to perform specific tasks, further expanding the scope of LLMs applications. Harrison Chase, co-founder and CEO of Landchain, described two major changes at LangChain: The Link Chain Expression Language (LCEL) was developed to simplify the construction of components or chains and to take advantage of the ability to call new functions. Participants will learn how to use this capability directly and how to build tools and dialogue agents. Get free course notes: https://t.me/NoteForYoutubeCourse

detail
{'title': '5. Functions, Tools and Agents with LangChain | Andrew Ng | DeepLearning.ai - Full Course', 'heatmap': [], 'summary': 'Course by andrew ng explores new function calling capabilities for developers working with llms and openai, covering lcel development, openai function calling, pydantic and langchain integration, ai message parsing, and building ai agent and conversational agents.', 'chapters': [{'end': 165.943, 'segs': [{'end': 124.314, 'src': 'embed', 'start': 66.821, 'weight': 0, 'content': [{'end': 73.343, 'text': 'This new capability, which OpenAI has named function calling, is really helpful for developers working with LLMs.', 'start': 66.821, 'duration': 6.522}, {'end': 81.231, 'text': "Harrison, you've described Langchain in other courses, but maybe you could describe what has changed and what you'd be covering in this course.", 'start': 73.763, 'duration': 7.468}, {'end': 89.098, 'text': 'Absolutely As you know, Langchain is an open-source library that helps developers bridge the gap between traditional software and LLMs.', 'start': 81.691, 'duration': 7.407}, {'end': 98.344, 'text': 'It allows developers to support any number of different LLMs and provides over 500 integrations to different language models, vector stores and tools,', 'start': 89.739, 'duration': 8.605}, {'end': 101.006, 'text': 'as well as supporting memory chains and agents.', 'start': 98.344, 'duration': 2.662}, {'end': 104.909, 'text': 'There are two significant changes that you will be getting into during the course.', 'start': 101.747, 'duration': 3.162}, {'end': 113.811, 'text': 'The first is the development of link chain expression language, or LCEL, that makes composing components or chains simpler and more intuitive.', 'start': 105.689, 'duration': 8.122}, {'end': 117.812, 'text': 'The second is changes to take advantage of the new function calling capability.', 'start': 114.611, 'duration': 3.201}, {'end': 124.314, 'text': "You'll learn how to use that directly, and we'll also show how it can be used to do tasks like tagging or extracting data.", 'start': 118.352, 'duration': 5.962}], 'summary': 'Langchain enables developers to work with llms, offering 500+ integrations and new function calling capability.', 'duration': 57.493, 'max_score': 66.821, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY66821.jpg'}], 'start': 2.998, 'title': 'Llms and function calling with langchain', 'summary': 'Introduces the new function calling capability for developers working with llms, covering the development of langchain expression language (lcel) and enabling the building of tools for llms and conversational agents.', 'chapters': [{'end': 165.943, 'start': 2.998, 'title': 'Llms and function calling with langchain', 'summary': 'Introduces the new capability of function calling, which is helpful for developers working with llms and covers the development of langchain expression language (lcel) and the new function calling capability, enabling the building of tools for llms and conversational agents.', 'duration': 162.945, 'highlights': ['Langchain is an open-source library that supports any number of different LLMs and provides over 500 integrations to different language models, vector stores, and tools, as well as supporting memory chains and agents. Langchain provides extensive support for LLMs and offers over 500 integrations to different language models and tools, making it a versatile open-source library.', 'The new capability of function calling, named by OpenAI, expands the capabilities of LLMs, allowing them to extract information from structured or tabular data and make it easier to call other code as a subroutine. The function calling capability significantly expands what LLMs can do, enabling them to extract information from structured or tabular data and call other code as a subroutine, enhancing their capabilities.', 'The course covers the development of Langchain expression language (LCEL) and how to take advantage of the new function calling capability, enabling the building of tools for LLMs and conversational agents. The course focuses on developing Langchain expression language (LCEL) and leveraging the new function calling capability to build tools for LLMs and conversational agents, providing practical skills for developers.']}], 'duration': 162.945, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2998.jpg', 'highlights': ['Langchain is an open-source library that supports any number of different LLMs and provides over 500 integrations to different language models, vector stores, and tools, as well as supporting memory chains and agents.', 'The new capability of function calling, named by OpenAI, expands the capabilities of LLMs, allowing them to extract information from structured or tabular data and make it easier to call other code as a subroutine.', 'The course covers the development of Langchain expression language (LCEL) and how to take advantage of the new function calling capability, enabling the building of tools for LLMs and conversational agents.']}, {'end': 970.626, 'segs': [{'end': 199.828, 'src': 'embed', 'start': 166.563, 'weight': 0, 'content': [{'end': 169.484, 'text': "So with that, let's go on to the next video and let's get started.", 'start': 166.563, 'duration': 2.921}, {'end': 179.473, 'text': 'In this lesson, we will go over function calling, a new capability added to the OpenAI API a few months ago.', 'start': 173.089, 'duration': 6.384}, {'end': 184.616, 'text': "We'll go over how exactly to use this and some tips and tricks for getting best results.", 'start': 180.293, 'duration': 4.323}, {'end': 191.62, 'text': 'OpenAI has fine-tuned some of the most recent models to accept additional parameters for function calling.', 'start': 185.476, 'duration': 6.144}, {'end': 199.828, 'text': "These models are fine-tuned to determine if it's relevant to call one of these functions and if so, what the parameters to those functions should be.", 'start': 191.92, 'duration': 7.908}], 'summary': 'Openai api now supports function calling with fine-tuned models.', 'duration': 33.265, 'max_score': 166.563, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY166563.jpg'}, {'end': 235.761, 'src': 'embed', 'start': 213.203, 'weight': 1, 'content': [{'end': 220.649, 'text': "To walk through this example, we're going to imagine that we have a function that we think is interesting to provide to the language model.", 'start': 213.203, 'duration': 7.446}, {'end': 226.614, 'text': "And we'll go over what interesting means later on because there are a bunch of different use cases for this new parameter.", 'start': 220.949, 'duration': 5.665}, {'end': 230.837, 'text': "Here, we're going to define a getCurrentWeather function.", 'start': 228.035, 'duration': 2.802}, {'end': 235.761, 'text': 'This is an example from OpenAI themselves when they first released this functionality.', 'start': 231.458, 'duration': 4.303}], 'summary': "Example of defining a getcurrentweather function from openai's release.", 'duration': 22.558, 'max_score': 213.203, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY213203.jpg'}, {'end': 292.29, 'src': 'embed', 'start': 253.174, 'weight': 2, 'content': [{'end': 256.935, 'text': "In this example we hard code the information that's returned,", 'start': 253.174, 'duration': 3.761}, {'end': 261.916, 'text': 'but in production this could be hitting a weather API or some external source of knowledge.', 'start': 256.935, 'duration': 4.981}, {'end': 265.797, 'text': 'So how do we pass this information to the language model?', 'start': 262.956, 'duration': 2.841}, {'end': 273.799, 'text': 'OpenAI has exposed a new parameter called functions, through which you can pass a list of function definitions.', 'start': 266.377, 'duration': 7.422}, {'end': 277.841, 'text': 'The full function definition for the above is right here.', 'start': 274.839, 'duration': 3.002}, {'end': 282.104, 'text': "As you can see, it's a list and then the element.", 'start': 278.722, 'duration': 3.382}, {'end': 288.748, 'text': "in the list there's only one, because we're only passing one function, and here it is this JSON object with a few different parameters.", 'start': 282.104, 'duration': 6.644}, {'end': 292.29, 'text': "You've got a name parameter and this is the name of the function.", 'start': 289.529, 'duration': 2.761}], 'summary': "Pass information to language model using openai's new 'functions' parameter.", 'duration': 39.116, 'max_score': 253.174, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY253174.jpg'}, {'end': 401.864, 'src': 'embed', 'start': 362.202, 'weight': 4, 'content': [{'end': 365.708, 'text': "First, we're going to create a list of messages to pass to the language model.", 'start': 362.202, 'duration': 3.506}, {'end': 385.032, 'text': "Let's make the first message we pass something that's related to the above function.", 'start': 381.249, 'duration': 3.783}, {'end': 386.433, 'text': "So let's make it a question about weather.", 'start': 385.052, 'duration': 1.381}, {'end': 393.258, 'text': "What's the weather like in Boston? Let's then import the OpenAI SDK and call the chat completion endpoint.", 'start': 386.693, 'duration': 6.565}, {'end': 401.864, 'text': "First, we're going to specify the model.", 'start': 400.263, 'duration': 1.601}], 'summary': 'Creating a list of messages for language model testing, including a weather-related question for boston, and using openai sdk for chat completion.', 'duration': 39.662, 'max_score': 362.202, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY362202.jpg'}, {'end': 636.329, 'src': 'embed', 'start': 602.406, 'weight': 6, 'content': [{'end': 604.866, 'text': 'That additional parameter is the function call parameter.', 'start': 602.406, 'duration': 2.46}, {'end': 607.427, 'text': "By default, it's set to auto.", 'start': 605.407, 'duration': 2.02}, {'end': 609.587, 'text': 'This means that the language model chooses.', 'start': 607.807, 'duration': 1.78}, {'end': 611.168, 'text': "This is what we've been doing so far.", 'start': 609.727, 'duration': 1.441}, {'end': 617.709, 'text': "There's two other modes that we can use it in.", 'start': 615.649, 'duration': 2.06}, {'end': 621.03, 'text': 'In the first mode, we can force it to call a function.', 'start': 618.169, 'duration': 2.861}, {'end': 627.171, 'text': "This is good if we always want to return the function, and we'll see some use cases for that later on in the lesson.", 'start': 621.97, 'duration': 5.201}, {'end': 629.404, 'text': 'As you can see,', 'start': 628.243, 'duration': 1.161}, {'end': 636.329, 'text': "because we're using auto and we're letting the language model choose what to do here it recognizes that it doesn't need to call the function,", 'start': 629.404, 'duration': 6.925}], 'summary': "Language model defaults to 'auto' parameter; two other modes available for function call.", 'duration': 33.923, 'max_score': 602.406, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY602406.jpg'}, {'end': 800.332, 'src': 'embed', 'start': 773.976, 'weight': 7, 'content': [{'end': 780.144, 'text': 'So if we run this, we can see that we get back prompt tokens 89.', 'start': 773.976, 'duration': 6.168}, {'end': 789.588, 'text': 'If we comment out functions and function call, we can see that prompt tokens goes down to 15.', 'start': 780.144, 'duration': 9.444}, {'end': 795.57, 'text': 'In this particular example, the function definition and the function description are taking up a lot of the tokens.', 'start': 789.588, 'duration': 5.982}, {'end': 800.332, 'text': 'This is important to note because OpenAI models have a token limit on them.', 'start': 795.971, 'duration': 4.361}], 'summary': "Running the code returns 89 prompt tokens, but commenting out functions reduces it to 15, indicating functions are consuming a significant number of tokens, which is important due to openai's token limit.", 'duration': 26.356, 'max_score': 773.976, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY773976.jpg'}, {'end': 855.122, 'src': 'embed', 'start': 820.46, 'weight': 8, 'content': [{'end': 827.702, 'text': 'This is important because oftentimes you want to use the language model to determine what function to call, then run that function,', 'start': 820.46, 'duration': 7.242}, {'end': 831.243, 'text': 'but then pass it back into the language model to get a final response.', 'start': 827.702, 'duration': 3.541}, {'end': 839.185, 'text': "So let's run this again and we'll get back this response that has this function call parameter in it.", 'start': 833.663, 'duration': 5.522}, {'end': 855.122, 'text': "We'll then take this message and we'll append it to our list of messages.", 'start': 850.993, 'duration': 4.129}], 'summary': 'Using a language model to determine functions, run them, and append responses to a list of messages.', 'duration': 34.662, 'max_score': 820.46, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY820460.jpg'}], 'start': 166.563, 'title': 'Openai function calling', 'summary': 'Introduces new capabilities of function calling with OpenAI API, explaining usage, tips, and focus on fine-tuned models and additional parameters. It also covers defining and using functions, passing parameters, creating message lists, analyzing responses, and using OpenAI SDK for function calling with various modes and token usage limits.', 'chapters': [{'end': 277.841, 'start': 166.563, 'title': 'Using openai api for function calling', 'summary': 'Introduces the new capability of function calling added to the openai api, explaining how to use it and providing tips for best results, with a focus on fine-tuned models and additional parameters.', 'duration': 111.278, 'highlights': ['OpenAI has fine-tuned some of the most recent models to accept additional parameters for function calling. OpenAI has updated models to accept extra parameters for function calling, enhancing their capability.', "We're going to imagine that we have a function that we think is interesting to provide to the language model. The tutorial discusses providing interesting functions to the language model, showcasing the potential use cases for this new parameter.", "OpenAI has exposed a new parameter called functions, through which you can pass a list of function definitions. OpenAI has introduced a new parameter 'functions' for passing function definitions, providing a way to integrate external knowledge sources with the language model."]}, {'end': 540.667, 'start': 278.722, 'title': 'Using functions with openai', 'summary': 'Explains the process of defining and using functions with openai, including passing parameters, creating a list of messages, and analyzing the response from a function call.', 'duration': 261.945, 'highlights': ['The chapter explains the process of defining and using functions with OpenAI The transcript provides an overview of defining and using functions with OpenAI, demonstrating the process of passing parameters and creating a list of messages to use in a call to the language model.', 'Creating a list of messages to pass to the language model It details the creation of a list of messages to pass to the language model, including an example message related to a specific function, such as a weather-related question about Boston.', 'Analyzing the response from a function call The transcript explains the process of analyzing the response obtained from a function call, including the information related to the function name, arguments, and the utilization of JSON dictionaries.']}, {'end': 970.626, 'start': 540.667, 'title': 'Openai function calling', 'summary': 'Discusses how to use the openai sdk for function calling, including the auto, force, and none modes for function call parameter, and the token usage limit implications, demonstrating the process of calling functions and passing the results back to the language model.', 'duration': 429.959, 'highlights': ["The chapter discusses the auto, force, and none modes for the function call parameter, demonstrating how the language model chooses whether to use a function or not. The chapter explains the default 'auto' mode for the function call parameter, where the language model chooses whether to use a function or not.", 'It explains the implications of the token usage limit, indicating that functions and their descriptions count against the token usage limit. The chapter highlights that functions and their descriptions count against the token usage limit and provides an example where the function definition and description consume a significant number of tokens.', "The chapter demonstrates the process of calling functions and passing the results back to the language model, showcasing a practical example of using the language model to generate a natural language response based on the function's output. It provides a practical example of calling the function, passing the result back to the language model, and generating a natural language response based on the function's output."]}], 'duration': 804.063, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY166563.jpg', 'highlights': ['OpenAI has fine-tuned models to accept additional parameters for function calling, enhancing their capability.', 'The tutorial discusses providing interesting functions to the language model, showcasing the potential use cases for this new parameter.', "OpenAI has introduced a new parameter 'functions' for passing function definitions, providing a way to integrate external knowledge sources with the language model.", 'The chapter explains the process of defining and using functions with OpenAI, demonstrating the process of passing parameters and creating a list of messages to use in a call to the language model.', 'It details the creation of a list of messages to pass to the language model, including an example message related to a specific function, such as a weather-related question about Boston.', 'The transcript explains the process of analyzing the response obtained from a function call, including the information related to the function name, arguments, and the utilization of JSON dictionaries.', "The chapter explains the default 'auto' mode for the function call parameter, where the language model chooses whether to use a function or not.", 'The chapter highlights that functions and their descriptions count against the token usage limit and provides an example where the function definition and description consume a significant number of tokens.', "It provides a practical example of calling the function, passing the result back to the language model, and generating a natural language response based on the function's output."]}, {'end': 1999.124, 'segs': [{'end': 1024.109, 'src': 'embed', 'start': 997.472, 'weight': 0, 'content': [{'end': 1002.935, 'text': 'A lot of the power of link chain comes from composing chains of different components together.', 'start': 997.472, 'duration': 5.463}, {'end': 1004.616, 'text': 'And we have a new way of doing this.', 'start': 1003.515, 'duration': 1.101}, {'end': 1009.259, 'text': 'And this is LCEL and a runnable protocol which define a few things.', 'start': 1004.856, 'duration': 4.403}, {'end': 1016.443, 'text': 'First, it defines an allowed set of input types and a corresponding allowed set of output types.', 'start': 1009.779, 'duration': 6.664}, {'end': 1024.109, 'text': "there's then a bunch of standard methods and we'll go over all of these, but these are standard methods that all runnables will expose,", 'start': 1017.043, 'duration': 7.066}], 'summary': 'Lcel introduces a new way to compose link chain components, defining input and output types and standard methods for all runnables.', 'duration': 26.637, 'max_score': 997.472, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY997472.jpg'}, {'end': 1083.526, 'src': 'embed', 'start': 1036.801, 'weight': 2, 'content': [{'end': 1043.286, 'text': "so it'll look something like this What is this interface that we expect all runnables to expose?", 'start': 1036.801, 'duration': 6.485}, {'end': 1045.768, 'text': 'First, there are a few common methods.', 'start': 1043.767, 'duration': 2.001}, {'end': 1049.711, 'text': "There's invoke, which calls the runnable on a single input.", 'start': 1046.328, 'duration': 3.383}, {'end': 1054.754, 'text': "There's stream, which calls it on a single input in stream's backer response.", 'start': 1050.371, 'duration': 4.383}, {'end': 1058.676, 'text': "And then there's batch, which calls it on a list of inputs.", 'start': 1055.334, 'duration': 3.342}, {'end': 1063.68, 'text': "For all of these synchronous methods, there's also a corresponding async method.", 'start': 1059.317, 'duration': 4.363}, {'end': 1066.602, 'text': "So there's a invoke, a stream, a batch.", 'start': 1063.82, 'duration': 2.782}, {'end': 1072.562, 'text': "There's common properties that all runnables have, namely an input schema and an output schema.", 'start': 1067.54, 'duration': 5.022}, {'end': 1077.944, 'text': 'And you can see a list of the input and output types for common components below.', 'start': 1073.122, 'duration': 4.822}, {'end': 1083.526, 'text': 'Why use LCL? There are a few benefits that this gets you.', 'start': 1079.244, 'duration': 4.282}], 'summary': 'Common methods like invoke, stream, batch with corresponding async methods. all runnables have input and output schema.', 'duration': 46.725, 'max_score': 1036.801, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY1036800.jpg'}, {'end': 1132.772, 'src': 'embed', 'start': 1108.007, 'weight': 4, 'content': [{'end': 1115.933, 'text': 'and so you can easily attach fallbacks not only to llms but also to entire chains, so you can have a fallback safety mechanism.', 'start': 1108.007, 'duration': 7.926}, {'end': 1119.095, 'text': 'third, parallelism llm calls can be time consuming.', 'start': 1115.933, 'duration': 3.162}, {'end': 1127.061, 'text': "they often take a while, and so it's really important to run them in parallel if you can, and the lcel syntax makes it easy to do that.", 'start': 1119.095, 'duration': 7.966}, {'end': 1129.831, 'text': 'And fourth, logging is built in.', 'start': 1127.99, 'duration': 1.841}, {'end': 1132.772, 'text': 'As these chains and agents get more and more complex,', 'start': 1130.211, 'duration': 2.561}], 'summary': 'Llm supports fallbacks, parallelism, and built-in logging for complex chains and agents.', 'duration': 24.765, 'max_score': 1108.007, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY1108007.jpg'}, {'end': 1322.953, 'src': 'embed', 'start': 1292.38, 'weight': 5, 'content': [{'end': 1296.123, 'text': "And so we'll replicate that same process using link chain expression language.", 'start': 1292.38, 'duration': 3.743}, {'end': 1299.224, 'text': 'First, we need to set up our retriever.', 'start': 1297.603, 'duration': 1.621}, {'end': 1301.867, 'text': "We're first going to create a really simple vector store.", 'start': 1299.725, 'duration': 2.142}, {'end': 1304.289, 'text': "We're going to do this by initializing it with two texts.", 'start': 1301.907, 'duration': 2.382}, {'end': 1308.172, 'text': 'First, Harrison worked at Kensho, and next, bears like to eat honey.', 'start': 1304.609, 'duration': 3.563}, {'end': 1315.118, 'text': "We'll then also pass in an embedding model, OpenAI embeddings, and then we'll create a retriever by doing vector store.asRetriever.", 'start': 1308.752, 'duration': 6.366}, {'end': 1322.953, 'text': 'As a reminder from the previous lesson, we can now call dot get relevant documents on this retriever, and we get back relevant documents.', 'start': 1316.147, 'duration': 6.806}], 'summary': 'Using link chain expression language to set up vector store with two texts and openai embeddings, creating a retriever to get relevant documents.', 'duration': 30.573, 'max_score': 1292.38, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY1292380.jpg'}, {'end': 1496.664, 'src': 'embed', 'start': 1465.286, 'weight': 6, 'content': [{'end': 1467.947, 'text': 'So we can just compose all those together with pipes.', 'start': 1465.286, 'duration': 2.661}, {'end': 1473.469, 'text': 'We can then name this as our chain, and we now have a new runnable.', 'start': 1468.547, 'duration': 4.922}, {'end': 1489.14, 'text': 'And now if we invoke this with a question like where did Harrison work, we can see that it responds with Harrison worked at Kensho.', 'start': 1483.277, 'duration': 5.863}, {'end': 1496.664, 'text': 'If we want to take a closer look at what exactly is going on under the scenes, we can just look at this runnable map directly.', 'start': 1490.561, 'duration': 6.103}], 'summary': 'Creating a chain with pipes, resulting in a new runnable that responds with harrison worked at kensho when invoked.', 'duration': 31.378, 'max_score': 1465.286, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY1465286.jpg'}, {'end': 1841.18, 'src': 'embed', 'start': 1817.836, 'weight': 7, 'content': [{'end': 1826.181, 'text': 'what we can do is we can create a final chain and we can start with the simple chain and then we can add fallbacks by using with fallbacks.', 'start': 1817.836, 'duration': 8.345}, {'end': 1832.965, 'text': "In with fallbacks, we'll pass in a list of other runnables.", 'start': 1829.003, 'duration': 3.962}, {'end': 1837.619, 'text': 'And basically what will happen is that first, The core runnable tries to get run.', 'start': 1833.466, 'duration': 4.153}, {'end': 1841.18, 'text': 'If an error is raised, then it goes through this list in order.', 'start': 1838.059, 'duration': 3.121}], 'summary': 'Create a final chain with fallbacks to handle errors and execute runnables.', 'duration': 23.344, 'max_score': 1817.836, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY1817836.jpg'}, {'end': 2005.249, 'src': 'embed', 'start': 1978.126, 'weight': 9, 'content': [{'end': 1985.552, 'text': "In the next lesson we're going to combine what we just learned here lane chain expression language with what we covered in the previous lesson open AI function calls.", 'start': 1978.126, 'duration': 7.426}, {'end': 1988.315, 'text': "And we're going to show how to use those two things together.", 'start': 1986.133, 'duration': 2.182}, {'end': 1999.124, 'text': "In this lesson we'll combine what we learned in the previous two lessons and go over how to use open AI functions with lane chain expression language.", 'start': 1991.758, 'duration': 7.366}, {'end': 2005.249, 'text': "We'll also introduce Pydantic, a Python library that makes it easier to construct open AI functions.", 'start': 1999.684, 'duration': 5.565}], 'summary': 'Combining lane chain expression language with open ai function calls and introducing pydantic.', 'duration': 27.123, 'max_score': 1978.126, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY1978126.jpg'}], 'start': 971.286, 'title': 'Lane chain expression language (lcel) and its benefits', 'summary': 'Introduces lane chain expression language (lcel) for composing chains of components, defining input/output types, and demonstrates its benefits including async, batch, streaming support, fallback mechanism, and parallelism.', 'chapters': [{'end': 1016.443, 'start': 971.286, 'title': 'Lcel: lane chain expression language', 'summary': 'Introduces lane chain expression language (lcel) which provides a new syntax for composing chains of different components, defining allowed input and output types, aiming to make it easier and faster to use this functionality.', 'duration': 45.157, 'highlights': ['LCEL is a new syntax that makes it much easier and more transparent to construct and work with different chains and agents, aiming to make it easier and faster to use this functionality.', 'It defines an allowed set of input types and a corresponding allowed set of output types, providing clarity and structure for composing chains of different components.']}, {'end': 1275.307, 'start': 1017.043, 'title': 'Lcl interface and benefits', 'summary': 'Introduces the lcl interface, which includes standard methods like invoke, stream, and batch, along with their corresponding async methods, common properties of all runnables, and the benefits of using lcl including async, batch, and streaming support, fallback mechanism, parallelism, and built-in logging.', 'duration': 258.264, 'highlights': ['The LCL interface includes standard methods like invoke, stream, and batch, along with their corresponding async methods. The LCL interface includes standard methods like invoke, stream, and batch, along with their corresponding async methods.', 'Common properties of all runnables include an input schema and an output schema. Common properties of all runnables include an input schema and an output schema.', 'Benefits of using LCL include async, batch, and streaming support, fallback mechanism, parallelism, and built-in logging. Benefits of using LCL include async, batch, and streaming support, fallback mechanism, parallelism, and built-in logging.']}, {'end': 1999.124, 'start': 1275.307, 'title': 'Creating complex chains with link chain expression language', 'summary': 'Demonstrates using link chain expression language to create complex chains for retrieval augmented generation, including setting up retrievers, creating pipelines, binding parameters, handling fallbacks, and interfacing with runnables for various asynchronous tasks.', 'duration': 723.817, 'highlights': ['Demonstrating retrieval augmented generation using Link Chain Expression Language The chapter discusses setting up retrievers with vector stores and embedding models, retrieving relevant documents, creating a prompt for language models, and using the retrieved context in a generation pipeline.', 'Creating and composing runnables for various components in the chain The chapter covers creating a runnable map to fetch relevant context, composing components such as prompt, model, and output parser using pipes, and binding parameters to runnables for language model functions.', 'Handling fallbacks and error handling in chains The chapter explains adding fallbacks to handle errors in chains, using with fallbacks to specify alternate runnables, and demonstrates how the system falls back to a different chain when an error occurs.', 'Exploring the interface of runnables for synchronous and asynchronous tasks The chapter discusses invoking runnables synchronously with invoke, calling batch on multiple inputs in parallel, streaming back responses with chain.stream, and using asynchronous methods like chain.ainvoke for asynchronous tasks.', 'Combining Link Chain Expression Language with Open AI function calls The chapter previews combining Link Chain Expression Language with Open AI function calls in the next lesson, highlighting the integration of the two concepts for advanced applications.']}], 'duration': 1027.838, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY971286.jpg', 'highlights': ['LCEL defines allowed input and output types for composing chains.', 'LCEL provides clarity and structure for composing chains of components.', 'LCEL interface includes standard methods like invoke, stream, and batch.', 'Common properties of all runnables include input and output schema.', 'Benefits of using LCEL include async, batch, streaming support, fallback mechanism, parallelism, and built-in logging.', 'Demonstrates retrieval augmented generation using Link Chain Expression Language.', 'Creating and composing runnables for various components in the chain.', 'Handling fallbacks and error handling in chains.', 'Exploring the interface of runnables for synchronous and asynchronous tasks.', 'Combining Link Chain Expression Language with Open AI function calls.']}, {'end': 2721.275, 'segs': [{'end': 2025.441, 'src': 'embed', 'start': 1999.684, 'weight': 0, 'content': [{'end': 2005.249, 'text': "We'll also introduce Pydantic, a Python library that makes it easier to construct open AI functions.", 'start': 1999.684, 'duration': 5.565}, {'end': 2006.45, 'text': "Let's get to it.", 'start': 2005.93, 'duration': 0.52}, {'end': 2011.675, 'text': 'So what is Pydantic? Pydantic is a data validation library for Python.', 'start': 2007.813, 'duration': 3.862}, {'end': 2014.856, 'text': 'It makes it really easy to define different schemas.', 'start': 2012.375, 'duration': 2.481}, {'end': 2018.578, 'text': 'And then it also makes it really easy to export those schemas to JSON.', 'start': 2014.956, 'duration': 3.622}, {'end': 2025.441, 'text': "That's going to come in handy because we can use the Pydantic objects to create the OpenAI functions descriptions.", 'start': 2019.038, 'duration': 6.403}], 'summary': 'Introducing pydantic, a python library for data validation and schema definition, facilitating easy json export.', 'duration': 25.757, 'max_score': 1999.684, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY1999684.jpg'}, {'end': 2182.824, 'src': 'embed', 'start': 2145.385, 'weight': 1, 'content': [{'end': 2152.735, 'text': 'Now if we try to pass in an invalid argument for age, if we try to pass in a string of value bar, we can see that it raises a validation error.', 'start': 2145.385, 'duration': 7.35}, {'end': 2154.878, 'text': 'So this is something that Pydantic is doing under the hood.', 'start': 2152.916, 'duration': 1.962}, {'end': 2160.727, 'text': "It's actually doing some additional validation on the inputs that we pass in, which is another nice benefit of Pydantic.", 'start': 2154.919, 'duration': 5.808}, {'end': 2165.555, 'text': 'One other thing we can do with Pydantic is we can actually nest these data structures.', 'start': 2161.693, 'duration': 3.862}, {'end': 2171.437, 'text': "So here we're going to define a class type, and this is a Pydantic model, so it's inheriting from base model.", 'start': 2166.295, 'duration': 5.142}, {'end': 2178.88, 'text': 'And then the only element on this class is going to be students, and students is going to be a list of puser, which we defined above.', 'start': 2172.057, 'duration': 6.823}, {'end': 2182.824, 'text': 'And now we can create an object that has this exact structure.', 'start': 2180.103, 'duration': 2.721}], 'summary': 'Pydantic provides additional validation on inputs, enabling nesting of data structures.', 'duration': 37.439, 'max_score': 2145.385, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2145385.jpg'}, {'end': 2651.254, 'src': 'embed', 'start': 2606.115, 'weight': 4, 'content': [{'end': 2614.78, 'text': "We'll then create a chain by just piping together prompt and the model with the function, so we'll still let it decide what to do.", 'start': 2606.115, 'duration': 8.665}, {'end': 2627.852, 'text': 'And now we can do chain.invoke with input what is the weather in SF.', 'start': 2619.545, 'duration': 8.307}, {'end': 2632.316, 'text': 'And it pipes it through the prompt and then to the model at the end.', 'start': 2629.093, 'duration': 3.223}, {'end': 2640.403, 'text': 'Our next step up is we can pass in a list of functions and let the language model decide which one to use based on the question context.', 'start': 2633.657, 'duration': 6.746}, {'end': 2645.89, 'text': "So here we're going to create another Pydantic model, this one called artist search,", 'start': 2641.707, 'duration': 4.183}, {'end': 2651.254, 'text': "and we're going to have the description and the usage be call this to get the names of songs by a particular artist.", 'start': 2645.89, 'duration': 5.364}], 'summary': 'Creating a chain to let the model decide what to do, invoking with an input, and passing in a list of functions for language model decision.', 'duration': 45.139, 'max_score': 2606.115, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2606115.jpg'}, {'end': 2696.615, 'src': 'embed', 'start': 2668.919, 'weight': 3, 'content': [{'end': 2679.732, 'text': "and we're going to create a new object called model with functions and we're going to do model.bind and we're going to attach the functions here.", 'start': 2668.919, 'duration': 10.813}, {'end': 2685.629, 'text': "And now let's try invoking this with different inputs and see what happens.", 'start': 2681.687, 'duration': 3.942}, {'end': 2693.033, 'text': 'So first ask it, what is the weather in SF? And here it responds.', 'start': 2686.47, 'duration': 6.563}, {'end': 2694.774, 'text': "It's going to do a function call.", 'start': 2693.113, 'duration': 1.661}, {'end': 2696.615, 'text': "It's going to call the weather search tool.", 'start': 2694.794, 'duration': 1.821}], 'summary': 'Creating a new model object with functions and testing with different inputs, including weather search tool invocation.', 'duration': 27.696, 'max_score': 2668.919, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2668919.jpg'}], 'start': 1999.684, 'title': 'Pydantic and langchain integration', 'summary': 'Introduces pydantic for openai functions, explaining its use and benefits, and discusses combining openai functions with langchain, including creating and binding functions to models, invoking functions, using function invocations in a chain, and letting the language model decide based on the question context.', 'chapters': [{'end': 2273.304, 'start': 1999.684, 'title': 'Introduction to pydantic for openai functions', 'summary': 'Introduces pydantic, a data validation library for python, explaining its use in simplifying the creation of openai function descriptions and demonstrating its benefits in data validation and nesting data structures.', 'duration': 273.62, 'highlights': ['Pydantic simplifies the creation of OpenAI function descriptions by defining schemas and exporting them to JSON, easing the process of dealing with complex JSON structures. Pydantic simplifies the creation of OpenAI function descriptions by defining schemas and exporting them to JSON, making it easier to handle complex JSON structures.', 'Pydantic performs data validation, preventing the creation of objects with invalid data and raising validation errors when incorrect values are provided. Pydantic performs data validation, preventing the creation of objects with invalid data and raising validation errors when incorrect values are provided.', 'Pydantic allows nesting of data structures, enabling the creation of complex objects with nested Pydantic models, providing a flexible approach to data modeling. Pydantic allows nesting of data structures, enabling the creation of complex objects with nested Pydantic models, providing a flexible approach to data modeling.']}, {'end': 2721.275, 'start': 2279.73, 'title': 'Combining openai functions with langchain', 'summary': 'Discusses combining openai functions with langchain, including creating and binding functions to models, invoking functions, using function invocations in a chain, and letting the language model decide which function to use based on the question context.', 'duration': 441.545, 'highlights': ["Creating and binding functions to models It explains how to create and bind functions to models, such as using the 'model.bind functions' method to attach functions to a model.", 'Using function invocations in a chain It demonstrates how to use function invocations in a chain by piping together prompt and the model with the function to let it decide what to do.', 'Letting the language model decide which function to use based on the question context It discusses the capability of passing in a list of functions and letting the language model decide which one to use based on the question context, as shown with the examples of weather search and artist search.']}], 'duration': 721.591, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY1999684.jpg', 'highlights': ['Pydantic simplifies the creation of OpenAI function descriptions by defining schemas and exporting them to JSON, easing the process of dealing with complex JSON structures.', 'Pydantic performs data validation, preventing the creation of objects with invalid data and raising validation errors when incorrect values are provided.', 'Pydantic allows nesting of data structures, enabling the creation of complex objects with nested Pydantic models, providing a flexible approach to data modeling.', "Creating and binding functions to models It explains how to create and bind functions to models, such as using the 'model.bind functions' method to attach functions to a model.", 'Using function invocations in a chain It demonstrates how to use function invocations in a chain by piping together prompt and the model with the function to let it decide what to do.', 'Letting the language model decide which function to use based on the question context It discusses the capability of passing in a list of functions and letting the language model decide which one to use based on the question context, as shown with the examples of weather search and artist search.']}, {'end': 3108.808, 'segs': [{'end': 2776.939, 'src': 'embed', 'start': 2721.375, 'weight': 0, 'content': [{'end': 2726.516, 'text': "So if we just say, hi, it should respond with something that doesn't use functions at all.", 'start': 2721.375, 'duration': 5.141}, {'end': 2731.636, 'text': "So hello, how can I assist you today? Now's a good point to pause and try this out.", 'start': 2726.716, 'duration': 4.92}, {'end': 2738.44, 'text': 'Try passing in a few different Pydantic objects to convert Pydantic to OpenAI functions, getting a list of multiple functions,', 'start': 2731.717, 'duration': 6.723}, {'end': 2743.003, 'text': 'binding those to a model and then passing in various inputs and seeing what the response turns out to be.', 'start': 2738.44, 'duration': 4.563}, {'end': 2748.887, 'text': "That's it for an overview of using Langchain Expression Language and OpenAI functions together.", 'start': 2743.924, 'duration': 4.963}, {'end': 2756.231, 'text': "In the next lesson, we're going to talk about one of the main use cases for this type of functionality, tagging and extraction.", 'start': 2749.907, 'duration': 6.324}, {'end': 2766.891, 'text': "In this lesson, we'll go over one of the main use cases that we see developers using OpenAI functions for, tagging and extraction.", 'start': 2759.608, 'duration': 7.283}, {'end': 2771.673, 'text': 'This allows us to extract structured data from unstructured text.', 'start': 2767.632, 'duration': 4.041}, {'end': 2772.934, 'text': "Let's have some fun.", 'start': 2772.334, 'duration': 0.6}, {'end': 2776.939, 'text': "The first use case that we'll cover is tagging.", 'start': 2774.537, 'duration': 2.402}], 'summary': 'Using langchain expression language and openai functions together for tagging and extraction.', 'duration': 55.564, 'max_score': 2721.375, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2721375.jpg'}, {'end': 2895.761, 'src': 'embed', 'start': 2867.714, 'weight': 4, 'content': [{'end': 2870.716, 'text': 'So our import list from typing to help us with type hints.', 'start': 2867.714, 'duration': 3.002}, {'end': 2873.277, 'text': "We're going to import base model and field from Pydantic.", 'start': 2870.736, 'duration': 2.541}, {'end': 2876.839, 'text': "And then we're going to import our convert Pydantic to open AI function.", 'start': 2873.737, 'duration': 3.102}, {'end': 2882.789, 'text': "Let's now create a pydantic model that we're going to use for tagging.", 'start': 2879.266, 'duration': 3.523}, {'end': 2883.77, 'text': "So we're going to call it tagging.", 'start': 2882.829, 'duration': 0.941}, {'end': 2887.694, 'text': "We're going to have a description where we just say tag the piece of text with particular info.", 'start': 2883.79, 'duration': 3.904}, {'end': 2891.758, 'text': "And then we're going to have a list of the attributes that we want to tag the text with.", 'start': 2888.094, 'duration': 3.664}, {'end': 2892.939, 'text': 'So first sentiment.', 'start': 2892.078, 'duration': 0.861}, {'end': 2895.761, 'text': "And we're going to have the description be the sentiment of the text.", 'start': 2893.699, 'duration': 2.062}], 'summary': 'Import pydantic, create tagging model with description and attributes.', 'duration': 28.047, 'max_score': 2867.714, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2867714.jpg'}, {'end': 2931.619, 'src': 'embed', 'start': 2902.507, 'weight': 5, 'content': [{'end': 2907.23, 'text': "So this is how we're telling the language model what the shape of the data that we're extracting should be.", 'start': 2902.507, 'duration': 4.723}, {'end': 2908.911, 'text': 'And then we also have a language tag.', 'start': 2907.29, 'duration': 1.621}, {'end': 2915.273, 'text': "And here we have the language of the text and we're again saying it should be in this ISO 639-1 code.", 'start': 2909.331, 'duration': 5.942}, {'end': 2922.636, 'text': 'If we then call our convertPydanticToOpenAI function method on this class,', 'start': 2916.393, 'duration': 6.243}, {'end': 2931.619, 'text': 'we can see that we get back a nice JSON blob with a lot of the name and description and parameters and properties that are necessary to pass into OpenAI functions.', 'start': 2922.636, 'duration': 8.983}], 'summary': 'Defining data shape and language tag for openai function', 'duration': 29.112, 'max_score': 2902.507, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2902507.jpg'}, {'end': 2983.996, 'src': 'embed', 'start': 2952.126, 'weight': 6, 'content': [{'end': 2955.369, 'text': "So we're going to import a chat prompt template in our chat open AI model.", 'start': 2952.126, 'duration': 3.243}, {'end': 2962.512, 'text': "We'll create our model really simply by setting temperature equals to zero.", 'start': 2956.345, 'duration': 6.167}, {'end': 2966.317, 'text': "and we're going to do this because when we're tagging things, we generally want it to be pretty deterministic.", 'start': 2962.512, 'duration': 3.805}, {'end': 2974.547, 'text': "We're then going to create our tagging functions which are just going to be a single one at this point.", 'start': 2969, 'duration': 5.547}, {'end': 2983.996, 'text': "And now we need the prompt that we're going to use to tell the language model how to do this.", 'start': 2979.113, 'duration': 4.883}], 'summary': 'Import a chat prompt template, set temperature to zero for deterministic tagging, create a tagging function, and specify the prompt for the language model.', 'duration': 31.87, 'max_score': 2952.126, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2952126.jpg'}], 'start': 2721.375, 'title': 'Using langchain expression language and openai functions', 'summary': 'Covers using langchain expression language and openai functions together, including passing pydantic objects, tagging and extraction use cases, and the process of extracting structured data from unstructured text.', 'chapters': [{'end': 2867.694, 'start': 2721.375, 'title': 'Using langchain expression language and openai functions', 'summary': 'Covers using langchain expression language and openai functions together, including passing pydantic objects, tagging and extraction use cases, and the process of extracting structured data from unstructured text.', 'duration': 146.319, 'highlights': ['The chapter covers using Langchain Expression Language and OpenAI functions together It covers the integration of Langchain Expression Language and OpenAI functions, showcasing the use of Pydantic objects and the process of extracting structured data from unstructured text.', 'The process of extracting structured data from unstructured text Describes the use of OpenAI functions to extract structured data from unstructured text, demonstrating the use cases of tagging and extraction.', 'Tagging and extraction use cases Explains the use cases of tagging and extraction, including the generation of structured output and the extraction of specific entities from text.', 'Passing Pydantic objects to convert Pydantic to OpenAI functions Discusses the utilization of Pydantic objects to convert them into OpenAI functions, emphasizing the process of passing different Pydantic objects to obtain a list of multiple functions.']}, {'end': 3108.808, 'start': 2867.714, 'title': 'Creating pydantic model for tagging', 'summary': 'Discusses creating a pydantic model for tagging text, utilizing the convertpydantictoopenai function to generate necessary data for openai functions, and demonstrating the process of tagging text using a chat open ai model with specific parameters and language settings.', 'duration': 241.094, 'highlights': ["Creating Pydantic model for tagging text The transcript explains the process of creating a Pydantic model called 'tagging' with specific attributes such as sentiment and language.", 'Utilizing convertPydanticToOpenAI function for generating data The function convertPydanticToOpenAI is used to generate a JSON blob containing necessary data for passing into OpenAI functions.', 'Demonstrating tagging text using a chat open AI model The process involves importing a chat prompt template and setting specific parameters for the language model, then demonstrating the tagging function with different languages and sentiments.']}], 'duration': 387.433, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY2721375.jpg', 'highlights': ['The chapter covers using Langchain Expression Language and OpenAI functions together, showcasing the use of Pydantic objects and the process of extracting structured data from unstructured text.', 'Passing Pydantic objects to convert Pydantic to OpenAI functions Discusses the utilization of Pydantic objects to convert them into OpenAI functions, emphasizing the process of passing different Pydantic objects to obtain a list of multiple functions.', 'The process of extracting structured data from unstructured text Describes the use of OpenAI functions to extract structured data from unstructured text, demonstrating the use cases of tagging and extraction.', 'Tagging and extraction use cases Explains the use cases of tagging and extraction, including the generation of structured output and the extraction of specific entities from text.', "Creating Pydantic model for tagging text The transcript explains the process of creating a Pydantic model called 'tagging' with specific attributes such as sentiment and language.", 'Utilizing convertPydanticToOpenAI function for generating data The function convertPydanticToOpenAI is used to generate a JSON blob containing necessary data for passing into OpenAI functions.', 'Demonstrating tagging text using a chat open AI model The process involves importing a chat prompt template and setting specific parameters for the language model, then demonstrating the tagging function with different languages and sentiments.']}, {'end': 4491.734, 'segs': [{'end': 3143.084, 'src': 'embed', 'start': 3109.068, 'weight': 0, 'content': [{'end': 3111.09, 'text': "And we know that we're always going to be extracting the structure.", 'start': 3109.068, 'duration': 2.022}, {'end': 3119.239, 'text': 'And so what we really want to do is add an output parser that takes in this AI message and basically parses out the JSON and just says that,', 'start': 3111.471, 'duration': 7.768}, {'end': 3120.861, 'text': "because that's the only interesting thing here.", 'start': 3119.239, 'duration': 1.622}, {'end': 3123.284, 'text': "We already know that we're going to be calling this function.", 'start': 3120.881, 'duration': 2.403}, {'end': 3126.508, 'text': "So the fact that content is null, that's not interesting to us.", 'start': 3123.705, 'duration': 2.803}, {'end': 3129.271, 'text': "The fact that there's a function calls made, that's not interesting to us.", 'start': 3126.588, 'duration': 2.683}, {'end': 3130.132, 'text': "We're forcing it to do that.", 'start': 3129.291, 'duration': 0.841}, {'end': 3133.496, 'text': "The fact that it's calling the tagging function also not interesting to us,", 'start': 3130.172, 'duration': 3.324}, {'end': 3136.219, 'text': "because we know that it's going to be calling the tagging function because we forced it to.", 'start': 3133.496, 'duration': 2.723}, {'end': 3143.084, 'text': "And so here what we really want is just the value of arguments, which is JSON blob, and it'll be really convenient if that was parsed into JSON,", 'start': 3136.9, 'duration': 6.184}], 'summary': 'Add an output parser to extract json from ai message.', 'duration': 34.016, 'max_score': 3109.068, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY3109068.jpg'}, {'end': 3162.498, 'src': 'embed', 'start': 3136.9, 'weight': 1, 'content': [{'end': 3143.084, 'text': "And so here what we really want is just the value of arguments, which is JSON blob, and it'll be really convenient if that was parsed into JSON,", 'start': 3136.9, 'duration': 6.184}, {'end': 3147.267, 'text': "because it's JSON in this JSON blob and we want to be able to use the individual elements.", 'start': 3143.084, 'duration': 4.183}, {'end': 3151.87, 'text': 'And so we have a nice little output parser in Langchain that can help with exactly that.', 'start': 3147.888, 'duration': 3.982}, {'end': 3162.498, 'text': "It's called JSON output functions parser, and so we'll import it from Langchain output parsers OpenAI functions.", 'start': 3155.433, 'duration': 7.065}], 'summary': 'Langchain offers a json output parser for extracting values from json blobs.', 'duration': 25.598, 'max_score': 3136.9, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY3136900.jpg'}, {'end': 3484.457, 'src': 'embed', 'start': 3440.978, 'weight': 2, 'content': [{'end': 3450.023, 'text': 'We pass in this new output parser and we pass it in with a key name equals people because this is the field that we want to extract.', 'start': 3440.978, 'duration': 9.045}, {'end': 3456.461, 'text': 'Now if we call this on the input again, We can see that now we just have this list.', 'start': 3450.763, 'duration': 5.698}, {'end': 3462.987, 'text': "So it's a minor improvement, but it will make it easier to use downstream if extraction is truly what we care about for this purpose.", 'start': 3457.162, 'duration': 5.825}, {'end': 3467.572, 'text': 'Now is a good time to pause and play around with both tagging and extraction.', 'start': 3463.688, 'duration': 3.884}, {'end': 3470.875, 'text': 'Add some different models, try some different input text.', 'start': 3468.512, 'duration': 2.363}, {'end': 3472.816, 'text': "There's a lot of cool things you can do here.", 'start': 3471.375, 'duration': 1.441}, {'end': 3477.481, 'text': "We're going to put this all together into a slightly more real world example.", 'start': 3474.238, 'duration': 3.243}, {'end': 3484.457, 'text': "So what we're going to do is we're first going to load a real article from the internet.", 'start': 3478.632, 'duration': 5.825}], 'summary': "Using a new output parser with key 'people' for extraction. testing models and input text for improving downstream use.", 'duration': 43.479, 'max_score': 3440.978, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY3440978.jpg'}, {'end': 3867.546, 'src': 'embed', 'start': 3839.867, 'weight': 4, 'content': [{'end': 3843.947, 'text': "And then we're going to pass those pieces of text to the language model individually.", 'start': 3839.867, 'duration': 4.08}, {'end': 3846.148, 'text': "And then we're going to combine all the results at the end.", 'start': 3843.987, 'duration': 2.161}, {'end': 3854.775, 'text': "so let's create some splits by calling split text on the page content of the document, and if we look at how many splits we have,", 'start': 3847.148, 'duration': 7.627}, {'end': 3858.438, 'text': 'we can see that we have 14 different splits,', 'start': 3854.775, 'duration': 3.663}, {'end': 3867.546, 'text': "and so what we're going to try to do now is we're going to create an entire chain using link chain expression language that does everything that i just said we're going to take in this page content.", 'start': 3858.438, 'duration': 9.108}], 'summary': 'Using language model, 14 splits created and combined.', 'duration': 27.679, 'max_score': 3839.867, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY3839867.jpg'}, {'end': 4012.99, 'src': 'embed', 'start': 3981.397, 'weight': 5, 'content': [{'end': 3987.76, 'text': 'So this is constructing a lambda where it takes as input and we want to pass the document as input,', 'start': 3981.397, 'duration': 6.363}, {'end': 3989.901, 'text': 'or rather the page content of the document as input.', 'start': 3987.76, 'duration': 2.141}, {'end': 3991.742, 'text': 'So the x here is going to be a string.', 'start': 3989.961, 'duration': 1.781}, {'end': 3998.666, 'text': "And what we're doing is we're creating a function that takes that string splits and constructs a list of dictionaries,", 'start': 3992.403, 'duration': 6.263}, {'end': 4003.068, 'text': 'where each dictionary is an input corresponding to the split.', 'start': 3998.666, 'duration': 4.402}, {'end': 4012.99, 'text': 'If we play around with this and see what it does, we can call it on a string and we get back a list of dictionaries.', 'start': 4004.508, 'duration': 8.482}], 'summary': 'Constructing a lambda to split a string into a list of dictionaries.', 'duration': 31.593, 'max_score': 3981.397, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY3981397.jpg'}, {'end': 4114.828, 'src': 'embed', 'start': 4087.714, 'weight': 6, 'content': [{'end': 4091.076, 'text': 'We can see that it will take a little bit, but it will respond with the answer.', 'start': 4087.714, 'duration': 3.362}, {'end': 4096.559, 'text': "And so what it's doing is first it's splitting it into 14 sections, and it's then passing that to the extraction chain.", 'start': 4091.116, 'duration': 5.443}, {'end': 4102.542, 'text': "When it passes it to the extraction chain, it's actually parallelizing a lot of those calls automatically.", 'start': 4097.319, 'duration': 5.223}, {'end': 4105.724, 'text': 'The default that it parallelizes by is five calls.', 'start': 4103.162, 'duration': 2.562}, {'end': 4109.685, 'text': "So it's not fully parallelizing, but it's speeding up a significant amount.", 'start': 4106.163, 'duration': 3.522}, {'end': 4114.828, 'text': 'When all of those calls are done, it will then get passed into the final flatten function.', 'start': 4110.446, 'duration': 4.382}], 'summary': 'Data processing splits into 14 sections, parallelizes with 5 calls, speeding up significantly.', 'duration': 27.114, 'max_score': 4087.714, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY4087714.jpg'}, {'end': 4219.998, 'src': 'embed', 'start': 4185.325, 'weight': 7, 'content': [{'end': 4188.246, 'text': 'Try it out on a PDF and extract some information from that.', 'start': 4185.325, 'duration': 2.921}, {'end': 4190.167, 'text': 'Try it out on another web page.', 'start': 4188.805, 'duration': 1.362}, {'end': 4192.568, 'text': 'Mess around with what you want to extract.', 'start': 4190.426, 'duration': 2.142}, {'end': 4198.23, 'text': 'Tagging and extraction are some of the most popular use cases for this structured data extraction.', 'start': 4193.028, 'duration': 5.202}, {'end': 4201.412, 'text': 'And so gaining familiarity with this will go a long way.', 'start': 4198.731, 'duration': 2.681}, {'end': 4205.133, 'text': "In the next class, we'll talk about something slightly different.", 'start': 4202.172, 'duration': 2.961}, {'end': 4210.616, 'text': "We'll talk about using OpenAI functions not for extraction, but for deciding what functions to call.", 'start': 4205.153, 'duration': 5.463}, {'end': 4211.416, 'text': 'See you there.', 'start': 4211.076, 'duration': 0.34}, {'end': 4219.998, 'text': "In this lesson, we'll cover a second big use case of OpenAI functions, tool usage.", 'start': 4214.795, 'duration': 5.203}], 'summary': 'Experiment with structured data extraction from pdf and web pages using openai functions.', 'duration': 34.673, 'max_score': 4185.325, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY4185325.jpg'}], 'start': 3109.068, 'title': 'Ai message parsing and data extraction', 'summary': 'Covers implementing an output parser for ai messages to extract json structure, using tagging and extraction models to retrieve information, splitting text for language model processing, and constructing a lambda function for data extraction, resulting in 14 different splits and a list of papers with titles and authors extracted from documents.', 'chapters': [{'end': 3136.219, 'start': 3109.068, 'title': 'Ai output parser', 'summary': 'Discusses the implementation of an output parser for ai messages to extract json structure, highlighting the exclusion of non-interesting aspects like function calls and null content.', 'duration': 27.151, 'highlights': ['The implementation of an output parser for AI messages to extract the JSON structure.', 'Exclusion of non-interesting aspects like function calls and null content.']}, {'end': 3766.719, 'start': 3136.9, 'title': 'Langchain: tagging and extraction', 'summary': 'Explains how to parse json output for tagging and extraction, then demonstrates the use of tagging and extraction models to retrieve information from text data, including a real-world example of extracting a summary and academic papers from an article.', 'duration': 629.819, 'highlights': ['The chapter introduces a JSON output parser in Langchain for parsing JSON output from arguments, making it convenient for usage with individual elements, and then demonstrates the use of a JSON output parser for tagging and extraction. The JSON output parser in Langchain is used to parse JSON output from arguments, making it convenient for usage with individual elements, demonstrating its relevance for tagging and extraction.', 'The chapter explains the process of defining and extracting multiple pieces of information, such as a person schema and the use of a class called information, and demonstrates the conversion of the class to OpenAI functions for extraction. The process of defining and extracting multiple pieces of information, such as a person schema and the use of a class called information, is explained, along with the demonstration of converting the class to OpenAI functions for extraction.', 'The chapter demonstrates the modification of an extraction chain using a new output parser to extract only the required field, showcasing an improvement in the extraction process. The modification of an extraction chain using a new output parser to extract only the required field is demonstrated, highlighting an improvement in the extraction process.', 'The chapter provides a real-world example of using tagging and extraction models to retrieve a summary and academic papers from a web-based article, showcasing the practical application of the discussed concepts. The practical application of the discussed concepts is showcased through a real-world example of using tagging and extraction models to retrieve a summary and academic papers from a web-based article.']}, {'end': 3980.401, 'start': 3767.079, 'title': 'Text splitting and language model processing', 'summary': 'Covers the process of splitting a long article into smaller pieces of text to be passed to a language model individually, and then combining all the results at the end, resulting in 14 different splits and a method of preparing the splits to be passed into the chain.', 'duration': 213.322, 'highlights': ['The process of splitting a long article into smaller pieces of text to be passed to a language model individually and then combining all the results at the end. 14 different splits', 'A method of preparing the splits to be passed into the chain. N/A', 'Creating a function that can join lists of lists and flattening them for further processing. N/A', 'Defining a function to convert a list of text into a list of dictionaries for processing. N/A']}, {'end': 4491.734, 'start': 3981.397, 'title': 'Constructing lambda for data extraction', 'summary': 'Discusses constructing a lambda function to split and create a list of dictionaries from a string input, passing it through an extraction chain to parallelize the process and return a list of papers with titles and authors extracted from documents.', 'duration': 510.337, 'highlights': ['The lambda function splits the input string and constructs a list of dictionaries, with the number of sections split being 14, and returns a list of papers with titles and authors extracted.', 'The extraction chain parallelizes the process, automatically making multiple calls and speeding up the extraction, with the default parallelization being five calls.', 'Using OpenAI functions for structured data extraction is popular for tagging and extraction use cases, providing familiarity with the process for real-world examples, PDFs, web pages, and gaining proficiency for various extraction tasks.']}], 'duration': 1382.666, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY3109068.jpg', 'highlights': ['The implementation of an output parser for AI messages to extract the JSON structure.', 'The chapter introduces a JSON output parser in Langchain for parsing JSON output from arguments, making it convenient for usage with individual elements.', 'The chapter demonstrates the modification of an extraction chain using a new output parser to extract only the required field, showcasing an improvement in the extraction process.', 'The chapter provides a real-world example of using tagging and extraction models to retrieve a summary and academic papers from a web-based article, showcasing the practical application of the discussed concepts.', 'The process of splitting a long article into smaller pieces of text to be passed to a language model individually and then combining all the results at the end. 14 different splits', 'The lambda function splits the input string and constructs a list of dictionaries, with the number of sections split being 14, and returns a list of papers with titles and authors extracted.', 'The extraction chain parallelizes the process, automatically making multiple calls and speeding up the extraction, with the default parallelization being five calls.', 'Using OpenAI functions for structured data extraction is popular for tagging and extraction use cases, providing familiarity with the process for real-world examples, PDFs, web pages, and gaining proficiency for various extraction tasks.']}, {'end': 5308.287, 'segs': [{'end': 4561.5, 'src': 'embed', 'start': 4519.392, 'weight': 0, 'content': [{'end': 4529.269, 'text': 'And so when we call this on a given tool, We get back a JSON blob which combines all those elements and now have the name,', 'start': 4519.392, 'duration': 9.877}, {'end': 4533.292, 'text': 'the description and then the parameters, which include the properties of the longitude and latitude.', 'start': 4529.269, 'duration': 4.023}, {'end': 4536.054, 'text': 'And this is the format that OpenAI functions expects.', 'start': 4533.332, 'duration': 2.722}, {'end': 4538.256, 'text': 'And again, this tool is callable.', 'start': 4536.775, 'duration': 1.481}, {'end': 4545.862, 'text': 'So if we pass in latitude and longitude there, this is now making a real request to the OpenMedia API and getting back a response.', 'start': 4538.436, 'duration': 7.426}, {'end': 4551.046, 'text': "The second tool that we're going to define is a Wikipedia tool.", 'start': 4548.664, 'duration': 2.382}, {'end': 4552.787, 'text': 'So this is going to search things in Wikipedia.', 'start': 4551.086, 'duration': 1.701}, {'end': 4554.808, 'text': "So it's going to take in a query.", 'start': 4553.708, 'duration': 1.1}, {'end': 4561.5, 'text': "And it's first going to call the wikipedia-python-library.search and it's going to get back a list of pages.", 'start': 4555.816, 'duration': 5.684}], 'summary': 'Two tools defined: 1. openai function with json blob for latitude and longitude. 2. wikipedia tool for searching with python-library.search.', 'duration': 42.108, 'max_score': 4519.392, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY4519392.jpg'}, {'end': 4684.289, 'src': 'embed', 'start': 4662.233, 'weight': 2, 'content': [{'end': 4670.719, 'text': "And so what we're going to show now is we're going to show how you can take one of these open API specs and convert it into a list of opening AI function calls.", 'start': 4662.233, 'duration': 8.486}, {'end': 4674.822, 'text': 'And again, this is really useful because a lot of functionality is behind APIs.', 'start': 4670.779, 'duration': 4.043}, {'end': 4681.527, 'text': 'And so having a general easy way to interact with those APIs is going to prove very useful.', 'start': 4675.403, 'duration': 6.124}, {'end': 4684.289, 'text': "So we're going to import two things.", 'start': 4683.048, 'duration': 1.241}], 'summary': 'Demonstrating conversion of open api specs into opening ai function calls for easier interaction with apis.', 'duration': 22.056, 'max_score': 4662.233, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY4662233.jpg'}, {'end': 4757.219, 'src': 'embed', 'start': 4732.148, 'weight': 4, 'content': [{'end': 4739.711, 'text': "We're going to load the open API spec from this text and we're then going to pass that spec into the open API spec to open AI function.", 'start': 4732.148, 'duration': 7.563}, {'end': 4740.852, 'text': "And we're going to get back two things.", 'start': 4739.811, 'duration': 1.041}, {'end': 4746.954, 'text': "First, we're going to get back the function definitions, the open AI function definitions that we can use.", 'start': 4741.372, 'duration': 5.582}, {'end': 4754.578, 'text': "And then second, we'll also get back a set of callables that we could actually call to invoke those functions because this is a made up spec.", 'start': 4747.475, 'duration': 7.103}, {'end': 4757.219, 'text': "Those callables aren't actually going to be real.", 'start': 4754.878, 'duration': 2.341}], 'summary': 'Loading open api spec to receive function definitions and callables.', 'duration': 25.071, 'max_score': 4732.148, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY4732148.jpg'}, {'end': 4795.508, 'src': 'embed', 'start': 4767.653, 'weight': 5, 'content': [{'end': 4773.799, 'text': 'We first have the list pets function, then we have the create pets function, and then we have the show pet by ID function.', 'start': 4767.653, 'duration': 6.146}, {'end': 4779.645, 'text': "We'll now show how we can use a language model to determine which of these functions to call.", 'start': 4775.08, 'duration': 4.565}, {'end': 4782.368, 'text': "So we're going to import our OpenAI model.", 'start': 4780.246, 'duration': 2.122}, {'end': 4791.065, 'text': "and we're then going to create a simple version of this model and we're going to pass in temperature equals zero again,", 'start': 4784.682, 'duration': 6.383}, {'end': 4795.508, 'text': "because when we're choosing between functions, we probably want to do that in a pretty deterministic way,", 'start': 4791.065, 'duration': 4.443}], 'summary': 'Demonstrating use of language model for function selection.', 'duration': 27.855, 'max_score': 4767.653, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY4767653.jpg'}, {'end': 4881.938, 'src': 'embed', 'start': 4856.199, 'weight': 3, 'content': [{'end': 4863.445, 'text': "We're going to use the OpenAI model to decide which function to invoke, and then we're actually going to do the step of invoking that as well.", 'start': 4856.199, 'duration': 7.246}, {'end': 4871.211, 'text': "This creates something called routing, which is where we're using a language model to determine which path to take and also the inputs to that path.", 'start': 4864.065, 'duration': 7.146}, {'end': 4876.471, 'text': "First, we're going to create the list of OpenAI function specs to use.", 'start': 4873.146, 'duration': 3.325}, {'end': 4881.938, 'text': "So we're going to call format tool to OpenAI functions on our two tools, search Wikipedia and get current temperature.", 'start': 4876.631, 'duration': 5.307}], 'summary': 'Using openai model to route and invoke functions, including search wikipedia and get current temperature.', 'duration': 25.739, 'max_score': 4856.199, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY4856199.jpg'}, {'end': 5045.145, 'src': 'embed', 'start': 5016.92, 'weight': 6, 'content': [{'end': 5020.721, 'text': "First, whether it's just a function call or whether it's a response.", 'start': 5016.92, 'duration': 3.801}, {'end': 5026.302, 'text': 'And then, second, if it is a function call, what is the function that should be called and what is the input?', 'start': 5021.221, 'duration': 5.081}, {'end': 5031.483, 'text': "We'll create a chain by combining the prompt above with the model in this new output parser.", 'start': 5027.522, 'duration': 3.961}, {'end': 5035.823, 'text': "And then we'll invoke the chain on the same input as before.", 'start': 5032.762, 'duration': 3.061}, {'end': 5038.544, 'text': "Let's now take a look at what result looks like.", 'start': 5036.283, 'duration': 2.261}, {'end': 5041.404, 'text': 'So if we look at the type of result, it is an agent action.', 'start': 5038.764, 'duration': 2.64}, {'end': 5045.145, 'text': "And that's because what it's doing is it's going to be calling one of these tools.", 'start': 5041.724, 'duration': 3.421}], 'summary': 'Creating a chain to call a function with input, resulting in agent action.', 'duration': 28.225, 'max_score': 5016.92, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY5016920.jpg'}, {'end': 5151.743, 'src': 'embed', 'start': 5124.886, 'weight': 7, 'content': [{'end': 5128.388, 'text': "in order to do that, we're going to define a route function.", 'start': 5124.886, 'duration': 3.502}, {'end': 5134.452, 'text': 'this route function is going to act on the result of the language model and do the corresponding steps.', 'start': 5128.388, 'duration': 6.064}, {'end': 5138.214, 'text': "so we're going to check if it's an agent finish, and if it is an agent finish,", 'start': 5134.452, 'duration': 3.762}, {'end': 5142.897, 'text': "we're just going to return the values of output if it's not an agent finish.", 'start': 5138.214, 'duration': 4.683}, {'end': 5144.818, 'text': 'so if it is an agent action,', 'start': 5142.897, 'duration': 1.921}, {'end': 5151.743, 'text': "we're going to look up the correct tool to use and then we're then going to run that tool with the tool input as specified.", 'start': 5144.818, 'duration': 6.925}], 'summary': 'Defining a route function to act on language model results for agent actions and finishes.', 'duration': 26.857, 'max_score': 5124.886, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY5124886.jpg'}], 'start': 4492.335, 'title': 'Openai functions and language model usage', 'summary': 'Demonstrates defining openai functions and api integration, including wikipedia and openmedia api. it also covers using openai model for tool invocation, showcasing routing and conversational agent creation.', 'chapters': [{'end': 4731.22, 'start': 4492.335, 'title': 'Openai functions and api integration', 'summary': 'Demonstrates the process of defining and converting tools into openai function definitions, as well as using open api spec to openai function for interacting with apis, with examples of wikipedia and openmedia api integration.', 'duration': 238.885, 'highlights': ['The process of defining and converting tools into OpenAI function definitions is demonstrated, including examples of longitude and latitude parameters, Wikipedia tool, and open API spec integration. The chapter explains the process of defining tools as OpenAI function definitions, such as converting longitude and latitude parameters, creating a Wikipedia tool, and integrating open API spec for interacting with APIs.', 'The integration of Wikipedia tool is illustrated, showcasing its functionality in searching and summarizing information from Wikipedia pages. The chapter provides an example of integrating a Wikipedia tool, which searches and retrieves information from Wikipedia pages, including constructing a list of summaries based on the search query.', 'The utilization of open API spec to OpenAI function is explained, emphasizing its usefulness in interacting with APIs and converting open API specs into a list of OpenAI function calls. The chapter discusses the usage of open API spec to OpenAI function for interacting with APIs, demonstrating the conversion of open API specs into a list of OpenAI function calls, highlighting its general applicability for interacting with APIs.']}, {'end': 5308.287, 'start': 4732.148, 'title': 'Using language model for tool invocation', 'summary': 'Discusses the use of an openai model to determine and invoke different functions, demonstrating routing using language models and creating a conversational agent.', 'duration': 576.139, 'highlights': ['The chapter explains the process of using an OpenAI model to determine and invoke different functions, showcasing routing using language models and creating a conversational agent. determining and invoking functions, routing using language models, creating a conversational agent', 'The process involves loading the open API spec, passing it into the OpenAI function, and getting back function definitions and callables. loading open API spec, obtaining function definitions and callables', 'The chapter demonstrates using the language model to recognize and call specific functions based on input sentences, such as listing pets or showing pet details by ID. using language model to recognize and call specific functions, examples of listing pets and showing pet details', 'Creating a chain by combining the prompt with the model and a new output parser, and then invoking the chain on specific inputs to get desired responses. creating a chain, combining prompt, model, and output parser, invoking chain on specific inputs', "Defining a route function to act on the language model's result and execute corresponding steps, which involves checking for agent finish or action and running the tool accordingly. defining route function, executing corresponding steps based on agent finish or action"]}], 'duration': 815.952, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY4492335.jpg', 'highlights': ['The process of defining and converting tools into OpenAI function definitions is demonstrated, including examples of longitude and latitude parameters, Wikipedia tool, and open API spec integration.', 'The integration of Wikipedia tool is illustrated, showcasing its functionality in searching and summarizing information from Wikipedia pages.', 'The utilization of open API spec to OpenAI function is explained, emphasizing its usefulness in interacting with APIs and converting open API specs into a list of OpenAI function calls.', 'The chapter explains the process of using an OpenAI model to determine and invoke different functions, showcasing routing using language models and creating a conversational agent.', 'The process involves loading the open API spec, passing it into the OpenAI function, and getting back function definitions and callables.', 'The chapter demonstrates using the language model to recognize and call specific functions based on input sentences, such as listing pets or showing pet details by ID.', 'Creating a chain by combining the prompt with the model and a new output parser, and then invoking the chain on specific inputs to get desired responses.', "Defining a route function to act on the language model's result and execute corresponding steps, which involves checking for agent finish or action and running the tool accordingly."]}, {'end': 6295.295, 'segs': [{'end': 5378.568, 'src': 'embed', 'start': 5308.708, 'weight': 0, 'content': [{'end': 5313.61, 'text': 'But you can have others like hard-coded rules, such as the maximum number of iterations or something like that.', 'start': 5308.708, 'duration': 4.902}, {'end': 5319.193, 'text': "In this lab, we're going to use the tools that we built in the previous lesson.", 'start': 5315.471, 'duration': 3.722}, {'end': 5325.716, 'text': "We're going to take that concept of routing and choosing and then calling the tools, and we're going to put that into our own agent loop.", 'start': 5319.693, 'duration': 6.023}, {'end': 5327.616, 'text': 'using the link chain expression language.', 'start': 5326.235, 'duration': 1.381}, {'end': 5335.158, 'text': "We're then going to use the agent executor class in link chain and show how that corresponds to the agent loop, but also,", 'start': 5328.876, 'duration': 6.282}, {'end': 5338.52, 'text': 'as in other things like error handling, early stopping and tracing.', 'start': 5335.158, 'duration': 3.362}, {'end': 5340.28, 'text': "Let's code it up.", 'start': 5339.6, 'duration': 0.68}, {'end': 5342.401, 'text': "First thing we're going to do.", 'start': 5341.441, 'duration': 0.96}, {'end': 5348.583, 'text': "we're going to set up the environment per usual, and then we're also going to set up the same two tools that we used in the previous lesson.", 'start': 5342.401, 'duration': 6.182}, {'end': 5350.844, 'text': "So we're going to import the tool decorator.", 'start': 5349.164, 'duration': 1.68}, {'end': 5355.246, 'text': "We're going to create our current weather tool.", 'start': 5353.445, 'duration': 1.801}, {'end': 5368.264, 'text': "We're going to create our Wikipedia tool, and then we're going to create our list of tools as just a list of these two tools.", 'start': 5358.663, 'duration': 9.601}, {'end': 5377.087, 'text': "where they're going to go ahead and create the logic that takes in the input and determines what tool to call.", 'start': 5372.401, 'duration': 4.686}, {'end': 5378.568, 'text': 'And so we did this in the previous lesson.', 'start': 5377.127, 'duration': 1.441}], 'summary': "In this lab, we'll integrate tools into an agent loop using link chain, including error handling and early stopping.", 'duration': 69.86, 'max_score': 5308.708, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY5308708.jpg'}, {'end': 5447.302, 'src': 'embed', 'start': 5419.835, 'weight': 6, 'content': [{'end': 5424.476, 'text': "And again, that's because the input here is specifically aimed at calling one of the tools.", 'start': 5419.835, 'duration': 4.641}, {'end': 5429.998, 'text': 'So what we want to do is create this loop that determines what tool to use,', 'start': 5425.457, 'duration': 4.541}, {'end': 5435.059, 'text': 'then calls that tool and passes it back in and repeats until some stopping criteria is met.', 'start': 5429.998, 'duration': 5.061}, {'end': 5437.4, 'text': 'If you look at the prompt,', 'start': 5435.819, 'duration': 1.581}, {'end': 5444.361, 'text': 'what that means is we need a place in the prompt to pass back in this history of tools that are called and the corresponding outputs.', 'start': 5437.4, 'duration': 6.961}, {'end': 5447.302, 'text': "So we're going to have to change the prompt a little bit to add that in.", 'start': 5444.641, 'duration': 2.661}], 'summary': 'Creating a loop to call and pass tools until stopping criteria is met, requiring changes to the prompt.', 'duration': 27.467, 'max_score': 5419.835, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY5419835.jpg'}, {'end': 5739.362, 'src': 'embed', 'start': 5702.855, 'weight': 2, 'content': [{'end': 5709.239, 'text': "otherwise it means it's an agent action and so we're going to look up the correct tool to use.", 'start': 5702.855, 'duration': 6.384}, {'end': 5714.223, 'text': "we're then going to call that tool with that tool input and get back an observation,", 'start': 5709.239, 'duration': 4.984}, {'end': 5721.309, 'text': "and then we're going to append that tuple of result and observation to intermediate steps and continue on our way.", 'start': 5714.223, 'duration': 7.086}, {'end': 5727.172, 'text': "we're going to make one minor modification to this chain before continuing to make it a true agent chain.", 'start': 5721.309, 'duration': 5.863}, {'end': 5732.957, 'text': "And basically what we're gonna do is we're gonna put this format to open AI functions logic in the chain itself.", 'start': 5727.212, 'duration': 5.745}, {'end': 5735.559, 'text': "That's gonna make it a little bit more portable.", 'start': 5733.477, 'duration': 2.082}, {'end': 5739.362, 'text': 'So it just has to take in the user input and the intermediate steps.', 'start': 5735.959, 'duration': 3.403}], 'summary': 'Agents action involves calling tool, getting observation, and appending to steps.', 'duration': 36.507, 'max_score': 5702.855, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY5702855.jpg'}, {'end': 5895.036, 'src': 'embed', 'start': 5866.288, 'weight': 3, 'content': [{'end': 5868.649, 'text': 'We pass it back into the language model and ask it to correct it.', 'start': 5866.288, 'duration': 2.361}, {'end': 5874.328, 'text': 'We can use this pretty simply by just importing agent executor from langchain.agents.', 'start': 5869.605, 'duration': 4.723}, {'end': 5878.23, 'text': 'We can then initialize this with agent equals to the agent chain.', 'start': 5874.728, 'duration': 3.502}, {'end': 5881.893, 'text': 'We pass in the tools because it needs to have the tools to be able to call it.', 'start': 5878.631, 'duration': 3.262}, {'end': 5885.315, 'text': 'And then we pass in verbose equals true to get some nice logging.', 'start': 5882.373, 'duration': 2.942}, {'end': 5891.613, 'text': "If we now call this on what is link chain, we can see that we get some logs of what's going on.", 'start': 5886.949, 'duration': 4.664}, {'end': 5895.036, 'text': 'We call search Wikipedia with query equals to link chain.', 'start': 5891.873, 'duration': 3.163}], 'summary': 'Utilize agent executor from langchain.agents to correct language model and call search wikipedia with query', 'duration': 28.748, 'max_score': 5866.288, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY5866288.jpg'}, {'end': 6114.715, 'src': 'embed', 'start': 6089.234, 'weight': 1, 'content': [{'end': 6094.796, 'text': "So this has the same functionality as before but we've added in this memory component that just remembers previous interactions.", 'start': 6089.234, 'duration': 5.562}, {'end': 6103.258, 'text': 'Now is a good time to pause and try it out on some questions of your own, try adding in some new tools, try adding in a different system prompt.', 'start': 6095.436, 'duration': 7.822}, {'end': 6105.679, 'text': "There's a lot of different things you can play around with here.", 'start': 6103.478, 'duration': 2.201}, {'end': 6111.814, 'text': "For our final bit here, we're going to put it all together and create a nice looking chatbot that we can interact with.", 'start': 6106.653, 'duration': 5.161}, {'end': 6114.715, 'text': "Let's add in another tool.", 'start': 6113.575, 'duration': 1.14}], 'summary': 'Enhanced chatbot with memory component, interactive customization, and new tools.', 'duration': 25.481, 'max_score': 6089.234, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY6089234.jpg'}, {'end': 6287.754, 'src': 'embed', 'start': 6259.89, 'weight': 4, 'content': [{'end': 6262.752, 'text': 'And I really look forward to seeing what you guys build with this technology.', 'start': 6259.89, 'duration': 2.862}, {'end': 6267.577, 'text': 'And that brings this course to a close.', 'start': 6265.495, 'duration': 2.082}, {'end': 6271.58, 'text': "It's a really exciting time in AI, and there's lots of advancements happening.", 'start': 6267.997, 'duration': 3.583}, {'end': 6276.664, 'text': 'In this course, we covered two of them, open AI function calling and lane chain expression language.', 'start': 6272.041, 'duration': 4.623}, {'end': 6283.59, 'text': 'We then showed how you could use those to do structured data extraction, and then also tool usage and tool selection.', 'start': 6277.465, 'duration': 6.125}, {'end': 6287.754, 'text': 'And we put that all together by building a conversational agent at the end.', 'start': 6284.231, 'duration': 3.523}], 'summary': 'Course covered open ai function calling and lane chain expression language for structured data extraction, tool usage, and building a conversational agent.', 'duration': 27.864, 'max_score': 6259.89, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY6259890.jpg'}], 'start': 5308.708, 'title': 'Building ai agent loop and conversational agent', 'summary': 'Covers building an agent loop using link chain expression language, incorporating tools for routing and choosing, and demonstrates the use of the agent executor class. it also discusses creating an ai agent chain, importing tools, creating a model, and implementing a loop. additionally, it covers building conversational agents in langchain, including error handling, conversation memory, and practical guidance to create a fully functional chatbot.', 'chapters': [{'end': 5348.583, 'start': 5308.708, 'title': 'Agent loop and tool execution', 'summary': 'Focuses on building an agent loop using the link chain expression language, incorporating tools for routing and choosing, and demonstrating the use of the agent executor class in link chain for error handling, early stopping, and tracing.', 'duration': 39.875, 'highlights': ['The chapter emphasizes building an agent loop using the Link Chain expression language, incorporating tools for routing and choosing.', 'Demonstrating the use of the agent executor class in Link Chain for error handling, early stopping, and tracing.', 'Setting up the environment and the same two tools used in the previous lesson.']}, {'end': 5847.776, 'start': 5349.164, 'title': 'Creating an ai agent chain', 'summary': 'Discusses the creation of an ai agent chain, including the importing of tools, the creation of a model, the logic for tool selection, and the implementation of a loop to call and process the tool outputs.', 'duration': 498.612, 'highlights': ['Creation of AI agent chain The chapter discusses the process of creating an AI agent chain, including importing tools, creating a model, and implementing logic for tool selection.', 'Logic for tool selection The chapter explains the process of determining what tool to call based on input, creating a loop to call the tool and pass back the corresponding outputs until a stopping criteria is met.', 'Implementation of tool outputs processing loop The chapter details the implementation of a loop to call the tool outputs, determine the correct tool to use, call the tool, and append the result and observation to intermediate steps.']}, {'end': 6295.295, 'start': 5848.037, 'title': 'Building conversational agents with langchain', 'summary': 'Covers the implementation of an agent executor in langchain, demonstrating how to handle error logging, tool errors, and conversation memory to create a conversational agent, using examples and practical guidance to create a fully functional chatbot.', 'duration': 447.258, 'highlights': ['The agent executor handles error logging and tool errors, ensuring smooth error handling for language model outputs and tool calls. It adds better logging, error handling for invalid JSON language model outputs, and error handling for tool errors, ensuring smooth handling of errors.', 'The implementation of conversation memory enables the agent to remember previous interactions, enhancing its conversational capabilities. The addition of a conversation memory mechanism allows the agent to remember previous messages and interactions, thereby improving its conversational abilities.', 'The course concludes with practical guidance on building conversational agents using Langchain, emphasizing the application of open AI functions and Langchain expression language for structured data extraction and tool usage. The course concludes by demonstrating the application of open AI functions and Langchain expression language to build conversational agents, emphasizing structured data extraction and tool usage.']}], 'duration': 986.587, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/Dwpv9raFJxY/pics/Dwpv9raFJxY5308708.jpg', 'highlights': ['Demonstrating the use of the agent executor class in Link Chain for error handling, early stopping, and tracing.', 'The implementation of conversation memory enables the agent to remember previous interactions, enhancing its conversational capabilities.', 'The chapter discusses the process of creating an AI agent chain, including importing tools, creating a model, and implementing logic for tool selection.', 'The agent executor handles error logging and tool errors, ensuring smooth error handling for language model outputs and tool calls.', 'The course concludes with practical guidance on building conversational agents using Langchain, emphasizing the application of open AI functions and Langchain expression language for structured data extraction and tool usage.', 'Setting up the environment and the same two tools used in the previous lesson.', 'Logic for tool selection The chapter explains the process of determining what tool to call based on input, creating a loop to call the tool and pass back the corresponding outputs until a stopping criteria is met.', 'The addition of a conversation memory mechanism allows the agent to remember previous messages and interactions, thereby improving its conversational abilities.', 'The chapter emphasizes building an agent loop using the Link Chain expression language, incorporating tools for routing and choosing.', 'The implementation of a loop to call the tool outputs, determine the correct tool to use, call the tool, and append the result and observation to intermediate steps.']}], 'highlights': ['Langchain is an open-source library that supports any number of different LLMs and provides over 500 integrations to different language models, vector stores, and tools, as well as supporting memory chains and agents.', 'The new capability of function calling, named by OpenAI, expands the capabilities of LLMs, allowing them to extract information from structured or tabular data and make it easier to call other code as a subroutine.', 'The course covers the development of Langchain expression language (LCEL) and how to take advantage of the new function calling capability, enabling the building of tools for LLMs and conversational agents.', 'OpenAI has fine-tuned models to accept additional parameters for function calling, enhancing their capability.', 'The tutorial discusses providing interesting functions to the language model, showcasing the potential use cases for this new parameter.', "OpenAI has introduced a new parameter 'functions' for passing function definitions, providing a way to integrate external knowledge sources with the language model.", 'The chapter explains the process of defining and using functions with OpenAI, demonstrating the process of passing parameters and creating a list of messages to use in a call to the language model.', 'It details the creation of a list of messages to pass to the language model, including an example message related to a specific function, such as a weather-related question about Boston.', 'The transcript explains the process of analyzing the response obtained from a function call, including the information related to the function name, arguments, and the utilization of JSON dictionaries.', 'Pydantic simplifies the creation of OpenAI function descriptions by defining schemas and exporting them to JSON, easing the process of dealing with complex JSON structures.', 'Pydantic performs data validation, preventing the creation of objects with invalid data and raising validation errors when incorrect values are provided.', 'Pydantic allows nesting of data structures, enabling the creation of complex objects with nested Pydantic models, providing a flexible approach to data modeling.', 'The chapter covers using Langchain Expression Language and OpenAI functions together, showcasing the use of Pydantic objects and the process of extracting structured data from unstructured text.', 'The process of extracting structured data from unstructured text Describes the use of OpenAI functions to extract structured data from unstructured text, demonstrating the use cases of tagging and extraction.', 'The chapter introduces a JSON output parser in Langchain for parsing JSON output from arguments, making it convenient for usage with individual elements.', 'The chapter provides a real-world example of using tagging and extraction models to retrieve a summary and academic papers from a web-based article, showcasing the practical application of the discussed concepts.', 'The process of splitting a long article into smaller pieces of text to be passed to a language model individually and then combining all the results at the end. 14 different splits', 'The chapter discusses the process of creating an AI agent chain, including importing tools, creating a model, and implementing logic for tool selection.', 'The implementation of conversation memory enables the agent to remember previous interactions, enhancing its conversational capabilities.', 'The course concludes with practical guidance on building conversational agents using Langchain, emphasizing the application of open AI functions and Langchain expression language for structured data extraction and tool usage.']}