title
19. Build LLM Apps with LangChain js| Andrew Ng | DeepLearning.ai - Full Course

description
The course comes from [https://learn.deeplearning.ai/build-llm-apps-with-langchain-js](https://learn.deeplearning.ai/build-llm-apps-with-langchain-js) created by Andrew Ng This is a course on building Large Language Model (LLM) applications using Langchain-js, taught by Jacob Lee. The course covers common steps for using large language models when building applications, and introduces LineChain.js, a popular open source choreographer that helps JavaScript developers build their LLM applications faster. The course covers the basic building blocks of LLM applications, including data loaders, parsers, hints, models, and more, as well as the Lang Chain Expression Language (L-CEL), which is used to easily combine these modules. The course also highlights the reasons why JavaScript is the language of choice for building LLM applications, as well as the two LLM types supported by LangChain: text LLM and Chat model. Finally, the course introduces how to optimize the output of LLM applications using stream, batch, and other methods.

detail
{'title': '19. Build LLM Apps with LangChain js| Andrew Ng | DeepLearning.ai - Full Course', 'heatmap': [], 'summary': "Course 'build llm apps with langchain js' by andrew ng covers the importance of langchain.js for javascript developers, langchain's expression language, ai-generated product names, text splitting, vector store initialization, and building conversational retrieval chains using deno, providing comprehensive insights into building context-aware applications with large language models.", 'chapters': [{'end': 244.573, 'segs': [{'end': 38.04, 'src': 'embed', 'start': 0.549, 'weight': 0, 'content': [{'end': 3.551, 'text': 'Applications are becoming so important and so mainstream.', 'start': 0.549, 'duration': 3.002}, {'end': 10.476, 'text': "I think it's important for JavaScript developers to really use these tools and build them into your applications as well.", 'start': 4.072, 'duration': 6.404}, {'end': 13.678, 'text': 'And I hope that this course will help you with that.', 'start': 11.116, 'duration': 2.562}, {'end': 19.642, 'text': 'When building applications using large language models, there are common steps that many developers use.', 'start': 13.998, 'duration': 5.644}, {'end': 24.746, 'text': 'LandChain.js make these steps much easier for JavaScript developers.', 'start': 20.162, 'duration': 4.584}, {'end': 34.376, 'text': "For example, if you're building a retrieval, augmented generation or RAC application, you'd have to choose an LLM to perform your task.", 'start': 25.626, 'duration': 8.75}, {'end': 38.04, 'text': "figure out how to retrieve relevant texts to fill the LLM's context,", 'start': 34.376, 'duration': 3.664}], 'summary': 'Javascript developers should integrate tools like landchain.js for building applications with large language models, such as retrieval, augmented generation, or rac applications.', 'duration': 37.491, 'max_score': 0.549, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8549.jpg'}, {'end': 82.741, 'src': 'embed', 'start': 57.576, 'weight': 1, 'content': [{'end': 66.921, 'text': 'LanChain is a very popular open source orchestrator for LM applications and will help you to build your LM applications much more quickly.', 'start': 57.576, 'duration': 9.345}, {'end': 69.363, 'text': 'The instructor for this course is Jacob Li.', 'start': 67.221, 'duration': 2.142}, {'end': 76.011, 'text': 'Founding Software Engineer at LandChain and Lead Maintainer for the open source LandChain.js.', 'start': 70.003, 'duration': 6.008}, {'end': 82.741, 'text': 'Jacob has worked with many developers to help them integrate OMS into their web and mobile applications.', 'start': 76.692, 'duration': 6.049}], 'summary': 'Lanchain orchestrator accelerates lm application development, led by jacob li.', 'duration': 25.165, 'max_score': 57.576, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B857576.jpg'}, {'end': 143.207, 'src': 'embed', 'start': 120.813, 'weight': 2, 'content': [{'end': 128.735, 'text': 'Models, which provide an abstraction layer on top of specific LLMs so that you can write applications that are themselves not vendor-specific.', 'start': 120.813, 'duration': 7.922}, {'end': 134.317, 'text': 'And then there are other modules to support RAG, such as text splitters and integrations with vector stores.', 'start': 129.555, 'duration': 4.762}, {'end': 143.207, 'text': "You'll also use the Langchain Expression Language, or LCEL for short, to easily compose complex chains of these modules.", 'start': 135.857, 'duration': 7.35}], 'summary': 'Models provide abstraction for non vendor-specific applications. also use lcel for composing complex chains.', 'duration': 22.394, 'max_score': 120.813, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8120813.jpg'}, {'end': 216.306, 'src': 'embed', 'start': 185.502, 'weight': 4, 'content': [{'end': 189.487, 'text': "And you'll learn a bit about how to combine them together to create chains with LanChain expression language.", 'start': 185.502, 'duration': 3.985}, {'end': 190.708, 'text': "Let's dive in.", 'start': 190.287, 'duration': 0.421}, {'end': 199.561, 'text': "Before we get too far in, let's tackle the elephant in the room.", 'start': 196.84, 'duration': 2.721}, {'end': 208.824, 'text': 'Why Langchain.js? Well, JavaScript is the biggest ecosystem of developers in the world, and many of them prefer, surprisingly, to use JavaScript.', 'start': 199.841, 'duration': 8.983}, {'end': 216.306, 'text': 'You also might choose it for some of the easy deployment tools and powerful scaling features of frameworks like Next.js,', 'start': 209.104, 'duration': 7.202}], 'summary': "Learn to create chains with lanchain expression language, chosen for javascript's large developer ecosystem and powerful scaling features.", 'duration': 30.804, 'max_score': 185.502, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8185502.jpg'}], 'start': 0.549, 'title': 'Javascript developers and llm applications', 'summary': 'Emphasizes the importance of landchain.js for javascript developers, introducing it as a tool for building applications using large language models, with a focus on retrieval, augmented generation, and rac applications. it also covers the building blocks of llm applications in javascript, including data loaders, parsers, models, and the langchain expression language.', 'chapters': [{'end': 82.741, 'start': 0.549, 'title': 'Javascript developers and landchain.js', 'summary': 'Emphasizes the importance of applications for javascript developers and introduces landchain.js as a tool to simplify building applications using large language models, with a focus on retrieval, augmented generation, and rac applications, highlighting its role as an orchestrator for lm applications and the expertise of the instructor, jacob li.', 'duration': 82.192, 'highlights': ['LandChain.js simplifies building applications using large language models, such as retrieval, augmented generation, or RAC applications, enabling developers to choose an LLM, retrieve relevant texts, tune prompts, and parse text output (Relevance: 5)', 'The course instructor, Jacob Li, is a Founding Software Engineer at LandChain and Lead Maintainer for the open source LandChain.js, with extensive experience in integrating LMs into web and mobile applications (Relevance: 4)', 'Applications are becoming increasingly important and mainstream, urging JavaScript developers to incorporate tools like LandChain.js into their applications (Relevance: 3)']}, {'end': 244.573, 'start': 83.782, 'title': 'Building llm applications in javascript', 'summary': 'Introduces the basic building blocks of llm applications in javascript, covering elements like data loaders, parsers, models, and the langchain expression language, with contributions from various individuals and the rationale behind choosing javascript for building ln-based applications.', 'duration': 160.791, 'highlights': ['You will learn about a number of elements in LLM applications, like data loaders, parsers, prompts, models, and other modules to support RAG, along with the Langchain Expression Language (LCEL) for composing complex chains of modules.', 'Contributions from individuals like Harrison Chase, Nuno Campos, Jeff Ludwig, and Ashmo Gargari made the course possible.', 'The chapter discusses the reasons for choosing JavaScript for building LN-based applications, highlighting its wide developer ecosystem, easy deployment tools, powerful scaling features, and support for building for multiple platforms.', 'The use of Deno Jupyter kernel, a JavaScript runtime with slight differences from Node and web environments, is mentioned for this notebook.']}], 'duration': 244.024, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8549.jpg', 'highlights': ['LandChain.js simplifies building applications using large language models, such as retrieval, augmented generation, or RAC applications, enabling developers to choose an LLM, retrieve relevant texts, tune prompts, and parse text output (Relevance: 5)', 'The course instructor, Jacob Li, is a Founding Software Engineer at LandChain and Lead Maintainer for the open source LandChain.js, with extensive experience in integrating LMs into web and mobile applications (Relevance: 4)', 'You will learn about a number of elements in LLM applications, like data loaders, parsers, prompts, models, and other modules to support RAG, along with the Langchain Expression Language (LCEL) for composing complex chains of modules.', 'Applications are becoming increasingly important and mainstream, urging JavaScript developers to incorporate tools like LandChain.js into their applications (Relevance: 3)', 'The chapter discusses the reasons for choosing JavaScript for building LN-based applications, highlighting its wide developer ecosystem, easy deployment tools, powerful scaling features, and support for building for multiple platforms.']}, {'end': 663.157, 'segs': [{'end': 276.503, 'src': 'embed', 'start': 245.213, 'weight': 0, 'content': [{'end': 246.894, 'text': "And when there are differences, we'll call them out.", 'start': 245.213, 'duration': 1.681}, {'end': 248.736, 'text': 'So a bit about Langchain expression language.', 'start': 247.255, 'duration': 1.481}, {'end': 257.221, 'text': 'Langchain uses this to compose chains of components together and components that implement and are usable with this language are called runnables.', 'start': 249.096, 'duration': 8.125}, {'end': 265.792, 'text': 'They define some core methods and an allowed set of input types and output types and allow you to use methods like invoke,', 'start': 257.781, 'duration': 8.011}, {'end': 272.679, 'text': 'stream and batch right out of the box, which are all common methods used when building and using LLM applications.', 'start': 265.792, 'duration': 6.887}, {'end': 276.503, 'text': "There's also means of modifying parameters at runtime with the bind method.", 'start': 273.099, 'duration': 3.404}], 'summary': 'Langchain uses an expression language to compose runnables, enabling methods like invoke, stream, and batch for llm applications.', 'duration': 31.29, 'max_score': 245.213, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8245213.jpg'}, {'end': 323.833, 'src': 'embed', 'start': 283.666, 'weight': 1, 'content': [{'end': 290.287, 'text': 'And some other benefits of the runnable protocol are that you get fallbacks and parallelism right out of the box with our batch method,', 'start': 283.666, 'duration': 6.621}, {'end': 294.648, 'text': 'as well as logging and tracing built in via LangSmith, our tracing and observability tool.', 'start': 290.287, 'duration': 4.361}, {'end': 299.189, 'text': "Throughout the course, we'll link to a few explorable traces in LangSmith that illustrate how different chains work.", 'start': 294.888, 'duration': 4.301}, {'end': 303.27, 'text': "Let's start with one of the most fundamental pieces of LangChain, the language model.", 'start': 299.669, 'duration': 3.601}, {'end': 306.571, 'text': 'And LangChain supports two different types of language model.', 'start': 303.81, 'duration': 2.761}, {'end': 312.898, 'text': 'In fact, There are text LLMs, which take a string as input and return a string as output.', 'start': 307.131, 'duration': 5.767}, {'end': 314.961, 'text': 'So string to string.', 'start': 313.278, 'duration': 1.683}, {'end': 323.833, 'text': 'And then there are chat models, which take a list of messages as input and return a single message output.', 'start': 315.561, 'duration': 8.272}], 'summary': 'Runnable protocol offers fallbacks, parallelism, logging, and tracing. langchain supports two types of language models: text llms and chat models.', 'duration': 40.167, 'max_score': 283.666, 'thumbnail': ''}, {'end': 581.146, 'src': 'embed', 'start': 555.936, 'weight': 3, 'content': [{'end': 561.557, 'text': 'but we can also create a prompt template for messages directly for finer grain control over what types of messages are passed to the prompt.', 'start': 555.936, 'duration': 5.621}, {'end': 570.035, 'text': 'For example, many models and model providers rely on a system message to define certain behavior, and this is a great way to format your input.', 'start': 562.318, 'duration': 7.717}, {'end': 571.739, 'text': "So here's an example.", 'start': 571.038, 'duration': 0.701}, {'end': 577.563, 'text': "We'll import some other methods here, and then we'll create our prompt slightly differently using these new message prompt templates.", 'start': 572.039, 'duration': 5.524}, {'end': 581.146, 'text': 'So prompt for messages will be a chat prompt template from these messages.', 'start': 578.103, 'duration': 3.043}], 'summary': 'Create prompt template for messages to control behavior effectively.', 'duration': 25.21, 'max_score': 555.936, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8555936.jpg'}, {'end': 673.741, 'src': 'embed', 'start': 646.126, 'weight': 4, 'content': [{'end': 649.891, 'text': "That's right, it's expression language, or LCEL for short.", 'start': 646.126, 'duration': 3.765}, {'end': 653.876, 'text': 'LCEL is a composable syntax for chaining langchain modules together.', 'start': 650.512, 'duration': 3.364}, {'end': 657.954, 'text': 'And again, objects that are compatible with LCEL are called runnables.', 'start': 654.833, 'duration': 3.121}, {'end': 661.656, 'text': 'We can construct a simple chain from the prompt and model we declared above like this.', 'start': 658.375, 'duration': 3.281}, {'end': 663.157, 'text': "We'll use this pipe method.", 'start': 662.116, 'duration': 1.041}, {'end': 670.94, 'text': "And what this is going to do is it's going to create a chain where the input is the same as the first step in the sequence, our prompt here,", 'start': 663.497, 'duration': 7.443}, {'end': 673.741, 'text': 'which in this case will be an object with a single property called product.', 'start': 670.94, 'duration': 2.801}], 'summary': 'Lcel is a composable syntax for chaining langchain modules together, using runnables and the pipe method to create a chain from the prompt and model.', 'duration': 27.615, 'max_score': 646.126, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8646126.jpg'}], 'start': 245.213, 'title': "Langchain's expression language and runnable protocol", 'summary': "Introduces langchain's expression language for composing components, discussing core methods, input/output types, provided methods like invoke, stream, and batch, and benefits of fallbacks, parallelism, logging, and tracing provided by the runnable protocol. it also covers langchain's language models, supporting text and chat models, providing examples of popular apps like chatgpt and gpt-4, and demonstrating the usage of prompt templates for formatting user input for model calls.", 'chapters': [{'end': 299.189, 'start': 245.213, 'title': 'Langchain expression language and runnable protocol', 'summary': "Introduces langchain's expression language to compose chains of components, discussing the runnables' core methods, input/output types, and provided methods like invoke, stream, and batch, as well as the benefits of fallbacks, parallelism, logging, and tracing provided by the runnable protocol.", 'duration': 53.976, 'highlights': ['Langchain uses an expression language to compose chains of components called runnables, which provide core methods and an allowed set of input and output types, along with methods like invoke, stream, and batch, commonly used in LLM applications.', 'The runnable protocol offers benefits such as fallbacks and parallelism with the batch method, and built-in logging and tracing via LangSmith, the tracing and observability tool.']}, {'end': 663.157, 'start': 299.669, 'title': 'Langchain language model', 'summary': "Introduces langchain's language models, supporting text and chat models, providing examples of popular apps like chatgpt and gpt-4, and demonstrates the usage of prompt templates for formatting user input for model calls.", 'duration': 363.488, 'highlights': ['LangChain supports two types of language models: text LLMs, which take a string as input and return a string as output, and chat models, which take a list of messages as input and return a single message output.', 'Examples of text models include popular apps like ChatGPT and GPT-4, making text LLM inputs and outputs easy to visualize.', 'The usage of prompt templates is demonstrated, allowing the formatting of user input for later model calls, with variable injection and the ability to generate both string input for an LLM and a message array for calling chat models.', 'The chapter introduces the concept of expression language (LCEL), a composable syntax for chaining LangChain modules together, providing a more elegant way to use prompts and models together.']}], 'duration': 417.944, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8245213.jpg', 'highlights': ['Langchain uses an expression language to compose chains of components called runnables, with core methods like invoke, stream, and batch. (Relevance: 5)', 'The runnable protocol offers benefits such as fallbacks and parallelism with the batch method, and built-in logging and tracing via LangSmith. (Relevance: 4)', 'LangChain supports two types of language models: text LLMs and chat models, with examples like ChatGPT and GPT-4. (Relevance: 3)', 'The usage of prompt templates is demonstrated, allowing the formatting of user input for model calls. (Relevance: 2)', 'The chapter introduces the concept of expression language (LCEL), a composable syntax for chaining LangChain modules together. (Relevance: 1)']}, {'end': 988.4, 'segs': [{'end': 744.444, 'src': 'embed', 'start': 717.324, 'weight': 3, 'content': [{'end': 720.346, 'text': "The final consideration we'll go over in this lesson is formatting our output.", 'start': 717.324, 'duration': 3.022}, {'end': 725.929, 'text': "For example, it's often easier to work with the raw string value of a chat model's output rather than an AI message.", 'start': 720.606, 'duration': 5.323}, {'end': 728.83, 'text': 'And the link chain abstract for this is called an output parser.', 'start': 726.309, 'duration': 2.521}, {'end': 730.491, 'text': "So we'll import it like this.", 'start': 729.19, 'duration': 1.301}, {'end': 736.154, 'text': 'This is one that will take our chat model output, a single message, and coerce it into a string.', 'start': 731.652, 'duration': 4.502}, {'end': 737.795, 'text': "So let's initialize that output parser.", 'start': 736.454, 'duration': 1.341}, {'end': 744.444, 'text': "And let's redeclare our chain like this.", 'start': 742.984, 'duration': 1.46}], 'summary': 'Lesson covers formatting output and using an output parser to convert chat model output into a string.', 'duration': 27.12, 'max_score': 717.324, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8717324.jpg'}, {'end': 822.068, 'src': 'embed', 'start': 790.476, 'weight': 2, 'content': [{'end': 795.198, 'text': 'So we see that we have a runnable sequence here, which corresponds to our chain, and that wraps a few steps here.', 'start': 790.476, 'duration': 4.722}, {'end': 803.644, 'text': "First, our chat prompt template, where it takes in our input, fancy cookies, that's a product, and injects it into the prompt.", 'start': 796.783, 'duration': 6.861}, {'end': 808.385, 'text': "It outputs a single human message wrapped in an array, and that's the format that our chat model expects here.", 'start': 803.844, 'duration': 4.541}, {'end': 811.726, 'text': 'So our input what are three good names for a company that makes fancy cookies?', 'start': 809.005, 'duration': 2.721}, {'end': 817.127, 'text': 'And the output an AI message with three good names for a company that makes fancy cookies.', 'start': 812.466, 'duration': 4.661}, {'end': 822.068, 'text': 'And then finally, a string output parser converts that from an AI message into a string.', 'start': 817.607, 'duration': 4.461}], 'summary': 'A sequence of steps to generate ai responses for a fancy cookie company name prompt.', 'duration': 31.592, 'max_score': 790.476, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8790476.jpg'}, {'end': 899.238, 'src': 'embed', 'start': 873.523, 'weight': 0, 'content': [{'end': 878.723, 'text': "That's a very important one for many of you in the web dev community, and we'll take great advantage of that later.", 'start': 873.523, 'duration': 5.2}, {'end': 884.988, 'text': 'But all these runnables and sequences of runnables themselves are also runnables, by the way, get a .', 'start': 879.024, 'duration': 5.964}, {'end': 888.07, 'text': 'stream method, which returns output from the chain in an iterable stream.', 'start': 884.988, 'duration': 3.082}, {'end': 894.454, 'text': 'And because LLM responses often take a long time to finish, this is useful in situations where showing feedback quickly is important.', 'start': 888.37, 'duration': 6.084}, {'end': 896.496, 'text': "And here's an example with the chain we just composed.", 'start': 894.715, 'duration': 1.781}, {'end': 898.217, 'text': "You'll see here that we call this .", 'start': 896.836, 'duration': 1.381}, {'end': 899.238, 'text': 'stream method instead of .', 'start': 898.217, 'duration': 1.021}], 'summary': 'Llm responses take a long time to finish, so using the stream method is useful for quickly showing feedback.', 'duration': 25.715, 'max_score': 873.523, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8873523.jpg'}, {'end': 970.726, 'src': 'embed', 'start': 943.879, 'weight': 1, 'content': [{'end': 948.16, 'text': 'This is useful for performing concurrent operations and multiple generations simultaneously.', 'start': 943.879, 'duration': 4.281}, {'end': 955.442, 'text': "To show what this looks like, we'll define a set of inputs as an array, and each one of these should match the syntax that our prompt template takes.", 'start': 948.18, 'duration': 7.262}, {'end': 966.764, 'text': "So product will be some new product, let's say large calculators, a little whimsical, and product alpaca wool sweaters.", 'start': 955.542, 'duration': 11.222}, {'end': 970.726, 'text': "And this time, instead of using streamer invoke, we'll use the dot batch method.", 'start': 967.524, 'duration': 3.202}], 'summary': 'Demonstrating concurrent operations with array inputs for new products using dot batch method.', 'duration': 26.847, 'max_score': 943.879, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8943879.jpg'}], 'start': 663.497, 'title': 'Ai-generated product names and chaining methods', 'summary': 'Covers generating ai-created names for products and the runnablesequence.from method for chaining, demonstrating formatting, internal workings, and efficient output processing.', 'chapters': [{'end': 822.068, 'start': 663.497, 'title': 'Creating ai-generated names for products', 'summary': "Explains the process of using a chain to generate ai-created names for products, demonstrating how to format the output and illustrating the chain's internal workings.", 'duration': 158.571, 'highlights': ['The process involves creating a chain to generate AI-created names for products, using a prompt template and a chat model to produce responses based on the input data. It showcases the generation of names for products like colorful socks and fancy cookies.', 'Formatting the output is discussed, emphasizing the use of an output parser to convert the AI message into a string, making it easier to work with the raw output of the chat model.', 'The chapter illustrates the internal workings of the chain by providing a visual representation, demonstrating the sequence of steps involved in the generation of AI-created names for products.']}, {'end': 988.4, 'start': 822.813, 'title': 'Using runnablesequence.from method for chaining', 'summary': 'Discusses using the runnablesequence.from method to construct a chain from an array of runnables in an expression language, illustrating the use of .stream method, async iteration syntax, and .batch method for efficient output processing.', 'duration': 165.587, 'highlights': ['The .stream method returns output from the chain in an iterable stream, useful for showing feedback quickly, demonstrated with async iteration syntax and individual string chunks as output. (Relevance: 5)', 'The .batch method is useful for performing concurrent operations and multiple generations simultaneously, as demonstrated with two sets of inputs resulting in string outputs for different companies. (Relevance: 4)', 'The chapter introduces the use of runnableSequence.from method to construct a chain from an array of runnables, providing an alternative for more complicated chains. (Relevance: 3)', 'The output parser transforms output chunks from the model as they are generated, resulting in string output chunks rather than models, enhancing efficiency in processing. (Relevance: 2)']}], 'duration': 324.903, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8663497.jpg', 'highlights': ['The .stream method returns output from the chain in an iterable stream, useful for showing feedback quickly, demonstrated with async iteration syntax and individual string chunks as output. (Relevance: 5)', 'The .batch method is useful for performing concurrent operations and multiple generations simultaneously, as demonstrated with two sets of inputs resulting in string outputs for different companies. (Relevance: 4)', 'The process involves creating a chain to generate AI-created names for products, using a prompt template and a chat model to produce responses based on the input data. It showcases the generation of names for products like colorful socks and fancy cookies. (Relevance: 3)', 'Formatting the output is discussed, emphasizing the use of an output parser to convert the AI message into a string, making it easier to work with the raw output of the chat model. (Relevance: 2)', 'The chapter illustrates the internal workings of the chain by providing a visual representation, demonstrating the sequence of steps involved in the generation of AI-created names for products. (Relevance: 1)']}, {'end': 1448.896, 'segs': [{'end': 1111.445, 'src': 'embed', 'start': 1084.255, 'weight': 0, 'content': [{'end': 1091, 'text': "And then we'll get into splitting, where we'll use some of LanqChain's text splitting abstractions to format our text in a way legible to the LLL.", 'start': 1084.255, 'duration': 6.745}, {'end': 1092.061, 'text': "So we'll start with loading.", 'start': 1091.3, 'duration': 0.761}, {'end': 1095.584, 'text': 'An example of one of the many LanqChain loaders is GitHub.', 'start': 1092.341, 'duration': 3.243}, {'end': 1098.165, 'text': 'So we can paste in an import here.', 'start': 1096.064, 'duration': 2.101}, {'end': 1104.89, 'text': 'And this particular loader requires a peer dependency used to support gitignore syntax called ignore.', 'start': 1098.966, 'duration': 5.924}, {'end': 1111.445, 'text': "Once we've loaded it, we'll instantiate it with a GitHub repo, in this case, of course, Langchain.js.", 'start': 1105.21, 'duration': 6.235}], 'summary': "Using lanqchain's text splitting abstractions to format text for lll, starting with loading github repo langchain.js", 'duration': 27.19, 'max_score': 1084.255, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81084255.jpg'}, {'end': 1265.21, 'src': 'embed', 'start': 1240.012, 'weight': 1, 'content': [{'end': 1245.697, 'text': 'And the idea here and the goal, as a reminder, is to try to keep semantically related ideas together in the same chunk,', 'start': 1240.012, 'duration': 5.685}, {'end': 1249.279, 'text': 'so that the LLM gets an entire self-contained idea without distraction.', 'start': 1245.697, 'duration': 3.582}, {'end': 1253.863, 'text': 'You notice the previous example with the CS229 transcript.', 'start': 1249.499, 'duration': 4.364}, {'end': 1260.348, 'text': 'there are many, many pages with a lot of text, and our LLMs only have a fixed amount of attention to give to each chunk.', 'start': 1253.863, 'duration': 6.485}, {'end': 1265.21, 'text': "So there are many different strategies for splitting on data, and it's going to really depend on what you're loading.", 'start': 1260.968, 'duration': 4.242}], 'summary': "Goal: keep semantically related ideas together for llm's attention and processing.", 'duration': 25.198, 'max_score': 1240.012, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81240012.jpg'}, {'end': 1362.772, 'src': 'embed', 'start': 1333.071, 'weight': 2, 'content': [{'end': 1334.532, 'text': 'And to show kind of like the alternative here.', 'start': 1333.071, 'duration': 1.461}, {'end': 1341.276, 'text': 'if we split naively, for example using something like spaces as a separator, we may get chunks, generating, for example, half a log statement,', 'start': 1334.532, 'duration': 6.744}, {'end': 1343.957, 'text': "which makes the LLM's job more difficult on final generation.", 'start': 1341.276, 'duration': 2.681}, {'end': 1345.498, 'text': 'And to show what this looks like,', 'start': 1344.297, 'duration': 1.201}, {'end': 1353.889, 'text': "let's import a more naive character splitter And we'll set similar parameters here where small chunk size and no overlap,", 'start': 1345.498, 'duration': 8.391}, {'end': 1356.03, 'text': 'and then a pretty naive separator of just spaces.', 'start': 1353.889, 'duration': 2.141}, {'end': 1358.831, 'text': "And we'll leave this on screen for comparison's sake.", 'start': 1356.59, 'duration': 2.241}, {'end': 1359.891, 'text': "We'll give it the same code.", 'start': 1359.011, 'duration': 0.88}, {'end': 1362.772, 'text': "And then let's just see what happens when we call split on it.", 'start': 1360.752, 'duration': 2.02}], 'summary': "Naively splitting text using spaces may generate half log statement, making llm's job difficult.", 'duration': 29.701, 'max_score': 1333.071, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81333071.jpg'}, {'end': 1421.032, 'src': 'embed', 'start': 1393.293, 'weight': 3, 'content': [{'end': 1396.674, 'text': 'but maybe we also want to get some of the function body as context as well.', 'start': 1393.293, 'duration': 3.381}, {'end': 1401.678, 'text': 'So we can try something pretty similar to the above, where we use our cursive text splitter,', 'start': 1396.914, 'duration': 4.764}, {'end': 1405.14, 'text': "but let's turn up the chunk size a little bit and let's also give it some overlap.", 'start': 1401.678, 'duration': 3.462}, {'end': 1410.344, 'text': "So let's try it with some tune parameters where we increase the chunk size and make the chunks overlap a little bit.", 'start': 1405.42, 'duration': 4.924}, {'end': 1412.546, 'text': 'So the idea is a little bit more split naturally.', 'start': 1410.364, 'duration': 2.182}, {'end': 1414.107, 'text': 'And this time you can see that,', 'start': 1412.766, 'duration': 1.341}, {'end': 1421.032, 'text': "even though it's a little bit less efficient in terms of how much in that we're sort of putting redundant information into these chunks,", 'start': 1414.107, 'duration': 6.925}], 'summary': 'Using cursive text splitter with increased chunk size and overlap for a more natural split.', 'duration': 27.739, 'max_score': 1393.293, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81393293.jpg'}, {'end': 1457.523, 'src': 'embed', 'start': 1430.819, 'weight': 4, 'content': [{'end': 1437.564, 'text': 'So link chain includes several different options for different types of content, including Markdown, HTML, JavaScript, which you just saw,', 'start': 1430.819, 'duration': 6.745}, {'end': 1439.549, 'text': 'Python and more.', 'start': 1438.248, 'duration': 1.301}, {'end': 1444.093, 'text': 'But for generic written text, the recursive character text splitter is a great place to start,', 'start': 1439.789, 'duration': 4.304}, {'end': 1448.896, 'text': 'since it splits chunks on paragraphs as natural boundaries for people to split up their thoughts and points.', 'start': 1444.093, 'duration': 4.803}, {'end': 1453.58, 'text': "So let's initialize one that we'll use on our previously loaded CS229 class.", 'start': 1449.337, 'duration': 4.243}, {'end': 1457.523, 'text': "That'll be a text splitter this time with a little bit bigger chunks.", 'start': 1453.82, 'duration': 3.703}], 'summary': 'Link chain offers options for various content types, including markdown, html, javascript, and python. the recursive character text splitter is recommended for generic written text, splitting chunks on natural paragraph boundaries.', 'duration': 26.704, 'max_score': 1430.819, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81430819.jpg'}], 'start': 990.32, 'title': 'Langchain modules and data splitting strategies', 'summary': 'Covers langchain modules, retrieval augmented generation, and data loading techniques. it also discusses data splitting strategies for language model training, demonstrating their impact on chunk generation and language model (llm) performance.', 'chapters': [{'end': 1260.348, 'start': 990.32, 'title': 'Langchain modules and retrieval augmented generation', 'summary': 'Covers the fundamental modules behind langchain and langchain expression language, the basics of retrieval augmented generation, the process of loading and preparing data, and techniques for storing and retrieving documents, with a focus on retrieval augmented generation (rag) and document loading from various sources, including github and pdfs.', 'duration': 270.028, 'highlights': ["The process of loading and preparing data involves retrieving documents from sources such as PDFs, databases, or the web, splitting them into small chunks to fit into an LLM's context window, and embedding those chunks in a vector store for later retrieval based on input queries.", 'LanqChain offers a variety of document loaders for bringing in data from different sources, such as GitHub, and provides text splitting abstractions for formatting text in a way legible to the LLM.', 'An example of using LanqChain loaders involves importing a GitHub repo, instantiating it with specific options, and then loading and logging the retrieved documents, demonstrating the flexibility to load various types of data including GitHub files, code, and PDFs.', "Another example showcases loading a transcript of Andrew Ng's CS229 course on machine learning using a peer dependency called PDF parse, with a demonstration of initializing and running the loader to retrieve and log the loaded documents, emphasizing the broad range of data types that can be loaded.", 'The splitting process aims to keep semantically related ideas together in the same chunk to provide the LLM with a self-contained idea without distraction, as demonstrated with the CS229 transcript, which consists of multiple pages with the goal of maintaining semantic coherence within each chunk.']}, {'end': 1448.896, 'start': 1260.968, 'title': 'Data splitting strategies for language model training', 'summary': 'Discusses the strategies for splitting data for language model training, demonstrating the impact of different parameters and separators on chunk generation, with examples showing the influence on function definitions and classes, and the implications for the language model (llm) performance.', 'duration': 187.928, 'highlights': ['The impact of different parameters and separators on chunk generation is demonstrated through examples, influencing function definitions and classes, and the implications for the Language Model (LLM) performance.', "The demonstration of splitting on code-specific delimiters for the GitHub JavaScript example yields four chunks with natural splits, facilitating the LLM's work, and contrasts this with a naive separator resulting in half a log statement and making the LLM's job more difficult.", 'The adjustment of parameters, such as increasing chunk size and introducing overlap, is shown to provide a more natural split, allowing the entirety of the function to be captured in a single chunk, providing full context for the LLM.', 'The recursive character text splitter is highlighted as a great starting point for generic written text, leveraging paragraphs as natural boundaries for chunk splitting.']}], 'duration': 458.576, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B8990320.jpg', 'highlights': ['LanqChain offers document loaders for various sources like GitHub, PDFs, and databases.', 'Data splitting aims to maintain semantic coherence within each chunk for LLM processing.', 'Different parameters and separators impact chunk generation and LLM performance.', 'Adjusting parameters like chunk size and overlap provides more natural splits for LLM context.', 'Recursive character text splitter is a good starting point for generic written text.']}, {'end': 2073.311, 'segs': [{'end': 1476.684, 'src': 'embed', 'start': 1449.337, 'weight': 0, 'content': [{'end': 1453.58, 'text': "So let's initialize one that we'll use on our previously loaded CS229 class.", 'start': 1449.337, 'duration': 4.243}, {'end': 1457.523, 'text': "That'll be a text splitter this time with a little bit bigger chunks.", 'start': 1453.82, 'duration': 3.703}, {'end': 1462.627, 'text': "So we're going to use 512 characters maximum and then a 64 character overlap.", 'start': 1457.983, 'duration': 4.644}, {'end': 1463.848, 'text': 'And there we go.', 'start': 1463.007, 'duration': 0.841}, {'end': 1469.361, 'text': "Let's split up our previously ingested transcript of Andrew Ng's course and see what happens.", 'start': 1464.368, 'duration': 4.993}, {'end': 1470.841, 'text': "And let's log the result.", 'start': 1469.841, 'duration': 1}, {'end': 1476.684, 'text': "We'll again take the first five so that our output doesn't get too long here.", 'start': 1471.322, 'duration': 5.362}], 'summary': 'Initializing text splitter with 512 max characters and 64 character overlap for cs229 class transcript analysis.', 'duration': 27.347, 'max_score': 1449.337, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81449337.jpg'}, {'end': 1563.627, 'src': 'embed', 'start': 1532.585, 'weight': 3, 'content': [{'end': 1537.347, 'text': "We'll split those chunks small enough to fit into an LLM's context window and avoid distraction with our text splitters.", 'start': 1532.585, 'duration': 4.762}, {'end': 1543.169, 'text': "And now we're onto embedding those chunks and storing them in a vector store to allow for later retrieval based on an input query.", 'start': 1537.587, 'duration': 5.582}, {'end': 1546.31, 'text': "And to do that, we'll use what's called a text embedding model.", 'start': 1543.749, 'duration': 2.561}, {'end': 1552.352, 'text': "We use OpenAI's hosted model in this demo, but again, you can swap them out for any embedding provider of your choice.", 'start': 1546.83, 'duration': 5.522}, {'end': 1556.253, 'text': 'A vector store is a specialized type of database with natural language search capabilities.', 'start': 1552.772, 'duration': 3.481}, {'end': 1563.627, 'text': "And when the user comes in with a query, We'll search the vector store for an embedding which we'll get to in just a moment,", 'start': 1556.733, 'duration': 6.894}], 'summary': 'Text chunks embedded into vector store for retrieval based on query.', 'duration': 31.042, 'max_score': 1532.585, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81532585.jpg'}, {'end': 1592.564, 'src': 'embed', 'start': 1563.627, 'weight': 2, 'content': [{'end': 1569.653, 'text': "similar to the query that they're asking, and then return relevant chunks for the LLM's final generation.", 'start': 1563.627, 'duration': 6.026}, {'end': 1574.355, 'text': "And we'll show off how to embed our previously split document chunks so that we can take advantage of these capabilities.", 'start': 1569.933, 'duration': 4.422}, {'end': 1578.237, 'text': 'So the first step of adding documents to a vector store is called ingestion.', 'start': 1574.655, 'duration': 3.582}, {'end': 1582.339, 'text': 'And again it uses an embeddings model, which is a specialized type of machine learning model,', 'start': 1578.557, 'duration': 3.782}, {'end': 1587.742, 'text': 'to convert document contents into a representation called a vector, which our vector store can then search over.', 'start': 1582.339, 'duration': 5.403}, {'end': 1592.564, 'text': "And we'll be using OpenAI's hosted embeddings, so we'll need to import our environment variables to get our key.", 'start': 1588.042, 'duration': 4.522}], 'summary': 'Demonstrating ingestion of documents into a vector store using embeddings model for document representation.', 'duration': 28.937, 'max_score': 1563.627, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81563627.jpg'}, {'end': 1700.29, 'src': 'embed', 'start': 1673.505, 'weight': 5, 'content': [{'end': 1679.49, 'text': "What are vectors used for in machine learning? So we'll use the same similarity cosine function.", 'start': 1673.505, 'duration': 5.985}, {'end': 1682.984, 'text': "and this time we'll use similar vector.", 'start': 1681.523, 'duration': 1.461}, {'end': 1693.127, 'text': 'And we can see the result is significantly higher than our dissimilar vector comparison, since both texts contain similar information.', 'start': 1685.545, 'duration': 7.582}, {'end': 1700.29, 'text': "Great So we'll prepare our documents using the techniques covered in the previous lesson.", 'start': 1694.908, 'duration': 5.382}], 'summary': 'Vectors are used for similarity comparison in machine learning, yielding significantly higher results for similar vectors.', 'duration': 26.785, 'max_score': 1673.505, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81673505.jpg'}, {'end': 1839.714, 'src': 'embed', 'start': 1804.335, 'weight': 1, 'content': [{'end': 1805.636, 'text': "Now let's talk about retrievers.", 'start': 1804.335, 'duration': 1.301}, {'end': 1812.792, 'text': "So we've just shown how to return documents from a vector store using similarity search directly,", 'start': 1807.608, 'duration': 5.184}, {'end': 1816.395, 'text': "but that's actually just one type of way to fetch data for an LLM of many.", 'start': 1812.792, 'duration': 3.603}, {'end': 1823.721, 'text': 'And Langchain encapsulates this distinction with a broader retriever abstraction that returns documents related to our given natural language query.', 'start': 1816.796, 'duration': 6.925}, {'end': 1828.665, 'text': 'Conveniently, we can instantiate a retriever from an existing vector store with a simple function call.', 'start': 1824.362, 'duration': 4.303}, {'end': 1831.863, 'text': 'as retriever just like that.', 'start': 1829.68, 'duration': 2.183}, {'end': 1839.714, 'text': 'and one nice trait of retrievers is that, unlike vector stores, they implement the invoke method and are themselves expression language runnables,', 'start': 1831.863, 'duration': 7.851}], 'summary': 'Langchain includes retriever abstraction for fetching documents related to natural language queries. retriever can be instantiated from existing vector store with a simple function call.', 'duration': 35.379, 'max_score': 1804.335, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81804335.jpg'}, {'end': 1923.39, 'src': 'embed', 'start': 1893.418, 'weight': 7, 'content': [{'end': 1896.541, 'text': "So to start, let's load our environment variables, of course.", 'start': 1893.418, 'duration': 3.123}, {'end': 1901.324, 'text': "And then let's split and load the CS229 lesson PDF transcript from earlier.", 'start': 1897.301, 'duration': 4.023}, {'end': 1909, 'text': 'For brevity, I factored out the vector store initialization code into a helper function that takes arguments for chunk size and chunk overlap.', 'start': 1902.315, 'duration': 6.685}, {'end': 1916.105, 'text': "You'll use bigger chunks this time to more simulate a production environment.", 'start': 1910.241, 'duration': 5.864}, {'end': 1923.39, 'text': "1536, and we'll use an overlap of 128 characters.", 'start': 1916.405, 'duration': 6.985}], 'summary': 'Loading environment variables, splitting and loading cs229 pdf transcript, using larger chunks of 1536 with an overlap of 128 characters.', 'duration': 29.972, 'max_score': 1893.418, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81893418.jpg'}], 'start': 1449.337, 'title': 'Text splitting, vector store initialization, and retrieval for language model', 'summary': 'Discusses initializing a text splitter, embedding document chunks, adding them to a vector store, and creating a retriever abstraction for a conversational question answering llm application, emphasizing the importance of vector databases in retrieval augmented generation.', 'chapters': [{'end': 1803.835, 'start': 1449.337, 'title': 'Text splitting and vector store initialization', 'summary': 'Discusses the process of initializing a text splitter, setting chunk sizes, embedding document chunks, and adding them to a vector store, emphasizing the importance of vector databases in retrieval augmented generation.', 'duration': 354.498, 'highlights': ['The process of initializing a text splitter and setting chunk sizes is described, with a maximum of 512 characters and a 64 character overlap.', 'The significance of vector databases in retrieval augmented generation and their role in storing document chunks for later retrieval is highlighted.', "The steps for embedding document chunks and adding them to a vector store are outlined, with emphasis on using OpenAI's hosted model for text embedding.", 'The concept of vector store ingestion, using machine learning models to convert document contents into vectors, is explained, with the importance of environment variables and key management highlighted.', 'The use of cosine similarity to compare similarity between vectors is demonstrated, showcasing the ability to search for relevant chunks based on input queries.']}, {'end': 2073.311, 'start': 1804.335, 'title': 'Retrieval for language model', 'summary': 'Discusses creating a retriever abstraction that returns documents related to a given natural language query, instantiating a retriever from an existing vector store, and constructing a retrieval chain for a conversational question answering llm application.', 'duration': 268.976, 'highlights': ["The chapter discusses creating a retriever abstraction that returns documents related to a given natural language query, instantiating a retriever from an existing vector store, and constructing a retrieval chain for a conversational question answering LLM application, which will retrieve chunks most similar to the input query via vector similarity search and present them to the LLM as context to ground the LLM's generation of the final answer.", 'The retriever abstraction encapsulates the distinction with a broader retriever abstraction that returns documents related to a given natural language query, and retrievers implement the invoke method and are themselves expression language runnables, which can be chained with other modules.', 'The chapter also involves loading environment variables, splitting and loading the CS229 lesson PDF transcript, initializing a vector store with documents, and constructing a chain that formats the inputs and outputs for the other steps, taking an object parameter for flexibility.']}], 'duration': 623.974, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B81449337.jpg', 'highlights': ['The process of initializing a text splitter and setting chunk sizes is described, with a maximum of 512 characters and a 64 character overlap.', 'The chapter discusses creating a retriever abstraction that returns documents related to a given natural language query, instantiating a retriever from an existing vector store, and constructing a retrieval chain for a conversational question answering LLM application.', 'The significance of vector databases in retrieval augmented generation and their role in storing document chunks for later retrieval is highlighted.', "The steps for embedding document chunks and adding them to a vector store are outlined, with emphasis on using OpenAI's hosted model for text embedding.", 'The concept of vector store ingestion, using machine learning models to convert document contents into vectors, is explained, with the importance of environment variables and key management highlighted.', 'The use of cosine similarity to compare similarity between vectors is demonstrated, showcasing the ability to search for relevant chunks based on input queries.', 'The retriever abstraction encapsulates the distinction with a broader retriever abstraction that returns documents related to a given natural language query, and retrievers implement the invoke method and are themselves expression language runnables, which can be chained with other modules.', 'The chapter also involves loading environment variables, splitting and loading the CS229 lesson PDF transcript, initializing a vector store with documents, and constructing a chain that formats the inputs and outputs for the other steps, taking an object parameter for flexibility.']}, {'end': 2779.739, 'segs': [{'end': 2151.392, 'src': 'embed', 'start': 2121.494, 'weight': 4, 'content': [{'end': 2124.015, 'text': 'And when a runnable map is invoked,', 'start': 2121.494, 'duration': 2.521}, {'end': 2133.901, 'text': 'it calls all the runnables or runnable-like objects that it has as properties here context and question corresponding to our document retrieval chain and then a simple extraction function here in parallel.', 'start': 2124.015, 'duration': 9.886}, {'end': 2139.405, 'text': "And it'll call each of those functions or runnables with the same input.", 'start': 2134.022, 'duration': 5.383}, {'end': 2144.168, 'text': 'Then the output is an object whose properties are the results of those calls.', 'start': 2140.586, 'duration': 3.582}, {'end': 2146.19, 'text': "So to show this off, let's try it.", 'start': 2145.069, 'duration': 1.121}, {'end': 2151.392, 'text': 'What are the prerequisites for this course?', 'start': 2146.57, 'duration': 4.822}], 'summary': 'A runnable map is invoked to call runnables or runnable-like objects with the same input, producing an object with the results.', 'duration': 29.898, 'max_score': 2121.494, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82121494.jpg'}, {'end': 2243.643, 'src': 'embed', 'start': 2210.136, 'weight': 3, 'content': [{'end': 2215.658, 'text': "And then we want to pass our question all the way through to our answer generation prompt, which if you'll recall, looks something like this.", 'start': 2210.136, 'duration': 5.522}, {'end': 2218.319, 'text': 'It requires variables for context and question.', 'start': 2216.418, 'duration': 1.901}, {'end': 2229.818, 'text': 'So. to do that, we extract the input as a question and pass the entire object, which looks like this output from the runnable map, into our prompt,', 'start': 2218.679, 'duration': 11.139}, {'end': 2232.099, 'text': 'pass that to the model and then parse the output as a string.', 'start': 2229.818, 'duration': 2.281}, {'end': 2234.28, 'text': "All right, now let's try invoking it.", 'start': 2232.539, 'duration': 1.741}, {'end': 2243.643, 'text': "So we'll ask for an answer and say what are the prerequisites for this course with our new retrieval chain?", 'start': 2234.64, 'duration': 9.003}], 'summary': 'Passing a question to answer generation prompt, then invoking for prerequisites.', 'duration': 33.507, 'max_score': 2210.136, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82210136.jpg'}, {'end': 2426.415, 'src': 'embed', 'start': 2400.77, 'weight': 1, 'content': [{'end': 2406.956, 'text': "Can you list them in bullet point form? And the LLM replied that it didn't see a specific question or prompt.", 'start': 2400.77, 'duration': 6.186}, {'end': 2416.065, 'text': "So what went wrong here? Well, the question is referencing past information, but the LLM has no memory of that past information and can't answer.", 'start': 2407.797, 'duration': 8.268}, {'end': 2417.847, 'text': 'So how do we fix that?', 'start': 2416.886, 'duration': 0.961}, {'end': 2423.132, 'text': 'One way is to rewrite or rephrase the new question, while being aware of the past chat history,', 'start': 2418.428, 'duration': 4.704}, {'end': 2426.415, 'text': 'into a standalone question free of external references.', 'start': 2423.132, 'duration': 3.283}], 'summary': 'Llm cannot answer questions referencing past information, can be fixed by rewriting or rephrasing questions', 'duration': 25.645, 'max_score': 2400.77, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82400770.jpg'}, {'end': 2631.858, 'src': 'embed', 'start': 2579.675, 'weight': 0, 'content': [{'end': 2589.27, 'text': "First, you'll build a prompt template Notice it has an input variable for context from the vector database and a spot for the question.", 'start': 2579.675, 'duration': 9.595}, {'end': 2594.971, 'text': 'Now you initialize your model and then you build your retrieval chain.', 'start': 2590.31, 'duration': 4.661}, {'end': 2603.294, 'text': 'It gets the context from the document retrieval chain and the input question and passes all of that to the prompt.', 'start': 2596.032, 'duration': 7.262}, {'end': 2607.275, 'text': 'And finally, to the model and the output parser.', 'start': 2604.674, 'duration': 2.601}, {'end': 2611.376, 'text': 'Great, so now we are ready to make our new chain.', 'start': 2608.195, 'duration': 3.181}, {'end': 2627.459, 'text': "So we'll make a new chain that's going to be purely responsible for rephrasing the user's input that may contain references to past chat history into a question free of references that both our vector store and later LLM calls can follow.", 'start': 2614.352, 'duration': 13.107}, {'end': 2631.858, 'text': 'And to do that, we will define a new prompt using something called a messages placeholder.', 'start': 2627.859, 'duration': 3.999}], 'summary': 'Building a prompt template and a new chain for rephrasing user input.', 'duration': 52.183, 'max_score': 2579.675, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82579675.jpg'}], 'start': 2073.311, 'title': 'Using runnable map and llm-powered qa', 'summary': 'Explains the use of a runnable map in the prompt template to call runnables with the same input, ensuring output as an object with results, and the augmentation of llm-powered question answering with retrieval, focusing on addressing conversation history shortcomings and integrating a new chain for rephrasing user input.', 'chapters': [{'end': 2167.4, 'start': 2073.311, 'title': 'Using runnable map in prompt template', 'summary': 'Explains the use of a runnable map in the prompt template to call all runnables with the same input, ensuring output as an object with results of those calls, facilitating context properties and question preservation, with an example of answering a question about course prerequisites.', 'duration': 94.089, 'highlights': ['The chapter explains the use of a runnable map in the prompt template to call all runnables with the same input, ensuring output as an object with results of those calls.', 'The use of a runnable map facilitates context properties and question preservation, ensuring that the question has been preserved for passing it along to a later step in the chain.', 'An example is provided where the runnable map is used to answer a question about the prerequisites for a course, showing the preservation of the question and the retrieval of context from the document retrieval chain.']}, {'end': 2779.739, 'start': 2168.161, 'title': 'Llm-powered qa augmented by retrieval', 'summary': 'Covers the augmentation of llm-powered question answering with retrieval, highlighting the shortcomings around conversation history and the techniques to remedy them, emphasizing the need for rephrasing questions and saving chat history for context, culminating in the creation of a new chain for rephrasing user input and its integration into the overall process.', 'duration': 611.578, 'highlights': ["The process involves rephrasing questions and saving chat history for context, which is crucial for the LLM to generate suitable answers, emphasizing the need for techniques to address the LLM's lack of memory of past information.", 'The chapter details the creation of a new chain responsible for rephrasing user input containing references to past chat history into standalone questions that can be followed by the vector store and LLM calls.', 'The chapter also outlines the steps to rebuild the components from the previous lab, including loading configuration, text splitter, vector store with documents, defining the retriever, and building retrieval chains to integrate with the LLM.', 'The chapter highlights the importance of utilizing a prompt template with input variables for context from the vector database and the question, initializing the model, and constructing retrieval chains for effective LLM-powered question answering augmented by retrieval.']}], 'duration': 706.428, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82073311.jpg', 'highlights': ['The chapter highlights the importance of utilizing a prompt template with input variables for context from the vector database and the question, initializing the model, and constructing retrieval chains for effective LLM-powered question answering augmented by retrieval.', "The process involves rephrasing questions and saving chat history for context, crucial for the LLM to generate suitable answers, emphasizing the need for techniques to address the LLM's lack of memory of past information.", 'The chapter details the creation of a new chain responsible for rephrasing user input containing references to past chat history into standalone questions that can be followed by the vector store and LLM calls.', 'The use of a runnable map in the prompt template facilitates context properties and question preservation, ensuring that the question has been preserved for passing it along to a later step in the chain.', 'An example is provided where the runnable map is used to answer a question about the prerequisites for a course, showing the preservation of the question and the retrieval of context from the document retrieval chain.']}, {'end': 3238.469, 'segs': [{'end': 2807.269, 'src': 'embed', 'start': 2779.739, 'weight': 2, 'content': [{'end': 2785.36, 'text': "and then let's define our answer generation prompt to also use a messages placeholder for chat history.", 'start': 2779.739, 'duration': 5.621}, {'end': 2789.981, 'text': "so it's going to look pretty similar to our existing answer chain but with the system template,", 'start': 2785.36, 'duration': 4.621}, {'end': 2796.203, 'text': "and we're going to use that same from messages method so that we can have a placeholder for the chat history.", 'start': 2789.981, 'duration': 6.222}, {'end': 2800.124, 'text': 'and this lets our final answer generation chain also take into account chat history.', 'start': 2796.203, 'duration': 3.921}, {'end': 2807.269, 'text': 'So you can see your experienced researcher, like before, using the provided context and chat history this time,', 'start': 2800.785, 'duration': 6.484}], 'summary': 'Answer generation prompt updated to include chat history placeholder for improved context understanding.', 'duration': 27.53, 'max_score': 2779.739, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82779739.jpg'}, {'end': 2850.276, 'src': 'embed', 'start': 2820.897, 'weight': 3, 'content': [{'end': 2825.44, 'text': "this final prompt and what we're going to use for our final generation requires those three inputs instead of two.", 'start': 2820.897, 'duration': 4.543}, {'end': 2833.527, 'text': "Let's try running that on its own and get a sense of what we need to pass this final answer generation chain to get a good output.", 'start': 2827.264, 'duration': 6.263}, {'end': 2840.071, 'text': "So we're just going to format the prompt purely with some dummy context.", 'start': 2834.728, 'duration': 5.343}, {'end': 2845.194, 'text': 'This is going to be a pretend output from the rephrase question chain and then some sample chat history.', 'start': 2840.091, 'duration': 5.103}, {'end': 2850.276, 'text': 'And if we invoke it, answer generation chain prompt.', 'start': 2845.494, 'duration': 4.782}], 'summary': 'Final prompt requires three inputs for answer generation chain.', 'duration': 29.379, 'max_score': 2820.897, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82820897.jpg'}, {'end': 2936.16, 'src': 'embed', 'start': 2904.902, 'weight': 0, 'content': [{'end': 2908.586, 'text': 'One method of doing this is with the convenience RunnablePassThrough.Assign method.', 'start': 2904.902, 'duration': 3.684}, {'end': 2914.077, 'text': 'In the diagram, step one is taking a history and a question and outputting a standalone question.', 'start': 2908.991, 'duration': 5.086}, {'end': 2920.644, 'text': 'But our prompt in step two also needs to receive that original input history as well as the revised standalone question.', 'start': 2914.617, 'duration': 6.027}, {'end': 2922.085, 'text': 'that was the output of step one.', 'start': 2920.644, 'duration': 1.441}, {'end': 2928.532, 'text': 'Essentially, we want to pass through the old properties while assigning a new one to the current state of the chain.', 'start': 2922.486, 'duration': 6.046}, {'end': 2931.015, 'text': "Let's describe this again, looking at the code.", 'start': 2929.093, 'duration': 1.922}, {'end': 2936.16, 'text': "So you'll notice we were using a couple of new things here.", 'start': 2934.178, 'duration': 1.982}], 'summary': 'Using runnablepassthrough.assign method to pass through old properties while assigning a new one to the current state of the chain.', 'duration': 31.258, 'max_score': 2904.902, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82904902.jpg'}, {'end': 3019.996, 'src': 'embed', 'start': 2995.224, 'weight': 4, 'content': [{'end': 3000.526, 'text': 'then wrap our chain in a manager that will automatically update the history, called Runnable with Message History.', 'start': 2995.224, 'duration': 5.302}, {'end': 3006.116, 'text': "Let's take a closer look at this runnable with message history class.", 'start': 3003.072, 'duration': 3.044}, {'end': 3012.445, 'text': 'It wraps another LCEL chain and adds persistent state by both updating and injecting chat history.', 'start': 3006.497, 'duration': 5.948}, {'end': 3017.713, 'text': 'It automatically updates the chat history by saving part of the input to the chain, which, in your case,', 'start': 3012.866, 'duration': 4.847}, {'end': 3019.996, 'text': 'is the user defined question field in the input.', 'start': 3017.713, 'duration': 2.283}], 'summary': 'A manager called runnable with message history updates chat history by saving user-defined questions.', 'duration': 24.772, 'max_score': 2995.224, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82995224.jpg'}, {'end': 3172.506, 'src': 'embed', 'start': 3147.422, 'weight': 1, 'content': [{'end': 3154.668, 'text': 'And to get a visual sense of what is going on here, because we have multiple calls going back and forth and a lot happening behind the scenes.', 'start': 3147.422, 'duration': 7.246}, {'end': 3157.991, 'text': 'we can look at a Langsmith trace to explore visually.', 'start': 3154.668, 'duration': 3.323}, {'end': 3161.698, 'text': "So here you can see visually what's going on internally.", 'start': 3159.537, 'duration': 2.161}, {'end': 3167.902, 'text': 'We have this runnable with message history class wrapping our runnable sequence here, which is our conversational retrieval chain.', 'start': 3161.818, 'duration': 6.084}, {'end': 3172.506, 'text': "And it's going to insert and load those history messages as parameters to that chain.", 'start': 3168.903, 'duration': 3.603}], 'summary': 'A langsmith trace visually illustrates the conversational retrieval chain with message history class and runnable sequence.', 'duration': 25.084, 'max_score': 3147.422, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B83147422.jpg'}], 'start': 2779.739, 'title': 'Building chat history and conversational retrieval chain', 'summary': 'Covers defining an answer generation prompt using a messages placeholder, integrating chat history into the answer generation chain, and creating specific inputs for optimal output. it also discusses implementing a conversational retrieval chain with a new type of runnable and message history class, providing a demonstration of the final version.', 'chapters': [{'end': 2885.064, 'start': 2779.739, 'title': 'Building chat history-based answer generation', 'summary': 'Discusses defining an answer generation prompt using a messages placeholder for chat history, incorporating chat history into the final answer generation chain, and providing specific inputs for the final generation chain to produce a good output.', 'duration': 105.325, 'highlights': ['Defining an answer generation prompt with a messages placeholder for chat history to incorporate chat history into the final answer generation chain.', 'The final prompt and the final generation requires three inputs instead of two.', 'Injecting a few messages for the chat history, and providing specific inputs for the final generation chain to produce a good output.']}, {'end': 3238.469, 'start': 2885.124, 'title': 'Building conversational retrieval chain', 'summary': 'Discusses building a conversational retrieval chain using a new type of runnable, runnablepassthrough.assign method, and runnable with message history class to streamline chat history tracking and sessions, with a demonstration of the finished version.', 'duration': 353.345, 'highlights': ['The chapter explains using a new type of runnable, RunnablePassThrough.Assign method, to pass through some of the inputs unchanged to the next step, which is a convenient shortcut for extracting one property from the original input and passing it as an object property to the next step.', 'It introduces the usage of Runnable with Message History class to automatically update and inject chat history, where it saves part of the input to the chain as a new human message and the output of the chain as a new AI message, and adds the current history messages to the input of the wrapped chain, along with the function getMessageHistory that returns a new chat history object based on the past session ID.', 'The transcript demonstrates the practical application of the conversational retrieval chain with a demonstration of invoking the final retrieval chain to get an original answer, followed by a follow-up question and logging the result in bullet point form, showcasing the usefulness of tracing and visual exploration using a Langsmith trace to understand the internal process of the chain.']}], 'duration': 458.73, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B82779739.jpg', 'highlights': ['Introducing a new type of runnable, RunnablePassThrough.Assign method, to pass through inputs unchanged.', 'Demonstrating the practical application of the conversational retrieval chain with a Langsmith trace.', 'Defining an answer generation prompt with a messages placeholder for chat history integration.', 'Creating specific inputs for the final generation chain to produce a good output.', 'Injecting chat history messages and providing specific inputs for the final generation chain.']}, {'end': 3880.666, 'segs': [{'end': 3298.548, 'src': 'embed', 'start': 3262.3, 'weight': 1, 'content': [{'end': 3266.263, 'text': "So you're encouraged to modify the above prompts and parameters for different models and data types.", 'start': 3262.3, 'duration': 3.963}, {'end': 3269.925, 'text': "In the next lesson, we'll show how to put this retrieval chain into production.", 'start': 3266.623, 'duration': 3.302}, {'end': 3275.19, 'text': 'including showing off some interactions with common web APIs and HTTP.', 'start': 3270.845, 'duration': 4.345}, {'end': 3278.915, 'text': "Now that you've composed your chain, let's ship it.", 'start': 3276.933, 'duration': 1.982}, {'end': 3285.843, 'text': "In this lesson, you'll go over some considerations for using it as a live web API, like streaming and personalized chat sessions.", 'start': 3279.656, 'duration': 6.187}, {'end': 3286.925, 'text': 'Away we go.', 'start': 3286.524, 'duration': 0.401}, {'end': 3298.548, 'text': 'Now pick up where you left off in the last lesson with loading and splitting our CS229 transcript into a vector store.', 'start': 3293.186, 'duration': 5.362}], 'summary': 'Demonstrate putting retrieval chain into production, including web api interactions and considerations for live web api usage.', 'duration': 36.248, 'max_score': 3262.3, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B83262300.jpg'}, {'end': 3524.336, 'src': 'embed', 'start': 3492.939, 'weight': 3, 'content': [{'end': 3495.9, 'text': 'This way, messages from different users will not get mixed up.', 'start': 3492.939, 'duration': 2.961}, {'end': 3507.677, 'text': "So additionally, We'll want to bear in mind that users should not share chat histories.", 'start': 3496.28, 'duration': 11.397}, {'end': 3515.565, 'text': 'And we should create a new object per session.', 'start': 3510.02, 'duration': 5.545}, {'end': 3519.789, 'text': "So to do this, we'll define and override this getMessageHistory function here.", 'start': 3515.585, 'duration': 4.204}, {'end': 3524.336, 'text': 'with one that takes into account previous sessions like this.', 'start': 3521.013, 'duration': 3.323}], 'summary': 'Ensure message separation by creating new object per session', 'duration': 31.397, 'max_score': 3492.939, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B83492939.jpg'}, {'end': 3584.306, 'src': 'embed', 'start': 3552.34, 'weight': 0, 'content': [{'end': 3556.723, 'text': "Wonderful So now let's set up a simple server with a handler that calls our chain.", 'start': 3552.34, 'duration': 4.383}, {'end': 3558.904, 'text': "And let's see if we can get a streaming response.", 'start': 3557.563, 'duration': 1.341}, {'end': 3561.886, 'text': "So we'll set a port here.", 'start': 3560.465, 'duration': 1.421}, {'end': 3570.731, 'text': "And instead of 8080, let's try 8087.", 'start': 3562.766, 'duration': 7.965}, {'end': 3577.475, 'text': "And then we'll create a little handler here that wraps and calls our chain in streaming fashion first parsing our question from a body,", 'start': 3570.731, 'duration': 6.744}, {'end': 3584.306, 'text': "an HTTP request body and a session ID also from the body, and then we'll create a stream using the.", 'start': 3578.78, 'duration': 5.526}], 'summary': 'Setting up a server with a handler to call a chain and get a streaming response on port 8087.', 'duration': 31.966, 'max_score': 3552.34, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B83552340.jpg'}, {'end': 3646.256, 'src': 'embed', 'start': 3613.893, 'weight': 4, 'content': [{'end': 3617.274, 'text': 'So a few more housekeeping things before we get too far into it.', 'start': 3613.893, 'duration': 3.381}, {'end': 3622.956, 'text': "Let's make a few helper functions that will make handling streaming responses a little bit easier from our server.", 'start': 3617.334, 'duration': 5.622}, {'end': 3627.158, 'text': 'One will be to allow us to read chunks in an async iteration fashion.', 'start': 3623.176, 'duration': 3.982}, {'end': 3631.919, 'text': "So we're going to keep reading based on a passed in reader until there are no more chunks to be read.", 'start': 3627.338, 'duration': 4.581}, {'end': 3637.661, 'text': "And a sleep function to ensure that our notebook doesn't shut down execution before the request finishes.", 'start': 3632.359, 'duration': 5.302}, {'end': 3639.322, 'text': "Now we're ready to call our endpoint.", 'start': 3638.122, 'duration': 1.2}, {'end': 3646.256, 'text': "We'll use the web native fetch function here to make a HTTP request to that server we just spun up.", 'start': 3640.165, 'duration': 6.091}], 'summary': 'Creating helper functions for handling streaming responses and using fetch function to make http request.', 'duration': 32.363, 'max_score': 3613.893, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B83613893.jpg'}], 'start': 3238.529, 'title': 'Building conversational retrieval chains and setting up a streaming server with deno', 'summary': 'Covers considerations for creating a conversational retrieval chain, such as loading, splitting, and querying data using web apis and http, as well as demonstrates setting up a streaming server with deno on localhost 8087, making http requests, handling streaming responses, and testing with different session ids to ensure no cross-contamination, culminating in the completion of the course on building context-aware applications with large language models.', 'chapters': [{'end': 3551.64, 'start': 3238.529, 'title': 'Conversational retrieval chain considerations', 'summary': 'Discusses considerations for creating a conversational retrieval chain, including loading, splitting, and querying data, using web apis and http, and managing message histories for individual users.', 'duration': 313.111, 'highlights': ['The chapter explains the process of loading, splitting, and querying data for a conversational retrieval chain, emphasizing the need to modify prompts and parameters for different models and data types.', 'It discusses putting the retrieval chain into production, showcasing interactions with common web APIs and HTTP, and transitioning to using it as a live web API with streaming and personalized chat sessions.', 'Emphasizing the importance of managing message histories for individual users in a web environment, the chapter details the creation of new message histories per session ID and the prohibition of sharing chat histories among users.']}, {'end': 3880.666, 'start': 3552.34, 'title': 'Setting up streaming server with deno', 'summary': 'Demonstrates setting up a streaming server with deno on localhost 8087, making http requests, handling streaming responses, and testing with different session ids to ensure no cross-contamination, culminating in the completion of the course on building context-aware applications with large language models.', 'duration': 328.326, 'highlights': ['The chapter explains the process of setting up a streaming server with Deno on localhost 8087, demonstrating the use of a handler to call a chain and achieve a streaming response.', 'It outlines the use of helper functions to handle streaming responses more effectively, including reading chunks in an async iteration fashion and implementing a sleep function to ensure request completion.', 'The transcript details the process of making HTTP requests to the server using web native fetch function, specifying the method, headers, and body parameters, including the type of body and content type.', 'It discusses consuming the streaming response by obtaining a reader from the readable stream returned from the server and using async iteration syntax to log individual chunks, followed by a housekeeping step to ensure request completion.', 'The chapter also highlights the testing of different session IDs to ensure no cross-contamination messages and encourages the audience to implement the lesson in their own projects, concluding with congratulations on completing the course and a message on leveraging unique skill sets to build amazing applications with language models.']}], 'duration': 642.137, 'thumbnail': 'https://coursnap.oss-ap-southeast-1.aliyuncs.com/video-capture/YjxlQr72_B8/pics/YjxlQr72_B83238529.jpg', 'highlights': ['The chapter explains the process of setting up a streaming server with Deno on localhost 8087, demonstrating the use of a handler to call a chain and achieve a streaming response.', 'It discusses putting the retrieval chain into production, showcasing interactions with common web APIs and HTTP, and transitioning to using it as a live web API with streaming and personalized chat sessions.', 'The chapter explains the process of loading, splitting, and querying data for a conversational retrieval chain, emphasizing the need to modify prompts and parameters for different models and data types.', 'Emphasizing the importance of managing message histories for individual users in a web environment, the chapter details the creation of new message histories per session ID and the prohibition of sharing chat histories among users.', 'It outlines the use of helper functions to handle streaming responses more effectively, including reading chunks in an async iteration fashion and implementing a sleep function to ensure request completion.']}], 'highlights': ['LandChain.js simplifies building applications using large language models, such as retrieval, augmented generation, or RAC applications, enabling developers to choose an LLM, retrieve relevant texts, tune prompts, and parse text output (Relevance: 5)', 'Langchain uses an expression language to compose chains of components called runnables, with core methods like invoke, stream, and batch. (Relevance: 5)', 'The .stream method returns output from the chain in an iterable stream, useful for showing feedback quickly, demonstrated with async iteration syntax and individual string chunks as output. (Relevance: 5)', 'The chapter explains the process of setting up a streaming server with Deno on localhost 8087, demonstrating the use of a handler to call a chain and achieve a streaming response. (Relevance: 5)', 'The course instructor, Jacob Li, is a Founding Software Engineer at LandChain and Lead Maintainer for the open source LandChain.js, with extensive experience in integrating LMs into web and mobile applications (Relevance: 4)', 'The runnable protocol offers benefits such as fallbacks and parallelism with the batch method, and built-in logging and tracing via LangSmith. (Relevance: 4)', 'The .batch method is useful for performing concurrent operations and multiple generations simultaneously, as demonstrated with two sets of inputs resulting in string outputs for different companies. (Relevance: 4)', 'The process involves creating a chain to generate AI-created names for products, using a prompt template and a chat model to produce responses based on the input data. It showcases the generation of names for products like colorful socks and fancy cookies. (Relevance: 3)', 'Applications are becoming increasingly important and mainstream, urging JavaScript developers to incorporate tools like LandChain.js into their applications (Relevance: 3)', 'LangChain supports two types of language models: text LLMs and chat models, with examples like ChatGPT and GPT-4. (Relevance: 3)', 'The process of initializing a text splitter and setting chunk sizes is described, with a maximum of 512 characters and a 64 character overlap. (Relevance: 3)', 'The chapter discusses creating a retriever abstraction that returns documents related to a given natural language query, instantiating a retriever from an existing vector store, and constructing a retrieval chain for a conversational question answering LLM application. (Relevance: 3)', "The process involves rephrasing questions and saving chat history for context, crucial for the LLM to generate suitable answers, emphasizing the need for techniques to address the LLM's lack of memory of past information. (Relevance: 3)", 'The chapter highlights the importance of utilizing a prompt template with input variables for context from the vector database and the question, initializing the model, and constructing retrieval chains for effective LLM-powered question answering augmented by retrieval. (Relevance: 3)', 'The chapter explains the process of loading, splitting, and querying data for a conversational retrieval chain, emphasizing the need to modify prompts and parameters for different models and data types. (Relevance: 3)', 'The .runnable map is used to answer a question about the prerequisites for a course, showing the preservation of the question and the retrieval of context from the document retrieval chain. (Relevance: 3)', 'The usage of prompt templates is demonstrated, allowing the formatting of user input for model calls. (Relevance: 2)', 'Formatting the output is discussed, emphasizing the use of an output parser to convert the AI message into a string, making it easier to work with the raw output of the chat model. (Relevance: 2)', 'The process involves creating a new chain responsible for rephrasing user input containing references to past chat history into standalone questions that can be followed by the vector store and LLM calls. (Relevance: 2)', 'The use of a runnable map in the prompt template facilitates context properties and question preservation, ensuring that the question has been preserved for passing it along to a later step in the chain. (Relevance: 2)', 'The chapter illustrates the internal workings of the chain by providing a visual representation, demonstrating the sequence of steps involved in the generation of AI-created names for products. (Relevance: 1)', 'Different parameters and separators impact chunk generation and LLM performance. (Relevance: 1)', 'Adjusting parameters like chunk size and overlap provides more natural splits for LLM context. (Relevance: 1)', 'Recursive character text splitter is a good starting point for generic written text. (Relevance: 1)', 'LandChain offers document loaders for various sources like GitHub, PDFs, and databases. (Relevance: 1)', 'The significance of vector databases in retrieval augmented generation and their role in storing document chunks for later retrieval is highlighted. (Relevance: 1)', "The steps for embedding document chunks and adding them to a vector store are outlined, with emphasis on using OpenAI's hosted model for text embedding. (Relevance: 1)", 'The concept of vector store ingestion, using machine learning models to convert document contents into vectors, is explained, with the importance of environment variables and key management highlighted. (Relevance: 1)', 'The use of cosine similarity to compare similarity between vectors is demonstrated, showcasing the ability to search for relevant chunks based on input queries. (Relevance: 1)', 'The retriever abstraction encapsulates the distinction with a broader retriever abstraction that returns documents related to a given natural language query, and retrievers implement the invoke method and are themselves expression language runnables, which can be chained with other modules. (Relevance: 1)', 'Introducing a new type of runnable, RunnablePassThrough.Assign method, to pass through inputs unchanged. (Relevance: 1)', 'Demonstrating the practical application of the conversational retrieval chain with a Langsmith trace. (Relevance: 1)', 'Defining an answer generation prompt with a messages placeholder for chat history integration. (Relevance: 1)', 'Creating specific inputs for the final generation chain to produce a good output. (Relevance: 1)', 'Injecting chat history messages and providing specific inputs for the final generation chain. (Relevance: 1)', 'The chapter explains the process of setting up a streaming server with Deno on localhost 8087, demonstrating the use of a handler to call a chain and achieve a streaming response. (Relevance: 1)', 'It discusses putting the retrieval chain into production, showcasing interactions with common web APIs and HTTP, and transitioning to using it as a live web API with streaming and personalized chat sessions. (Relevance: 1)', 'Emphasizing the importance of managing message histories for individual users in a web environment, the chapter details the creation of new message histories per session ID and the prohibition of sharing chat histories among users. (Relevance: 1)', 'It outlines the use of helper functions to handle streaming responses more effectively, including reading chunks in an async iteration fashion and implementing a sleep function to ensure request completion. (Relevance: 1)']}