today we're going to build a research assistant from scratch This research assistant from tavil AI is one of my favorite llm applications it's complex it's more than just chat it's actually useful and it performs well um and and so we're going to build this from scratch using Ling chain and uh take a look at what goes into all the decision points along the ways so if you're not familiar with tavil AI research assistant it is a web hosted platform although there's also an open source version and we'll talk about that but it's a web hosted platform where you can give it a task such as what is the difference between Lang train and Lang Smith that will do a bunch of research from the web and then write this really long report so this is much longer than uh uh you know you might get back from chat GPT or or one of the other search engines it takes a lot longer as well as we'll see there's a lot going on behind the scenes so you're not going to get back this instantaneous response so there's there's pros and cons but I really like this because you are able to do a lot of research behind the scenes you are able to put more work in and that generally allows better and more interesting responses so tavil AI is based off of this open source uh repo called GPT researcher um we've worked very closely with them at linkchain um and and to understand what's going on they have this great diagram and so what's going on in the hood is it takes in a given task and then for that task it generates a bunch of of research questions so it will generate a bunch of research questions and for each one it will then look that up online it will get a bunch of web pages so this is a web research agent we'll talk about how to make it more generic than that but this is a web research agent so we'll get those web pages it will then summarize those web pages and then using all the summaries it will generate a final report so that's going on what one the hood a lot of moving pieces a lot of it can be parallelized as well so I'll talk about how to do that so it gets a little bit faster and uh uh yeah let's uh let's jump into it and build it from scratch we are going to be using open AI so if you don't have an open API key you can go ahead and get one um from open AI uh themselves um and then we're also going to be using lsmith a bunch to kind of like show what's going on under the hood so this is a logging debugging tool that we built out Ling chain if you don't have um access to this yet feel free to shoot me a message and we can get you off the wait list so the first thing I'm going to do is I've got my environment I've got my code editor set up what I'm going to do is set a few environment variables I'm going to set the open AI API key and I'm going to set my lsmith keys I want to pause the video and do that now so you don't see my secrets all right done so now uh we're going to install a few packages um we're going to install L chain open Ai and then Duck Duck Go as well so we're going to use them as our search engine got to look up what their python package name is here it is duck duck go- search so we're going to install that um we made some more libraries but we'll install that um to get started while that's going let's think about what the first thing that we want to implement is so let's start from one of the smaller kind of like parts which is basically the summarization of a web page so at the core what it's doing in the middle is it's taking a question it's taking a web page and it's summarizing the response and the reason that we want to do that is a few fold one there's going to be a lot of different web pages and a lot of different information here so maybe we could pass this all to the final report agent maybe the 128k context Windows enough but even if it is it's generally more performant a split up into these smaller tasks um and ask ask the language model to kind of perform those it's more focused for one um you can paralize it better for two um and then three you know you can use other models besides the really long context window ones so let's focus on this first task where we're going to take in a web page take in a question and summarize the response so let's write this so and then once we get this working we can work worry about kind of like scaling that up so just as an example let's go to the Lan chain log um let's pretend we're asking about the difference between L chain and Lang Smith Let's Pretend one of the sub questions is like what is link Smith so linkchain blog link Smith let's find one of the blogs all right so let's take this URL let's just set this as the URL um what are we going to import from Lang ch ch. chat models import chat open AI from Lang chain do prompts import chat prompt template um well let's uh make some prompt template um hold I'm going to pause it and fix my python interpreter issues to avoid all those R Squig cool all right we're back so let's make our template let's say our template is going to look something like summarize the following question based on the context we're going to update this this just a placeholder we're just going to get started let's put question question context context prompt we're going to set chat prompt template from template template um okay so that's that now we need to figure out how to get the content from this URL in order to do that we're going to need actually two more modules we're going to pip install requests and then we also going to want let's look up beautiful soup I may have spelled that wrong um yes so this is the one we want um and let's add that these are going to help us scrape some uh web pages all right so we've added those we're going to go back we're now now going to create a function to get the content of this web page so I'm going to put this here import requests um from PS4 import beautiful soup this is just a nice little helper function that's going to scrape text from a web page it takes in a URL then hits that with their requests um then scrapes it with beautiful soup if it was able to otherwise it just uh prints that and prints that prints the error and says that it's failed to retrieve the web page cool so we've got that we can now do something like um page content equals scrape text URL um this could be really really long so what I'm going to do is I'm just going to take the first like 10,000 characters that should probably work um I am am then going to create a little chain to get this working so let's do prompt chat open AI um model let's use let's use a longer context one so open AI models um let's see what we can use Let's uh use let's use this one you know cheaper longer content window that sounds fantastic so let's use that um we then so this is going to return a chat message we just really want the string from here so from Lang chain. schema.
help parer import stir out wither this is really simple just parses the message into a string um let's call this our chain and then let's do chain dot in v um uh question what is BL Smith content um page content so again we're using this pretty basic prompt we'll update it it'll get better let's just make sure this works right now um so let's do Pyon main. py we run it okay we get some error context cool update that run it okay I forgot to print out what it was luckily we can go to Lang Smith we can see here we've got this um and we can see the chain really simple one only three steps prompt template L um output parser input is this uh input is basically this dictionary of context and question and then the output um and then uh we've got the answer here the question is asking from Lang Smith which is described as a uni platform for debugging testing evaluating monitoring all applications cool it's working um let's update the prompt that we're using a little bit awesome so I've updated the prompt now this is actually taken from uh gbtc researcher repo um so we're going to yeah we're going to use that heavily and and and um take a lot of inspiration from that so we've got this summary template it's you put the text up here you ask it to answer the question um updated down here um let's uh run this again make sure it's working notice that I changed the context key to just text to update the prompt template that they were using if we go back here we check yeah so we can see that we now have a different response um and looks a little bit better than before the previous one uh said that it was doing a summary this just responds that's better cool um let's make this um let's make this chain a little bit more involved so what we're going to do now is let's actually um add in a step that is going to look up the and get the content by itself so we're going to import from linkchain um schema runable will import runable pass through and so what runnable pass through is going to do is going to take the current input and add into it so the current input will just be a question and then what we're going to do is we're going to set text equals to Lambda um and actually let's make this not only a question but also a URL URL so we'll pass in url here we'll go over how we're going to get those URLs in a little bit um so let's set text equals to Lambda X and then basically this bad boy right there so now we're actually doing so we're moving more and more of it into the chain and this is useful because we're going to package this up into one big application later on so let's run this now we get a here text um why is that happening that is happening because okay so what was happening was basically I'd forgotten to put this assign thing here um assign is the thing that actually uh does the assignment of text to a text variable before that it wasn't doing anything I also updated this to be uh uh to pluck the URL key from the input X and pass it to scrap text before I had just hardcoded this here but of course that's wrong we want it to be as part of the input we can run it and we can see that we get something like this where we pass in the URL on the question and then it basically uh runs uh behind the scene gets this output for text um passes it into the prompt template passes it into the passes into the output parer all right so we've done one part which is we have uh we've gotten to a place where we can take a URL and a question and and we can uh get content for it and and then summarize it so let's now go uh one step further and let's do this for this question we're first going to look up stuff using duck Dogo and then we're going to get the results from duck. go take like the three URLs and then we're going to apply this to each one so let's see what that looks like all right so I've added in a few things I've imported duck ducko search API rapper from Lang chain this makes it super easy to work with um and defined um uh this wrapper and then also Define this web search function which is going to take in um the query the number of results that we want to get and let's just set this to three um as a default to make it easy and then uh it's going to return a list of links so what we want to do is for a given query um we want to look up the list of URLs and then we basically want to apply this function to that okay so what that's going to look like is let's do web search um and we're going to wrap this in a runnable Lambda and basically what the runnable Lambda is it's going to let us Define a function and then from there um but it's defining this function in this uh and using Lang chain expression language so that we can easily compose it with other things um so let's do X web search x.
question um cool this is going to return remember a list of URLs from there what we want to do is we want to turn that list of URLs um into something that we can pass in in to uh uh this chain so the chain takes in dictionaries with questions and URLs so what we're going to now do is actually I think yeah so what we're going to do is actually let's use runable pass through again because we just want to assign um let's assign URLs equals to that um then so this is going to add a new key called URLs then what we want to do is is we we're going to have this be a dictionary it's a question and then a list of URLs we want to turn that into a list of question and URLs so let's we can write some function to do that Lambda X um we can do uh question X question um URL U for you in X URL so We're looping over the URLs that are passed in and then for each one we're constructing a little dictionary where there's just that URL and then that question so this is going to give us the input and it's a list and so then what we can do is we can call let's name this something like um and summarize chain what we can do is we can do this and we can do this map thing this map is going to apply this chain to every element in the list so let's now call this chain that um let's do oop that's the wrong thing let's you chain. invoke we can now remove this URL because we're going to do the researching of the URLs under the hood let's run this see what happens okay so it took a little bit let's check out Lang Smith and see what actually happened so we had this input and we get back in output which is a list of things um so it's a list of things and so what's going on under the hood is first we have this thing which is getting a list of uh blogs to look at this is then getting converted into a list of dictionaries with each element in the dictionary having a URL and then a question then for each of these elements we're applying this chain and this is the exact same chain that we were working with before so if we look into it it this is the one where we're asking um things to we're asking to summarize the above text in short with the following question perfect this is working great so we've gotten to a point now where we have something and we can generate some answers still returning a list um that's probably fine what we want to do now is this is still just one question what we want to do is generate a bunch of sub questions and then from there we want to apply the same chain to all of them that's going to get back a list of lists of things then we can then combine all of those things into one big thing and then pass that to the final response okay so there a lot to digest but to start we're going to work on adding a uh llm call that'll generate a list of sub questions to go research all right I now added this new search prompt which and again this is taken from GPD researcher so it's going to look up uh basically um uh let's remove this part so they have an agent prompt we're going to skip that I'll I'll link in this uh I'll link in the description to their repo and then also our Lang chain template which uses this it's more advanced basically we're going to write um we're going to use this one which just writes uh three Google search queries to search online that form an objective opinion from the following question so we've got that that um we can now construct a search uh search question Chain by doing search prompt um we can then pass this into chat open AI then from here what we want to do is we want to split this out into so this is returning like a list and then each one is a uh it's returning a list of quoted search questions or search queries so we want to now parse that bit out that bit is actually just Json so we are just going to import Json where then going to use the string out parser to just get the string and then we can do json. loads I believe this should work and this will now return a list so let's double check that that is right and let's just run it on this and let's start now changing this up what is the difference between L Smith and Lane chain let's run this let's take a look at what's happening this is now again just a simple list and we get back out as our final output we get uh some questions Lang Smith versus Lang chain similarities and differences pros and cons of using Lang Smith user reviews and experiences with L chain so these are all sub questions that we now research to answer this overall question and so in order to do that what we're going to do is we want to take the output of this and we want to pass it to this chain up here that we created let's call this like web search chain and we want to pass each element into this um again we need to do the thing where this is returning a list of questions each one takes in each of these takes in a single question so what we want to do is let's do search chain and we'll type that into web search chain.
map again let's call this let's see what happens cool okay so we got uh some error with the input to runable pass through. assign must be a dect let's look up to where we're using runable pass through. assign which is right here and so what's going on is this returns a list this actually expects the input to be a list of dictionaries but this is returning a list of strings so we can just map that pretty quickly um with a nice little function Lambda X um and then we can do question Q for Q in X um try that we to see what's going on we can actually see this while it's running um so we can see it start to populate still pending looks like it's done yep all right success we can see now that uh we have this question it uses this it gets back a list of um questions I'm going to set temperature equals to one just to be easier to uh debug so I set that to that um and we can see that the final answer is this list of lists things um cool okay so we get this list of lists um let's um now pass this in to a final prompt and ask it to basically generate a final report so what we're going to do is we are going to need something to flatten that list of lists into a nicely formatted uh string that we can pass in and then we're GNA need a prompt that we can pass into that string so let's first get that prompt cool all right so we've taken some prompts again from GPT researcher we've got the system prompt and then we've got this uh more in-depth prompt that writes a certain style of research report you can easily swap these out but we'll construct this with the writer system prompt and the research report template this takes in two variables research summary and question so question we already know what it is um and so now we need to add research report summary this is going to be coming from this chain let's name this full research chain but we want it to be a string right now we have a list of lists so what we're going to do is let's use runable pass through.
assign again let's call This research summary because that matches the variable here let's do uh full research chain and then let's do Lambda X um we want to be joining things together um so let's this is just going to be some simple string formatting we can easily change that if we want um so let's join together um various things the things that we're going to want to join together are actually let's just write a little hper function for this to make this easier um def collapse list of lists list of lists content equals that for l in list of lists content.
