at the end of this video you've built your own chatboard with streamlit and this chatboard is going to answer questions based on documents you provided and we're also going to work with a semantic database that we haven't used before and that's pine cone now pine cone runs fully in the cloud and this means that we can actually use it in production now before we dive into the code I want to take a moment to explain you what wre actually is and why it is so popular the concept of w was first introduced in this paper in 2021 I'm going to assume that most of you know how expensive and how complex this to train your own large language model or even to find unit and that is exactly why R was introduced so I'm going to explain rag with a very simple example there is a user and the user wants to know what the color of a banana is so everything starts with the user asking what is the color of banana and this question is then sent to a semantic database now this semantic database will try to find texts in the database that are similar to the question so it will try to find text with the word banana or maybe the word called as you can see we get back one result from the semantic database a banana is a delicious Jello-O piece of fruit so there is some information in this text which is not really relevant to a question for example that is very delicious but there is also some relevant information here such as the color of the banana now at this moment we have only worked with the semantic database and we haven't made any call to the large language model yet now that's going to change because now we're going to make a call to a large language model and we're going to ask it please answer the below question with the answer I'm providing you so the the question is still the same what is the color of banana and the answer we're also providing to the large language model which is a banana is a delicious jell piece of fruit now the large language model is going to use the answer that we are providing it to answer the question that we have just asked it and the large language model is now going to answer the color of a banana is jello so before starting to build a chatboard I first want to show you a really simple example of a semantic database and I also want to show you how the retrieval process works because the retrieval process is so important now as usual you can find all the Python scripts that we're going through today on like GitHub so you can find all the scripts and all the installation instructions right here of course if the script is useful for you please don't forget to store it and then let's navigate to Pine Cone and in order to follow this video you going to need a pine coin account so navigate to Pine coin. io and then you can create an account there that's entirely free then navigate to API keys and this is where you're going to create an API key so it just give some kind of a description click on create key um copy the key then you're going to rename the env. example file to justv so remove do example then paste your pine cone API key right here and you also need an open AI um API key you can get the open AI API key right here on platform.
open. com API Keys create new secret key so just make sure that you have those two API keys and you don't need anything more in order to follow this video then as soon as you've added the API keys right here you've saved the documents then let's navigate to sample ingestion dopy it's going to check whether the index already exists and if it's not existing yet it's going to create it and it's going to create it with these specs so you see here that the region is Us East um it's hosted on AWS and this is really important you see here the dimension which is 372 and this Dimension is related with the embedding model uh that we're going to use so in a second we're going to um add these text to the index and for every text the embedding model is going to generate some kind of code and this code represents the semantic meaning of this text and basically the more Dimensions this embedding head the better it's able to represent the semantic meaning of this text so you see here that Dimension is 372 and this is related to the embedding model I'm just searching for the embedding model this is the embedding model that we are using so that's text embedding three large it's a model from open AI but if you are using another model you can use another model but then you also have to change Dimension and I'm just going to navigate back to Pine Cone because you can also manually generate a database so if I navigate to database here click on create index thas test like this and then it's asking for the embedding model that we are using and you see that if I select for example text embedding 3 small this model has only 1536 dimensions and the text embedding 3 large model that we are using at the moment has 372 you can also use Ada 002 which also has 1536 Dimensions so the dimensions that you provide um right here are related to the embedding model that you are using and of course we have to to use the same embedding model for the ingestion so for filling the database as for um the retrieval process but I'm going to tell a bit more about that later so the documents that we are adding these are just example documents that are coming from the Lang chain website you see that you can also provide metadata so for example here there is a source and you can use this metadata in order to filter so you can ask your retrieve augmented generation model to only fetch news or only F fetch tweets for examples but you can as well use it in order to store information about the document where it's coming from and ask your llm to also provide the source where this piece of data is coming from um so adding metadata is really important and the better you structure with the more information of course the r can provide to the end user um then you see that we are adding all these documents to a list right here and then there is one more thing that we need to do we have to calculate a unique identifier for every uh document I'm just doing this by here and starting with ID Zer id1 id2 etc etc and this will allow you to later edit or delete a specific text if that's what you want I'm going to execute this piece of code and you will see that the index to which I'm ingesting uh these um these embeddings that's going to be sample index and of course for our chatboard we're going to use a different index so let's run this piece of code it can take a few seconds and now the uh embedding should be ingested into the database so let's navigate to the database and if you refresh the page you should now see that an index has been created so let's click here now you will see here that we have 10 results and you will see that all the text have been loaded and have been embedded in the database okay now comes the second part and that's the retrieval process so let's open sample retrieval py this script is linked to the same index um you see here a sample index we are initiating here the pine cone database and as I just mentioned for the retrieval part you need the same embedding model if you use a different embedding model it's not going to work and you see that we create here an object which is uh the vector store and this this is basically the database that we are referring to now the first thing that I want to show you is that it's really easy to um do a similarity search so this means that we we are going to search for text that are similar um to the text that we are providing then we are using this method here similarity search with score and that's because I also want to get back a score and this score is going to tell us how similar the text that we are providing is with the text that is retrieving you can as well specify the number of rows that you want to get for now I'm just going to leave this out but you can specify that you want to get like only one or maybe three um texts and if you go to the different embeddings and you will see that this text is the text that has uh the most similarity with the question that we are asking so let's run the script and let's see what we get back and this is exactly why I want to show you this um we're getting back four texts which have some kind of similarity with the question that we asked um but I want you to take a look at this last number here which is the similarity score so of course if I if I'm asking what did you have for breakfast um this is the first record that pops up and it has a similarity score of 0. 57 now we get three other uh similar answers but you see that the similarity score is really low it's like lower than .
05 so they are really not related to uh what we just asked and now we can use this similarity score to filter on the records that we want to get I'm just going to comment out this similarity search so let's do it like this and another component that Lang chain is providing us rather than the similarity search with score is the retriever but the retriever is something that you will use most of the time if you work with Lang chain because it is something that can be called and it's called give back some results now you will see here that um the search method is similarity score with threshold you'll see here that I want to get Five results back and will see that the score threshold is 0. 5 so this is really important so um the records that were not relevant were something like were below this and record that was relevant was a bit higher than this but just a bit higher right so you can change it number 0 to five makes sense 0 to3 or 0 to2 also makes sense um but just find something that you feel comfortable with I'm just going to go with 0. 5 and I'm going to ask the same question and now you will see that we only get relevant answers so let's try this out and now you will see that even with the score threshold on 0.
5 I still get a lot of Records which are not really relevant and this is probably because I left out the filter so let's change this one to 0. 6 let's run the script again and now you will see that we get only back one result which is the relevant one okay so that was a lot Theory I think it's good that we didn't skip it because the quality of the W that you're going to build highly depends on the ingestion process and as well the retrieval process so let's start working with real life data and let's start building our chatbot so you have seen the demo of this process but let's now do it with real documents and with a real chatbot the document that I'm going to use um in this demo is this one it's the research paper about retrieval augmented generation so how this script Works everything that you store in this folder doents is going to ingest in the database so you will see that we create a new um Index right here and the name of the index you can Define in the EnV file and then as soon as the index has been created you're going to load all the PDF files in the documents folder then you see here text splitter so this text splitter recursive characters text splitter this is a component from Lang chain and it's going to chunk all the text in the PDF file in chunks of 800 characters you see as well that there is an overlap of 400 and this basically means that every chunk overlaps the other chunk so in case a text is split in the middle we should also have a chunk that contains all the information and this is where we use the text splitter in order to generate the chunks from the world documents we call them documents you've seen this um little script earlier where we generate a unique identifier for every chunk and then we add them to the database so the script is entirely the same as the script that you've seen earlier the only difference is that we now as well have an actual document and that we are going to chunk that document into smaller parts then let's run the script in order to starty inje process and as soon as the inje has finalized let's navigate to the pine cone database as five the page and you will see now that there are two indexes and this is the one that we've just created L chain sample index now you will see that um we have now 165 records and you will also see that there is way more metadata so you see that there is the source here which is the name of the PDF file there is a producer basically anything that has been retrieved out of the PDF file and as you can see there is as well a page here so if you provide the page um The Source Etc then you can as well as the large language model to as well provide a source about the information that is providing to you so that you can uh yeah check if it's correct you see that there is as well a field text and this is where the actual text is stored that the uh rack is going to use let's navigate back to vs code because now we've done the ingestion process I also want to show again the retrieval process but this time we're going to ask something about this documents so you see this entire python file is exactly the same as the other uh retrieval python file um here you see that I have a score threshold of 0. 5 and my question is what is retrieval augmented generation so let's run this file and if I scroll up you see now all the different chunks that have something to do with retrieve augmented generation and I think if I remove ks5 here it's probably going to gave us way more documents about retrieval augmented generation so for now we see that the retrieval method is working so let's navigate to the actual chatbot now the chatbot that you see here is exactly the same chatbot as we've built in one of my previous videos so if you want to know exactly how it works then definitely check out that video for now I'm just going to show you a couple of things so of course we're going to start with a system message you are an assistant for question answering tasks then um you will see here that we we get the input from the user and as soon as we receive the input from the user then we're going to append this to the messages and then this is where we initiate the llm the large language model it also has temperature etc etc and then you see here that we use the same retriever that we've also used in the previous script uh we invoke the Retriever and then we're going to combine all the documents together in docor text and then you see here that we provide a second system prompt and in this system prompt we're going to as well add the context and Doc text is of course the combination of all the documents that we have retrieved from the Retriever and I see here that I put the system prompt on the screen just because I want to see uh based on what it is answering me and then you will see here that I append the system prompt to the streamlit messages and because I'm only putting the messages from either the human or the AI message on the screen um this system prompt is then not shown to the user and that's of course because the human is only interested in the answer human is not interested in like how everything works on the back side okay so now in the messages we have now the message from the user so this is the actual question and then we have the system prompt that includes all the answers that we have just retrieved from the semantic database and then this is where we invoke the large language model we provide the entire chain like everything that happens so the question from the user um the system message that includes everything that we have retrieved from the semantic database and then this is where we then add to the screen the answer from the large language model so let's try this out and you can run this file by typing streamlit run and the name of the Python file which is chatboard W.
