Kotaemon - Easy Local RAG UI - GraphRAG with Ollama - Tutorial

hello everyone and welcome to the channel if you're looking to do retrieval augmented generation or rag in a easy way on your local system then maybe have a look at this tool called as caman caman is a word from Japanese language it means good luck gate Kota means good luck and Imon means gate this is what I have been told if I'm wrong please correct me in the comments anyway this is an open- Source tool which does only one thing and that is to offer you a clean and customizable rag where you can talk with your own documents if you don't know what rag is in very simple words whenever you provide the context around your own data your own documents to the large language models it means you are creating a rag because all of these llms they have been trained on huge amount of data but that data not doesn't contain your own system company's data or your personal data if you want to extract intelligence out of that data and if you want to give that data as a context to the cery being ques being asked from llm then you need to do the rag in in which you take your data use chunk it or split it you convert it into numerical representation you store it in the vector store and then whenever then user asks secury the data is retrieved from that Vector store in terms of similarity and there are few other algorithms and then Sim that similar data is augmented with the user secury and given to llm and that is how llm responses are then more grounded and focused on your own data so that is what rag is all about so in this video we are going to install this codan tool locally on our system and then we will see how it works one problem with this project is that uh and that is the problem with many open source project unfortunately is that many of the instructions in the GitHub repo are outdated for example there are few instructions which are for end users but if you apply them they won't work so hopefully this video will help you in order to get it installed locally and then we will see how it works before that let's um let me allow to offer a huge thanks to Mast compute who are sponsoring the VM and GPU for this video if you're looking to rent a GPU on affordable prices I will drop the link to their website in video's description plus I'm also going to give your coupon code of 50% discount on range of gpus so please do check them out let me take you to my terminal where I'm running 1220 2. 04 and this is my GPU card and VTR RTX a6000 let me clear the screen first up let me create a virtual environment with python 3. 10 let me run it it is going to take bit of a time to get activated so let's wait and the cor environment is activated next up let's get clone the repo and then CD into it that is done let me clear the screen the next step is to install some of the prerequisites from this repo and for that let me paste the PIP command where I'm installing everything from this caman so let's wait for it to finish and again this is going to take bit of a time it installs so many things and that is done took Fair bit of time around 5 minutes to get that installed and we need to install few more prerequisites from this lips so let me run it and let's wait for it to finish this is again going to take bit of a time okay that is good this was quick let me clear the screen next thing we need to do is to set our environment so they already provide a v file in the root directory of repo so just to va.

EnV and you will see sorry if you go to the top that you can set your APN open a here of course you would need the open a key which you can grab from open a and that's a paid option and then this is the Azure if you are using open a in the Azure cloud or for cair or for any local model with Lama cp. Python and so it means that you would need nomic embed for for that or you could go with the graph rag so what I'm going to do in this one I'm just going to use graph rag but if you have Lama CPP running you can download these models and then run it with the graph rag uh or this uh kodon but I'm going to go with the graph rag in this one similarly you can also grab your PDF GS which I will show you how you can do that so this is primarily for in browser viewing of PDF files so let me set my key here and then I will save the file and clear the screen so I have set my key here now next up if you want to set in browser PDF viewer what you need to do is to go to this directory okay there is none here so we'll just go manually C CD km again km and then assets and now do LS dltr then there's no pre-built directory so we'll just create a pre-built okay oops I just made a typo here I'll just say make directory prebuilt and CD pre-built let's clear the screen and now here what we need to do we need to download this PDF GS extension let me paste the link here and by the way this is available in in there GitHub repo which you can just simply get it and this should have been done by here there you go and all you need to do is to just simply unzip it here and that is done cool so now we have our extension to so let's go back and now all the installation is done if you do ls- ltrr here you will see that they already have provided um app. py and this is the file which we need to run in order to start it and by the way you can even uh run it with the docker so now all you need to do is to run this Python 3 app.

py in order to start it so let me run it okay it has you see that it has opened it in the browser here and I have used the user admin and password is also admin and this is a console of kodaman in front of you and now in order to do do the rag all you need to do is to just upload your file from here from left so for example if I just go grab one of my own PDF file and I'm just going to go with this PDF file hopefully it is going to yep so it has been uploaded as you can see and it has also indexing it which means that it is just storing it in the vector store after converting it into vectors you just have to wait for a bit that is done and then you can see that indexing has been completed and now if you want to chat with it all you need to do is to just uh chat with it here you can uh so for example if I ask it who is bad mza because that file just contains information about myself so you see as soon as I pressed enter it said that thinking and then uh on the right hand side due to the extension which we uploaded it is showing us the um PDF file and then also the relevance score is one so it means that it found a perfect match and it has given us the data from my own file otherwise the model doesn't know about me right and it is using a graph rag at the back end which is quite cool so if you click on files here at the top you will see that this is the file we have uploaded and this is a loader and all that stuff it is telling us about you can upload many files here as many you like you can if you have changed your file you want to reindex it you can do that and these are all the file types which are supported like PNG JPEG and all that stuff if you click on resources again so we are using graph track so that is why it is showing us there if you want to change your llm so you see I'm using uh AMA is not default but you can make it and I will shortly show you how you can also add more if you like from other vendor like Lama app chat Azure chat open a and all that stuff and I think this is going to increase this is embedding um model which it is using so embedding model it converts your text into numerical representations and it's a multi-user stuff so right now we have admin users with the password admin you can change it of course just by going in settings and from there if you click on user setting you can change your password and you can create more us users now this is a retrieval one this is what I was referring to so you can just keep file loader as is but if you want you can use Adobe or Azure document intelligence for llm 1 you can go with o Lama here you see instead of open here you can go with AMA and then you can just simply save your changes if you don't know what AMA is olama is one of the easiest and fastest tool to run large language models locally it's all free and you can use lot of models there so if I quickly open a new window in my terminal and do all Lama list here you will see that ama is already running and if you don't know how to install AMA I have done heaps of videos on it so you see that I have this Lama 3.