In this comprehensive generative AI course you'll dive deep into the world of generative AI exploring key Concepts such as large language models data pre-processing and Advanced Techniques like fine-tuning and rag through Hands-On projects with tools like hugging face open Ai and Lang chain you'll build real world applications from text summarization to custom chatbots by the end you'll have mastered AI pipelines Vector databases and deployment techniques using platforms like Google Cloud vertex Ai and AWS Bedrock Baker Ahmed B created this course JV is the most demandable skills nowadays across Industries if you see all the industries
have started using genv in their product development if you want to level up your skill and if you want to crack good job nowadays with a good package definitely you should know about genbi if you're looking for well Organized n2n complete genv course I have a very much good news for you recently I have published one amazing n2n genv course so here I have covered everything you need to know to master the genbi this course will cover Basics to Advanced concept of genv so here we are not only going to focus on the theoretical part
we'll be also focusing on the Practical implementation of genbi we'll be exploring different different kinds of large language model with that We'll be creating different different kinds of gen VI based application so let me show you the course content of this course so guys as you can see this is the course content as I already told you we'll be starting from very basics of the Gen and we'll be covering till Advanced part of the Gen so here we'll be starting from uh introduction of the Gen VI so if you're completely new to this field no
need to worry I'll cover each and everything you need to know About gen VI then we'll be learning about data pre-processing and emitting because going forward whenever you'll be using large language model and to give your data to the large language model first of all you have to know how to process this data and how to generate the embeddings of the data and to process our data we need to know some of the technique so here we'll be covering these are the technique in this section then we'll be starting with the large Language model we'll
be learning about different kinds of large language model we'll be learning about commercial large language model we'll be also learning about open source large language model we'll be starting with the hugging F platform and its API because if you see the hugging pH hugging pH is having all kinds of large language model whether it's a commercial model whether it's a open source model all kinds of model are available inside the hugging face Platform then we'll be also learning about open Ai and its platform because if you see nowaday open AI is evolving a lot they
are coming up with different different kinds of large language model like they are coming up with image model language model OKAY different different kinds of model they are bringing so in this course we'll be learning about the complete openi and its platform so that you can use openi platform to implement any of genv based application we'll be Learning about prompt engineering because if you see prompt is everything for the large language model so if you are designing a good prompt definitely you will be getting a good response from the large language model so here we'll
be learning how we can designer efficient prompt and we'll be also learning about different different kinds of prompting inside large language model in this course we'll be mastering the vector database we'll be learning Different kinds of vector database like Pine con we chroma files and so on this Vector database will help you to create a knowledge base so whenever you will be creating any kinds of rack based application any kinds of LM powered application this Vector database concept will help you a lot in this course we'll be mastering different different kinds of genbi based framework
like we'll be mastering Lang chain llama index CH leit and so on these are the fr work will Help you to implement different different kinds of jni based application in this course we'll also learn one very important topic inside jtbi called retable augmented generation that means rack so we'll be learning how we can create different kinds of rack based application with our custom data even we'll be also learning how we can find you any kinds of large language model on top of our custom data after completing all the topics inside genbi we'll be Starting with
some n2n project implementation with the deployment so here we'll be implementing these kinds of project from scratch we'll be using modular coding to implement this kinds of project at the last of the course we'll be also discussing about llm Ops if you see inside genbi llm Ops is the most trending topic nowadays we'll be learning how we can use different different kinds of llm Ops platform like we'll be learning about Bedrock vertex Ai and we'll be learning how we can use these are the platform to implement efficient large language based application so yes this is
the complete curriculum of our course so if you want to start your career with jvi guys so this would be the amazing course for you so make sure you complete this course till till the end and this is my promise guys you will become Champion inside geni with that guys all the best and I will see you in the course hi everyone Welcome to the course of generative VI my name is BK Ahmed Bui and I am a data scientist and Mentor I'm having more than four years of working experience in the field of machine
learning deep learning generative mlops and so on I have already worked with so many multinational company as a developer even I also work with so many attech company as a mentor I have also taken lots of bats related generative machine learning deep learning computer vision National language processing and so on so throughout the entire course I'm going to be your Mentor so as I already have some experience from the industry so in this particular course I'm going to share my knowledge and experience with you so in this course we'll be starting with the very Basics
concept of generative and we'll be completing till Advanced part of the generative a so if you're a beginner if you want to start your career with the genbi so this Course is for you I have designed this particular course in a such a way so that anyone is coming from any kinds of domain they will be also able to understand so if you're from non-tech background if you're from Tech background it doesn't matter so if you are interested in Genera only you just need to start with this course everything I will take care apart from
the theoretical understanding I'm also going to show you lots of practical in This particular course we'll be doing lots of Hands-On in this particular course we'll be solving different different problem statement in this course so that your understanding would be more clear hello everyone welcome back with another video in this video we'll be discussing about what is gener tvi so before discussing about generative a first of all uh let me introduce some of the real world application of the generative you are Using in your day-to-day life so I think you know about chat GPT Google
Jin metal Lama right so these are the application you are using in your day-to-day life I hope everyone has used chat GPT at least right since at chat GPT what we can perform we can give any kinds of prompt we can uh do the conversation we can do the summarization we can generate the content we can generate the code we can do the Tex summarization any kinds of task we can perform in the chat gbt Right so similar wise Google has developed their own product called jini Okay Google jini so with the help of Google
jini also we can perform the same things the things actually we perform usually in our chat gbt so the same thing we can perform with the help of metal amar2 as well that means chat GPT is developed by open AI uh Google J mini is developed by Google and Lama is developed by meta okay meta that means Facebook so all the TS company iies are Working on generative day by day they are improving this particular generative they're bringing actually different different large language model and they're uh launching different different application okay for the users nowadays
all the companies are using generate to implement their product So currently in the market uh genbi having more popularity so that's why you have to know what is genbi exactly how genbi works and definitely if you want to work In the current job market you have to know about this genbi you have to learn about generative value okay this is the idea see all the application I showed you here like chart GPT Google jini metal Lama so they are using something called large language model in the back end okay with the help of large language
model they're able to perform these are the operation now first of all let's try to understand what is generative a exactly see generative a is nothing but Generative AI generates new data based on training samples generative model can generate image text audio video and so on data as an output so I think you have already worked with machine learning deep learning computer vision and so on right so there you used to use something called discriminative model and what is discriminative model based on some input actually it will predict the output of that particular data okay
so this was actually discriminative model and to Train the discriminative model actually we used to use something called label data that means that means whenever we used to prepare the data for the training we used to prepare the let's say uh input data as well as the output data that means you have to pass the X data and Y data that means input output both okay and if you uh give input and output both it will learn the relationship between input and output it will learn the pattern from the data Then it will able to
predict something on top of the test data so that was the idea in the discriminative model that means in the prediction model right but generative models are different okay so generative models can generate a new data based on the training samples so here you'll be giving some training sample which is called unstructured Data So based on this training sample data it will try to generate some new data that means inside jna when whenever you are Giving any kinds of unstructured data as an input your generative model will try to understand this unstructured data it will
learn the pattern from the unstructured data and it will try to generate something from that particular sample you are giving okay this is the idea and the output can be anything it can be text it can be audios it can be videos and so on okay as I already told you inser generative we not only work with the text but also we also work with The image videos audios and so on okay so all the unstructured data can be used inside generate this is the idea so that's why I generative AI is a very huge
topic inside generative AI actually we are having generative image model as well as the generative language model that means you can work with the languages that means text you can also work with the images videos okay and audios so there is another model you can consider generative Audio model but at The end you are converting this audio to the text representation that means language representation that's why I haven't mentioned this Audio model separately okay this is the idea because audio is nothing but it's a frequency and from the frequency we can convert to the textual
representation okay this is the idea there are so many API you will get even Google has the API you can let's say convert SP to text okay with the help of is to text you can easily Convert any of audio to textual representation okay this is the idea that means what is generative a Genera a is nothing but it generates new data based on the training sample you are giving and generative model can generate any kinds of output whether it can be image it can be text or it can be audios okay now let me
tell you how they got the idea like how this generative model can work see this idea had taken from real life only let's say uh if I give You 10 different scat book okay let's say if I give you 10 different cats book and if I tell you just try to read all the 10 different cat book let's say you have read all the 10 different cat books now if I'm asking anything related cats okay you will be able to give me the answer because you have already studied about the cats like uh 10 different
books I have given and this was the enough data for you right to learn about the cats so that's why if I'm asking Anything related to the cats you'll be able to give me the response so here you have become one cat model okay you have become one cat model and that is why if I'm giving any kinds of question you will be able to give me the response okay so similar concept applied in the generative also so they implemented one model called generative model in the generative model they feed actually tons of data that
means huge amount of data they uh trained okay that particular Model and then actually they were uh giving some kinds of question and that particular model was able to give the response okay this is the idea that means here you are feeding tons of unstructured data and your model is able to uh let's say learn the pattern from the unstructured data and there would be a position your model will be able to capable enough to give any kinds of response okay based on the question you are asking now you can ask me why Generative models
are required see I can give you thousands reasons of the generative model requirement but here I have listed down some of them so the first thing you can consider understand the complex pattern from the data because nowadays you can see people are using unstructured data a lot unstructured data means it can be text Data it can be audio data it can be videos data right so this is called actually unstructured data and in Today's world actually people are using internet broadly and in in Internet actually people are generating huge amount of unstructured data okay they're
using different different social media they are using different different platform so that's how actually they're generating tons of unstructured data so that's why it's very hard to understand the pattern from these kinds of unstructured data with traditional machine learning model okay so that's Why this gentic model comes into picture so it can easily understand the complex pattern from these kinds of unstructured data the second thing you can consider the content generation that means that means your generative model can generate any kinds of content it can generate code it can generate let's say any kinds of
story it can generate any kinds of music any kinds of videos I think nowadays you have seen like there are so many application came in the market so If you give any kinds of prompt it will generate the video for you okay the complete video for you that means it is generating some content okay it is generating some content based on the prompt prompt you are giving and that's how for the content creator also it is becoming one of the very uh used tool okay for their content generation and all let's say you are a
Blog writer now you don't need to write the blog from scratch so what you can do you Can just pass the topic and your model will generate the blog for you and what you can do you can just modify this particular blog with respect to your requirement and you can publish okay anywhere you can also generate any kinds of videos you can also generate any kinds of skills okay everything is possible nowadays with the help of this generative models okay so that is why content generation is one of the very uh important let's say features
inside Generative models okay I can talk about now the third thing you can consider build powerful application as I already told you we are already using uh tons of powerful application in our day-to-day life like CH GPT gini okay then Lama so different different actually powerful application we are using in our day-to-day life and just try to think about when we didn't have these kinds of application we had to complete of work actually manually but nowadays we are Having this kinds of powerful application now you can do any kinds of work actually in a few
seconds okay it is possible let's say if I give you one example as a developer whenever let's say you are getting any kinds of error okay from your let's say application what you can do you can copy that error and you can ask through the chat GPT chat GPT will give you the solution okay how you can solve it but previously when we didn't have any kind of chat GPT what We used to do we used to search that particular error inside Google we used to open this track overflow and we used to see lots
of let's say Solutions then we used to solve that particular let's say error and nowadays actually we are using these kinds of application and it is actually saving lots of our time so we don't have to spend actually lots of time to fix any kinds of bugs uh in a few seconds actually it is possible nowadays uh if we are using these kinds Of powerful generative AI based application okay this is the idea those who have already studied about artificial intelligence machine learning deep learning maybe you have already seen these kinds of V diagram okay
this is the V diagram of our AI the complete AI now you can see machine learning is a subset of artificial intelligence deep learning is the subset of machine learning and now here you can see gen VI is the subset of deep learning okay it Is the subset of deep learning that means machine learning is the subset of artificial intelligence deep learning is the subset of machine learning and generative is the subset of deep learning okay this is the complete V diagram of the complete artificial intelligence okay I hope it is clear now let's try
to see the difference between discriminative model and generative model you can see uh discriminative model is nothing but it's just a Prediction model let's say here you have given two input one is the cat image other is the dog image what your discriminator model will do it will try to predict whether it's a cat image or whether it's a dog image that means you are trying to do some prediction here I think you already use these kinds of model in your deep learning let's say whenever you used to perform something called image classification there we
had different different model like resonet 50 Inception V3 right mobile net vg6 so with the help of this particular model we can perform the image classification task on the other hand generative model can take actually any kinds of sample data any of noise data and from that particular noise data it will generate a complete new image for you okay new data for you now let's say here you are passing one noise data noise data means it can be any kinds of unstructured data and what generative model will do it Will try to understand the pattern
it will try to understand the pattern from these kinds of data okay then it will generate a new data for you new data point for you this is the idea okay this this is the idea of the generative model see I have kept another example here let's say so here we are using distributive model to classify the music type you can see here we are uh giving actually different different music rock classical and romantic so here your Discriminative model that means your discriminator deep learning model will try to classify whether it's a rock music whether
it's a classical music or whether it's a romantic music on the other hand generative model what it will do you will just give some music sample okay and generative model will try to understand the pattern and it will generate a complete new music for you maybe you have already heard of AI has created Music AI has created songs AI Has written Story how that's how actually they're using something called generative model they're actually feeding lots of training samples okay and this model is learning and it is generating a new content okay with respect to the
data they are giving this is the idea of the generative model and the distributive model I hope now it is clear now if I discuss a little bit low level like how things are working here see maybe you have heard of something Called supervised learning okay in supervised learning uh what we do we give the X data as well as the Y data that means input and output and here we try to find the relationship between our input and output okay this is called actually supervised learning and whatever actually let's say discret model actually we're
using it is called actually supervised learning on the other hand we are having another learning called unsupervised learning in The unsupervised learning what we use to perform we us to perform something called clustering technique let's say here we are only giving the X data that means the input data and what my model will try to do it will try to make a cluster let's say this is uh one cluster this is another cluster this is another cluster that's how it is separated out my data okay that's how it will find out the relationship between the
data okay this is the idea now you can consider This unsupervised learning as a generative model so generative model will also work in the same way so instead of actually finding the relationship what it will try to do it will try to make a class chart okay it will try to make a cluster from the unstructured data actually will be feding this is actually lowlevel actually understanding I'm giving you how things are working here but on top of that they have added some more Technique to actually build this kinds of genbi model okay this is
the idea so in summary actually you can see generative VI is a subset of deep learning and generative models are trained on huge amount of data as I already told you while training the generative model we don't need to provide any kinds of label data we only give the unstructured input data because whenever uh you are working with huge amount of data it's not possible to Label the data okay that's why you are giving the unstructured data as an input to the generating model so here you can see in generative AI we give the unstructured
data to the large language model for the training purpose and whatever things I explain so far you can see what your distributive model will do it will try to uh actually predict whether it's a dog image or whether it's a cat image okay by finding the relationship on the other hand your Generative model will try to uh make a cluster okay let's say this is a cat cluster this is a dog cluster okay then with the help of that it will able to generate a new content from you I think you have heard of something
called gan the generative aders neural network okay so Gan is also called generative model okay because here you will be giving noise data and from the noise data itself it will able to generate a new content okay new data this is the idea Of the Gans so so far I told you about generative model but what exactly this generative model see generative model is nothing but it's a large language model which is also called llm see a large language model is nothing but uh it's a foundational machine learning model that uses a deep learning algorithm
to process and understand the natural language these models are trained on massive amount of Text data to learn patterns and entity relationships in a Language it can be also considered in a image data as well okay not only text Data you can also use llm for the image as well because I told you we are having two kinds of model uh generative language model generative image model okay so you can also use image data here not only text data and it is a language model which is responsible for performing tasks such as text to text
generation text to image Generation image to text generation generation as I Already told you as I already told you generative model can support actually all kinds of data you can generate text to text you can generate text to image even you can also generate image to text okay everything is possible with the help of this large language model nowadays okay and we also call it as multimodel that means you can perform multiple task here not one specific task you can perform multiple task like text to text text to image and image to text Generation apart
from that you can perform uh some more tasks let's say you can perform uh language translation you can perform text generation you can perform let's say text classification you can perform sentiment analysis name entity recognization all kinds of task you can perform with the help of only one model and which is nothing but our large language model okay and large language model is nothing but our generative model okay this is the Complete idea about our generative AI I hope it is clear now now let's try to understand what makes llm so powerful as I already
told you in case of llm one model can be used for whole variety of the task okay I I told you now uh Tex generation you can perform chatboard you can perform summarization you can perform translation you can perform code generation you can perform that means whatever task actually you have inside the NLP all the task you can perform With the help of one model which is nothing but large language model so that's why we call this llm is so powerful and that makes llm so powerful because in our traditional model whatever model we used
to use that means the language model only we can only perform one specific task let's say you are you want to do actually language translation for language translation you need to only 10 mon language translation model that model can't do the code Generation or let's say t summarization that model can perform but if you're using large language model you can perform all kinds of task with the help of one model only okay so that's why llm is so powerful because of this particular idea so as I already told you if you're using your traditional actually
language model it can only perform one specific task at a time let's say you want to build a sentiment analysis so one model you have to train For the sentiment analysis which will only able to do the sentiment analysis whether this particular let's say sentiment is positive neutral or negative let's see you want to perform language translation you have to read another model for the language translation and that model will be able to translate any kinds of text okay that means you have created two separate model okay for two separate task but if you're using
large language model you Need to only train one specific model okay and that can perform multiple task this is the idea of large language model so let's see some of the large language model so here you can see we are having gini GPT uh xlm then we are having t five Lama mral Falcon apart from that there are so many large language models are available over the internet I'm going to discuss I'm going to show you each and everything so here I just mentioned some of the large language Model just to show you but I
will tell you uh how you can see all the large language models are available over the Internet even I will also show you how you can access those model and how we can use those model to create your application okay on top of it everything I'll try to clarify so yes guys this is the introduction of generative I now I think you got the clear-cut idea about generative like what exactly the generative Ai and inside generative a Actually what are the things are available and why it is actually mostly used technology inside any kinds of
software product nowadays and inside generative models actually uh what is uses it is uses something called large language model and not body I'll discuss about large language model uh in detail like how large language model works and inside that what are the architecture they're using everything I will try to clarify and now you can ask me what is N2 pipeline of generi no need to worry I will explain uh each and everything what is Pipeline and all and why pipeline is super important whenever you are working in the generative field see it's not like that
whenever you are getting uh some problem statement from your company whenever let's say you are receiving the data it's not like that you will be directly applying the model on top of it before applying the model we have to perform some specific task okay and Those stepbystep task is called pipeline so you can see gener TBI pipeline is a set of steps followed to build an n2n genni software okay that means here you will be breaking the problem statement into several sub problems then you will be trying to develop a step byst step procedure to
solve them and here you can see since language processing is involved okay inside generative AI so we would also list all the forms of text processing needed at each step this step Byst step processing of the text is known as a pipeline okay that means uh that means whenever you are getting any kinds of problem statement from the company first of all break that problem statement in a several sub problems okay then try to solve one by one and to solve that you have to perform some steps okay you have to perform some steps and
those steps is called actually pipelines okay pipelines inside generative a now let's see one end to End pipelines actually we have to follow for all the projects we'll be developing in Futures see in Genera pipeline we are having these are the step we have to follow the first step actually we have to follow the data acquisition because here data is everything right at the end you have to work with the data here so first of all you have to get the data and this steps falls into data acquisition part okay then after that data acquisition
we have to perform Something called Data preparation okay what is data preparation I'll will let you know that means here you will be doing the cleaning of your data okay after that you'll be starting with something called feature engineering inside feature engineering there are so many steps you can follow like so the main uh step actually will be doing inside feature engineering called text representation that means you will represent your text to the Vector that means number so that your model can take the input then after that you'll be doing the modeling then you have
to select different different model and you have to fit the data there okay this is the idea after uh modeling what you have to do we have to evaluate that particular model Because unless and until you are not evaluating the model on top of test data you won't be able to decide whether this particular model is suitable for our production or not okay Then once let's say you got your uh let's say production model what you can do you can do the deployment deployment of your model you can poost your model so that other people
can use your model OKAY other user can use your model and they can give the feedback okay based on the feedback last step what you can do you can perform monitoring and model updating that means you have to monitor you have to uh keep on monitoring your application like uh I think you saw the Chart GPT right so previously I think you saw in chart GPT whenever let's say you are giving any kinds of prompt so it will give you one actually let's say input form it will tell you are you enjoying this CH GPT
or any kinds of feedb feedback you want to give or not not so they actually they were taking the input in real time okay this is called actually monitoring okay monitoring their application that means they're monitoring whether this Application is working fine or not in the production if something going wrong in the application if let's say user is giving negative feedback that time what they will do they will try to update this model again okay this is the idea so it's not like that you are only building your model you are only deploying your model
it's not like that you have to monitor the model in production and based on the feedback you have to keep on updating the model Itself okay this is the complete idea and these are the steps involves inside the generative VI pipeline I hope it is clear now let's break down all the let's say pipeline steps one by one and try to understand what we have to do in each and every step the first step was actually data acquisition so inside data acquisition you can follow some step to collect your data so the first thing you
have to check whether you have available Data or not so available data available data means let's say you are having a CSV file directly you are having let's say text file you are having let's say PDF documents you are having docs documents then you might also have something called Excel Excel SX okay H so if you have these are the files apart from if you have any other file format it's completely fine but first of all you Have to check whether you have available data or not okay this is the idea now if you don't
have the available data what you have to do you have to look for other data okay other data other data means you can collect the data from the database you can see in the internet whether someone is having the data set or not then you can also use the API let's say someone will give you the data but you have to fix the data Through the API that time actually what you can do you can collect the data from the API itself okay then you have then you can perform something called scrapping that me web
scrapping let's say uh you don't have the data in database internet API then what you can do you can do the web web scrapping okay we web scrapping means uh you'll be scrapping the data from a website okay this is the idea now there is another possibility you have no data that time What you will do okay that time what you will do that time you have to create your create your own data create your own data means either you can collect the data okay either you can do the survey you can collect the data
or what you can do you can use large language model to generate the data okay LM to generate data nowadays actually you will see people are using open AI okay open AI GPT model OKAY GPT model to generate the data okay so with the help of llm Also you can generate your own data but whenever you are uh creating your own data there is a possibility let's say uh your data will be less let's say here I will just write you have to note let's say whenever you are collecting your own data your data quantity
might be less so that time what you can do so let me write here if you have less data then you can perform data augmentation I think you know what is data augmentation okay so data augmentation is nothing but You will be augmenting the data let's say uh I'll give you one example so the first augmentation technique you can perform so let me give you the example you can do something called replace you can do replace with synonyms okay now how to do the syon see let's say here is one example I'll will give you
let's say I am a data scientist okay so let's say this is my text now what you can do you can just Replace the synonym so here you can write I am a AI engineer that means you are replacing this particular uh entity with this entity so this is called actually replace with synonyms okay this is the idea I hope you got it because here you will be working with textual data mostly and if you have the image that time what you can perform so there are lots of image augmentation technique as well let me
show you so you Can simply search image augmentation so there are different different technique you can follow see let me show you one example let's say let's say this is the example I can show you let's say this is your original image so what you can do you can do the horizontal flip you can do the vertical flip you can do the rotation of the image you can also do the negative rotation you can do the blower operation You can change the brightness you can add some noise you can add some darker okay so that's
how actually you can change the original image okay you can change the original image and make different different variants okay this is called actually image data augmentation because I already told you inside jni we use both kinds of data whether it can be textual data it can be image data it can be audio data okay anything actually you can consider here Now the next technique you can uh apply which is called diagram flip Bagram flip now you can ask me sir how to perform Bagram flip see it's very easy let's say I'll give another example
uh I am buppy all right now you can write this sentence like that puppy is my name okay so this is called actually Byram plate that's how actually you can also do the data augmentation because at The end you are increasing the data that is the idea and whenever you are increasing the data it should have the meaning it doesn't have the meaning that means there is no use of the data so you have to make sure whenever you are performing these are the technique this data should have some meaning I hope you getting my
point now the next thing you can perform third step you can perform back translate Back translate now how to perform back translate let me show you so let me go to my uh translator so I'll open the translator Okay Google translator let's say here if I write one sentence or I can just copy one story let's say we whatever will do let's say I will copy this uh copy this text and here let me paste it now you can see it is uh translating to the Bengali now what I Will do I'll copy this Bengali
transl translation and here uh the input language I'll be selecting the Bengali right now so let me select the Bengali and I can paste it here now here you can select any other language let's say I'll select Spanish now see Spanish text it is giving now let me copy okay now what I can do again uh I can come here I can past the Spanish text and let's say I want to convert to English again so here let me select the English you got your English text now so here if you see carefully uh some
of the word would be different okay then your previous actually text you have copied after doing the Byram flip that means you will be converting um it a language to another language that language to another language then uh again in your English language so that's how if you do multiple time translation you will see some word would be changed and see this this will work if you have actually lots Of text actually lots of sentence not like that you are only copi one to two line if you have multiple let's say line multiple text that
time actually the reflection actually will able to see okay I'm just only showing you how we can perform the back translation because there are so many um API nowadays inside python you can use for the back translate I think translation API is already there with the help of that you can do the translation first of all Let's say you have the English text first of all try to translate to the Hindi Hindi to let's say uh Spanish Spanish to English again that's how you can do the back translation and you can increase the data so
this is another amazing data mentation technique you can follow okay I hope you're cleared the last step actually you can follow something called add additional data or additional noise how you can add additional noise let's say here you have One sentence I am a data scientist so with this sentence you can add one more line let's say uh I love this job okay so this is your additional data or additional noise you have added okay with your data again this is called actually data augmentation technique and again uh it's an amazing technique you can follow
okay Al together so these are the step actually you can perform inside data acquisition okay so first of all Try to check whether you have available data or not if you don't have try to see the other data if you don't have try to uh create your own data for this you can use large language model even I will I will also show you how you can generate your data with the help of large language model and after generating let's say if you have less amount of data in that case you can perform something called
Data augmentation technique inside data augmentation Technique these are the step you can perform okay so this is the first step of the pipeline now let's discuss the second step of the pipeline okay the Second Step I think you remember data preprocessing so inside data pre-processing you can perform cleanup operation okay cleanup operation that means you can remove HTML tags then you can remove let's say Emoji you can do the spelling check okay spelling correction fine so these are The technique actually you can perform inside cleanup now there is another uh step you can follow called
basic pre-processing I'll tell you what what are the basics pre-processing are available even I'll also show you the Practical how you can perform then you can also perform something called Advanced preprocessing because nowadays the application actually we have let's say chat GPT Jin so these are the actually Advanced application advanc Let's say large language model so for this we have to also perform some Advanced pre-processing technique okay let's say whenever I'm having that data nowadays because let me show you let's say if I open my chat GPT okay if I open the chat GPT see
if I give any kinds of emoji nowadays inside chat GPT still it will be able to understand let's say if I give this haha okay haha let's Emoji now if I send it to my chat GPT see it is automatically detecting What's so funny okay so that's kinds of a text also we need to handle because whenever we are uh collecting the text Data it will have emojis it will have HML tags okay so many things it will have so sometimes it is also necessary to keep the Emojis because I told you now this kinds
of application is Advanced so it is also understand the Emojis any kinds of text so we also need to handle these are the let's say uh I mean text in that time now inside basic Speed processing let me just write here inside basic pre-processing so inside basic pre-processing the very important things you have to perform something called tokenization okay so you have two kinds of tokenization one is called sentence level tokenization other one is Word level tokenization so I'll be discussing each and everything no need to worry and there are some optional pre-processing Are available
so inside optional pre-processing the first thing you can perform stop word removal okay stop word removal then the second uh things you can perform something called steaming and this is like very less used nowadays okay less used actually technique the third thing you can perform something called latiz and the lemmatization actually more used more used okay I'll tell you what is steaming LZ whenever I'll show the Practical that time I'll discuss then the fourth uh things you can perform something called punctuation removal so punctuation means let's say you have uh question symbol you have dot
you have comma you have uh exclamation sign okay you have dollar symbol so these are the things are actually punctuation so you have to also handle the punctuation then you have to perform something called lower case then the sixth you can Perform something called language detection because nowadays if you open the chat GPT or gini if you give any kinds of language it will first of all detect in which language you are asking the question B not that it will give you the reply okay in the same language so uh so they are also applying
this language detection technique okay in the back end so that's actually one real world product Implement okay it's not like that you are getting the data you Are directly applying the model it's not like that so you have to follow the entire pipeline to build the endend software okay that's why I'm showing you these are the pipeline why this gen VI pipeline is super important and what are the steps are uh involved inside the pipeline so that after learning it it would be easy for you to develop any kinds of software in future and this
is my promise if you understand this okay if you can do this you can design any Kinds of software in future okay because what is generative Genera is nothing but just a technology so you have to use that technology you have to learn how to use the large language model how to prepare the data for the large language model okay how to use let's say Lang chain how to use Vector database you have to connect everything together then you have to build one application that is the idea only but the root things available inside the
pipeline how you're Designing the pipeline whether pipeline has all the steps available or not I hope you getting my point right so now I think these are the steps are clear uh but I think some of them still you are getting confused like what is steaming what is let's say LZ what is let's say why we have to perform the lowercase operation and what is this uh tokenization see let me just discuss all of them one by one so first of all let me discuss the tokenization that means Let's see you are having a sentence
here let's say my name is buy so this is a cent right this is a complete sentence right so whenever I will be converting this sentence to number that means Vector representation first of all I have to I have to actually do the tokenization tokenization means you will be uh taking the individual words let's say like that my comma Name okay comma is comma buy this is called actually Word level tokenization okay Word level tokenization okay that means you had one complete sentence you just did the tokenization and you got the word as a token
okay everywhere now you can take the word and you can convert to number there is some technique we can follow I'll tell you those are the technique what are the technique you can follow to Convert your text to number representation okay and there is another technique called uh let's say sentence level tokenization let's say you are having a sentence like that let's say my name is bbby uh I am a data scientist okay now if I perform sentence level tokenization so sentence wise actually I have to convert so it will come like that my name
is buy so this is my one token then Comma then I am a data scientist okay this is another token that means two token I'm having in the list okay this is called actually sentence level tokenization sentence level tokenization okay so we'll be using this word level tokenization a lot sentence level you can also use sometimes you have to use sentence level tokenization sometimes you'll be using Word level tokenization But most of the application I saw people are using what level tokenization that is the idea so this is what actually our tokenization okay now let's
try to understand what is steaming see steaming is nothing but see steaming is nothing but you are bringing one word in a root form let's say here we are having three kinds of word play played and playing so if you see them carefully meaning are same yes or no meaning are Same only the ver representation is different now instead of taking actually three okay separately what I can do I can um convert in a root form so root form is nothing but play okay because player represents Sports okay Sports so this is called actually Ste
steaming that means you have different different uh actually word form that means different different V form and you are trying to bring in a root representation okay this is called Actually uh steaming and again it's important because it will reduce the dimension okay it will reduce the dimension whenever let's say will be converting your data to the vector representation because if you have three word so it will create actually three dimensions separately but if you convert it to one let's say word only so it will only have one dimensional and we know that inside artificial
intelligence Dimension is like the issue so if you Have let's say more Dimension that time your model will get confused because there is a concept of curs of dimensionality so you have to also take care about the dimension okay that's why we perform this streamming operation and LZ is the same okay the way actually perform the steaming LZ is also same only whenever you will perform the LZ your root meaning of the word would be readable but whenever you will perform actually let's say streaming sometimes It won't be readable I'll tell you whenever I'll do
the Practical of it that time actually will get it okay what I'm trying to say that is the idea now you can ask me what is lower casing okay because fation you already know now you can ask me why lower casing is important why you have to perform the lower case operation let's say if I write two sentence buppy is very good boy and Buppy is a data scientist let's say these kinds of sentence we are having now here you can see I think buy and buy both name are same but here you can see
the first character it's a Capital One the second character it's a l one so as a human actually you can consider these two buy are same but whenever you will be passing to the model that means in your computer your computer will be considering this two word is a different word because here you are using capital And here you using lowercase character because all the character is having their asky value right that means the asky code uni code so with the help of that it will consider these two words are separate so that time this
bu would be different this bu would be different although they are same okay this is another issue so that's why we perform lower case operation so whenever actually we're having the upper case we'll try to convert to the lower case Okay we'll try to convert so let me show you we'll try to convert to the lower case Okay so this is now you can see this buy and this buppy will become same okay now if you give pass to the computer also your computer will identify this buy and this buy is the same right now
okay so that's why we perform the lowercase operation so that my model won't be getting the confused okay this is the idea so I think most of the things I clarified and yeah so Everything I'll show you as a practical as well how we can perform as a practical I'll tell I'll discuss it okay no need to worry now apart from basic pre-processing I also told you we have something called uh Advanced preprocessing as well now let's try to see what are the steps are involved inside Advan preprocessing see inside advance preprocessing advance pre-processing we
have something Called parts of space tagging parts of speech tagging so if you're good with English grammar I think you know what is parts of speech okay then you are also having something called parsing then you have something called Co reference okay Co reference resolution now if you want to understand them let me show you one uh image I think then this part would be cleared so guys I think you can see this image so here you can see uh this is the um input text we are having chaplain Wrote directed and composed the music
of uh for the most of is film now if I perform tokenization and LZ so I already told you what is tokenization you will be taking individual word see I have taken individual word here and if you perform the LZ LZ means it will bring the uh let's say any kinds of verb in a root word now we can see root root if I bring to the root what it will be right directed it would be direct okay then compost it would be compos it would be Compos so that's how actually you can understand the
tokenization and LZ now what is po is tagging P tagging means parts of spe tagging the things I showed you in my Advan preprocessing so this is what actually parts of spe tagging that means you are assigning what is noun what is verb what is adjective okay so that's how you are defining different kinds of part parts of speech if you're good with grammar I think you will be understanding better than me uh just try To consider here you are uh tagging everything that means you are tagging all the let's say noun verb okay then
pronoun everything you are tagging now there is another one called parsing okay I think I showed you parsing so this is the parsing tree that means you will be creating a tree and inside this tree you will have the entire sentence word you can see chaplain Road directed okay all the words so in the parts three you will be again assigning the POS tagging apart From that you will be also dividing these are the word in a separate separate form again this is a advanc actually grammatical rules uh you have to learn but no need
to worry let's say when will be working in the industry there you will have something called domain expertise okay let's say language expertise actually people will get so they will help you to do it because I know as a developer you know you don't Know about let's say uh this English grammar and all but whenever you'll be designing the software that time language expert will help you okay how we can perform the pursing how we can perform the P tagging everything they will help you out now this is the last example core reference resolution so
cor reference resolution means let's say Chaplin wrote directed and composed the music for the most of his film now here chaplain and his it is indicating the Both person uh sorry it is indicating the same person right so this is called actually core reference resolution Solutions sometimes you your model has to understand uh this Chap and his so it is mentioning the both uh it is mentioning the same person okay so this is called actually core reference resolution because nowadays if you see the chat GPD G so these kinds of application is able to understand
okay this kinds of input disc kind of prompt Okay so in the back end actually they're handling like that I hope it is clear now then the third actually step were available inside uh gen pipeline which is nothing but feature engineer ing feature engineering so inside feature engineering you can perform text vectorization so to perform the text vectorization you can follow some technique like uh you have tfidf Tfidf you have something called bag oford you have something called word to you can also perform something called one not encoding then you can also use some Transformer
okay Transformers model to perform the text vectorization as well because these are the very old technique so if you're using any deep learning and machine learning model that time you can use these are the technique but if you are creating advanc application like Let's if we using large language model Transformer based architecture like say encoder and decoder architecture that time you have to use Transformer model to perform the text vectorization technique okay I hope it is clear so these are the steps are available inside feature engineering so inside feature engineering you'll just try to convert
your text to Vector representation it can be also uh work with the image as well let's say you are having an image Okay let's say you having an image and in that image actually you are having a cat image let's say this is a cat let this is a cat okay now image is nothing but it's a pixel so if you just see the image so here you will see different different pixel value okay different different pixel value so that time what you can do you can use any kinds of vision Transformer model to convert
this image to the vector representation okay Vector Representation and you can pass to the model okay so that is the idea now fourth step actually I think you saw after feature engineering you can do the modeling okay modeling modeling means you have you'll be training the model that means uh you'll fit the data to the model okay so here you can choose different model you can choose actually different kinds of model so here you can either use open Source okay open source LM that means large language model either you can use paid one okay paid
paid model so so what is the difference between open source and paid see if you're using paid see if you're using paid one this model would be available in the server that means you don't need to download this model in your machine okay so you don't need to download instead of that what you will do you will only let's say Pass the data okay let's say you are using open a because I think you know opena provides actually paid model that's a GPT model so in opena you will upload your data and in their server
it will train the model okay you don't need to download the model in your machine and you don't need to train there because I know that people onlyon be having good GPU in their system okay it's not possible actually to buy uh expensive GPU for training these are Kinds of large language model because if you see this large language it's very huge okay if you see the parameter size is very huge so we can't download in our machine and we can't train it it's not possible so that's why most of the time we'll be using
Cloud platform to train the model okay so that time actually you can use the paid model because paid model if you're using paid platform if you're using everything they will take care only you have to upload your data There but if using open source large language model that time you have to download this model in your machine you have to prepare everything you have to set up the environment and then you have to train the model and for this you need a good GPU okay good GPU good CPU good memory okay everything you need and
after training this model you have to deploy this model uh in a cloud platform okay manually you have to deploy this model in a cloud platform and again you Will get the challenges there like uh you have to buy a good instance then you have to set up the load balancing and all okay then you have to deploy this particular model but if you're using Cloud platform they directly will get one option to deploy the model in their server only okay so that people can access the model in their platform only okay so you don't
need to take care about the infrastructure you don't need to take care about the instance Everything they will take care that is the difference got it we'll be also understanding how we can use paid model how we can use open source model and I will also discuss the difficulty level actually we'll be getting whenever you'll be using open source llm whenever you will be using let's say paid llm everything I'll try to clarify no need to worry uh train your model you have to perform something called Model evaluation okay evaluation so inside Evaluation you have
to part from two kinds of evaluation one is like inin evaluation other is like extrinsic evaluation so intrinsic Evolution means you'll be using some metrics okay you'll be using some metrics to evaluate the model I think you know we are having different different metrics for the evaluation let we are having accuracy score R2 score then we are having AOC card so these are the different different metrics we are Having okay so after trading a model you have to perform something called intrinsic evaluation and this is performed by the and this evaluation will perform by the
jna engineer okay jna engineer because they training the model and after training the model they have to actually uh evaluate the model with the help of these are the matrixes and the extensive evaluation when they have to perform after doing the deployment okay after doing the Deployment deployment so this uh evaluation can be applied during the production okay let's say whenever this model is in the production people are using it that time this extensive evaluation can be performed let's say I already told you chat GPT will uh launch one input sometimes it will uh ask
you give the feedback not only chat GPT if you see any kinds of application so if you just consider your phone keyboard so whenever you use the keyboard okay so Initially I think you won't be getting any of suggestion but whenever you will be using that particular keyboard okay um long time then you will see that automatically you will get the suggestion because it is learning okay how what kinds of input you are giving it is learning and sometimes it will ask for one kinds of feedback whether you are enjoying this app or not okay
any feedback you want to give or not and whenever you are giving the feedback Whenever you are giving the five rating four rating that time so that time actually they're getting this application is working fine there is no issue but whenever you are giving negative feedback that time they're getting actually there is some issue with the application that time what they will do again they will they will uh retain the model okay they will retain the model and they will again push the model Through the production okay so that's how extensive Evolution works that means
after deployment after uh going to the production okay after going to the production of that model you will be performing this particular extensive evaluation this is the idea and the sixth I told you we have to perform deployment okay deployment and inside deployment you will be also doing the monitoring monitoring and retraining okay retraining Because you'll be doing the monitoring in the extensive evaluation and if something going wrong you'll be doing the rning of the model this is the idea so these things are involved inside so the entire generative AI pipeline I hope it is
clear so first of all data acquisation then uh data preparation then uh feature engineering modeling evaluation and deployment and here one thing you have to remember because going forward I'll be using this Trum a lot so Let me just write here common Trum let me call copy this text and let me paste here let's say this is my data now here you will see some of the actually let's say common term the first ter actually we'll see uh called cordus okay Corpus then the second term you will see which is nothing but vocabulary and the
third term you will see which is nothing but documents and the fourth you will see Something called word okay word now what is Corpus Corpus is nothing but the entire text entire text that means the entire text is called Corpus okay and what is vocabulary unique word unique word now you can see this is unique word this is unique word this is unique word this un this un okay so these are the uh what is unique and if you see any of the word is coming duplicates only you have to take the Unique word okay
this is called actually vocable okay then what is documents document is nothing but one row that means one line only so till here actually one documents D1 then this is called actually D2 okay D2 then D3 so this is called actually documents so I can write one line okay one line is the documents okay or one row and what is what what is n this single word okay single word is called word so you can see this is a single Word is a single word this called actually word so these are the common term you
have to remember because going forward whenever we'll be actually playing with the data that time actually I may um tell you these are the actually common term okay that time you won't be getting confused what is corus what is vocabulary what is documents what is what okay that's time so that's why I showed you okay here in this video so yes Guys these are the actually Generative VI Pipeline and uh here I have just showed you as a theoretical so in the next video we'll be doing the Practical of it like how we can perform
different different text cleaning technique like different different text pre-processing technique after that we'll also see how we can perform feature engineering technique as well okay different different feature engineering technique because what I feel like before starting with our large Language model first of all we have to understand the data first of all we have to know how to prepare the data for the model okay then we'll be starting with the large language model okay so this thing actually is uh so this uh thing actually very important before learning uh your gen before learning the large
language model and application building so that's why I'm teaching you this part uh in our previous video I already discussed about end to end generate TBI Pipeline so there I told you what is the use of data preprocessing right because if you want to use the model that means large language model the first thing you have to do the data P processing Because unless and until you are not processing the data how your model will try to understand that one right that is the idea so here we'll be learning various kinds of technique uh to
clean up about data so for this what I have done I have created a beautiful uh collab notebook So there actually I have uh added all the examples you can uh use for the data cleanup operation so guys now let's try to see how we can do the data preprocessing part so guys as you can see this is The Notebook I already prepared so here you can see I'm using one data set uh from kagle so let me open the link um this is the link so the data set name is IMDb uh data set
and it is having actually 50k uh movie reviews Okay 50k actually movie reviews if you are already from let's say machine learning deep learning background I think you know about this data set right it's like very common data set and why I took this particular data set because in this data set you will see uh there are so many U actually unnecessary text okay because it's a movie review so what they did actually they extracted they scra this data from the IMDb website if you don't know this is the IMDB website IMDb So in this
website you will see uh all the movies reviews and rating and what they did actually they published this data set in the Kagel website so that if anyone is working in the field of let's say genv or natural language processing they can use the data set it now here what you just need to do you just need to download this data okay it's around uh I think 27 MB just click on download button it will download okay so I already downloaded this data set so it Is available inside my download folder so I'm going to
upload in my collab notebook so here first of all what you have to do you have to connect this particular notebook so there is a connect button just try to connect and no need to worry I will also share this notebook Link in the resources section from there you can open it up so my notebook is connected so first of all let's import some of the library first of all I need something called pandas Because if you see the data uh it's a csb data okay it's a csb data so let me upload and let
me show you so if you want to upload anything in the Google Drive just try to right click and there is a upload button click on upload and try to upload this data here now see it's a CSV file okay and if I want to load any kinds of CSV file Json file or let's Excel file whatever file you can use the pandas package for that so you can see my data set is Uploaded now if I want to load the data set what I have to do I have to assign the path and see
you don't need to execute deser the code because uh this code I have added let's say if your data set is available in your Google Drive that time what you have to do you have to Mount Your Google Drive first of all Mount means you will be connecting with your Google Drive then will'll be relocating the folder okay like inside which folder you kept your data okay With the help of CD command CD means change directory okay then after that what he will do he will assign the data path but here I haven't kept my
data inside my Google Drive I kept inside my col collab actually you can see drive this is the collab disc I'm using here you will get around 74 GB of space so here you also you can keep your data so I don't need to execute this code what I will do I'll just go below and if I check my current working directory that Means PWD you'll see I'm inside content content means this is the directory right now now here I'll just simply Define my data path I'll copy the path copy and let me paste it
here okay that's it now let me execute now see if I now load the data pd. csb I'm doing now see it will load the data see it has loaded the data if you want to see the shape of the data this is the shape okay you have around 50 uh 50k actually movies movies reviews In this particular csb file and two columns two columns means one is the reviews column other is like the sentiment column okay sentiment means whether it's a positive sentiment whether it's a negative sentiment this kinds of sentiment you will get
here I hope it is clear now see here I'm having 50k movie reviews but here I'm not going to use all the reviews here I'm going to show you the demo like how we can perform the text pre-processing and if I'm taking all the 50k reviews so it will take like lots of time um to process those are the text so what I will do I only take the 100 example okay the first 100 example I I'll be taking and on top of that I will perform all the text processing task right so this is
the code you can execute so it will load 100 example now if I show you the shape see now we are having 100 example and only two columns fine now if I want to show you the data see this is the Data so I think you remember in my uh theoretical class I was discussing about some pre-processing technique uh here I think H so here you can perform something called HTML tag removal Emoji handle art then spelling correction then in the basic preprocessing we saw that we can perform something called tokenization then we had some
optional preing as well like stop word streaming LZ punctation lower case Okay language detection each and everything so first Thing we'll be learning how we can perform the lower case operation and why lower case important I already explained here I think you remember let's say let's say if one of the name is containing uppercase character it will consider these are actually separate name okay these are actually separate entity that is the idea so that's how we have to bring everything in a lower case character so that's why you have to apply this lower operation and
how to Perform lower operation I think you already know in Python we are having a function called Lower with the help of lower also we can do it right now see here let me show you one example let's say I'm taking the reviews three so here I'm taking the three three three number rows this is the three number rows now you can see some of the uh character is uppercase character here so this is uppercase this is uppercase okay so that's how actually you will see Different different uppercase character would be there now if I
want to make them lower case what I have to do first of all I have to select my column like in which column you have to apply the lower function I have to apply on top of my review column now first of all I'm converting everything to the string okay string data type then I'm applying the lower because lower is a string method okay okay L is string method I think you already know that then whatever changes Actually I'm doing I'm saving inside my column that means I'm just doing the permanent change okay inside my
column so that's why I have given review again that is the idea now if I execute now see if I show you the data now see guys all the character has become lower case right now now if I want to show you the now if I again execute that review three you will see that all the character has converted to the lower case character okay that is The idea now the next thing will be learning how we can handle the HTML tags that means how we can remove the HTML tags for this here I have
written a function here I'm using something called regular expression regx okay inside regx you can give a pattern let's say here you are giving a pattern if you are getting this kinds of symbol okay if you are getting this kinds of symbol it's a like HTML tag you have to remove those HTML tags and you have to replace with Empty okay empty string that is the idea now this is the function we can use now if I execute now let's say this is a one text I have prepared you can see in this text actually
we are having lots of HTML tags now if I pass this particular text to my function see it will automatically remove all the tags now see I'm only getting the text okay relevant text that is the idea so this notebook I prepared in such a way so that you can use it as A template let's say whenever you need anything any kinds of functionality you can come here you can copy those function so please try to keep this particular notebook with you because this is going to help you a lot okay whenever you'll be developing
the projects this is going to help you a lot now if you want to apply on top of the entire data set again just call this column name let's say review column okay and there is a function called apply and Inside that just try to pass this function okay apply function takes actually uh one function object now here I'm giving the function object so what it will do it will try to apply this particular function on top of the entire uh rows you are having in your data set okay now see if I execute and
and the changes actually I'm doing I'm saving everything permanent okay now let me execute now if I show you any kinds of random let's say rows now you'll see all The uh now let's see if I also show you um seven okay seven number rows now just try to see there is no HTML tags in this particular text you can pick up any kinds of let's say index so let's say if I show you the 10 nowhere you will see the HTML tags here now we'll be learning how we can remove actually URL let's say
if you are having some URL in the text how we can remove it for this again I'm using regular expression and there is a Pattern I have given if you are getting this kinds of let's say word HTTP then slash then ww that means it is a URL and you have to remove the URL with the empty string okay this is the function now let me execute now here I have just mentioned some of the URL so this is my YouTube channel URL this is my LinkedIn URL and google.com and kaggle.com okay now these are
the text I have prepared one by one now let's say if I'm passing any kinds Of text inside my remove URL function it will remove that URL let me show you see I'm giving the text to that means my LinkedIn URL now you can see check out my LinkedIn I hope it is clear see again I'm telling you it's not a mandate things let's say you need URL in your data at that time you can keep it let's say you need actually HTML tags you can keep it but if you don't need it you can
remove it because I already told you uh nowadays actually we are having Advanced Genbi application it also supports all kinds of text like emojis HTML okay everything it supports so if you're creating these kinds of advanced let's say u i mean application that time you need Des are the data you don't need to remove it okay but sometimes actually you also need to remove this data set so that's why I'm sure you how you can remove it and if you want to keep it you can also keep it it's up to you okay you have
to decide based on your project Architecture that time this is the idea now we'll be learning how we can handle the punctuation so if you want to see the punctuation so there is a string package you can use now if you just write string. punctuation you will see all kinds of function are available now what I have done I just stored these are the function in a variable called exclude now here I have written a function okay here I've written a function called remove punctuation and Here I've written a for Loop so I'm just uh
looping through this punctuation one by one and user is giving the text and I'm just checking whether if there is any punctuation okay I'm just replacing with the empty string now let me show you how it will work now let's say this is one text string with a punctuation you can see there are so many punctuation I have assigned now if I pass this particular text to my punctuation function it will remove the Punctuation now see there is no punctuation right now even I'm also calculating the time like how much time it is taking to
remove the punctuation because there is another way you can follow to remove the punctuation I'll tell you how you can do it see this is the function guys uh so here you can use something called text. translate inside that just try to use this particular function okay make translate and inside that just try to mention the punctuation So what it will do it will uh take your text and it will remove all the punctuation so this is another approach now see if I calculate the time of this function you will see that this is the
time that means this function is taking less time than your this function because here you are using one for Loop and for Loop has the linear time complexity I think you know that if you are familiar with DSA con I you know that it is having linear time time Complexity okay so this particular approach is good so let's say if you're are applying this particular Logic on top of 50k data set just try to think how much time it will take this full loop and in other hand if you're using this particular method it will
take very less time to perform the operation now if you want to see the time difference you can also see the time difference now if I show you my uh text now see inside this text Actually I'm having lots of punctuation now what I will do I will use my uh remove function one that means this particular function and inside that I'm going to pass my entire review now see it will remove the punctuation okay see all the punctuation has been removed now you can also pass the entire data like that you can also pass
the entire data it will remove all the punctuation inside your entire data now we'll be learning how we can handle the chat Conversation see sometimes whenever we perform the chatting operation we give uh like lots of shortcut let's say if I want to write as far as I know what you will give you will give AF a ik then let's say away from keyboard AFK then as soon as possible ASAP that's how we use lots of chat keyword okay we use lots of chat shortcut keyword okay here I have listed down some of them you
can see for your information FYI so that's how I have listed all the chat conversation Short uh word now let's say you are working with actually social media data set in the social media data set you will see these kinds of data a lot people will be using short form okay short form so how we can handle this particular short form for this first of all try to create a dictionary like that the entire dictionary you have to create then after that here I've written a function so this is the function chat conversation handle so
inside that I'm Just taking the text okay I'm just taking the text and I'm checking and inside that with the help of this for Loop I'm checking if we are having this kinds of word okay if you're having these kinds of word what we doing we're just mapping with the value that means let's say if anyone having actually this particular let's say shortcut word let's say FYI what I will do instead of FYI I will just return for your information because this is a dictionary if I if I Want to get the value I want
to call the key okay that's how we are mapping now see if I execute the program now let's say here we are giving a short masses uh do this work as up now see it will return me do this work as soon as possible okay because here it is coming here and it is mapping and it is giving me this value okay this is the idea now let me see how we can handle the incorrect text let's say sometimes what happens whenever you are using real time Data there would be lots of spelling M mistake
right let's say this is one example s 10 so certain spelling is not correct then condition spelling is not correct that's how so that's how you'll see different different word is having the spelling mistake so how to handle this spelling mistake for this you have to use one package called text blob so let me import this text blob and now see here I'm having my incorrect text now if I pass this Incorrect text to my text block and if I call this particular function called correct now see it will uh handle everything now see certain
conditions during several see all the word has been corrected so this amazing package you can use if you want to handle the spelling okay spelling of any word now the next thing we'll be learning how we can handle the stop words so inside English language not only English language inside all kinds of language We're having some stop words okay now here we're working with the English language so let me show you some English related stop wordss for this I will be importing stop wordss from the nltk nltk is a so nltk is nothing but it's
a natural language toolkit Library so with the help of NLP also you can perform lots of NLP related task so here you can see I'm importing the stop word so here I'm performing nltk do download it will download all the stop wordss from the Internet now see now see if I uh click this cell it will download the stop wordss now if you want to list down all the stop wordss you can just write stop words. wordss and here you can specify the language so here I working with the so here I'm working with the
English language so here I've given the English let's say you are working with Hindi Bengali you can give that language also it will give you the stop what related that language so you can uh see the Documentation of anal we'll see that how we can pass the parameter here now see these are the words we are having inside English so this is called stop words so this word doesn't have any kind of meaning okay in a sentence okay we use it uh to represent one sentence but there is no meaning so let me show you
why I'm telling there is no meaning let's if you want to perform uh sentiment analysis so let's say there is one uh review we are having so let me Just write this movie is awesome I loved it now just try to see here you are having some stop word right like is is there this is there I is there it is there right but if I want to get the sentiment of this particular let's say reviews I can see movies is required awesome is required loved is required now if I'm getting these kinds of word
actually this kind of positive word that Means it's a positive sentiment okay it's a positive sentiment yes or no no right so here actually I don't need these are the stop word okay I don't need these are the stop word to understand whether this particular reviews is a positive or negative and if you're using these are the stop word in the sentence what will happen whenever you will perform the text vectorization it will make actually extra Dimension okay it will make the extra dimension in The data and whenever it is making the extra Dimension that
means if Dimension is increasing that time uh your model might get uh difficulties right because we know that there is a concept of card of dimensionality so so we always need to reduce the dimensionality somehow I think you have learned in your machine learning okay machine learning let's say topic so that is why we don't need these are the stop WS so we have to remove this particular stop W sometimes and Sometimes actually we have to also keep it so these are the stop W guys you saw now let me show you the length so
around we are having 179 stop words present inside English now here I've written a function so this function will remove the stop parts from a text now let me show you let's say this is a text I have given so inside that you can see different different stop wordss are there now if I pass to my function so see now see there is no Stop word in this particular text right now okay that's how you can use this particular function now again this is my data and if I want to apply on top of my
entire reviews I can do it I can use the apply function and I can pass my uh function object remove stop words and if you want to uh store permanently what you can do you can write this code okay uh DF reviews is equal to DF reviews apply stop words okay it will save everything permanent But I don't want to save it permanent as of now because I'm showing you as a demo if you want to do the permanent you can do it so now guys I think it is clear how we can uh remove
the stop words now the next thing we'll be learning how to handle the Emoji okay see Emoji is nothing but it's a uni code actually uh character okay if I show you if you just write Emoji uni code you will see different different Uni codes of different different emojis so let me open it up see uh here we are having different different emagis based on that we are having the uni code okay unic code character okay Unicode character so that's why I just collected some of the Unicode character like for the emotions emojis symbols then
uh pictur graphs then we're having transport map symbol flag okay so these are the emagis related uni code I collected here and I Just written a function okay and again I'm using the regular expression so here I'm telling if you're getting this kinds of unic code in a sentence that means it's a Emoji so what you have to do to remove this emoji with the empty space okay now let me show you so let's say this is one text I'm giving love the the movie it was flying Cas okay now see if I uh pass
this particular text now see it will remove the Emojis automatically now this is the next one now see only LMO is coming now sometimes actually I told you emojis are required because let say I told you in the chat GPT example so if you pass any kinds of emoji through the chat GPT it will also able to understand okay what you are trying to say let's say if I pass any kinds of emoji here let's say I'll give this Emoji hey there what's on your mind today see it is trying to understand my feelings right
now if I want to handle These kinds of situations what I have to do I have to keep the Emoji that time what I can do I can uh extract the meaning of the Emoji so for this I can use one Library called uh Emoji okay now let me show you first of all you have to install it now just import the Emoji and inside Emoji you are having one method called demo eyes okay now inside that just try to pass the text python is fire I'm giving the fire Emoji now it will Automatically convert
to the word python is fire okay that means it is trying to extract the meaning of that particular Emoji right now I hope you you cleared so that's how whenever you are passing any of emoji as input to the chat GPT it is trying to convert this Emoji to the word and is trying to understand what you are trying to say now there is another example I have given love this movie it was flying case see see love this movie it was face blowing a keys Now we'll be understanding the tokenization I told you we
are having two kinds of tokenization sentence level tokenization and Word level tokenization so let's try to see see if you want to perform the tokenization uh directly you can use the split function because spit also will give you the individual word in the list see if I have one sentence now if I perform the split operation see I'm giving I'm getting actually individual part so with the help of Split also you can perform the tokenization so similar wise you can also do the sentence level tokenization this is the word level tokenization now for this you
have to mention this particular fully stop sign so whenever it is getting the fully stop that means this is sentence okay now see if I execute so here I'm having three sentence so I am going to DHI I will stay there for 3 Days Let's uh hope the tree to be great okay see three sentence I'm getting so this is called sentence level tokenization so again some of the example with the split function now with the help of regular expression you can also do the um tokenization so this one example I give with the regular
expression so here is my sentence and it will give you the individual word this is the sentence level tokenization okay and this is the pattern for the word level this is the pattern for the sentence level okay this Is the idea now you can also use something called nltk okay that means natural language toolkit library for this tokenization as well inside that we are having to fun two actually function mod level tokenizer and Cent level tokenizer now let me and if you want to perform the tokenization you have to download this particular thing called p
n KT okay this particular thing you have to download okay now see here is my sentence and I want to perform Word Level tokenization I will pass this so it will give me the B level tokenization so automatically this function will handle now if you want to perform sentence level tokenization you can use send tokenizer okay from here and you are getting the sentence level tokenization okay so again some of the example you can try I have given and I created this notebook in a such a way you can use it as a template I
already told you I have written like like the Function wise right if you need any kinds of let's say um I mean cleanup technique you can copy from here and you can use it in your code that's that's the idea now there is another package you can use called Spacey with the help of Spacey also you can perform the tokenization fine so we can explore this part now let me go to the steamr okay I already told you what is steamr steamr means you are trying to bring the different different word in the root Form
that that means uh let's say you are having play playing played so what you will do you will Apple steamr and you will just try to convert it is to the and you'll just try to convert to the root word that means play right because it is meaning the same okay in the sentence so now let me show you so inside nltk we are having this steamr post poster pter steamr now we are importing pter steamr and here we have created a function steam word so Whenever you will give any kinds of let's say uh
sentence it will perform the streaming operation now see here I'm giving work Works working work now if I give this particular sentence it will give me the root form that means work work work and all okay now see here I kept one sentence so this is the entire sentence you can see now I want to perform streaming operation on top of the sentence so I'm using my function inside that I'm passing the sentence now See uh it will give me the um steaming word right now now see probably it has been uh probable okay probable
that's why I told you streaming is not readable sometimes you will get some kinds of what it's not readable but your model will try to understand okay but as a human it's not readable see favorite has been converted to favorite right now okay so in case actually what you can do you can use something called LZ okay LZ will handle this kinds of situation it Will give you readable word actually now see this is the LZ code so it is available inside nltk I'm importing the LZ initializing the LZ and if you want to use
latiz you have to download these at the things wnet and OMW and now if I execute this code you will see that it will give you the LZ for all the word see so this is the word this is the LZ this is the word this is the lemmatization now see it's readable okay it's readable then your streaming one I Hope it is clear fine so so that's why I just written uh streaming and LZ are same to retrieve root words but LZ is work uh good LZ is slow and streaming is fast because lemmatization
will give you the readable output that's why it's little bit slow than your streaming okay I hope it is clear so yes these are the technique you can follow for the text pre-processing if you're having any kinds of text you can perform text pre-processing with the help of these Are the technique you can clean up your text okay and again I'm tell telling you it's not required to perform all the text cleaning uh let's say uh I mean technique sometimes if you need anything just try to keep it as it is that is the idea
fine so yes this is all from this video and what you can do right now you can download any other data set from the Kagel kaggle.com because kagle is having different different data set not only movie data set you will also get Something called Twitter data set I think Twitter data set is also available see Twitter data set is also available just try to download the Twitter sentiment data set and try to apply these are the text preprocessing on top of the twetter data data set okay so this should be one task guys from my
side please try to attempt Because unless and until you are not practicing things would be more complicated and whenever you will be doing the Practical Okay by yourself things would be more clear here so in the next video we'll be learning how we can perform the tech data representation that means text representation how we can vectorize our text that means we'll be converting our text to numbers okay for the model so I think you already know what is data representation that means uh inside generative AI you will be working with textual data so not only
textual data you'll be also working with something Called image kinds of data now these are the actually raw file if you see any kinds of text Data any kinds of image data okay so these are actually raw data these are actually unstructured data so I can't directly pass these kinds of data to my model because uh model is nothing but uh at the end it's a mathematical equation only right and mathematical equation can only support uh numerical data so what we have to do uh whatever text Data whatever image Data we are using here we
have to convert them to the vector that means we have to convert them to the number okay numerical representation so this concept we call it as a data representation you can also call it as a data vectorization okay so here we'll be uh seeing couple of Technique we can follow to perform these kinds of uh vectorization technique on top of our data even I will also show you some drawbacks uh with different different technique and what Which technique actually at the end you have to follow for the generative VI if you're working in the generative
VI if you're working with the large language model so which technique would be suitable for your task okay so here everything I'll try to clarify so please try to uh watch this video till the end and if you have any kinds of doubt you can also ask me in the comment section so let's start guys with our data representation topic so guys in this Video I'll be discussing about like what is feature extraction from text you can also consider as image then we'll be discussing about why we need it then third I'm going to discuss
why it is so difficult then we'll be discussing about what is the core idea and Some techniques so so this should be my entire video agenda so first of all we'll be discussing about what is a feature extraction from text you can also consider as image because I told you not in genbi you can use any kinds of data whether it can be text image audio okay anything see first of all we have to understand what is feature extraction see feature extraction is nothing but let's say you are having a unstructured data let's say you
are Having a text data or let's say image data so from this data you are extracting some features to represent that particular data so I think those who are already familiar with with computer vision uh you already learned about convolutional neural network so with the help of CNN what we can perform we can perform a feature extraction that means we just try to extract some of the features from a image so what kinds of features let's say the ages okay then uh You'll be also extracting some patterns so these are the actually features so with
the help of this particular features my model will able to understand about the data and it will be able to predict something okay on top of the test data so this is the idea so in generative way also uh what kinds of data actually we're using we also need to perform something called feature extraction because whether it can be Text data image data any kinds of data The first thing what you have to do you have to extract the features from that particular data okay this is the idea and feature extraction means you are converting
your textual data to numerical representation that means to the vector representation okay this is the idea then the second thing actually you can see why we need it because as I already told you uh in general tvi whatever model you are using whatever large language model you are using every Model is a mathematical equation and mathematical equation can take actually your textual data or image data directly so for this definitely you have to convert to the number okay numbers you can also call it as a vector okay vectors so if you can convert to the
numbers or vectors then your model will be able to take this particular data as an input and it will learn okay what kinds of uh actually pattern it is having based on that it Will try to generate something this is the idea okay now why it is so difficult so whenever I'm talking about text Data okay whenever I'm talking about text Data it would be little bit difficult for you to extract the features from the data why so let me show you some example let's say if I'm considering let's say uh ml in machine learning
I think you remember inside machine learning what kinds of data we used to use I think you remember we used to use something called Tabular Data okay tabular data uh tabular data like we used to use something called csb Data then Excel fine so these are the actually tabular data that means here you will have a table so let's say this is your table so it will have one fixed table so in this table you will have the fixed column size let's say you are having here uh 1 2 3 four column and some of
the rows so 1 2 3 four columns 1 2 three four five rows okay so here your uh Table dimension is 5 cross 4 okay so this is the table dimension that means it's a fixed size it's a tabular data okay it's a tabular data so here you will have the column okay here you have the column and here you will have the row I hope you cleared now you will restore the data fine now whenever you are taking any kinds of let's say uh machine learning model let's say this is your Machine learning model
your ml model so what you will do so how so what is the X feature size X feature size is nothing but your input size okay input size that means the independence variable let's say you are doing house price prediction so let's say this is uh number of room let's say this is the area of the house let's say this is uh number of bathroom and this is the price price of the house so this is your y variable okay and these are your X Variable okay so X variable I'm having how many one 2
three 3x variable I'm having okay that means you will give three input to the model and only one output you will get which is nothing but y hat okay this is the idea that means to the model you have to give X data three column and Y data one column which is nothing but your price and this is your target variable okay and once you pass the data you will get the output that means here your column size is Fixed so here what you saw your column size is fixed that means your input size is
fixed okay to the model okay there is no issue that means we can easily use our tabular data okay for in the machine learning or you can also use in the Deep learning for this you have to use something called Neal Network that means artificial uh neural network right now let's consider uh image data let's say inside computer vision we use something Called image data okay image videos and whenever I'm talking about videos videos is nothing but it's a frame of image right it's a sequence of image so you can convert videos to image then
you can analyze again you can convert to videos so let's consider image only here so let me show you see image is nothing but it's a pixel okay inside image actually we will see different different pixel value so let me show you one example so guys here you Can see here I kept one image and this is a Mist uh digit image so here you can see three has been written so if you just zoom this image what you will see you will see these kinds of pixel okay and inside pixel you are having different
different numbers you can see these are actually numbers so this number range would be 0 to 255 okay 0 to 255 this is the range of the number and one thing whever you can see the zero okay zero number you can see it's a White pixel okay it's a white pixel and whenever you see some number okay it's a black pixel you can see whenever you can see much darker color it's a that means it is close to 255 and whenever you see some mild like darker color that means it is close to zero okay
so this is the idea let's see your three looks like that let's say this is the three in this image and here is the three representation in the number that means if you zoom this image you will see some Numbers that some value pixel value would be there okay so that's how actually your three looks like so whatever image you are taking you'll be getting this kinds of pixel value okay at the end now whenever I'm talking about image data so you can easily convert image to a table representation because it's a pixel value at
the end so what you can do you can create a fixed table you can create a fix table let's say this image image Dimension is 28 Cross 28 okay now just try to the pixel value based on that just try to create the table and add all the number here add all the pixel now if you calculate all the pixels uh here so if you just multiply 28 cross 28 you will get something called 7 84 okay I think that many of pixel so this should be your input size okay this should be your input
size now what you can do you can create a neural network okay dot dot dot 7 uh 84 now just pass all the data one By one okay pass all the data one by one and so on okay till uh 784 okay so that's how again you can create a easy neural network and here input size is also fixed which is nothing but this is the input size because because all the image will have the dimension fixed Dimension okay I hope it is clear that means there is no issue with the input size now the
next data let's talk about audio data okay Audio data so audio audio is what audio is nothing but so what is audio audio is nothing but it's a frequency you can see uh it's a frequency so whenever you will see some audio you will get get this kinds of frequency response and the parameter of the audio would be DB okay that means decibel okay this is the frequency of the audio now you can easily convert this audio signal to a number representation because here you can see Let's say this is your uh Hearts okay this
is your hearts frequency HS and this is your TB that means decibel like now you can see this is the uh actually frequency now just try to see the graph and based on that just try to collect the number let's say you have created one table let's say this is your table so one column you just consider let's say this is the frequency and this is the decibel okay DB now just try to collect the data Let's say here you can see 400 Heartz and the decibel value was let's say minus 10 DB okay that's
how just collect all the data okay collect all the data now again you will see we will have one fixed table okay we'll have one fixed table again you can take a deep learning model here let's say artificial neural network and you can fit the data you can fit the data and you can get the prediction okay again you don't have any issue with the input size but whenever I'm talking about let's say Tex data whenever I'm talking about Text data so here you will get the difficulty how so let me show you so let's
say this one sentence I'm having my name is buy so here just try to see how many tokens you are having so we are having uh 1 2 3 4 that means four tokens let's say somehow you converted this text to a number representation that means Vector representation let's say my you are Considering as zero name you are considering as one e is considering as two and byy you are considering as three now you'll be creating a network let's say this is your Neal Network 1 2 3 and four then what you will do he
will feed the data let's say my will come here name will come here e will come here and buy will come here okay now let's you have taken some hidden NE Network and here you will get the Yad okay now let's say there is another sentence how are You now see here you will get the difficulties because in the previous sentence how many tokens I had I had four tokens now if you count this sentence you are having one two three only three tokens now let's say you have represent presented this tokens four 5 6
okay let's say this is your vector representation now tell me can you can I pass this particular data to the model right now because what is the model input size four but here what is the Input you are having three so here you will get one dimensional issue okay Dimension Dimension issue you will get okay so it will give you one error so it is expecting four dimensional input but you have given three dimensional input now just try to consider whenever we write any kinds of text it doesn't have any kinds of input length similarity
okay you can write anything you can write my name is buy how are you I'm okay so it should be any kinds of length Sentence okay it can be any kinds of length sentence and there actually we usually face the difficulty whenever we use these kinds of textual data because it doesn't have any kinds of input size because it's a completely unstructured data and representing this uh text to numbers is very much difficult because I can't assign 0 1 to three like that because if you're Ling 0 1 to three like that your model won't
be able to understand it's not a meaningful number But previously the data I showed you we can easily we can easily convert them to the numerical representation because tabular data is already a numerical representation data right then image is already a numbers because at the end you are getting a pixel here then audio is already a number you are getting a number here but what about text text is a string but how we can convert string to a number representation so that it will it will have the same meaning of my Sentence let's say here
I have written a my name is buy so after my name is coming after name is is coming after is buy is coming so with the help of number how I can uh let's say contain that particular relationship is it possible yeah it is possible but it would be little bit difficult for you initially because you don't know like what kinds of let's say vectorization you can perform here to represent this particular sentence to a let's say Meaningful representation that is what I'm trying to say right so now you can ask me sir we can
apply actually two techniqu we have learned previously I will tell you what kinds of Technique we can apply you told me let's say the first Technique we can apply something called one hot encoding one hot encoding so this encoding technique you have learned in your machine learning right that's whenever you have any kinds Of categorical data used to perform this one not encoding right then the second technique you can also perform something called bag of word bag of word we can also call it as a bow okay bag of work so now let's try to
discuss this two technique uh how it can be applied and what are some drawbacks okay if I were applying this these kinds of technique okay so first of all let me discuss why not encoding so what I'll do I'll just create a table here so let me create a Table so here I will take uh two columns and four rows as of now I'm only taking four example just just to show you okay how things will be working but in actual way you will have uh more than actually four rows okay that means you might
have thousands rows let's say lacks of rows okay uh it doesn't matter let's say this is your data one so here you are having a sentence let's say people watch let's say I'll give my YouTube Channel name DS with buy so if you don't know guys this is my YouTube channel name if you're interested you can also so Vis it then let's say data two we are having DS with buy much DS with buy I'm writing any kinds of sentence guys as of now whatever things I'm thinking in my mind now it's a data three
you are having People write comments and data 4 I'll be writing D4 DS with buy right comments okay let's say this kinds of data we are having initially fine so that means how many rows we are having we are having Row one row two then Row three and row four that means only four row Pro we are having as Of now okay four example we are having four data points we are having right now if I want to perform one not encoding so first thing what I have to do I have to take the entire
Corpus okay and I already told you what is Corpus uh I think in my previous session the text uh pre-processing session at the last I was discussing about some like key important things you have to remember okay so whenever I'm talking about Corpus that means your entire data so just try to Collect your entire data let's say people then you have the watch then you have the with puy then you are having um then again you are having D with buppy Ds with buppy then again watch then again DS with buy then you are having
people then write then comments then again DS with Buy write comments okay so this is your entire Corpus okay Corpus okay Corpus of your data now from here you have to figure out how many unique word you are having so we are having people we are having watch DS with buy then uh write and comments so these are the unique words that means 1 2 3 4 and five that means n is equal to five okay five unique word we are having now what we Have to do we have to uh create a table let's
say I'll create a table with this five unique words so the first word will come people the second one will come watch then the third B will come let's say DS with buy then write then comments okay comments okay so that's how your one encoding will try to represent the data okay let's say Whenever you using any uh let's say library in Python so how this Library works exactly to perform the one encoding that is what I'm explaining here fine now see how it will try to convert your text data to numerical represent ation so
first of all it will take the row one that means D1 let's say people now just try to find people here so you can see people is there that means this should be one and everything would be Zero okay then watch now just try to find out the watch so here is the watch so it should be one and everything would be zero again okay then D is with BP so it should be one and everything would be zero okay so here the number actually we got this is for the D1 that means data one
row one so here I can write like that let's I'll just simply write D1 okay D1 is equal to so here I got one two dimensional Vector so here 1 0 0 0 uh zero okay then comma comma 0 1 0 0 0 then comma I got 0 0 1 0 0 okay so this is my D1 that means sentence one okay this is for the sentence one okay I hope you're clear now again you will take the sentence two that means D2 okay D2 now you'll just try to find the numerical representation now let's
find the D2 as well so let's say d is with B so again let's say this one would be um one and everything would be zero Okay so this is for actually D1 and now we are doing for D2 now again it will see the watch so watch is here it would be one and everything would be zero again now the next thing would be DS with B again so let's say this is a DS with buy and everything would be zero again okay so this is what actually your D2 so again I can represent
like that let's say D2 is equal to uh 0 0 1 0 0 comma 0 1 0 0 0 okay then comma I'll take 0 0 1 0 0 okay so This is my D2 so that's how you have to complete for all the sentence that's how you have to complete for all the sentence and at the end you will get all the vector representation okay you will get all the vector representation I hope it is clear okay how one encoding can be applied now let's say you have completed for all the sentence and you
got your vector now what you will do you will just try to create a newal network now just see the input size the input size For the model you can see uh input size would be 1 2 3 4 5 because we are having five unique W only so everywhere you can see five dimensional Vector 1 2 3 4 5 that means you'll be taking a neural network like that 1 2 3 four and five okay inside that you will take some hidden layer and you will get your y hat here okay so that means
you will pass the data okay you will pass the data in this particular layer that means you will take this particular Vector so First of all you will you will do for the D1 you will take this particular Vector you will pass it here okay then the second one you will pass it here then the third one you will pass it here okay that's how you'll be passing for all the sentence okay you'll be passing for all the sentence but let's say in case in your Corpus okay in your Corpus the number of unique word
is 1,000 number of unique word is 1,000 in that case what Should be the input size of the model definitely it will also become 1,000 that means you have to take one dimensional sorry uh 1,000 1,000 neuron okay at the very first that means again complexity is increasing let's say in your entire Corpus you are having 2,000 okay 2,000 uniqu so you have to take 2,000 actually neurons so if you're using one in coding that means the number of unique word you are having in that particular Corpus based on that Your input layer would be
decided okay it's not a recommended because if your input if your let's say unique word is inre ining your uh neurons is also increasing and whenever you are increasing the neurons your uh calculation would be increased also because at the end you are doing uh neuron calculation you are doing the computation here so computational cost will also increase right so that's why it's not a recommended way again there Is another issue let's say I'm having a sentence and which have actually the word which is not present in my Corpus let's say I can write my
name is buy let's say this sentence actually I'm having okay now just try to see these are the word is having in my entire Corpus no these are the word is not present inside my entire Corpus okay that means what will happen it would be out of vocabulary issue that means the vocabul You are having right now it is not present inside your Corpus so your model won be able to understand these are the what because you haven't used this data during training so whenever you are giving doing testing definitely it won't be able to
understand what kinds of sentence you are giving so again it is called actually out of vocabulary issue O oov Okay out of vocabulary issue so if you're using one encoding so you will get this kinds of issue okay out of Vocabulary issue and another issue I think you saw here you are having lots of zeros you are having lots of zeros and zero is what it's a sparity problem that means it's a SP sparse Matrix you are having okay and whenever you are doing the computation and whenever you are using zero okay to perform any
kinds of operation with number so there is uh no meaning right that's why zero is unnecessary number here zero is unnecessary number here so you are Unnecessarily increasing the computation here because with because you can see zeros is increasing the dimension in this particular case so let me just write some of the drawbacks of the one encoding so here I'll just write the drawbacks so the first drawbacks you can see the sparity that means here you are having lots of zeros then the second you you can see the no fix sized because anytime your sentence
might be anything okay Let's say here I have taken uh people watch DS with buy I can take also my name is buy then I'm a data scientist now just count of the sentence length you will see that it would be bigger okay so you can write no fixed size then the third you saw o that means out of vocabulary issue and the fourth important things it is not not capturing semantic mining meaning semantic meaning semantic meaning Means uh after people watch is coming after watch DS with buy is coming before DS with buy watch
is coming before watch people is coming these kinds of semantic information semantic relationship this is not capturing because it's just a number and whenever this word is present it is giving one otherwise everything it is giving as zero that means this particular number is not capturing the semantic information of your sentence but whenever we are using Text data so Definitely it will have the semantic relationship that means it will follow on grammar what is what kinds of grammar let's say whenever I'm writing any kinds of sentence let's say my name is buy so my is
a subject so my is a subject name is noun okay is is verb bu is also noun that's how it is following on grammatical rule grammatical rule okay that means after my name will come after name uh is will come after is buy will come okay so it is following one Relationship it is following one relationship like that right so this kinds of relationship it won't be able to capture if you're using one not encoding okay so that's why we can't use this particular uh method okay we can't use this particular method whenever we want
to convert our data to numerical representation that means Vector representation okay inside generative so this technique you don't have to follow because it's like very Basic technique if you're doing any machine learning project if you have some if you have some level of categorical data in a column you can use this uh I mean one not encoding that but whenever you are working with completely textual data inside Genera VI I own suggest don't use this particular method because I already showed you some of the drawbacks you will get here so now the second technique we'll
be discussing about bag of word okay bag of word now Let's try to see the back back of word example like how it will work so for this I'm going to copy the same table so again uh you can consider the same things Corpus so I'm using the same table that means it will have the same Corpus again again the number of let's say uh uni word would be five and you have to create a five uni word table so let me again copy now how this bag of word will work see see this bag
of word will work based on the count okay based On the count of the what present in your entire Corpus so let's say first of all it will see people um so now just try to tell me how many times people you can see in this particular sentence uh I can see only one time so people is coming only one time okay now now the next word watch so how many time watch is occurring in this particular sentence only one then DS with buy one time and right is there no so right is not there
it would be zero And comment is not also there it would be zero okay then the next sentence you can see DS with buy only one uh DS with buppy now occurring two times so it would be two then watch is occurring only one time one and uh people is not there right is are not there comment is not is not also there right now the third sentence people is coming one time right is coming one time okay comment is coming one time and watch is not there the is not there now Now the last
sentence DS with the p only one time write is only one time comment is only one time DS with uh sorry uh watch is not there people is not there okay so this is how actually you'll be generating the vector okay this is how actually be generating the vector now just try to match with your previous Vector what you have observed you see the bag of what technique you have applied it is having very less zeros than your one not encoding okay that Means it is slight better than your one not encoding okay because it
is working based on the counting okay based on the counting okay counting of the word present in the sentence okay this is the idea now research proven research okay research already proven that if if you if you want to do something called classification kinds of task let's say sentiment analysis okay sentiment analysis then let's say we are Performing positive negative reviews okay these kinds of task you can use this uh V for technique to perform the text vectorization because this is suitable for this particular task because if you see any kinds of sentiment analysis problem
let's say here I'm having some sentiments this movie is wow or you can also write this movie is amazing Amazing performance okay performance let's say this kinds of reviews you are having now if you want to find the positive okay positive or it's a negative I can see the let's say word so movie amazing amazing performance okay you can see amazing this particular positive ver is occurring like multiple time that that time I can consider it's a positive review okay it's a positive review so that is why it's a frequence frequency Based actually approach so
it will always try to see the frequency frequency count in in a sentence and based on that it will decide whether it's a let's say positive sentiment or whether it's a negative sentiments okay so that is why it is recommended if you're performing any classification kinds of task inside a text you can use this particular approach okay but again um one drawbacks you'll see it is not capturing the semantic information like After DS with buy watch is coming after watch DS with buy is coming all people watch is coming afterward with bu is coming so
this kinds of semantic information it is not able to capture so again it's a drawback okay and some of the zeros still you can see okay because again it will increase that uh your computational cost because zero doesn't have any kinds of meaning if you're doing any kinds of operation right this is the idea fine so apart from that we Are having some more Technique we can follow like we having something called TFI DF so we are having tfidf then we are also having another technique called word uh word to V so this one use
one statistical let's equation to perform let's say vectorization and this one is the de deep learning based actually vectorization technique okay and from this technique actually lots of idea came how we can use the Transformer model to perform the feature extraction Because uh whenever I'll teach you this word to B technique I'll show you the feature extraction technique so how it will extract the features from a sentence let's say from a text okay so from here actually idea came then people have uh actually launched Transformer based actually Transformer based actually encoding technique that means with
the help of Transformer model you can perform the text representation that means you can Convert your text to Vector representation and whatever large language model you using okay llm you'll be using this Transformer based approach only to convert your data to the numerical representation so whenever we'll do the Practical that time definitely I'll will show you okay we'll be downloading the model from the hugging face hugging face is also having lots of tokenizer that means these kinds of uh encoding encoding model okay with The help of that we can easily convert our data to the
numerical representation fine so as of now let me uh show you all the Practical that means this bag of word then TF IDF then what to B okay how we can use it and all but before discussing tfidf and what to B I just wanted to show you this bag of word practical so for this what I will do I will uh open up up my collab notebook and there I'll show you this particular practical so guys here you can see I Already prepared one collab notebook for you so one not encoding I'm not going
to show you because uh you already know how to perform one encoding so with the help of Panda's Library also we can perform one one encoding even uh we also have one let's say class inside esal and Library okay to perform the one encoding so I'm expecting if you already work with machine learning you'll know how to perform one encoding and again we are not going to use one encoding technique In our analysis but bag of what we be using sometimes because I told you now if you're doing any kind of sentiment analysis and all
that time you can perform because it's easy to implement okay at the end now see uh bag of word for this I need two library npay and Panda so let me import them so with the pandas actually I'll be creating a data set first of all so the same data set example I have taken the data set I showed in my whiteboard okay so you can See people watch this with buy D with what DS with buy okay these kinds of example I have taken now let me prepare the data frame okay so this is
my data frame and here I also have taken another let's say column called output let's say whether it's a positive whether it's a negative so that's how have just given a number okay it's just a assumption number as of now um don't try to think it's a actual number fine one means it's a positive Zero means it's a negative okay but whenever you will be using actual data there will have the positive negative previews okay based on the text you are having I hope it is clear now if you want to perform bag of word
technique you have to use one uh Library called escalan from escalan you you have to use something called count vectorizer okay so count vectorizer is nothing but it's a bag of word technique so I I know that many of you already use this count Vectorizer in your machine learning but you don't know this is a bag of word okay so now just try to remember it's a bag of word technique only right now we have created the U like uh counter vectorizer object now here I just need to fit my data and I want to
fit my data on top of my text that means Text data now here I'm fitting the data now it will return me all the vocab now see uh what will happen here it will just uh assign some of the index number okay it Will just assign some of the index number with all the words now who whoever having actually zero that means this is the first index that means this word would be considered at the very first then uh this one that means number one then two three and so on right now how it is
generating this particular array so let me just try to discuss now just try to see which word is coming at the very first based on the index comments now let's try to see comment is There in the first sentence no comment is not there that's why it is coming as zero got it now the second sentence you can see now the second word you can see DS with buy now tell me DS with bu is there yes how many time only one time that's why it's coming as one okay then the next one people people
is there yes how many time only one time okay then the next word watch watch is there yes is there how many time only one time then The next one I think last one right right is there no right is not there that's why uh it is coming zero okay now let's consider the second sentence okay now again you have to consider based on the index again comments is there no comment is not there so that's why it's zero now next thing yes with B yes it is there how many time two time that's why
it's coming as two okay then uh you can see the next one people people is not there that's why it's coming as zero Okay it's coming as zero zero then after people actually we are having watch watch is there is there how many time only one time okay then after that uh we are having something called um this one right right is there no right is not there that's why it's coming as zero that's how it you have to consider for all the sentence I hope it is clear how it is generating this particular number
okay and this is your vector okay this Vector actually you can pass to the Model okay this Vector you can pass to the model so if you're only printing this B object see what you will get you will get one object here okay you will get one sperse Matrix object and if you want to get the uh number that means the complete array you have to use this particular method called two array that's why every time I'm doing this one okay just to show you uh to show you all the number that is the idea
fine now if I want to perform On top of my new data let's say this is one sentence I've given B watch this with buy uh so I have to use this particular transform okay transform function that time cv. transform pass your data and convert it to two again you will see your data representation fine now what you can do you can store your X and Y data let's say x is nothing but my entire B and uh Y is nothing but my output so this is my output okay positive or negative now you can
pass This data to the model okay this is the idea got it now there is another things inside bag of for called NRS okay what is NRS NRS is nothing but here you see NRS is nothing but here you do the same thing only you give one NRS parameter so let's say if engram parameter is two so what will happen let's say engram parameter is equal to two that means it will consider two two word as a one token okay two word as a one token see previously I I took individual word Right now it
will consider two word as one token that means here it will come people watch okay inside this box people watch will come then watch DS with buppy will come okay so that's how you have to take pair of word pair of word okay and you have to perform the same thing so this is called actually n Gams now let's say n is equal to three that time three pair will come that means three word you have to consider okay but I saw that people are using only two uh parameter Because sometimes uh it is required
so let me show you one example why it is required so let's say here I'm having a review movie is very good and here I can write it this movie is not very good now this one is the positive uh sentiment you know this one is the negative sentiment you know okay but how about your model okay how it will Identify because I told you based on the frequency count it can decide let's say let's say you are having some word let's say very good movie okay that means it's a positive one and if it
is coming to the second sentence let's say this is the first sent the second sentence again it can consider movie very good and it will ignore this one because it's a stop word I think I taught you the stop word concept right not would be considered stop word so again your model will try To consider it's a positive okay it's a positive review but all it's a negative review but how I can capture this negative information so if I am doing this n gs that means if n is equal to let's say n g is
equal to two it will consider two word as a pair okay two word as appear so the first two word then second two word then these two word and this two word okay so it's a positive one you can see now you can see it's a positive one so this movie very Good okay now again if I just uh make this this sentence let's say this word this word this word this word and this word now you can see easily it is capturing is not okay is not now see it is easily capturing this is
not that means not word as well okay now easily your model can identify it's a negative okay it's a negative sentiments okay so that's why sometimes we have to perform this particular n Gams operation okay because if you're giving nram parameter Is equal to two that time it will consider two tokens okay in one just let's say representation and it can capture some of the hidden information okay this is the idea now let's see the practical so here you can see the same uh example only you have to use the counter vectorizer I'll convert to
the text now see whenever I used counter vectorizer here I have given a parameter NRS range is equal to 2 2 comma 2 that means it will consider two words okay Now you can see people watch watch DS with PDS with B P Watch okay now it is considering two tokens okay uh in just one representation that is the idea if you give three so if you give three here it will consider as a three word okay three word as a count okay that's how you can consider fine and if you're using three three word
that means it would be considered as a tyram okay tyram and the if you're using two words if you're using two it would be Considered as a pams okay this is the idea now there is another technique I already show you TF IDF that means TR frequency inverse document frequency so this will work based on one equation based on one let's statistical equation so simply in Google you can search for this equation TF uh TF I DF so you'll see this particular formula so yes this is the formula guys it use okay to perform this
uh text representation technique okay so It will work based on the let's say weight let's say let's say if anyone what is having let's say uh frequent count okay in a sentence it will give more weight okay it will give more weight now if I show you the representation so you have to use this escalan to initialize TF tfidf vectorizer now inside that you can pass your data let say text dataa I'm passing again I'm converting to the array representation now see you are getting The entire array okay now see none of the number is
zero complete zero you will still get some number okay it would be a floating number so this technique helps us to reduce that sparse Matrix dimensionality issue because uh if you're are using zero okay zero in your let's say Vector you are doing unnecessary computation that's why it will assign the weights and weight is nothing but it's a floating number okay and everywhere you will see the floating Number so this particular issue will be resolved because still after performing bag of word you are generating some zeros but if you're using tfidf you are not generating
any kinds of zeros okay so this is the B benefit here and this is better than your uh let's say this one uh bort okay sometimes you can also use this tfidf technique again I won't be recommended to use this one because we have some better option to do next uh thing I'll be teaching you that what to V technique that means we'll be using deep learning model for the fure extraction okay and this will uh contain the santic information because again if you see this TF IDF is not able to uh actually contain the
centic information that means after let's say people watch is coming after watch D with B is coming these kinds of santic information is not able to capture okay so again I won't be recommended to use this TF IDF technique here as well fine and here is the some TF IDF value okay if you are understanding this formula you'll get it okay what kinds of things it is generating but initially I won't be suggesting you to Deep dive in this equation because again because again it's not a recommended we won't be using it okay we'll be
understanding whatever actually we need okay for our large language model that's that idea so yes these are some actually practical I showed you for the text representation Technique and word embedding technique now the next thing we'll be discussing about uh this one a word to V technique a word to V technique okay now let's try to see how word to V technique works okay so guys now we'll be discussing about word to V so this is the Deep learning approach actually so how what to vake uh works and how it will extract the features from
the uh data itself I'll try to clarify see what to V actually use uh Two kinds of architecture so if you want to see just write word to V architecture so you'll see it has actually two kinds of architecture called cow so here you can see guys uh it uses two kinds of architectur cow and Skip gram so it's a neural network only right so let me show you the code mechanism of what to W like how it extract the features let's see you are having a documents let's see you are Having entire Cor and
in the Corpus let's say you are having some unique word so let me create a table so I can create a table just a minute so here I'll be adding all the features and this side I'll be adding all the unique word let's say you are having the word of King let's say queen let's say [Music] man let's say woman and let's say Monkey so as of now just try to consider these are the unique word you are having in in that particular entire corpus now how your word to V will try to extract the
features so see it's a neural network and neural network uh Works based on the back propagation because every time it will adjust the weight based on that it will learn and if you see the neural networks uh you can't see like what kinds of feature it is extracting because it's a completely Hidden thing if you see for the let's say CNN if you see the for the NN so these feature extraction things is a completely hidden you don't know what kinds of feature it will extract but as of now let's consider uh it has let's
say generated some of the features let say the first features it has generated based on this word gender because gender features can match with all the words you can see because all the let's say word I'm having let's Say all the entity I'm having it has the gender yes or no right then it has also generated another features let's say wealth now you can see wealth is also matching with all the words now we can also consider power then weight then speak see as of now I'm just assuming these are the features my model has
generated because here we are using deep learning model I think I showed you now See that's how actually it will identify the features that means the vector representation first of all it will see King has gender or not yes King has gender and it's a male king let's say male king queen has gender yes let's say zero zero means female man as gender yes one woman yes that means zero because man I'm considering as one and women I'm considering as zero right monkey has also genda let's say one now wealth King has wealth yes King
has wealth let's say I can assign as one because King is having lots of wealth Queen is also having wealth let's say because Queen is a part of King right man is also having wealth but less than king and queen so I can give let's say 0.5 okay or I can give let's say 0.2 fine because again they are king and queen right woman has also wealth that's a 0.2 less than man monkey doesn't have any wealth I will give zero Now power yes definitely King has the power Queen Also has the has the power
okay but I can just decrease a little bit let's say less than K King okay I can give 0.7 man has also power let's say 0.3 women has also power let's say 0 point um 0.2 monkey no monkey doesn't have any power now wait yes King has weight let's say I will give 0.8 let's say this king is little bit obese Queen yes she has Al weight I can give let's say 0.5 very slim Queen man let's say 0. 7 wom let's say 0.5 monkey let's say 0.3 Fine King can speak yes King can speak
Queen can also speak man can also speak woman can also speak monkey let's say can't speak okay now see beautifully I have generated some of the vector okay beautifully I have generated some of the vector and I haven't generated it has generated by your what to fake algorithm okay part to V model that means this architecture that deep learning architecture you saw cow okay CBO and Es skip Gram okay so these are the architecture let's say it has generated this features now if I tell you just try to give me the king Vector so what
it will give me it'll give me let's say king is equal to let's say this is the king Vector so I can write here one then one then one then let's say 0.8 let's say one this my king Vector you can also give me the queen Vector so here is the queen Vector 0 1 0.7 let me just write 0 1 0.7 then 0.5 and one okay this is the queen Vector you can also give me the man Vector yes the man Vector 1 0.3 0.3 0.7 1 0.3 0 3 0.7 and 1 okay then
woman 0 0.2 0.2 0.5 0 0.2 0.2 0.5 and 1 then let's say this is the monkey Vector 1 0 0.2 monkey one sorry should be one then 0 0 0.3 0 0 0.3 sorry 0.3 and I think zero okay see all the vector I got now if you plot them in a dimensional space let's say this is my Dimension as of now I'm just drawing two dimensional Dimension okay because uh 3D 4D I can draw here that's why I just consider two dimensional space now here you can uh let's say this is your x
coordinate this is your y coordinate now let's say first of all I want to let's say represent King here so Let's say King King is here let's say this is the King this is the king Vector okay this is the king Vector I can write this is the king now if you just now if you just uh plot the queen also you will see that Queen will come close to King okay Queen will come close to King why because if you see the vector okay if you see the vector they are very closely related Vector
you can see let's say if you calculate the queen Distance with respect to all the vector you'll see that Queen will have very less distance with the King because both are similar okay both are similar okay because uh you can see Queen is Queen is the part of King so all the features are already messing with the king that's why it is coming to close so that means it's a it's a cluster okay it's a cluster of king and queen so they are falling in the same cluster now similar wise if you plot the man
and woman Vector you will See that they will uh they will come nearly let's say this is your man okay man vector and let's say this is the woman [Music] Vector because if you see carefully men and women Vector they are closely related okay they uh they are almost similar so that's why they are coming in the same cluster so again it's a cluster okay for men and women and if you see the monkey okay monkey is a completely Different entity you can see the vector is also different so let's say monkey will come here
so I can take a new pen let's say this the monkey monkey okay and monkey and monkey it is present in the different cluster okay so that's how it is capturing the centic information okay between multiple word now let's say there is another sentence you are writing let's say I am a princess okay princess now tell me where This princess will come in which Vector me which space okay in which cluster it will come definitely inside this particular cluster because if you uh just extract the features from The Princess you'll see that uh it would
be close to King and Queen okay so that's how uh this word to V can uh actually understand the semantic information of a word okay so that's why it is uh so that's why what to V is the much better than your previous whatever technique You have learned so far okay and from here actually your actual concept starts like whatever things you will be learning in future let a Transformer based encoding like how Transformer model extract the features that's how it extract the features okay because at then it's a deep learning model and by the
back propagation itself it will generate deser the features but as a human you won't be able to see the features but here we are assuming let's Say these are the features your model May create and based on that that's how it will generate the vectors embedding okay and when it comes to centic information that's how it will uh let's say capture the centic information because it will plot the all the vector in a dimensional space and then it will try to find the relationship whether king and queen are same or not man and moment same
or not monkey is a different or not okay I hope it is clear now so Let me show me show you one image actually uh experiment of what to V you can see this is one experiment for the word to so here you can see we are having three kinds of word King man and woman and whenever what to is applying on top of it it is generating some features you can see so this color represent the features let's say uh these are the features it has generated let's say this is uh wealth this is
uh Power this is speak this is and let's Say this is the weight you can see King uh color is different than your man and woman now see man and women color they're closely related you can see all the color is messing okay all the colors are messing that's how it is making a separate cluster of man and woman and King is a different entity here okay I hope it is clear now so now let's see the Practical of what to V how we can use what to V in a python code so for this
I'm going to use one data set so Let me show you the data set here so the data set name is games of ss book data set okay data set in Kagel I think you know what is games of th okay it's a series actually so if you don't know you can simply search on image see uh this is one actually uh series okay this is a movie series actually games of THS and it is having different different characters so if you have already watched this particular let's say series you will know okay About this
particular character and all So based on this particular series one book has been published and uh this data set is avable in the Kagel also this book data set now this book contains all the dialogue okay present in that particular series okay see all that dialogue it is having different different txt file you can see based on the let's say season okay it is having different different dialogue from all the characters so what I'll do I'll just try to download one txt file and I'll just try to apply what to V on top of it
and I'll try to see what the Character Are closely related that means I'll see these kinds of let's say uh this kinds of say representation that means whether some character are uh in the same clust on not because if you see this series now some of the character would be closely related okay let's say character one that's a character two they're similar kinds of Character so these two Vector will come closely okay that's how we want to see the relationship okay between the data so I already downloaded one txt file you can also download just
try to search for games of th books in kagle you will see this particular data just try to download one txt file so let me show you the notebook so guys this is the notebook and here we'll be showing you the what to fake now let me connect and this data set is already available in This link I have already given now let me download and let me upload in my Google collab so here I can upload this data so I'll create a folder here called data and inside that I can upload this txt file
now first of all let's import some Library naai pandas and Jim I'm importing because what V is there inside genim Library okay if you want to use whatto you have to use this particular Library called Jim inside that you have the whatto architecture now let me import them now if you're using Google cab first of all you have to update the genim because otherwise you will get some of the issue so let me update okay update is done now I'm importing like sentence tokenizer from the nltk and I'm also importing simple pre-process from the genim
library because uh before passing my data to my Let's say uh whatto model first of all I have to apply some processing okay processing means it will do some cleaning operation and all then uh you can pass the data so let me import them now this Cod spp it will load your data from the data folder you can see okay so here I'm having test.txt so it will load the data and it will extract all the let's say uh word that means all the sentence one by one and it will perform from the Tokenization okay
it will perform the tokenization that means every word will have the tokenizer okay now if I show you the story that means start story list now see you got your all the sentence okay see this is the entire sentence and inside the sentence you you have done the word level tokenization okay this is the idea now if you check the length of the entire story you have more than 8,000 uh actually let's say example here So here you are having more than 8,000 sentence that means 8,000 dialogue okay now we can see the entire story
now if you want to see the story one that means the first uh let's say dialogue the first sentence this is the first sentence okay you can see and again we have done the word level tokenization okay individual token now we have to initialize the genim model so that's how we can initialize the genim model so let me Initialize then if you want to let's say convert to the vector representation just try to call this particular function bu uh build vocab now inside that pass your data then you have to train the model so you
have to give your data total example then total example you will get total example from the model. Corpus count that means it will count the entire Corpus and it will give the count and you have to give the epoch okay Epoch You can take the default actually Epoch from the model itself now see my training is completed now if I want to see the similar so similar of Deniz so Deniz is a character inside this particular let's say uh Series so see she is the character now you can see the similar score of the D
with these are the word okay these are the word but see again you will get some let's say random output because here we are only using one Dialogue that means one txt file I'm not using the entire txt entire dialogue Because unless and until my model is not getting the entire dialogue it would be a little bit difficult for uh it to match those character yes or no right so if you're doing it just try to take all the data and try to perform this job but here just to show you I have taken only
one THD file now you can see with still word it is having 0.99 okay per relationship okay din eyes then and Prince so Prince is another character prince in gamesof songs see with him actually uh her character is matching if you watch the series actually you will see that with him her character is matching okay mostly so you can see then deny then uh doar key also is a character see okay so that's how you can actually match now see it's not actually Properly visible so what I can do I can also plot them in
a so what I can do I can also plot them in a uh visual graph representation I'll tell you but before that let me show you if you want to let's say see the relationship between two character you can use the similarity function inside that give two character let's say Arya and Sansa so if I give Arya and Sansa so they are having this is the relationship okay now if you want to get all the Vectors you can execute this code it will give me it will give you all the vectors okay all the vectors
for the entire text you have given okay and the shape of the vector see now let me show you the visualization I apply the PCA that means principal component analysis to reduce the dimension that means I want to plot in a threedimensional space right now so here I'm initializing PCA component now pca. fit I'm giving my data that means Vector so it will give Me the uh three dimensional Vector right now that means from 100 Dimension it has reduced to three dimensional Vector I think you know what is PCA principal compound analysis in machine learning
we can use it to reduce the dimension size now you can see the dimension right now so this is the dimension 3 and here we are having 3,840 example in this particular entire text now see if I want to plot it I can use plotly library now inside that just Pass your data and set the colors it will show you the interre representation of the data see that's how you are getting a 3D space see this is the 3D space now see the see the different different uh word okay different different word and their similarity
so here you can see they are closely related these are the cluster so this word is nothing but so this red is nothing but just try to see it's a take and this blue is nothing but it's a put That means put and take they're falling in the same cluster because they have the relationship okay if you just find uh in a grammatical way you will see that they have the relationship and apart from that you will also see some character okay you can also match the character you will see that they're falling in the
same cluster so you can play with this particular graph uh I'll share the notebook you can Play with this particular graph see that's a beautifully it has stored the data in the dimensional space okay and that's how your model will identify okay the semantic relationship okay between two wordss that means I showed you now here here the centic information from the word itself I hope it is clear now fine so yes so that's how actually we can use this word to V and if you're using word to V you have to pass this this particular
data sorry uh after this Data okay the vector we are getting the entire Vector that normalized Vector okay and this is your vector representation this Vector you have to pass to the model but again we won't be using that we have just learned this part to V just to understand how my Transformer architecture will be working how it will extract the features and how it will get the let's say semantic information so in Transformer architecture you use something called Attention mechanism let's say attention model with the help of attention model uh uh it can actually
understand the semantic information in a good way okay this is little bit Advanced concept okay that's the idea so yes this is what actually we can perform our data representation that means text representation and that's how we can perform the vectorization technique so why we learned this because going forward whenever we'll be learning about Let's say large language model and all uh before understanding large language model I just wanted to let you know how data would be prepared for the model okay now I think you got it how we can prepare our data for the
large language model okay if I have any kind of raw text how we can convert to the numerical representation and which approach would be better because finally we saw that this what to V approach is better because it is able to extract the Semantic information okay semantic information because it is a deep learning model okay this is the idea so yes I hope guys you got it now in the next video we'll be using all the let's say topic we have learned so let's say text pre-processing and text representation technique uh and we'll be doing
one small project with the help of machine learning we'll be doing one text classification project as of now we have learned about text pre-processing and uh Text representation technique like how we can convert our textual data to new medical representation that means the vectorization right so we'll be using all the concept we have learned so far and we'll be uh bring one hands on that means we'll be doing one practical so here we'll be implementing one text classification model so in this video my objective is to show you how we can prepare your data set
for the model that means how we can apply all the Preprocessing technique how we can apply uh these kinds of text factorization techniqu so that you can use those data okay for the model training but model wise actually I'm not going to use large language model as of now I'm going to use simple machine learning model later on I will also show you how you can use large language model okay because it's a like very initial phase we are doing as of now we haven't learned about the llm that's why I'll be using simple machine
Learning model so let's uh open open up our Google collab and uh let me show you the text classification practical there so guys as you can see here I already prepared one notebook for you and the data set actually I'm going to use so this is the link of the data so the data set name is IMDb movie review data set so this data set is having uh 50k movie reviews so you can download this data from the kagle so kagle website it is available so it is having reviews based On the sentiment whether this
reviews is a positive or negative okay so it's a classification problem statement so I already downloaded the data I have already given the link you can download the data so let me upload the data here so I'll just try to upload so this is my data so here let me upload it okay my data set is uploaded here now the first thing what we have to do we have to import some of the libraries so here I've already listed down all the Libraries you need uh for this experiment so I need naai pandas mat plot
leave okay then I also need uh 10est Le counter vectorizer tfidf Transformer okay so everything I have just imported here and as of now I'm not going to use this Keras so let me just remove it it's not required so here I already told you I'll be using simple machine learning approach so I don't need to use any kinds of deep learning library as of now so fine you can see Guys I have imported all the library required Library I need and definitely I will explain whenever I will use it I'll tell you okay why
we're using this particular library now let me import all of them I think naai and pandas we already imported here so so I don't need it I can delete it now you don't need to execute this line because uh here I have my data in my uh Google collab you can see Google collab space but if you have Your data in your Google Drive that case uh what you can do you can mount your Google Drive so I'm not going to execute this line so directly I going I'm going to assign my data path so
I'll copy and here I'm going to mention the path of my data okay so let's say this is my data part now I'll execute now I will load the data with the help of pandas Library now see guys this is the entire data you are having and here you are having the uh this one your reviews as well as the Sentiment of that reviews now if you want to see the shape so here you are having 50,000 movie reviews and two columns uh reviews and sentiment now see here uh I don't want to take all
the data because again if I'm taking all the data it will take lots of time to process so I'll only take the 10,000 example okay from uh this 15,000 example so here you can see I'm taking only 10,000 example first 10,000 and now you can see this is my Data now if I show you the data shape now see only 10,000 example I have taken now if you want to see any kinds of reviews you can also check this is the first reviews now if you want to see the sentiment value count that means how
many sentiment it is having it is having two sentiment um positive and negative you can see so positive sentiment is having around uh 5,000 and negative sentiment is having around uh it is also around 5,000 you can see 4 4,000 972 that means it's a balanced data okay we don't have any issue with the data now let's see whether it is having any missing value or not for this I can use is Nal function now see there is no missing value now let's see any duplicates is there or not so you can see we are
having a 17 duplicates what I can do I can drop the duplicates for this you can execute this line of code drop duplicates and in place is equal to true that means you want to permanently Delete the duplicate now if I execute now see there is no duplicate uh data in my data set right now now let's perform some basic pre-processing so as I already told you inside basic pre-processing you can perform HTML tag removal then lower case operation stop remember so these are the thing you can perform now see uh from my previous notebook
that text prosing notebook I have taken the same function that removes tag function that means it will Remove all kinds of HTML tags from the data because if you show if I show you the data it is having lots of HTML tags because it's a it's a extracted data from the website right so let's apply this function I will apply on top of the reviews so it will uh remove all the let's say HTML tags now see this is the clean data I'm having now see there is no HTML tags present inside this data now
the next pre-processing technique will be performing the lower casing Operation so let's apart from it so here you can see I'm applying lower casing now if I show you see all of my word has become lower case right now fine now what we'll be doing we'll be removing the stop Parts because as I already told you some stop word doesn't have any kinds of meaning u in a reviews whenever you are performing any kind of sentiment analysis because it will see the positive and negative words okay to differentiate whether this sentiment is A positive or
negative so here you can see uh here I have written a Lambda function so it is doing the same thing you can also use the previous function I showed you in that notebook I you can also follow this particular approach okay so here I'm using Lambda function and Lambda is a on line function okay in the one line I'm taking all the word one by one and if any stop words is there okay I'm just trying to remove that stop W with the empt space that is the idea Now let me execute now see there
is uh no stop word present in this data right now fine now let me show you my head of the data now what I have to do I have to uh just uh separate out my X data and Y data that means my independent variable and dependent variable so independent variable means this is the review one independent variable means this is the sentiment that means this sentiment is my level right now so I'll store inside X and level I will store inside y now this is my X that means the entire review and this is
my y that means my sentiment now one issue you can see sentiment is nothing but it's a text okay it's a text you can see it's a string but what I have to do I have to convert to the numerical representation so here I can simply apply uh label encoder so it will assign positive with one and negative with zero so let me Apply so for this I'm using level encoder from escalan initializing the object and I'm fitting my Y data now see your data will become 0 by one okay that means it has converted
to the numerical representation then what I have to do I have to perform the 10 test speed because I have to keep my data for my training as well as the testing and the test size is I have given 0.2 that means 20% data I'm keeping for the testing and 80% data I'm giving for the Training so let me perform now this is the training site and this is the testing size fine now I'll be applying first of all bag of word okay I already taught you the bag of word I think you remember so
I can use the counter vectorizer for this now I have to apply the bag of for on top of my reviews okay reviews is my uh like input data right then I'm also performing on top of my test data because I'm having two kinds of data Training data as well as the testing data and I think you know whenever you are performing on top of training data you have to use feed transform and whenever you are performing on top of testing data you have to use transform okay Ma function for this then I'm converting to
the array representation now this is my training uh vector and if you want to also see the testing Vector this is your testing Vector okay now here we'll be using one machine learning model called KN bias inside KN bias we are having uh one NAB version called goian NB okay goian NAB we'll be using this particular algorithm so here you can see I'm fitting my data so X train and Y train now let me train my data so here I'm only showing you how you can pre-process your data how you can prepare your data for
the model training and all but going forward here We'll be using uh actually Transformer based architecture okay that means large language model we'll be using here now this is the training I have done now we'll be doing the prediction on top of my test data then we'll be calculating the accuracy score okay now let me execute see this is the ACC score I'm getting uh if I'm using goian NB and if I'm using bag of word okay bag of word technique so I'm getting 63% accuracy now you can see this is the confusion Metrix now
if I'm applying random Forest classifier on top of it so now let's see the accuracy I got so guys you can see after applying random forest classifier and uh I was using bag of word I I'm getting this accuracy 84% accuracy now see uh we'll be using bag of word only now instead of uh see taking all the features what I will do I'll take maximum features uh 3,000 okay now you can ask me what is maximum feature 3,000 I think you saw The entire Corpus right I I already told you the Corpus okay about
the Corpus corpa means the entire data and from the entire data I'm only considering 3,000 unique word okay 3,000 unique word 3,000 frequent uni word okay this is called maximum features because see I don't need all the features because if I'm using all the features my Dimension will increase right and I don't need to increase my Dimension okay if my Dimension is increasing that means my Model will get more complexity right to perform the prediction on top of it and here I'm using a machine learning model okay I think you are uh you can see
so that's why I'm taking most frequent 3,000 uh maximum features instead of taking all the features right now it will create uh 3,000 dimensional space okay 3,000 dimensional space instead of taking all the uh unique word right now see again I'm performing the bag of word operation now again I'm taking random First classifier and I'm training and checking the accuracy now if I execute uh so you can see this is the result but this result is not good enough so we are getting close related accuracy only if I'm using all the let's say um features
and if I'm taking 3,000 features the accuracy I'm getting it is almost same now let's try with NRS because I already told you about NRS right what is NRS exactly so here let me uh pass the range of 2 two because I want to consider a Pair okay pair of word that's why I have given two two that means it will consider two word okay as a one pair that's the idea and here I'm only taking uh 5,000 uh most reent word okay now again I'm performing back back of word operation then uh random for
classifier then I'm uh calculating the accuracy now let me show you so guys you can see uh this is the result actually I'm getting so now in the summary you can see uh your like bag of word is working fine Instead of taking the grams so in some case actually bag of word will work fine in some case engrams will work fine it's completely experimental things okay you have to perform now let's try with TF IDF okay um tfidf technique so here is the TF IDF code I think you remember so first of all we
have to initialize the TF IDF then I will apply on top of my data okay training data as well as the testing data so it will give me the vector and again I'll be using random First classifier to train the model and I will show you the accuracy score now see if I show you the accuracy see so here we are using machine learning model that's why performance is not good but if you're using any kinds of deep Landing model you will see that your model will perform good now here you can see this is
the performance of the now you can see this is the accuracy I got so here I showed you almost all the vectorization technique we have Learned but one thing I want you to explore this uh what to vake one so I already shared you the code of the what to vake so what you have to do just try to apply what V get the vector and try to train with random first model and just observe the accuracy okay so this should be your task guys from my side Because unless and until you are not doing
anything things would be um much difficult for you to learn right but if you're practicing by yourself I think This will help you a lot to learn any kinds of topic so yes this is how actually we can perform the text classification with the help of machine learning and that's how actually we can perform the data cleaning task like data preprocessing and text representation okay everything we can perform like that but again I'm telling you guys this is not ual way we create our application this is just a basic introduction I have given like how
we can uh like do the Data preparation okay this is my main objective now in the next video we'll be learning about large language model like what exactly this large language model and how this large language model got trained okay everything we'll be trying to discuss so as of now we have seen so many things like we saw the entire genbi pipeline we saw the text preprocessing as well as the data representation and uh I also did one practical uh I just implemented one text Classification uh with the help of machine learning model but I
already told you uh see it was just experiment uh but going forward actually we have to use something called large language model because here we are learning generative and inside generative VI the core component is the large language model now let's try to understand what exactly uh the large language model is and what is the code architecture behind this large language Model so guys as you can see a large language model uh is a trained deep learning model that understand and generate text in in a humanlike fashion okay LMS are good at understanding and generating
the human language as I already told you inside generative by we having large language model we also call it as a generative model and what is the use of generative model generative model can generate a new data okay so that is why here I have mentioned a large Language model is a trained deep learning models that understand and generate text like a human fashion that means whatever model you used previously let's say whatever traditional model you used let's say U machine larning model it can be any kinds of traditional deep larning model those are the
model can't generate the text like a human fashion okay it can only do the prediction because I told you the difference between descriptive model as well as the Generative model but here this large language model can generate a text okay and it would be human fashion that means the way human generate any kinds of text let's say currently I'm speaking right I'm giving you the understanding so the same concept can be applied in the large language model also so large language model can also mimic the human behavior okay this is the main idea here and
LMS are good at understanding and rting the human language as I already told you and How this llm works see the core working mechanism inside llm uh it will generate the next word so here I have given example let's say the garden was full of the beautiful now see this particular sentence you are giving to the large language model okay and large language model is able to generate flowers that means it is trying to generate okay next word always okay is trying to generate next word that is the main uh core idea inside large language
model that means Whatever let's say prompt we are giving okay whatever prompt we are giving whatever let's say uh text it is generating it will try to generate the next word always okay this is the main U logic behind the large language model so here are some more example you can see it's raining cats and so this is my let's say input to the large language model large language model will uh try to uh actually generate this particular word called dogs okay now there is Another one I have two apples and I at one I'm
left with so this is the input you are passing to the large language model so Palm is nothing but it's a large language model and it is able to predict one okay that means it is able to generate the next word of the prompt you have given okay this is the main working mechanism of a large language model apart from that there are some more advanced technique it usually used I will uh also explain so I'll will also Explain like how chat GPT was trained and how it is working and all okay I think this
concept would be more clear as of now this is the high level understanding I'm giving you how this model is working okay let's say whenever it is generating something new data how it is working this is the main idea here now let's try to understand why we call it as a large language model because you can see um because of the size and complexity of the neural network as well As the size of the data set that it was trained on that means if you see any kinds of large language model OKAY over the Internet
any kind of LM model you will see it is using very big and complex neural network okay very big and complex neural network right even it is trained with massive amount of data okay and the model actually we are using it is called actually pre-end model that means this model is already trained with massive amount of data okay it is called Pre-rain model and we call it as a transfer learning I think you have already learned transform learning in your uh previous days right in you know deep learning computer visual machine learning so what is
transform learning transform learning means we are using an existing model okay that model is already trended with some kinds of data set okay this is called actually transfer learning so here so this transfer learning makes this gen is so Powerful because they are training these kinds of complex neural network with the massive amount of data set so as you can see researcher started to make this model large and trained on huge data sets that they started showing impressive results like understanding the complex natural language and generating language more eloquently than ever okay this is the
main idea that's why we call it as a large language model now you can see the large language model Code architecture now many people uh has this kind of question what is the architecture actually llm is using okay see this is the core architecture all kinds of llm are using a large language models are based on the Transformer a type of neural network architecture invented by Google those who have already learned about natural language processing right those you have already learned about attention mechanism Transformer so you are already familiar With this architecture yes or no
right so you are already familiar with this architecture and this architecture is known as Transformer architecture okay and it is invented by Google okay and this was the main breakthrough in the field of uh natural language processing even in the field of generative because based on this particular architecture all kinds of large language model you can see over the market okay everything has been Developed fine so this is the main architecture here so it is having two kinds of part one is the encoder part and other is like decoder part and inside that you are
having something called multi-head attention okay no need to worry I will explain this particular concept uh I will uh create a dedicated video for this how uh this Transformer architecture is working inside a Transformer architecture what the component it is having okay everything I'll try to clarify okay for this I'll keep a dedicated uh video okay so that you can understand this Transformer in depth fine as of now just try to consider this is the core architecture behind all kinds of large language model okay and it is invented by Google and this is known as
Transformer architecture and it is having two layer called uh um your encoder layer and is like decoder layer so this one left left side this is Called encoder layer and right side this is called decoder layer okay inside that you can see different different component like input in coding multi head attention add normalization feed forward okay that's how you are having different different component inside that I hope it is cleared fine now on top of this architecture what they did actually they added some more layer they added some more functionality and they brought these kinds
of large language Model different different large language model now let's try to see some large language model but before that uh first of all let me tell you what makes large language model so powerful because I already told you in case of llm one model can be used for whole variety of task like you can perform Tech generation chatboard summarization translation code generation and so on with one model you can perform all kinds of task but whenever we used to use our Traditional NLP model that means only the language model we can only perform one
specific task let's say you want to do language translation for this you have to only use language translation model and that model can't do the code generation okay but here in the large language model you can do whole variety of task I think you saw the chat gbt let me show you one example so guys this is our chat GPT now inside that I can give any kinds of input let's say I'll give Uh give me the German translation of this text now here you can pass the text let's say how are you fine now
see if I give this prompt so it will give me the translation in German of this text okay now see this is the translation in German now you can also do the sentiment analysis so here I can write give me the let's say um Sentiment okay of this text see it's a neutral sentiment now you can also perform something called language detection so I'll give give me the language or I can write detect the language okay detect the language of this text now see it it should give me English fine so that's how with the
help of one model you can do variety of task you can also generate the code so I'll give give Me a python code for adding two numbers see it can also generate the codes so this is how actually large language model works okay and that makes llm so powerful now let's try to see some of the large language models are available over the internet see I have just listed down some of them but you will see thousands of models are available I will also tell you where you will get all the model list and all
okay so as of now just try to see guys uh we Are having different different large language model like jman so I think you know jini right ji was developed by Google even we are having also uh one application of the Jin let me show you so in Google if you search for J.G google.com so you'll see these kinds of application and this is the same kinds of application like your CH GPT okay here here also you can give the prompt and here also you can get the response let's say the previous prompt I have
Given here I can also pass it here okay and in the back end they're using something called jini model OKAY jini large language model then we are having something called GPT okay GPT stand for generative pre-end Transformer okay the model was developed by open AI that means the chart GPT you are using it is using something called GPT based model okay we are having GPT 3 4 okay these are the model we are having even uh recently actually GPT published one Amazing model I think GPT 40 model okay with that you can also generate videos
you can also generate like so many things okay if you just search on Google you will see different different uh version of the GPT even uh there uh application and all like okay everything you can see there then we are having something called xlm that means cross lingual language model okay so this is one of the large language model it is is Al also available now we're having Something called llama okay and it was developed by m so llama is having different different variant like llama 2 llama 3 okay recently Lama 3 was published 3.1
is published I think and uh this model is uh amazing uh it can also let's say uh generate text like a human even uh going forward we'll be also learning how we can use Lama 23 model OKAY in our application with the help of that we'll be building different different application now you can see There is another model called Megatron so Megatron is a large language model and it's like very powerful Transformer model OKAY developed by um Nvidia research team I think you know Nvidia so we are using Nvidia gpus in our system right so
Nvidia uh team has developed this particular Megatron model and this is one of the large language model now we having something called M2M 100 so this is called actually multilingual encoder and decoder sequence to sequence Model okay and it was developed by Facebook so these are the models actually we are having apart from that I told you now we are having thousands of model let me show you so this is the GitHub guys so the GitHub name is open llms okay now now what uh this guy did actually this guy actually listed all the large
language models are available over the Internet okay now see he has uh created one beautiful GitHub repository and in the rme file he listed down all The large language model as well as their uh checkpoints let's see want to download this model you can also download this model from this particular link if you want to read the papers and blog of this particular model he has also given the link apart from that the parameter size context length and as well as the license so every information you will get uh of any kinds of large language
model now see T5 is one of the large language model then RW kv4 okay Then GPT neox then yl2 Bloom chat glm okay now see different different large language model you are having see okay Falcon is also there then Lama 2 I already told you then Mistral is also there okay see different different large language model and it is having so many large language model guys so many large language model if you want to explore open this particular link just write open LM inside Google you will able to see these Kinds of GitHub okay and
you can also keep this link with you so that if you need any model later on um if you want to develop any kinds of projects with the help of any kinds of large language model you can use this repository later on so that you can see the uh paper blog link okay it will help you to understand this particular model so guys as I already told you uh all the large language model using one code architecture which is nothing but Transformer architecture and inside Transformer architecture I showed you there are two kinds of layer
called encoding layer and decoder layer okay now see uh here is the Transformer architecture let's say and it is having two kinds of part like encoder and decoder now I think I showed you different different large language model like thousands of large language model now see all the models are not using both layer that means encoder and Decoder layer some of the models are using encoder layer some of the models are using decoder layer okay and some of the models are using both layer that means encoder and decoder both now here I've listed down some
of the models so this will give you the idea what kinds of models are using encoder layer only what kinds of models are using decoder layer only and what kinds of models are using both layer okay you can see encoder uh based architecture these are The encoder based architecture like distill b b Robata okay ex xlm then Alber Electra okay then darta so these are the model based on the encoder layer only okay now you can see the decoder layer model so GPT gpt2 gpt3 okay whatever GPT series you are having it is using something
called decoder layer only okay decoder architecture only fine then both actually layer that means encoder and decoder both layer so we are having something called T5 B then we are Having something called uh see this is Bart actually b a r t not b r t okay this is another model now M2 m00 and Big B so these are the models are using encoder and decoder see it's not possible to list down all the models here that's why I just listed down some of them but there are so many models that using like both architecture
there are so many model that using only encoder architecture there are so many model that are using decoder Architecture okay and this is called actually Transformer T okay this is called actually Transformer T so with the so by referring this particular T you can easily understand um what kinds of models are using which kinds of architecture okay this is the main idea here now let's try to see some open based large language model uh because going forward we'll be using something called open platform okay we'll be generating open key and we'll be Implementing different different
projects now see inside open we having different different models like GPT 4 then GPT 3.5 GPT based Deli whisper embeddings model even moderation model gp3 Legacy model so these are the models are available in the open AI platform apart from that I think recently they have released more model because it's a uh continuous research fi it's not like that uh it will be like uh fixed research every day they bringing new new Technology in their platform okay so if you want to learn all of them you have to keep updated with their technology always for
this you can you can refer their website okay you can refer their website and try to see what are the models they're bringing okay so like that actually you'll be learning a lot it's not like that you only need to rely on your Mentor okay as a mentor I can give you the entire path how we can get started with I will teach you how you Can use them but as a learner what you have to do you have to explore by yourself unless and until you are not exploring uh by yourself it would be
very much difficult for you okay um to learn any kinds of topic so that should be my suggestion please try to explore from your end uh after this uh actually video what you can do you can search open AI you can search different different open source website and you can see what are the models are came Actually recently okay that's how actually you can learn and you can also create a GitHub repository like that and you can list down all the models okay you will be exploring okay in future that is the idea now let's
try to see some other open source large language model see there are two kinds of large language models are available one is like commercial model commercial model means this is the commercial model like open AI based model so if you want to Use these are the model you have to pay okay you have to pay uh you have to take the subscription okay but but if you're not interested in paying what you can do you can use open source our language model so these are the models are available in the hugging phase platform so if
you're using these are the model you don't need to pay okay freely you can access but there you will get some difficulty because these are the B you have to download manually in your system You have to set up everything okay let's say you want to F tune this mistal what you have to do you have to download the M model in your system now just try to see what is the size of the m model OKAY more than 30 to 40 GB okay so you should have good uh actually resources in your system let's
say you should have good CPU good GPU good memory otherwise you can ever train this particular model but if you're using commercial model okay let's say you are using openi model okay That time what you can do you can use their platform like open a platform to find you these are the model okay you don't need to download these are the model in your system so going forward I will teach you how you can use commercial model as well as the open source model both then you can decide which one you will be using okay
it's up to you now these are some open source large language model like Mr Lama gini Falcon okay cloudy then MPT 30b and St STM okay these are the models actually we are having as an open source model apart from that there are so many model I will tell you there is a platform called hugging face inside hugging face you will see all kinds of large language models are available there fine uh what can llm be used for I already told you you can use large language model for the text classification uh for the text
generation for text summarization conversation like chart board question Answering speech recognization speeech identification spelling correction and so on that means whatever task you can see in the field of natural language processing okay all kinds of task you can perform with the help of large language model only one model I think I showed you one example of the Char GPT now there is another uh concept uh you will be getting inside large language model which is nothing but prompt engineering and prompt designing now you Can ask me what is prompt engineering and prompt designing so you
can see all the Tes that we feed uh into an llm as an input it is called actually prompt and this whole art is known as prompt designing or prompt engineering which is about figuring out how to write and formate prompt takes to get llms to do what you want to do okay that means whatever input you will be passing inside large language model okay this is called actually prompt and the way Actually will be giving the prompt the way actually will be preparing the prompt this is called prompt designing or prompt uh engineering okay
now let's say here two example I have kept it's raining cats and okay let's say this is your prom you are giving this particular input to the large language model and your large language model will try to predict dog I think I showed you okay that text uh uh word generation okay next word generation then I have another Example I have two apples and I at one I left with this is the input now your model will try to generate one okay now apart from that we are having some types of prompt like few short
prompt zero short prompt okay these are the types types of prompt prompts are available let me show you so this is called actually zero shot learning zero shot prompting that means here you are only giving the you you can see the approach uh using a single command to get an LM To take on a behavior is it uh it is called actually zero short learning that means you are only giving a input only giving an input and your large language model will give you the output let's say this is your prompt write me a poem
about add a less in the style of Shakespeare okay this is the input now there is another one explain the quantum physics to me like I am 5 years old now you are giving this prom to the large language model and large language model Is generating the uh response for you the answer for for you okay this is called actually zero short learning okay zero short learning now there is another learning called few short learning in few short learning in addition uh to just providing an instruction it can be helpful to show the model what
you want to uh what you want by adding the example it is called actually F short learning so in F short learning what you will do you will give some example okay The Prompt actually you are giving let's say let's say here you are giving a prompt convert the text from English to France so with this prompt you will pass some example like English to FRS example let me show you so let's say this is the few short from example let's say this is the prompt you are giving convert the text from English to frames
now you are giving English as well as the frames English as well as the frames okay that means you are passing some example okay Instruction as well as the example and your model will try to uh give you the response okay after that that means you're teaching your model how you're expecting the results okay this is how actually you can pass the few short learning or few short prompting inside large language model okay so going forward we'll be also learning this this thing how we can add our system prompt how we can add the user
prompt okay how we can give few shot actually prompting How we can give let zero shot prompting we'll be learning whenever we'll be using large language model and always try to remember if you want to get a good response from the large language model you have to go with the prompt engineering and prompt designing this is the main idea here so yes guys this is all about our uh large language model okay which is uh llms so I hope you got the clearcut understanding about large language model what exactly large Language model is and what
is the core architecture it is using uh in the back so in the next video I'll try to um explain that uh core architecture the Transformer architecture how Transformer architecture works and and what are the components are available inside Transformer architecture so here we'll be discussing one amazing research paper called attention is all you need so This research paper published by Google and Google actually brought this particular Research called Transformer so here is the research paper guys attention is all you need if you are interested uh read this particular paper what you can do simply
search by this name you will get this research paper you can see this is from Google brain okay and uh you can see this is the architecture Transformer architecture but I know that first time uh it would be a little bit difficult for you uh understanding this uh architecture from this particular paper So what I will do I'll just try to break down this uh actually architecture and I will discuss each and everything all the components like what is input encoding like what is multi-ad attention what is ad normalization okay everything I'll try to discuss
okay then this part would be more clear then once we got the understanding then if you come to this paper this would be very much easy for you to understand this architecture okay like how they have uh let's say created This architecture and if you're interested learning the mathematics behind it you can go through this paper okay if you want to Deep dive into this you can go through the paper it will give you the comprehensive idea now to explain this uh actually Transformer let's say concept I'm going to refer one amazing blog by J
alar okay so this is one of the uh amazing guy uh and he has created so many content related actually NLP so this is one of of the best uh Resources you will find over the internet if you want to understand the Transformer architecture so this guy actually used so many visualization so many image to explain this architecture okay so we'll be referring this blog to understand the entire architecture I could have written everything on my Blackboard but I thought if I show you the visualization it would be more uh let's say good than the
writing on my Blackboard right so if you want to Understand this Transformer architecture so what I feel like this is this is going to be one best resources for you because it is having so many visualization okay we can easily understand what is happening inside Transformer architecture so now guys let's try to understand this architecture now let's say this is your Transformer architecture as of now just try to consider this is a black box okay it's a transform architecture and you Don't know uh what are the component it is having so here you are giving
one input data okay and here you are getting one output data that means you are performing language translation operation so here you are passing this FR input and you are getting English output okay this is the idea now let's let's try to see what inside this particular Transformer okay Transformer block now see inside Transformer block you are having two kinds of layer one is The encoder layer and and another is the decoder layer as I already told you uh Transformer architecture is having two kinds of layer one is encoded layer another one is decoder layer
now you are passing the input okay you are passing the input to the encoder layer see this PR uh input you are passing to the encoder layer and the output you are getting that English translation you are you'll be getting from the decoder layer okay this is the main idea now let's try To break this encoder and let's see what are the component we are having inside encoder layer now see if I show you this entire architecture okay in a low level now you can see inside encoder we are having some more encoded layer okay
we are having some more encoded layer how many encoded layer you can see 1 2 3 4 5 six that means six encoder layer we are having and how many decoder layer we having again we are having six decoder layer that means the architecture you Can see here okay the box you can see here encoder box and decoder box inside that you are using six okay six stack of encoder and six stack of decoder layer okay and inside that your input is going and it is processing this input and you are getting the output okay
this is the so that's how this architecture consist of okay I hope it is clear now let's try to see what we are having inside the encoder layer okay now if I show you the encoder layer see inside encoder layer We are having two kinds of layer one is like self attention other is like fit foral neural network and what is fit for neural network I think you know that means the artificial neural network okay if you're already familiar with an and you know what is f foral neural network okay it's a artificial neural network
at the end now let's try to understand the self attention because I know that many people will have the confusion what is self attention here okay now let's see Uh now you can see this is the encoder inside encoder we are having self attention as well as the feed forward and inside decoder also we are having self attention and another actually additional layer we having which is called encoder and decoder attention and the last layer you are having which is nothing but feed forward Nal Network so this part we'll be discussing later on as of
now let's try to uh discuss this encoded layer and this self attention Part okay now let's say you are passing the friends input you can see this is your friend input I don't know how to spell them but just try to consider this is your friend input so the first thing what you have to do the first thing you have to perform the text pre-processing then text representation yes or no so here first of all you have to convert your textual data to the vector representation okay so based on the paper they're using something called
What to V technique to perform the text representation technique that means they're converting their text to vectors with the help of what to V technique and what is the dimension okay what is the dimension of the vector they're getting 512 Dimension okay 512 dimension for the each of the vector okay this is the idea now let's say they have converted to the vector representation now what they are doing they're passing this particular input to the self attention layer you Can see they're passing the input to the self attention layer self attention layer is uh giving
one output which is nothing but Z1 Z2 and Z3 and this Z1 Z2 and Z3 will pass to the FED for all neural network okay now let's try to understand how it is encoding okay how it is encoding this particular input now see to simplify this particular understanding what J is doing he's using two words actually which is nothing but thinking and machines okay let's say you Having two words thinking and machine first of all what you have to do you have to convert this text to Vector representation with the help of word to V
and what dimensional Vector you will be getting 52 I think I showed you here 512 dimensional Vector it will return and this Vector will be passing to the self attention layer self attention layer will give you Z1 and Z2 then we'll be passing to the feed foral neural network again feed foral neural network Will give you some of the output and that output again you will pass to the encoded two layer that means I showed you now we are having stack of the encoded layer here see stack that means after uh getting one output from
one encoder layer it will come it will go to the another encoder layer then again it will pass to the another then another then another that's how you have to that's how actually it will pass through six encoded layer okay so that is what Actually he's explaining here you can see so here you can see after one encoding it is passing to the second encoding layer okay this is the idea now let's try to understand what we are having inside self attention now see self attention at high level let's say here we are having a
sentence the animal didn't didn't cross the street because it was true tried now here just try to understand the animal didn't cross the street because it was true tied okay it Was true tied now who was true tied here okay definitely the animal was true tied yes or no so as a human actually we can understand it means this is the animal it is indicating the animal but what about the machine how machine will identify whether this animal is the it or not okay so for this it used something called self attention Okay self attention
mechanism now let's try to understand how it will uh use that particular concept but before Understanding self attention just try to see how it will give the self attention to the word let's say here you are having the input so this is the input the animal didn't cross the street because it was true Tri okay now whenever I'm referring this it now here what it represents definitely it represents the animal okay you can see this darker color that means it is more focusing on the the animal part okay the animal part because the animal is
Nothing but it okay that means your machine will try to understand okay if I want to consider it I have to give the vage to the the animal didn't okay these are the actually word okay even two also okay two Triad also these are the word it has to give the attention okay unless and until it is not giving the attention to this word how it will identify whether this eight is represents the animal or not okay now you can ask me how it is identifying whether it has to Give the more weightage to this
words or not okay so for this we'll be understanding the self attention in detail so here you can see he has given one example let's say this is your input input means your two words thinking and machine so the first thing what you have to do you have to convert them to the embedding representation that means the vector representation then what will happen your Transformer will generate actually three weights okay you can see You can see it will generate three words W okay WQ w k and WV and how weight initialization happens I think you
know okay there are different different techniques for the weight initialization and these weights would be updated during back prop propagation okay BP I think you already know about artificial neural network okay how it got trained usually right how it do the back propagation and all this like very basic concept I'm expecting you are familiar With already right so these are the weight would be initialized at the very first and this weight should be updated during back propagation technique okay now you can see it is generating some more actually Vector the first Vector it is generating
queries keys and values okay now you can ask me how it is generating these are the values see see first of all what it will do it will take this particular X1 okay it will take this particular X1 and it will Multiply with the WQ okay it will multiply with the WQ so it will return you q1 okay then it will take the X2 it will multiply with the WQ it will return Q2 okay so that's why actually you will be getting something called query Vector okay query Vector then again it will multiply X1 with
w k okay and it will return you K1 then it you will multiply uh X2 with WK it will return return new K2 so here you will be getting Keys Vector okay then you will be multiplying This X1 with WV and it will return you V1 then X2 with w v it will return you V2 okay so that's how you'll be getting values Vector okay so that's how this three Vector would be generated queries keys and values okay I hope it is clear now okay now let me clear all the uh line here yeah now
let's see the next actually uh mechanism see I think you got it uh that means we are having our input uh data uh that means input word thinking and machine and first of all we Have to generate the EMB Bings then after that with the help of X1 and X2 that means this Vector we are generating queries keys and values okay and how I think you got it the multiplication actually strategy now the thing is like it will generate one score okay how it will generate the score see to generate this code it will use
this q1 okay q1 vector and it will multiply with the K1 okay that means q1 will multiply with the K1 okay this Vector that means Keys Vector where will multiply with the keys okay and you will get one score let's say as of now this is the Assumption score he has given here 112 F then for the next word you can see it will uh multiply q1 uh with K2 okay q1 with K2 and here the score let's say 96 okay now from this score you can see it is giving more score to the thinking
word okay to the thinking word than the machine word okay that means it will more refer this thinking word it will more give the Attention to the thinking word okay and that's how actually it is calculating this particular score I think you saw this graph now so how it is giving the attention to the word okay that's how actually it is calculating okay with the help of this particular Vector calculation it is creating this score and based on the score value it is giving the uh attention to that specific word let's say we are having
two words thinking and machine so it will give the More attention to the thinking word now see after getting this Cod what it will perform it will perform this uh s Max operation now see what what will happen you are getting this score now this score will be uh divided by uh8 okay that means you will divide this score by 8 now if you divide this number you will get 14 and if you divide this number okay you will get 12 now you'll just calculate the soft Max okay I think you know the soft Max
okay soft Max is an Activation function it's an equation the probability score and if you just add this probability score you will see that it would be 100% probability okay always so here you are getting actually 88% and here you are getting only 12% that means it is giving more attention to the thinking word that means 88% attention it is giving to the thinking word and only 12% attention it is giving to the machine word now I think it is clear how it is giving the attention to a specific Word now in the sixth step
what will happen see uh I think you saw we are getting the sof Max output now with the sof Max output what it will do it will try to multiply this value that means the value I think you are getting remember this particular value so it would be multiplied and after multiplying you will get one vector called V1 and again it will perform for the next word as well so here you will get V2 now let's see you having multiple Word so for all the words actually it will do the multiplication with the value so
here he only used two words but you can consider more than two word as well okay the working mechanism will remain same then after getting this V1 and V2 what it will apply it will apply the sumission operation that means it will sum up this two vector and it will get Zed one and Z2 okay and now I think you remember what is this zed1 and zed2 so let me show you That high level architecture um here see this is the Z1 and Z2 that means this Z1 and Z2 is the output from the self
attention Okay self attention layer and how self attention is working as of now whatever discussion I have given you whatever let's say operation I showed you this is the work of self attention and that's why actually it will perform all the mathematical operation and it will return Z1 and Z2 and this Z1 and Z2 will pass to the next okay next layer Okay next layer next layer means I think uh this layer yeah to the fit foral neural network okay this is the idea now whatever I have discussed so far if you want to see
as a matrix representation see the same thing that means you are getting your input okay you are getting your input that means here we are giving our input it will first of all convert it to the vector representation then it is generating some weights okay WQ WK and WV I think remember then we are getting query keys and values okay then it was multiplying this query with keys okay and we are uh dividing with eight okay and we are calculating the Sol soft Max okay and soft Max and after that whatever softmax output we are
getting we're performing the addition operation with the values okay and it will give me Z okay Z that means the output now so far whatever actually operation you saw this is for actually single head Attention Okay single head attention that means only one self attention Okay only one self attention mechanism now inside Transformer architecture they're using something called multi-head attention you can see so here is the TR called multi-head attention so what is multi-head attention see uh if I show you this uh graph again see if I'm considering it that means it is giving attention
to the the animal did okay these are the word even you can see It is also giving some attention to this word as well but see this thickness is like very uh I mean less here because it is not able to give the proper attention to this word okay because it is using only single head attention okay and and it is only using this three weight okay this three weight is not enough let's say you are having huge amount of data your input length is very big that time I only this three Vector is not
only this three weight is not actually enough So for this actually what we have to do we have to apply something something called multi multiple actually weights okay and if I want to apply multiple weights so what we have to do we have to use something called multi-head attention so uh Google Inu actually multi-ad attention so they're telling instead of only using one head okay one head attention so they're telling instead of using only single head attention we'll be using multi-head Attention to perform the same operation that means what will happen see this is the
attention head one that mean single attention and you'll be taking another attention head okay so this called attention head one that's how you'll be taking another attention and how many attention head it will be taking I'll I'll tell you as of now let's try to consider we are having multi-ad that means only two head okay zero and one now the same thing they are doing that Means they're generating this weights okay also this side also they are generating the weights and the same operation would be happening as here as well that means the operation I
showed you now previously the same operation will be happening here as well now you can ask me what is the main objective to perform this multi-ad attention because if you see in the Transformer architecture we can give very big size input let's say my input prompt would be Very big I think you saw in the chat GPT we can pass a very bigger prompt okay very bigger prompt to the uh this one our uh chat GPT input right that means to my GPT model so how it is processing that that particular bigger input because it
is using this multi-head attention concept that means with the help of multi-ad attention concept it it is able to give the attention to all the specific word actually present in that particular Pro okay so this is the main Idea that means if you're only using single head it won't be able to handle that particular bigger promt that's why you have to use multi-head attention here okay this is the main concept now see that's how actually things are working let's say you are passing two words thinking and machine so you you are passing as a vector
representation so this Vector will go to the different different attention and now here you can see they're using eight different Attention heads eight different attention heads that means whatever things we have learned so far we have learned for only one one uh self head attention right B self attention Okay but they're using something called eight different self attention eight different self attention here you can see okay attention head 0 1 dot dot dot 7 because it is starting from zero that's why it is ending till 10 now if you count it they're using something called
eight Different attention head here okay so inside eight different uh attention all the calculation is happening after that whatever z z 0 Z1 Z3 they are getting they're doing the sumission operation okay and this is your final Vector okay this is your final Vector that means the Z that means this one let me show you this one okay now inside that they're using eight different s attention and this is the Z One output you are getting okay I hope you got it Now see how beautifully uh Jammer created this blog how beautifully he has used
all the visualization okay so to explain this particular concept so that's why I feel like this is the best resources for the Transformer okay if you want to understand you can use this particular blog now I think we have understood uh till here that means we are getting the Z now see if you see the entire architecture that's how it is working so you are Having the input you are uh it is initializing the weights okay so it is generating the weights for eight different actually self attention Okay then uh you can see the query
Keys values then the Z okay that means the output from my self head attention then it is concatenating all the vector and we are getting only one vector which is nothing but Z now see after using multi detention that is this is how your output looks like now see it is able to Give the attention to all of the word actually it needs okay to represent this particular it right now now see this uh line is more darker right now okay this line is more darker that means it is performing better than previous that means
previous we are using only one head attention now we are using multi-ad attention and the performance is very good right now okay this is the main idea now see uh he has captured some of the images okay experimental images you Can see here now let's try to understand uh another important thing uh I told you now so let's say whenever we are using any let's say traditional embedding model let's say TF IDF what to V or let's say any other embedding model let's say this one bag of word so it won't be able to capture
the semantic meaning okay semantic meaning of a sentence let's say here we are having a sentence this is the let's say France sentence z swiss uh uh a to dend I don't Know how to spell but let's say this is my sentence now after Z this particular word is coming after this word this word is is coming okay how it will remember that means how it will understand the contextual meaning that means the sequence meaning okay sequence meaning of a sentence so to understand that sequence uh actually meaning they're using something called positional encoding okay
positional encoding now let's try to understand what is Positional encoding see let's say this is my input it is converting to the vector now you can see this particular Vector there adding with the positional encoding positional encoding is nothing but it's a vector only okay it's a vector only you can see T1 T2 and T3 and there will and they're doing the addition operation okay after that they getting the embedding with the time signal that means whenever we write any kinds of let's say sentence let's say my Name okay is buy so it always follows
one time sequence let's say my I have written at the time of T1 name I have written at the time of T2 is I have written at the time of T3 and byy T4 okay that's how it follows on sequence okay so that's why Text data can be also called as sequential data okay it's sequential data because it follows the time okay so that's why they are telling embedding with the time signal that means whatever Input you have given okay whatever input you have given it is having one time let's say T1 T2 and 33
and so on right so on so that they're adding this positional encoding now how this portional encoding helping us to uh identify whether after Z this word is coming after Swiss this word is coming because if you see positional encoding nothing but it's a vector okay and after adding okay after add in this Vector with my embedding so again we are Getting another Vector yes or no okay now if you calculate the distance okay between this three Vector you will see that this vector and this Vector would be closely uh together because if you just
do the subtraction okay if you just find the distance you'll see that it will get very less distance between these two Vector okay and you will you will see that this vector and this V Vector distance will be more okay because uh you can see the number okay So that's actually it is identifying after Z this particular word is coming after this word this word is coming okay based on this particular Vector okay that's how they're doing distance calculation and they're trying to understand the sequence information of a sentence okay let's say after my bu
is coming sorry after my name is coming after name is is coming after is BU is coming okay I hope it is clear so that's how they're handling this particular SE Uh sequence okay now now I think you got it whenever we are passing any against of bigger prompt to the large language model how it is identifying that particular contextual meaning how it is remembering that particular sequence let's say after this sequence this sequence is coming now he has added some more graph so as of now let's skip okay it's not required okay now the
next thing we'll be discussing about the residual block those who have learned About like computer vision so I think you learn one model called resonet okay so in resonet also we had these kinds of architecture okay I think you remember so I think you remember so let's say here we are having different different layer okay different different layer fine and we also had something called residual block so that's how actually it will look like that means whatever input actually you give usually so sometimes Let's say if this block is not working good okay let's say
this block is not working good not good then what will happen it will escape this block and this output will go to the next block okay this is called actually skip connection okay skip connection okay and this is called actually residual block so we already learned this residual Block in our CNN architecture so the the same residual concept has also applied in this Particular Transformer architecture as well so here we are also having residual connection you can see let's say this is the input we are giving okay let's see if this particular self attention layer
is not working good what will happen this input will go to the next okay next layer now they are also using something called add and normalization and what is normalization I think you know with the help of normalization we just uh bring our data distribution in a normal okay Normal distribution I think you remember right this is whatever weight actually we're having so if you if you just visualize them you'll see it will be it will be a normal distributed okay normal distributed output so it helps us during back propagation whenever it will try to
find the global Minima okay that time actually this concept helps a lot now you can see again we are having another residual block connection skip connection okay that means uh if any of The layer is not working good this uh this will actually skip this part and it will go to the next okay next layer so this is the idea of the residual connection because it's a very huge layer because you can see this uh Transformer architecture it's a very huge layer so sometimes let's if you are using any simple data very simple data so
you don't need all the layer now you don't need all the layer to process your let's say input so sometimes uh whenever You are having the simple data it will go to some of the layer and it will perform better okay that time actually it will skip these are the let's say layer okay so this called actually skip connection and rual block and if you want to see the normalization how it is performing the normalization so here you can see the normalization so it will do the addition operation um X and Zed okay Zed vector
and it will perform the normalization like that okay then Whatever output it is having it will pass to the fit forward and F Feit forward again will pass to the normalization and again it will pass to the next okay layer this is the idea now here you can see the entire architecture guys so far whatever we have discussed now he has given inside one block only now we are giving our input we are adding the positional inod then self attention then we are also adding something called add normalization then It will pass to the fit
forward then add normalization again then it will pass the next encoder okay and inside that we are also having some resist connection okay then whatever output we are getting okay let's say whatever let's say encoder we are using like I think I showed you six encoder layer we're using so this output will go to the decoder okay now see directly is going to the decoder layer now at the same time this output is also going to the encoder and Decoder attention now you can ask me what is encoder and decoder attention see this is the
same whatever things we have learned encoder and decoder the same architecture they're using in this particular layer okay that means whatever output you are get here again passing to this particular encoder and decoder again it will process those are the input and it will pass to the next layer okay so that's how actually your output is passing to the encoder and Decoder as well as the decoder and how many decoder we had we had six decoder layer now let's try to see the decoder side now see here we are passing the input so this is
the input we are passing to the encoder encoder will pass this uh output to the decoder but before passing this output it will add some Vector you can see K vector and V Vector okay it will first of all multi mly then it will pass to the decoder okay and decoder will pass to the linear and soft Max that means it will apply the soft Max function on top of it now one thing will happen let's see if you're doing actually language translation so the input you are giving to the encoder so this output actually
you're are passing to the decoder and decoder will predict something see decoder will predict something it will generate something let's say I so you are giving friends input and you're getting English output let's say Z means I Swiss mean M and it Means a student okay so that means it is first of all see if you see the first iteration let me show you it's animation if I show you the first iteration see this is the first iteration first of all it is predicting I so this I again it is coming as an input to
the decoder side you can see I then it is predicting M again this m will go to the input of the decoder side okay then it will predict something called a that means based on this two Word it is predicting the next word okay based on this word it is predicting the next next word okay I already showed you now that llm concept like it will predict the next word always okay that's how it is predicting the next next word okay unless and until it is having the EOS sign that means end of the sentence
okay EOS okay this is the tag I think you know okay what is eos and there is another one called SOS okay start of the sentence and end of the sentence Whenever it is getting the tag that means this sentence is complete and it will stop the execution that time okay so that's how your encoder and decoder is working okay and inside encoder whatever things were having I already expand inside decoder also whatever things you have I already expand okay that's how that's how it is uh actually building one amazing architecture called Transformer okay Transformer
and and that is why Transformer is so powerful because it is having so many component inside that it is using something called multi-head attention Okay that is the main concept here so this is the entire understanding guys and I hope guys you got the high level understanding of this Transformer like how it is working and what is the internal mechanism of the Transformer now if I open the actual architecture in the paper now I think you will get it See this is the encoder site okay this is the encoder site and now I think you
know what is input embedding and what is the positional encoding what is multi attention what is AD normalization and what is these are the arrow that means this is the skip connection residual block then feed forwall again uh add normalization again you are passing to the MTI attention and you are also passing this in uh output to the decoder side you can see this is the decoder Side and inside decoder we are having multi-ad attention again add normalization M attention feed forward okay s Max and so on okay and whatever output you are getting you
are again passing this output to the input of the uh decoder input okay this is the idea now I think this architecture is clear how they have developed okay now you simply just try to go through this paper once just try to read or you can also read this particular blog okay so it Will give you the clearcut idea about the Transformer architecture okay so this should be my suggestion guys please go through uh one time this particular blog just try to read everything okay see the way actually has written if anyone is from college
background they will also able to understand okay that much easy it is so yes this is all about our Transformer interation I hope you got it how Transformer is working so uh based on this Transformer today's Actually whatever large language model we are having they have developed now in the next video we'll be learning about like uh how chat GPT got trained because you can see chat GPT so chat GPT is using one large language model called uh GPT okay so we'll be understanding how chat GPT actually the train like what is the internal mechanism
of the chat GPT okay we'll be discussing this part so I think you are already familiar with chat GPT right so chat GPT is a product of Open AI so inside chat GPT we can pass any kinds of prompt and we can get the response so it is kinds of conversational let's say chatboard they have developed but uh by that particular chatboard what you can do you can perform all kinds of NLP task I think I showed you one example I think you remember we can perform language translation text summarization we can part from sentiment
analysis okay we can also let's say generate the codes every Everything is possible because I already told you now llm why it is powerful because it can perform multitask right multiple task it can perform with the help of one model only task wise it is clear like how uh it will able to perform the task right but what about the conversational agents let's say whenever I'm passing any kinds of prompt how it is identifying okay how it is identifying what kinds of let's say prompt I'm giving and what kinds of Tasks actually I'm expecting from
my uh chat GPT okay so this is what actually will be understanding uh in this particular video because because if this understanding is clear so going forward whatever model you'll be learning whatever application you'll be building this would be more clear in your mind so first of all you have to remember one thing see chat GPT is not a model okay so chat GPT is just a application internally it is using a llm which is GPT 3.5 or GPT 4 it has trained on large amount of data which is also available over the internet that
means that means over the Internet till 2023 whatever actually data we are having so all the data they have used to train this gpt3 or GPT 4 model right now to train this chat GPT they actually follow three step the first step is generative preing the second step is supervised fine tuning and the third step is reinforcement learning I'll discuss all of them one by One no need to worry but first of all just try to see as I already told you Char gbt is not a model it's just a application it's just a interface
right so whenever you are giving any kinds of prompt to the CH GPT it is using one open API key right I think you know what is openi API key with the help of openi API key it is sending that particular uh request to the model you can see it is it can access different different model of the GPT like GPT 3.5 GPT 4 okay or Any other model actually GPT is bringing in future right so this is the idea let's say you want to powerform language translation you are giving the prompt here so this
chat GPT interface receiving your prompt then it is sending that particular prom to the uh model with the help of open API key that is the idea okay so that means CH GPT is an application and it is using uh internally it is using some kinds of llm which is known GPT 4 or GPT 3.5 now as I Already told you um they used actually three step to train this model now you can see the first stage is generative pre-training so here you can see this base model GPT trained on a brnch of stuff from
internet for a whole bunch of different things by uh using the Transformer architecture that means they're using Transformer architecture internally and they're training on top of the internet data okay or Internet documents data you can Talk about then they are getting something called Bas GPT model OKAY B GPT model then this B GPT model then again they doing something called supervised fine tuning okay in the St two they're doing supervised fine tuning so what is supervised fine tuning I'll discuss uh everything don't need to worry so next with the human AI trainers you get to
have conversation where they play Both Sides you and AI assistant okay now after performing the Supervised fine tuning they get one model called fine tune chat GPT model now they'll be performing something called reinforcement Landing through human feedback so here you can see next uh let's take the model to the next level by optimizing it's even more reinforcement learning by training it against a reward model okay that means they will be creating something called reward model and why they'll be creating the reward model I will tell you then After that they will be getting the
CH GPT model and this is what actually we are using okay this is what we are using in our day-to-day life this is the idea now let's discuss all the St one by one now first of all let's try to discuss generative pretaining okay now see what is generative pretaining as I already told you they're using a bunch of data from the internet and they're training one Transformer model okay so they're using the code Transformer architecture Only on top of that they're adding some more layer okay and they're creating one GPT model okay this is
called a GPT model now this model will be able to uh do actually various kinds of task like text summarization sentiment analysis sentiment sentence completion translation and so on but what is our expectation from our chat GPD so here user can pass any kinds of prompt he or she can chat with uh chat and conversation with this uh let's say Application yes or no right but after training this particular let's say Transformer model we are only getting the uh we are only getting the actually task output let's say this model only can perform Tex summarization
s analysis sentence completion and language translation apart from that actually it can't do anything let's say it can't do the chat conversation with the human Okay this model can't do so this is our expectation okay with that this is our Expectation so for this actually what they did actually they follow another stage called supervised fine tuning in supervised fine tuning what they have done so they have actually taken two human so one human called human agent acting like a bot another human the actual human agent okay that means both are human but one human actually
will act like a bot and another human will act like a human only right so this particular human will give some request Okay let's say he will put some let's say text let's say how are you so this human will give the reply let's say I'm fine what about you again he will give I'm also fine then he will tell I need help from your side this particular let's say human will uh tell so this particular human will tell how I can help you okay so that's actually they will be doing the conversation the way
actually we do the conversation with our chat GPT okay so initially we have to Create our data that means chat data okay chat data so to collect the chat data they're using this particular technique called supervised finetuning okay supervised fining they're supervising the model okay this is the main idea now see after uh doing these kinds of conversation you will have different different data let's say uh from The Human Side you will get lots of request and from the B size that means human Agent B size you will be getting Lots of response now you'll
be using uh this particular data to train again your base GPT model okay you'll be again fine tuning your base GPT model and you'll be using something called stochastic gradient decent Optimizer okay AGD Optimizer they're using and they're training one model called sft Char GPT model that means supervised fine tuning model okay this is the main idea here and this is called actually sft and supervised fine tuning but one issue Actually they observed after uh doing the supervised fine tuning this model can only perform uh related uh to the training data let's say let's say
if someone is giving any kinds of input which was not present in the training example that time this model was giving some random output okay so to actually overcome this issue and to make this particular model more good they followed something called reinforcement learning through human Feedback technique so inside uh reinforcement learning uh through human feedback what they are doing whatever let's a conversation uh having between this human agent and sft chat GPT model so they're trying to rank the results okay rank the response you can see let's say this sft model will give actually
multiple response now there would be a human okay there would be a human agent so this human agent will rank the result let's say here we are getting uh 1 2 3 Four four different response let's say a b c and d now between these four results which one is the good okay which one is the good and which one is the let's say suitable with the user input user has given now he will just try to rank the results now let's say uh B is B response is good then a then D then C
okay this is the rank response that's how actually he will try to prepare then what they have done they created one reward model and what is reward model reward model is Nothing but it's a reinforcement learning model those so those who are familiar with reinforcement learning I think you know what is reward model so here you can see they're giving actually different different response and they're uh actually generating one score okay like what is the score for this response what is the score for this response okay that's how they're generating the score now whoever actually
having like more score this particular response would be Considered as my final output okay that's how actually they're doing reinforcement planning through human feedback and they're improving the accuracy of the model this is the main idea that means if you see uh in high level so you are using reward model and you are ranking uh this particular let's say response based on that you are and based on that actually you are providing the actual response and this reward model will uh give some policy let's say If this response is good if user is liking that
particular response so definitely it will give some reward otherwise it won't be giving any kinds of reward so sometimes in the chat GPT will see that it will ask for your feedback whether uh you are getting the correct output or not if you give the let a low rating that time they will consider this model is not working good that time they will give negative reward so again they uh This reinforcement learning will automatically learn okay how to improve the quality of the model and if you're giving positive fitmap that means it will give the
positive reward that time okay so that is how actually reinforcement learning works so here in the environment you just put on agents that particular agents learn the environment and whatever activity you are doing it will try to learn and based on that it will uh try to update the Reward model this is the main idea right so yes this was actually core uh concept to train this chat GPT model and that's how actually not only we get that task specific model we can also perform the conversation with that particular model okay because they have done
because I I think I showed you now how they have done this particular entire strategy and how we can perform the conversation so whenever we are giving any kind of prompt first of all it will trying to Understand my prompt because of that particular supervised fine tuning let's say I'm giving a prompt PRP I want to do the language translation French to English so it will try to understand that particular prompt then based on that it will decide what kinds of task you want to perform okay apart from that if you're doing some casual conversation
also it can also handle because again it was trained with huge amount of data okay that's the idea so yes guys I hope This makes sense now and you got how Char gbt was trained and now this concept will help you to understand whatever let's say model will be understanding in future okay so I think you enjoyed so if you don't know about hugging face hugging face is a platform so with the help of this platform you can do genbi project you can do NLP project even you can also perform computer vision related project so
here we are more focusing on the generative AI so we'll be using this hugging face platform for the gener vi related task so this hugging face platform is having all kinds of large language model even it is also having so many data sets you can use to train these kinds of large language model and hugging F provides one pipeline so with the help of this pipeline you can easily perform any kinds of task and hugging fish is also having one python Library called Transformer with the help of Transformer Library you can easily Implement any kinds
of genbi based project so let me show you this hugging face platform what are the things it is having so guys if you want to open the hugging face uh platform just search for hugging face okay hugging face in Google you will see this first link just try to open it up and make sure you create one account here so for me I already have one account so what I will do I'll just try to log in with with my account so if you Don't have the account you just need to do the sign up
operation okay so with your email address you can also create an account so let me login with my account so guys as you can see I already logged in with my account and this is the interface of the hugging face now as I already told you uh hugging face is a platform so with this platform you can perform uh lots of task you can perform geni related task you can perform natural language processing related task Even you can also perform computer vision related task okay even you can also do audios kinds of task let's say
you are having audio data okay you can also do these kinds of task in this hugging face platform okay now it is having actually different different Services the first the very popular Services it is having called models that means this hugging face is having all kinds of large language model let me show you if I click on the model section Now see it is having that many of model just try to see the count that many of models are available and these are the models are completely open source so you don't need to pay here
okay if you want to use these are the model you don't need to pay here only you just need to use hugging face python library for this okay with the help of hugging face python Library you can easily access these other the model even I will also tell you how we can access different Different open source large language model and how we can Implement different different application on top of it right now if you see left hand side it is having different different task you can select the multi model that means if you want to
generate image text to text if you want to do visual question answering if you want to do let's say document question answering if you want to let's say do video text to text everything you can perform here then it is having Computer vision related task you can perform object detection image classification image segmentation text to image okay all kinds of computer vision related task are available even nowadays whatever multi model you can see let's say uh large language model will generate image large language model will generate videos these kinds of models are also available in
this hugging F platform now you can see it is also having natural language processing Related task that means you can perform text classification to token classification summarization feature extraction Tex generation code gener everything you can perform here okay apart from that I already told you it is also having some audios model let you want to perform text to speech text to audio okay automatic speech recognization these are the task also you can perform here apart from that it also provides tabular kinds of task Let's say you can perform tab classification tabular regression time series forecasting
these are the task you can also perform then they they have added another amazing technology the reinforcement learning so those who are interested in reinforcement learing you can also perform reinforcement learning operation with the help of hugging face platform so it is having all kinds of reinforcement learning model you can also perform something called graph Machine learning with this hugging face platform so here I'm not going to focus on all the actually task here because we are more focusing on the generative and for generative actually we'll be more focusing on the NLP that means national
language processing and we'll be focusing on the multimodel section okay because it is having different different large language model as I already told you now you can select any kinds of task let's say you want to perform let's say Uh this one you want to perform let's say a text classification okay just select the text classification now see automatically it will suggest you all the text classification related model now see these are the model from different different organization see this model is from meta AI okay Meta Meta Lama see with the help of meta Lama
you can also do text classification and this is one of the large language model it is having 86 million parameter and This is the download see this is that that many times actually this model got downloaded okay so you can also start you can also download this model if you are let's say I you like this model you can also start this you can also like this even they have already given the model card let's say you want to read about this model like what this model can perform how many parameters it is having what
kinds of data it got PR okay everything they have given okay Everything they have given even they have also given the code snippit you can use okay let's say you want to test this model you can copy this code snippit and you can execute it here okay so it's one of the amazing platform guys if you are working in the field of generative I so definitely you should learn the hugging face platform and know it to body everything I'm going to teach you here so let me show you what are the things we'll be discussing
here see inside Hugging face actually hugging face we have to learn some of the very important concept the first concept we have to learn the pipeline because inside hugging face everything is all about pipeline okay whenever you'll be using one python library of the hugging fish called Transformer okay Transformers see Transformers is the architecture but they have also named it as a library okay because it is having All the model which is based on the Transformer architecture only so that's why they name this particular Library as a Transformer Library okay so inside Transformer Library whenever
you will be writing the code let's say you want to perform language translation summarization you have to create a pipeline for this okay so we have to discuss the pipeline then we'll be discussing different different NLP task you can perform with the help of this Hugging face library then we'll be discussing very important concept called tokenization okay because in hugging face everything is all about tokenization so whatever pre-processing whatever textt representation you have to do with the help of tokenization only then we'll be discussing about the data set like what are the data set it
is having then we'll be also discussing about another super important uh service called spaces let's say you don't have Good configuration PC but you want to train one large language model so what you can do you can also take the hugging face spaces that means hugging face infrastructure but definitely you have to pay for this so there actually you can train any kind of large language model okay and this machine is like very good configur tion machine you can take there then the fifth topic we'll be discussing about fine tuning okay fine Tuning llm okay
uh with the help of hugging F that means we'll be learning how we can fine tune any kinds of llm okay with help of hugging face platform then at the last we'll be seeing some of the project implementation okay so with the help of hugging face we'll be doing some of the project implementation that means we'll be doing Tex summarization language translation even we'll be also doing the research papers summarization so these are the project we'll be Implementing month by month so I hope this is enough to know know about hugging F and if you
understand these are the concept definitely you can use hugging F platform um if you want to build any kinds of application okay later on and trust me guys if you want to use open source large language model that means open llm okay open llm so hugging face is the only way okay you can use open llm okay hugging face is the only platform uh it will help you to Work with the open source large language model but if you want to work with paid large language model let's say if I want to take any kinds
of commercial large language model that time you can use something called open AI okay open a platform even there are some Cloud platform as well let's say Bedrock is there okay Amazon Bedrock is there then vertex AI is also there so we'll be also learning these are the things one by one no need to worry first of all let's try To understand the hugging face because uh this is the starting point okay this the starting point in the field of generative so what I feel like if you want to understand the if you want to
understand the large language model if you want to uh let's say use them as a practical so I will suggest first of all start with hugging face platform then try to learn openi and whatever Services you are having okay everything I going to teach you no Need to worry okay so apart from that it is also having data sets guys as you can see different different data sets let's say you want to train one language translation model so simply you can select the task here let's say I want to uh do the language translation I
select the task here so this is the translation now see these are the data related langu language translation okay see these are the data related language translation so it is having all kinds of language Translation data you can see so see this data set is containing Canadian actually text as well as the English text that means you can part from Canadian to English otherwise English to Canadian I think okay apart from that you are having different different data set even you can also sarch here if you want to let's say find any kind of data
set you can also search here it is also possible now apart from that you can also select different different libraries let's say I want to perform one task let's say I want to find a model let's say I want to perform something called uh text classification now I want to use one specific Library so see it is having different different Library you can use P torch then you can use transer flow then you have safe transource okay then you are having kasas different different Library it supports let's I want to perform with the help of
pytorch okay py toor framework so these are the model Related pytorch so if you want to use these are the model you have to use pytorch code so similar wise if you want to perform with the help of let's say tensor flow you can select the tensor flow it will give you all the model related tensor flow that time okay that time you have to install tensorflow Library okay not the py library so that's how it has given different different flexibility different different functionality so that you can Filter out filter out the specific model filter
out the specific data okay everything you can do here and it is having another uh amazing Service as I already told you the spaces so spaces will give you let's say uh this infrastructure if you want to train any kinds of large language model you can use their spaces service so there you can buy their instance and you can train your model see it is having different different spaces even you you can also Create your own spaces now apart from that it is having another amazing documentation now see if you want to Deep dive uh
in this hugging face platform like how you can use this hugging face platform properly so they are also having their official tutorials if you just go to the documentation so there they are having one uh documentation and inside that you will get one section called task now let's click on the task now see let's say I Want to perform image classification task so I'll just simply click here now see it will give me the official tutorial so they have their the YouTube videos you can also refer the YouTube videos even you can also refer their
blog okay how we can perform the text classification that time sorry how we can perform the image classification so everything they have written here okay so this is one of the amazing actually let's say platform I personally prefer If I want to implement any kinds of project and if you want to learn anything let's say I don't know about let's say zero short image classification I will go I'll come here I'll go to their tutorial section and easily I can learn okay how we can use this particular model okay so that's how actually you can
also Deep dive but if you cover this video I hope you'll be learning everything about this hugging F platform but still if you want to dip Dive you can refer their tutorial as well fine and if you want to take their premium subscription you can also see their pricing and all you can also check it up but as of now premium subscription is not required we'll be using free services okay free services is enough for us because here we'll be only accessing the models data set okay these are the thing as of now I'm not
going to take this spaces everything I'll be doing in my Google collab because in Google collab we'll be getting free gpus okay that is the idea so yes guys this is the introduction of the hugging phase now I think you got it what exactly this hugging phase is and uh we saw like different different Services of the hugging face now in the next video uh we'll be doing some handson on top of the hugging face like how we can use the hugging face with the help of Python Programming so there we'll be installing my amazing
Library called Transformer Okay with the help of Transformer we'll be using the hugging face and it is also give you the API functionality guys if you want to take the hugging F API key you can also take for this you have to go to the settings now here you having something called access token okay you can also take the access token so with the help of access access token also we can access the model let's say some of the models you will be getting here those model actually you can directly Download for this you need
this hugging F access token okay so we'll be also learning how we can use the access token to access any kinds of model and how we can generate these are the access token as well so here I already created one uh collab notebook and here I have already mentioned each and everything you need uh for the hugging face okay practical understanding so as I already told you if I want to use hugging face platform form with the help of python we need to Install one Library called Transformers okay so this is the library so Transformers
Library is having all the functionality uh present inside the hugging face platform right so if you want to install Transformer Library so you have to execute this command called pip install uh Transformers okay now let me first of all connect this notebook and the resources should be shared guys in the resources section so from there you can download and you can Also um code with me okay it is connected now now the first thing what what you have to do you have to select the run time uh to GPU okay because I'm I'm not going
to use the CPU configuration machine here I'll be taking the GPU machine because here we'll be using large language model so definitely uh you have to select the GPU based machine there fine so let me select the GPU machine and if you're using free collab you'll be getting T4 GPU and if you're using collab Pro uh there also you can take actually different different uh see let me show you if I click on since run type see you can also take a00 GPU and L4 GPU as well okay these are the GPU you can also
access so now guys you can see my notebook is connected now if I want to check the GPU you can execute this command called Nvidia okay hen SMI so this will uh show you the GPU configuration you got so Here I'm using free collab that's why I got Tesla T4 GPU now first of all I have to install the Transformer Library here see Transformer is already installed now now see inside hugging face we are having different different task as I already showed you so if I let's say click on the models and if you see
the left hand side you can see you are having different different task so here you are having different different task let's say uh here we are focusing On generative AI that means the natural language processing so here we can perform different different tasks let's say we can perform text classification uh token classification okay all kinds of tasks actually can perform whatever task actually you in the field of NLP or in the field of let's say generative all kinds of task you can perform here so if I want to perform these other the task I have
to use one uh functionality from the hugging fist called pipeline okay so Here you can see I have imported this pipeline from the Transformer okay so it is available inside Transformer now let's say uh we are having various uh NLP task as I already showed you text classification then we are having uh here you can see text classification token classification uh table question answering question answering zero shot classification translation okay now see I have listed down all the task here now let's say if I want to perform text Classification so what you have to do
inside pipeline you have to mention I want to perform text classification so this Transformer Library what it will do it will automatically try to understand you want to perform test classification so all the code actually it is having in the back end it will execute that particular code CIP it that means it's a high level raer okay on top of let's say hugging face it's a high level rapper this particular Transformer Library so You don't need to write the code from scratch let's say you don't need to write the code I want to perform uh
text classification so code you don't need to write from scratch only just need to give this particular input to the pipeline and pipeline will automatically take care you have to do the uh text classification task and for this whatever uh let's say strategy you have to follow everything it will take care okay I hope you got it now see similar Wise if you want to perform token classification you can mention token classification if you want to perform question answer you can also give question answering if you want to perform text generation summarization translation even also
you can select the model let's say which model you want to use as I already showed you now now we are having so many model now let's say I want to perform uh translation now see you are having different different model OKAY different different model now see you can also give the model OKAY model ID now let's say I want to use this model I'll just click here and I'll copy the model ID this the model I'll copy and here I just need to mention so it will automatically download this model from the hugging face
so you don't need to manually download it as well so that must powerful it is okay so that is why I told you you have to master this pipeline concept because going forward Whatever let's say you'll be implementing you need this particular pipeline approach fine now see I have given different different actually example computer vision example also let's say you want to part from image classification in the pipeline just mention image classification okay even also you can specify the model it's up to you now see different different example I have given just for your reference
so that you can refer this Notebook later on now let's perform one specific NLP task let's say I want to perform sentiment analysis so here you can see the task as well as uh you will also get the all kinds of task in the document section so here you are having one tab called task now see all the task is also visible red computer vision naal language processing audios and multim model okay everything is there now let's say I want to perform sentiment analysis and sentiment analysis is what kinds of Task it's a text classification
task I think you saw it's a text classification task okay now either you can give the text classification okay either you can pass the text classification here either you can also give the sentiment analysis okay both it will work so I have given sentiment analysis okay I want to perform sentiment analysis in the pipeline now it will give you one object called pipeline object and this is my classifier now inside that you have to Pass the input let's say here I given the input I was uh so not happy with the last mission impossible movie
now this is the sentence I have given now see if I execute this line of code so it will automatically download one specific model now see here I haven't given any kinds of model name so it will download one default model and it will try to predict whether this particular sentiment is a positive or negative okay let me show you see if I execute this Code see it is using one model default model called um distill bbased case Okay F tune SST to English model and this is the link of the model if I open
it up see it is using this model this is the default model okay and here is the prediction guys it's a negative sentiment and it is telling 99 99% confidence it's a negative sentiment okay that mean it is performing amazing see that much actually it is easy to use this Transformer library and that's how Actually pipeline helps us to perform any kinds of task I want to perform so I don't need to write the code from scratch only I just need to mention the specific task name it will automatically do it for me now there
is another way you can pass the input let's say so let's say here is my pipeline inside pipeline you can mention the task I want to perform let's say a sentiment analysis and the second parameter you can give the input let's say this is the Input I have given I was confused with the Barbie movie okay now now it will give you the same output so either you can follow this approach either you can follow this approach this is the one line approach okay so both it is possible now if you are having any big
text that time what you can do you can also use multi-line actually string there also you can pass the text like that see again it's a positive review and this is the confidence score Fine now let me show you how we can specify any kinds of model so let's say now I want to perform sentiment analysis and I want to use any other model let's say I want to use this model Facebook Bart large M let me search this model you'll see this is one of the large language model you can see but large uh
mnl see again it's not a large language model I can say it's a language model because this model only can perform one specific Task which is nothing but the um sentiment analysis but going forward we'll be using large language model like we'll be using Lama mistal Falcon jini okay so these are the model actually we'll be using and these are the model is multitasking model model okay that means large language model so there are two kinds of models are available guys uh just try to remember one is the large language model that means llm okay
and this is like only LM that means language Model only so language model can only perform one specific task okay one specific task and large language model can perform multiple task multiple task even it can also perform chat operation okay chat operation because I already told you now how large language model was trained okay because of that supervised fine tuning reinforcement learning through human feedback these are the technique has applied but Whatever LM model you can see it has only trained with the transform Transformer architecture okay Transformer architecture okay I think I showed you the
transform architecture example I think you remember fine so these are the model we using this is called actually language model only it can only perform one specific task so that's why here we are mentioning the task name sentiment analysis now see here we are giving the model name now You can use any kinds of model just go to the model section go to the model section now let's say I want to perform sentiment analyis for this I have to use text classification now just try to get any kinds of model let's say you want to
use this model copy the model ID try to paste it here okay it will download that model and it will do the operation now let me show you see first of all this model will be downloaded from the hugging F now see it is downloaded and See the model size it's around 1.63 GB this is actually sentiment I'm getting this neutral sentiment okay it's not a positive it's not a negative it's a neutral sentiment and this is The Confidence Code you are getting fine great now how to perform the batch actually sentiment if you're having
batch data let's say multiple let's say uh sentence that time how we can do you can use something called B sentiment analysis so again just try to create a Pipeline and whenever you are preparing your input data just try to uh give inside a list you can see it's a python list and inside that I'm having different different sentence okay different different sentence now you have to pass this particular uh list inside your classifier now see it will take one by one all the sentence and it will show you the uh sentiment now see this
is the first sentence sentiment second sentence third sentence and Fourth sentence okay I hope it is clear but the model actually it is using the default model uh it is having only two level which is nothing but positive and negative but if you see this sentence it is having some emotion okay it is having some different different sentiment and if you want to capture different different sentiments you can use any other model let's say I will be using this model robot based go emotion model so this can actually detect multiple uh Sentiment from a sentence
okay you can see all kinds of sentiment they have mentioned see these are the sentiment it is having okay so that many of sentiment it can detect now let me use this model and let me again apply the same data now see it will give you different different sentiment see again it is downloading the model after that it will do the sentiment analysis now the first sentence it is uh admiration second one confusion then uh Amusement then anger okay I hope you're cleared so I can say this is one of the very powerful Library okay
if you want to work in the field of gen or natural language processing now you can also perform T generation task so for this in the pipeline you have to mention T generation only okay I think I showed you different different task that's I want to perform TCH generation right now so Tech generation I think it is there natural language processing text Generation okay now see they have also mentioned in the tutorial you have to give Tex generation in the pipeline okay and you can also give the model so the same thing I'm doing TCH
generation model I'm using distill but and inside that I'm passing my input today is the rainy day in London now it will refer this sentence and it will generate the text okay based on this particular content now if I execute see see guys beautifully it has Generated the text today is rainy day in London and one that no other City can remember orever uh even no a few days year ago the police arrested okay see beautifully it has generated the text okay so that's actually you can also perform Tex generation now if you want to
perform question answering just give question answering and you have to give the question as well as the context okay so based on the context it will uh so by refering this context it will uh give You give you the answer okay let's say this is my question what is my job and here I'm giving the context I'm developing an AI model with the python okay now if I give my question as well as the context it will give me the answer again you can check their uh document ation for the question answering question answering so
they're telling like that so you have to give the question context and you have to pass to the uh pipeline okay now see uh Answer is developing AI model that means this is my job okay I hope it is clear so I hope guys uh now it is clear how we can use this uh hugging face platform if I want to perform different different task and how we can access different different model now let's see want to use large language model that means llm so I think I showed you one uh amazing uh GitHub open
llm GitHub so you can open this GitHub and you can see different different large language model Name let's say you want to use this model let's say you want to use um let's see we want to use this Falcon model falcon is a large language model you can copy the name and simply you can go to the hugging face go to the model section and simply s here okay simply s here now see Falcon model would be there see Falcon models is there and Falcon is nothing but it's a large language model it's not a
language model okay with the help of Falcon actually we can perform different different task even we can also perform chart operation not only that all kinds of models are available let me show you let's say Lama 2 if you want to use l 2 I'll copy and I'll come here I'll search for the Lama 2 see Lama 2 is from M okay like that we are having some another model Also let's say mral is there then we are having something called Jimma so this model is from Google site so that's how we are having different
different Large language model and no need to worry going forward we'll be using these other the model but before using these are the model uh we have to learn some additional let's say um tools and Technology like we'll be learning some gen VI framework like Lama index then langin then we'll be also learning about Vector database okay then we'll be learning these are the model as well fine so yes guys this is all about from this video I hope it is uh clear now so In the next video we'll be understanding about tokenizer uh tokenizer
inside actually hugging face what exactly this tokenizer okay and why we need it okay why tokenization is required so as I already told you inside hugging face actually we're having another uh super important concept called tokenization and now we'll try to understand why tokenization is important and why we use tokenization inside hugging pH I think you remember I taught You one concept called text reprocessing and text representation so there uh whenever I was passing any kinds of text to my model first of all I was doing the processing then I was converting that particular text
to the vector representation right and for this I was using different different technique like for text vectorization I was using TF Ida what to F bag of word then that time actually I told you we are also having some other technique which is nothing But Transformer based technique okay because it is having something called attention mechanism and uh then I taught you the attention mechanism concept the Transformer architecture so there I show you how it will give the attention to a specific word now it's time to show you this part as a practical that means
now we are using Transformer based architecture Transformer based model and whatever data see whatever data actually we're passing as an input to my model Okay so internally how it is processing that okay internally how it is processing that internally how it is converting to the vector representation so it is using something called tokenization technique okay tokenizer so inside actually Transformer we are having something called Auto tokenizer okay so inside Auto tokenizer we have to first of all give the model name like which model you have to use for that particular tokenization task so To explain
this tokenization I I just simplified this particular code so first of all I'm importing some of the library necessary Library I need and here you can see I'm importing Auto tokenizer okay so let me import all the library if you want to use any kinds of tokenization so first of all you have to define the model name like which model you want to use so here I I'm using this model bbased multilingual an sentiment model so this model will uh so with the Help of this particular Transformer based model first of all it will perform
the tokenization that means this input we are giving to the model right now yes or no right so first of all it will um this tokenizer what this tokenizer will do it will first of all pre-process let's say if it is having some HTML tags or any kind of word it will try to remove then it will try to convert this text to the vector representation okay then it will pass to the model and model Will uh actually do the sentiment analysis now see if I execute the code so first of all it will download
the tokenizer okay then after downloading the tokenizer it will also initialize the model after that it will do the sentiment analysis so this is the high level understanding and I will also show you the low level how it is generating the vector even we can also visualize the vector as well see now we can see this Is the result we are getting fine now let's try to see how tokenizer is working so if you want to use tokenizer you have to import one Library called Auto tokenizer from Transformer the first thing what you have to
do you have to load one pre0 model OKAY pre-end Transformer based model so here let's say we are using bbased Case Model so this is one of the Transformer based model okay you can see this one of the Transformer Bas model so we're giving The model ID here then this is my example text let's say I'm giving into my tokenizer okay so inside tokenizer we're having another function called tokenize now after tokenize it I'm just printing okay after tokenizing I'm just printing now let me show you what it will give me the output so I
can card these are the example now let me execute as of now H now see when you apply tokenizer do tokenize it will give you the Individual token that means the tokenization we used to perform with the help of NL iCal Library I think I remember right sentence level and level tokenization now it is possible with the help of this tokenizer class only okay that's why I told you Transformer is like very powerful so it's a high level rapper inside that they have written all kinds of code it will handle each and everything you don't
need to write any kinds of code from scratch okay now if I Want to convert this textual representation to the vector what I have to do I have to pass this particular token to the convert tokens to IDs this particular method okay and it is available inside tokenizer object the tokenizer object we have find here now it will give you the input IDs now let me show you this input IDs I can take different cell and here I can execute see this is the input ID and this is nothing but your representation Of the text
the text actually you have given I was so not happy with the Barbie movie now you can see this is the vector representation of this particular text and it is using Transformer model okay it is using Transformer architecture in the back end and it's like more powerful okay it's like more powerful technique than your bag of word or TF IDF or what to whatever you have learned okay and going forward we'll be using this particular approach only okay I hope you Cleared but whenever I have to pass this particular Vector to my model I won't
be using this particular method for this you have to use this particular method so here there is another function uh so here you have to pass your data uh to the tokenizer okay directly you have to pass to the tokenizer and tokenizer will give you this output let me show you see this is the entire output encoded input now see input IDs now it is familiar so this is the input IDs and here one additional token you can see one1 what is one1 this is the start of the sentence and start of the sentence represent
101 and end of the sentence represent one2 and in between actually you can see your entire Vector the vector actually you got okay now it is having another actually uh things called token type IDs as of now just try to skip this part it's not required and this is the important things guys attention mask okay now I think you Remember what is attention mask that uh attention mechanism concept that means uh whenever let's say it will pass this particular data to my model in which word it will give the more attention Okay it is telling
now see it is giving the attention to all the word because every word is one one one one you can see that means this one will get the same attention this one would give the same attention Okay this one will give the same attention okay and how it is Getting to know how to give the let's say attention uh I think I showed you the calculation that uh I mean one one calculation I was discussing in that uh J our blog okay that's how it is calculating the score and it is able to decide okay
I have to give the attention to this particular specific word okay I hope it is clear so these are the actually output you'll be getting after doing the tokenization now if you want to decode The tokenization that means if you want to again convert this uh let's say Vector to text representation you can use decode function for this now I'll give my input ID so whatever input ID I'm giving getting I'm I'll pass in in the decode function and decode will give me the previous text actually you can see I was so have uh so
I was not happy with the Barbie movie okay I hope it is clear now now if I execute all of them all together you can see this is the Output this is your tokens this is input IDs this is the encoded input this is the decode output that means this tokenization will first of all perform the pre-processing then it will perform the tokenization then it will convert your let's say Tex to Vector representation okay and additionally it will also give you some of the let's say number which is nothing but attention mask okay based on
this attention mask your model will decide in which word it Has to give the uh let's say attention mode so now I think guys this uh token tokenizer is clear like what is tokenizer exactly inside hugging face because many people will have this question later on that's why I just clarified right now because going forward whatever experiment we'll be doing we'll be using the tokenizer okay from the Hing itself so for this again I kept one example so here we'll be fine tuning uh one data set called IMDb okay IMDb movie review data set and
and we'll be using one uh language model OKAY from the hugging face itself so for this year we'll be using hugging face data set only because I already told you hugging face is having data set okay data set actually services so you can also get see different different data set and this is the data set count guys okay now you can search any kinds of data set now let's say I want to use something called IMDb data set so IMDb so this data set Is already available see this is the IMDB data set okay so
this is a movie reviews data set so here you have the movies and based on that you have the uh reviews okay whether it's a negative or positive so here we'll be fing one uh text classification model that means we'll be performing sentiment analysis okay this is our main objective so if you want to use actually hugging face data set for this you have to install one Library called data set okay so First of all try to install data set and make sure you also install Transformers okay because I already installed Transformers so that's why
I'm not going to install again now you have to restart the run time now see it is restarting the on time after that we'll load the data and loading data is very much easy only you just need to give the data set name okay so there is a load data set uh function inside data set you have to call it and Inside that you have to pass the data set name it will automatically download the data from the hugging face okay now see this is the data set name IMDb just copy the name and give
it here okay it will automatically download now if I show you so not only this data you can use any kinds of data even I think you remember in the previous video I showed you that tokenizer so in the tokenizer also you can take different different Model OKAY different different model for the tokenization so there I was using BB Cas model you can also use any other model but remember whatever actually model you are using for the tokenizer the same model you have to use for the model training as well for the model inference as
well okay this is the uh idea now let me show you huh see my data set is downloaded now if you want to see the data set metadata so this is the data set now we are having um uh Training data take and labels we are having and this is the example that means 25,000 actually rows we are having inside training example and test data we are having 25,000 and here we are also having unsupervised data so here we'll be using this data only train and test data okay uh for my analysis now first of
all let's do the pre-processing task so for this I think remember we have to perform the tokenizer so here I'm Importing Auto tokenizer so again I'm using this model but best Case Model okay so first of all I initialize my tokenizer then here I have created a function tokenize function inside that I'm just tokenizing my example now which example I have to tokenize guys I think remember if I show you the data let's if I show you the first example you can see this is the text okay this is my reviews okay this is my
reviews and this is the sentiment so I have to apply on top of The reviews only okay so that's why I'm giving text example text because inside example we are having the text okay text uh Keys actually you can see because it's a dictionary format okay that's the idea and here I'm adding pad padding max length let's say what is padding I think remember let's say there is a sentence this movie is good now now let's say there is another sentence I Hit this okay now just see the actually input length 1 2 3 4
1 2 3 so this input length actually having one word list so in case actually what you can do you can add some padding padding means zero value okay so that's why actually we adding this particular parameter padding is equal to max length that means first of all it will try to figure out the maximum length sentence based on that it will decide what would be the padding size okay this is the Idea now uh here we are applying this particular function you can see this tokenized function on top of my entire data set okay
and for this I'm using map function and map will apply on top of the entire rows okay this is the idea now if I execute see uh this tokenizer will apply on top of my entire data set then I will get all of my vector now previously I only had this many of example now after tokenization you will see uh you will you will have some more Actually Keys like your attention uh attention mask then you'll also get the input IDs okay these are the thing you'll also getting so guys as you can see my
tokenization is completed now if I show you my tokenized data set right now so see now I'm having uh all of the features like input IDs token type IDs and as well as the attention mask okay uh everything now if I want to show you the data how it will look like so here You can visualize that's I want to see my training data let's say I want to show you the first example now you can see the first example this is the raw text okay and after doing the tokenization uh you got the input
IDs and input ID is nothing but is the vector representation of the entire text okay you can see this the vector representation and it will also have something called uh this one attention mask let me show you token type IDs Would be there as well as the attention mask see attention mask okay I hope it is clear now that means we have successfully converted our uh textual data to Vector representation now we can start the training but before that I have to first of all set some of the training argument and Transformer provides actually one
uh actually function called training arguments okay inside that you can mention training arguments so these are the default Actually arguments you can keep as of now no need to change anything because again it's a hyper parameter tuning only just uh change you can do number of epoch let's say how many Epoch you want to train let I want only want to train only let's say one Epoch as of now okay only one EPO I want to train and output directory so it will create a folder here called output directory inside that it will save all
the models and everything right now let me set the Training arguments so again this is kinds of template kind of code you don't need to remember anything all the codes are available in the hugging face platform so from there you can refer okay you don't need to remember anything here F now see that many of actually training argument you can set here okay after printing you can see okay but I don't need to set all the training arguments we can only set these are the training Argument F now we can initialize the model so here
to initialize the model first of all you have to import to library Auto so here you have to import one Library called Auto model for sequence classification because here we are doing text classification for this you have to use this particular class then you have to also import the trainer now first of all you have to load the model you can see this is the model part based case model I'm loading okay so if You give it here it will automatically download and number of labels I'm giving two because we are having only two labels
positive and negative we are doing the sentiment analysis now of the IMDb data set and if you see the sentiment only positive and negative fine if you're having let's say four four sentiment that time you can give four okay I hope you're clear now simply we can initialize the trainer inside trainer give the model give the the Training arguments okay the training arguments we're getting from here this is the training arguments then we have to pass the train data so this is my train data tokenized data train okay training sample and evaluation data I'm giving
my testing data fine now let me click here now see it will download the model and it will initialize all the trainer now see training has not started yet because if you want to start the Training you have to call this particular things okay trainer. train now see if I click here now it will start the training and after training it will save all the artifacts in the result folder okay so it will take some time guys so let's wait once training is completed I will come back if you're using free collab so training will
take some time so you have to wait now what you can do you can also evaluate uh the model on top of the test data so just Try to execute this code so again it will take some time let's wait after this execution you will see the evaluation metrics and in between let me show you the training loss and validation loss you got after training so this is the training loss and this is the validation loss in only one Epoch so here if you increase the epoch size you'll see that this loss would be decreased
so try to increase the EPO size whenever you are uh Training your actual model as of now I'm only training monip because I just wanted to show you okay this the idea now let's see the evaluation metrix uh what is the evaluation metrix we get here so guys as you can see this is your evaluation result so this is the evaluation loss this is the evaluation runtime uh that means uh that many seconds actually took to execute this entire code and you can see we only train manip okay so you just only need To see
this losses like how much loss you are getting so this loss should be close to zero if it is close to zero that means your model is performing better that time now you can also save the model the model you have trained you can also save the model even you can also save the tokenizer why you have to to save the tokenizer because uh let's say uh in future user will come here and they will be using your model so they will give some input right and input is What it's a text so again you
have to convert that particular into the vector representation yes or no right so to uh convert this input to the vector representation I need to use my tokenizer my train tokenizer okay not the previous tokenizer okay that's the thing you have to remember and as well as I also need to save my model now let me save both of [Music] them now you can see in the results Folder we are having all the check points of our model training and all I think you already know if you have already learned previously like how to train
uh computer vision model tensorflow model I think you know what is checkpoints okay now see we have successfully saved our model now if I refresh now see this model has been saved here now inside uh this folder actually we are having different different actually file sequency config Json then uh save Tor okay Json so this is how actually uh your hugging face save the model okay it will contain actually multiple file inside config you will have the entire configuration of the model okay this is the main idea so see we have successfully saved our model
as well as the tokenizer now let's see actually one uh project now let's try to do one small project so here we'll be using uh this uh archive actually so I think you know archive is a website so Here you will get different different research paper okay all kinds of research article um you will get here so first of all we'll install this Library uh because in Python we have this library with the help of this Library we can uh actually download all kinds of article right we can get all kinds of article data so
now let me import now see if I want to now see if you want to use this particular Library archive first of all you have to prepare One query you need AI or artificial intelligence or machine learning related let's say paper here you are giving the query now here you have to search okay you have to search in the archive now here you can see I've given the query maximum result I need 10 and you have to pass this particular parameter called sorted by just try to mention submitted dat as of now now it will
give me different different page and I'm looking through the page and I'm extracting the Published date title and abstraction and category of the paper okay then whatever data I'm getting I'm converting to the data frame and I'm just plotting it here now let me execute and let me show you so guys you can see I got the results now this is the publish date this is the title of the paper and this is the abstraction of the paper Okay abstract of the paper and this is the category now let's say I want to uh Create
a research paper summarization system so what I can do in the pipeline I can mention I want to perform summarization okay and which things I want to summarize I want to summarize this abstraction okay this abstraction I got you can see this is the abstraction column okay abstract column now that's why I'm passing the abstract column from the data F and here I'm using this Facebook part large CNN model okay inside that I'm passing my abstraction Now see will give me the summary so this model actually they train on the summary okay summarization task you
can open it up and you can see the description of the model they're doing the summarization task okay so execution is completed now I got all of my summarization result now let's say I only want to uh see the first one okay first summarization text now I can execute this code you can also print all of them it's up to you but I'm only Showing the first example see this is the summary of the first results okay so I hope guys uh it is clear how we can perform the let's say fine tuning operation on
top of a pre-end model and no need to worry going forward also will be exploring lots of model and we'll be doing the fine tuning operation we'll be implementing different different application okay on top of it we'll be implementing different different projects in this video I will show you How we can generate hugging face API key because I told you sometimes we need this API key with the help of API key will be accessing the model okay so for this uh we have to learn how we can generate a key so as of now we'll
be generating the key later on we'll be using that so for this uh definitely first of all you have to login with your hugging face account now click on the profile icon now go to the settings now left hand side you will see one option Called access key okay access token now click here now see previously I already created some of the access token but for you it would be empty now if you want to create a new one just click on create new token now you can give the token name let's say I give
uh my new okay my new token now you can uh give the permission now here you can uh provide the permission whether uh you want to create For the read access or whether you want to create for the right access okay so let's say uh as of now I want to create for the read access I only want to perform the read operation and uh later on I will also show you how you can push your train model to the hugging face Hub so that time actually you need the right access okay so as of
now let's create my token this is your token just try to copy and save it somewhere don't no need to share with anyone otherwise they will Also access your account okay so it should be private I'll delete it after the recording that's why I'm showing you now let me uh close it so as of now uh I showed you the hugging F demo like how we can use this hugging face platform how we can access different different model even how we can also fine tune those are the model on top of our custom data so
in this video we'll be uh using the same technique and we'll be implementing one project called takech Summarization okay so here is The Notebook guys I already prepared so the first thing what you have to do first thing just connect the notebook and make sure you change the run time to the GPU because here you need the GPU if you want to let's say uh implement this project because here will be fine tuning one uh Transformer based model okay we'll be fine tuning one LM model here on top of our custom data I'll tell you
which model we going to f tune even on Top of which data set actually we're going to find tune but first of all uh let me tell you what is take summarization take summarization means let's say you are having a bigger paragraph and you are uh summarizing that particular paragraph I think you know okay what is the summary right what is the T summarization project so in Internet actually you will be getting different different summarization actually application let's say if I Write text summarization online so there are so many application you will see see this
is another application from qu bot actually so here actually you can also give any kinds of text and it will give you the summary so let me show you if I search for English story so what I can do I can copy this text as of now as it is and here let me paste it now if I click on summarize button You'll see that it will give me the summary of the entire text now you can also control like how much uh summary you need so let's say I need a little bit bigger summary
now if I again res summarize now see this result I'm getting uh this is the actually little bit long than our previous summary okay so that's how actually we're having different different application over the Internet so we'll be implementing this kinds of project but here I'm not going To implement a user interface user interface wise will be implementing later on so as of now we'll be only doing the experiment on the collab notebook okay we'll be using uh Transformer based model we'll be using uh LM model and we'll be doing this kind kinds of project
okay this is the main objective here so let me so let me get back to my collab notebook and first of all let me execute this command so this command will uh show you which uh GPU Actually you are using so here I'm using Tesla T4 GPU because here I'm using free Google collab now the first thing what you have to do you have to install Transformer Library okay why because here we'll be using hugging face actually platform and to use hugging face platform I already showed you you have to install Transformer Library apart from
that you need some additional package like sentence pieces okay then you need uh scare blue then row score Okay and Pi 7 Zer and data sets why data set because with the help of data sets we'll be downloading the data set from the hugging phe itself okay that is why and these are the dependency package you also need to install here fine so let me install all the package one by one so my installation is completed so the next thing what you have to do you have to update some of the package the first package
you have to update called accelerate then you have to uninstall Your older version of Transformer and accelerate and you have to install the latest one because what happens actually in Google collab sometimes uh they're using older version of the Transformer package and accelerate package okay so that's why I'm updating it again okay that's is the things you have to do and what is accelerate see accelerate will help you actually to access your GPU that means here you are using GPU you're using something called cuda okay so to Use your Cuda uh actually instance this accelerate
will help you so whenever it will do the find un operation that time actually it will access your GPU right so that's why actually this package is required now let me update all of them so as you can see uh update is successful there is no error that means everything is working fine now the next thing you have to import some of the necessary library to test whether everything is working fine or not so Here you can see I'm importing pipeline then I'm also importing load data set from the data set okay and apart from
that I'm also importing some additional Library okay some extra Library also I'm importing here it is not required but what the library actually I'm going to use like later on I'll tell you okay why I'm using that one so as of now let me import all of them see this execution is fine that means there is no error we have successfully install all the Package now I think I already imported torch so I don't need to import this line I can delete HM now the first thing what I have to do I have to check
the device okay whether it is utilizing my Cuda or whether it is utilizing my CPU so as I already showed you I have changed my runtime to the GPU right and here you can see it is connected with my T4 GPU that means it should use my Cuda so let me check see it is utilizing my Cuda so Here I'm using torch pytor library to check it whether Cuda is available or not if yes print CA otherwise print CPU so you don't need to manually set the your machine type so if you have the like
GPU install in your system and if all the tools and let's say services are already installed so it will utilize that particular GPU that means your ca otherwise it will automatically utilize your CPU okay so if you see any kinds of pyos implementation pyto use this Particular cod in the pit okay to map with your Cuda or with your CPU I hope it is clear now the first thing what I have to do I have to load my tokenizer okay tokenizer and what is tokenizer I already explained with the help of tokenizer we'll be preprocessing
our data and I will be converting my text to Vector representation so for this I'm going to import auto tokenizer from Transformer library and another class I have imported Auto model for sequence to Sequence LM because here we'll be uh loading one LM model that means we'll be loading one Transformer based model and to load the Transformer based model actually you have to use this particular class fine now let me import this package now here is the model checkpoint guys so here we'll be using one model called Google pagas CNN daily mail okay so this
is the Transformer pre- model and this is the model guys this is from Google so Google has trained this model And this model can perform T summarization okay now if you want to test it here is the inference API now here you can see it will give you the summary okay summary of the entire text now you can see like what kinds of data set they use to train this model every information they have given okay about the model so if you're interested uh to De dive you can check this documentation you can check this
model card you will see each and every details now if you Want to see the model just click on files and version you will see the GB of the model it is around 2.28 GB and there is another file it is around 3 3 GB okay so total I think 5 to 6 GB this particular model so whenever I will download this model you'll see that okay what is the like model size as of now let's let's get back so now let's load the tokenizer and I already told you uh the model actually will be
using the same model you have to use for the Tokenization as well fine this is the idea now let me load the tokenizer so for this I'm using Auto tokenizer from pre0 and here I'm giving the model checkpoint it will automatically download from the hugging face itself so fine I have already downloaded the tokenizer now we'll be downloading the model okay so to download the model I'll be using this class Auto model for sequence to sequence LM this class and there is a function called from preent You have to give the model checkpoint and two
device like in which device you want to load the model I want to load the model inside my GPU okay that's why I'm giving GPU two device is equal to my device now let me load it load it okay my model has also loaded now we'll be loading the data set and data set wise we'll be using my data set named Samsung data set so this is the data set guys so this data set is from Samsung actually so they have published This data and the Samsung data is nothing but it's a data set containing
about uh 60 16k messenger like conversation with somebody that means it is having actually different different conversation okay and uh in the another column actually it is having the summaries okay of that particular let's say conversation so if I show you let's say this is the conversation okay this side you are having the conversation and this side You are having the summary so let's say this is the conversation and this is the summary okay again this is the conversation this is the summary so that's how they have collected the entire data and it is having
16k actually messenger like conversation data fine and the data set name is Samsung data and if you want to see the data set is split so it is having around 14732 example for the training validation that many example and testing That many example fine so again you can uh see this documentation if you want to learn more about this data now let's download the data so for this I'll be using load data set from my data set uh package now let me download the data as well now I'll give yes and I'll press enter it
should download the data okay now if I show you the data set metadata so this is the metadata guys so as I already told you it is having dialogue and Summary and that many of example you are having for the training testing and validation fine now if you want to see any kinds of let's say dialogue that means your entire let's say conversation you can also visualize so let me visualize one of the example so this this is the dialogue guys you can see Olivia is telling who are you voting for this election then Olivia
is saying Liber as well and Olivia me too Olivia great okay so this Kinds of conversation it is having and based on that it is having one summary so let me show also show you the summary so I'm printing the summary here you can see so Olivia and Oliver are voting for the Liberals uh in the in this election okay so this these kinds of actually data set they have prepared now this particular quote cipit will print the number of example you are having in this data set as as well as the split length and
the let's say enter dialogue and ENT Summary now let me show you see this is the split length this is the training validation and testing and these are the features it is having like ID dialogue and summary and this is the entire dialogue you can see okay and this is the summary of the dialogue so this is the preview of the data that's why I just printed let I just wanted to show you okay how what kinds of data they collected now let's say you are having some different kinds of data so What you can
do you can give your let's say entire let's say paragraph As a dialogue and the summary as a summary okay and you can follow the same approach okay it doesn't matter now the next thing what I have to do guys I think you remember I have to pre-process my data and convert to the vector representation so for this what I have to use I have to use my tokenizer okay I think you know and on top of which actually let's say column I have to Apply my tokenizer under dialogue because dialogue is my uh dialog
is my actually paragraph and again if you see the summary summary is also at text okay so I also need to apply the tokenizer on top of the summary as well so here I've written a function convert example to Features so it will take the example batches that means your dialogue and summary and it will perform the tokenization operation you can see so first of all I'm applying Tokenization on top of my dialogue then I'm applying tokenization on top of my summary then whatever results actually I'm getting I'm just returning that means you will get
input IDs attention mask as well as the labels I think I showed you okay from that example now let me execute and now let me map this this function on top of my entire data set okay so see here I'm doing the map operation now first of all let me convert uh to vectorization then I will Show you okay how it will look like so see execution is done now if I show you my data set right now now see it is having different different features ID dialog summary input IDs attention mask and labels okay
because I converted to the vector representation right now now if you want to see the input IDs so these are this is nothing but your input IDs that means this is the vector represent of the first first dialogue okay and if you want to see the Attention mask you can also print attention mask so this is the attention mask that means uh it is giving all the word as the same attention I already explained this part fine and if you want to print the level as well that me the summary you can also print it
let me show you so here I will print the labels so this is the labels okay that means this is the summary of this dialogue fine I hope it is clear now we'll start the training but before that What I have to do I have to set some training arguments I think you remember right but before that one thing you have to do if you're using uh large amount of data so you have to uh initiate this data collator okay so what data collator will do data cator will help you to load your data as
a batches in the memory see here you are having so many example I showed you now in the training validation and testing so I don't need to load the entire data in my memory Because here if you see my memory size uh I got actually uh 12gb RAM so if you're having huge amount of data so 12gb Ram is not enough so for this what you can do you can load your data as a batches that means that means some amount of data you will be loading you'll be training again you'll be taking another batches
again you will be training that's how you can perform the operation so this data collator will help you to load your data as a batches And if you want to define the data collator you have to use this class from Transformer data cator for sequence to sequence so here you can see I'm passing my tokenizer and model inside the data C letteror fine then you have to initialize this data call letteror object so let me initialize it right now now the same thing you have to set the trading arguments so as I already told you
just keep all the arguments as same no need to change anything only you can Change the number of epoch okay you will be training so as of now I'll be showing you only one Epoch training because again uh it will take time we are we are using free Google collab okay and the output directory I want to create Pegasus Samsung okay so it will create our output directory Pegasus inside that it will save all the ARs fine so let me initialize the argument as well then I will initialize my trainer object so it will take
the model your training Argument tokenizer data cator and you have to pass the training data now see here one small hack actually I'm doing see if you see the training data size it's very huge so as I already showed you here so that many training example it is having almost 15,000 so if I'm taking 15,000 data for the training it will take lots of time and this is my validation and testing so what I'm done actually uh instead of taking the Training data I used actually uh test data as my training data because inside test
data I'm having very less example so that I want to show you the quick training but if you're training actually okay if you are doing the actual training that time you just use train data okay not the test data so this is the hack Guys small hack I'm doing because if you want to experiment something and you don't have that much of time so you can do you can take a Small amount of data and you can perform the let's experiment okay so that is what actually you can you can perform and here you can
see evaluation data I'm giving my validation data it's completely fine okay now let me initialize my trainer now if you want to start the training you have to call trainer. train now it will start the training of the model so it will take some time so I will pause the video and once training Is completed I'll come back okay so my training is completed now here you can see my training losses and uh different different matric that it is giving but you have to take the training losses so this loss should be close to zero
guys okay if it is close to zero that means your model is performing well now if you want to evaluate this model on of of the test data you have to use this function calculate Matrix on test DS okay so here Uh you'll be calculating something called row score okay what is row score see row score is a matrix for the uh summarization model take summarization model you can search on Google you will find the row row score actually equation but as of now I won't suggest just don't deep dive into the equation just try
to consider it just a evaluation Matrix like for the classification actually what evaluation Matrix we used to use like accuracy score then we used to use Confusion metrics AOC carve right and here we are performing something called T summarization and T summarization uses one Matrix called row score if you want to evaluate that model so similar wise for language translation we use another kinds of matric for let's say name entity recognization we use another kinds of Matrix for let's say a conversational agents we use another kinds of metrics okay all kinds of task is having
their different different Evaluation metrics okay this is the main idea here now before that let me show you the models actually it has trained now inside Pegasus Samsung folder this is the checkpoints inside that you will you can see this is the model actually it has train now we'll be saving this model as well just uh let me first of all show you the evaluation matrics then I will say uh tell you how we can save the model and now you can ask me where I got this function see if you just go to The
hugging face that documentation go to the T iation task you will see that they are suggesting this function okay so they're suggesting if you want to perform the evaluation on top of the let's say your Tex summarization model you can use this row score function okay this is the idea now here we'll be calculating our row score so here you are having actually Four kinds of row like Row one row two row large and row L sum okay now let me calculate the row Score first of all I'll be loading all the row matrixes I
have to give yes then I'll be calculating the score on top of my testing data okay now let me execute and again I'm only taking 10 example guys because again because uh it is having actually lots of data and it will take time that's why I have taken only 10 example but if you're performing actually so what you can you can remove this line that means you are performing on top of the entire data right now fine This is the small small hack you can follow if you're doing the experiment only now see guys this
is my row code I got now guys you can see I got the row score and this row score should be close to one okay if it is close to one that means your model is performing better so as of now we used our test data for the training and we only train one Epoch that's why this uh Roy score is not good okay but if you're using actual training data and if you train multiple EPO you Will see that you will get a good Roy score here fine now let's save the model my Pegasus
model so I'm using save F pen function and here I'm giving the name it will save my model so now if I refresh you can see my model has saved so this is my model Pegasus model my train Pegasus model then I also need to save my tokenizer because uh later on I'll be giving my uh let's say testing data and if I want to do the let's say Summarization task that time it will also convert that data to the vector representation now see this is my tokenizer I also save fine now if you want
to load any kinds of pretend tokenizer what you can do you can use uh Auto tokenizer from pretin now you can give your tokenizer location okay now see here I'm giving my tokenizer location not the tokenizer we downloaded from the hugging pH okay now let me load my tokenizer so this is my custom Tokenizer right now now this is the prediction code okay now to perform the prediction first of all you have to generate some arguments now you have to set the length penalty then number of beams then max length so you don't need to
change this parameter just keep it as it is only you can change this length panalty what is length penalty I think you remember so here we had one let's say interface like we can generate short output we can generate long output now If this parameter is close to one that means it will uh generate actually long output if it is close to let's say zero it will generate short output okay this is the idea now here you can see I'm taking a sample text from my test data I'm taking actually dialogue one uh dialog the
first dialogue okay you can see from the testing data I'm picking up the first dialogue let me show you so if I execute it here so this is the first dialogue I'm Taking from my test data and I'm Al also taking the reference that is the actual summary of it because I want to match this summary with my actual prediction of my model this is the actual summary right then what I'm doing I'm creating a pipeline I think you know what is pipeline in hugging Fist and here I'm telling I want to perform summarization right
now and I want to use my model the model train so this is the model train I'm giving the name of the model you can see okay Pegasus Samsung model and here I'm passing my tokenizer the tokenizer actually I loaded here F then I'm creating the pipeline now here I'm printing my sample text that means the text actually we have loaded this uh first dialogue as well as the uh reference that means my actual uh summary then I'm predicting my uh model you can see I'm passing this sample text to my pipe object okay my
Pipeline and Here I'm giving all the parameter I set here okay then whatever output I'm getting I'm only printing the summary text of it now let me show you if I execute this code see this is is my actual dialogue this is my actual summary now my model is predicting okay the prediction summary let's wait and let's try to match okay how much accurate it is now see guys I got the prediction now just try to see now you can see this is the Model summary that means my model has uh given me the summary
now see it's not very accurate but still it is close to because again I told you we only trained one oke and we used our testing data for the training okay that is why uh it is uh actually it is not giving actually good performance okay and if you want to increase the performance what you can do you can increase Theo size and you can test and you have to train on top of the Training data okay not on top of the test data okay this is the small modification you can apply so I hope
guys uh it make sense right now how we can actually Implement different different projects with the help of hugging face platform now okay that means I showed you how we can use different different model whether it can be LM model whether it can be llm model how we can use different different data sets okay and how we can use use the Hugging face pipeline okay to create your application so this was my main objective the proper use of hugging face okay because going forward we we'll be utilizing this particular let's say technique only to perform
all of the let's say project okay whatever project we'll be doing we'll be following this particular approach only and going forward I will be also showing you how we can use large language model so see now we are using LM model only okay Language model only but going forward we'll be using large language model that means we'll be using uh uh Lama model okay we'll be using Lama model from meta we'll be using let's say mral model okay we'll be using Falcon model different different like large language model will be exploring okay so I hope
guys this is clear now how we can uh perform this TCH summarization project now one task I want to give you see I already told you now we are having different different Data sets in the hugging phase so you can select based on the task let's say if I go to the task let's say I want to perform summarization I'll click the summarization now see these are the data related summarization now what you can do you can pick up any kinds of data set from here okay and you can perform that t summarization on top
of the data see here I used Samsung data but I want you to use some many other data okay so please try to attempt this task Because Unless and until you are not doing like a practical from your s it would be little bit difficult for you so if you want to uh actually generate image from the text so you have to use something called diffusion model OKAY diffusion model and what is diffusion model diffusion model is kinds of actually large language model it is available in the hugging phas okay and if you want to
use diffusion model you have to use one Library called diffuser okay diffuser so This diffuser will help you to actually load these kinds of diffusion model and you can perform these kinds of text image generation so again it is available inside hugging face only so you don't need to go to the any any other platform okay so additionally we just need to install another Library called diffuser and with the help of that we can easily access that diffusion model as of now I don't need the gradio here I can remove it so these are the
Library actually I have to install here so as you can see diffusers is the hugging face page for uh using the diffuser diffusion model from the hugging face Hub so here you can see let me show you the page actually I have already um given the link so see this diffusers is available inside huging face only so they have already uh written like how we can install it okay and how we can load the diffusion model and all every example They have already given here fine so I have followed this documentation guys and I prepared
one notebook for you and let me show you how we can perform this texture image generation with the help of different different large language model and here the model actually will'll be using guys this is actually multi model that means this model uses both kinds of architecture your uh NLP architecture as well as the computer vision architecture because here you Have to give the prompt that means text is nothing but your prompt okay and it will generate the image that means first of all it will process the text then it will process that particular image
okay image output so that's why it is using something called hybrid architecture okay so it is called actually multimodel I hope you cleared now let me install all required package I need here now let me import this stable diffusion pipeline for diffuses uh the Same way actually we used to load our pipeline I think you remember from the Transformer so here also we have to load the pipeline for the defur okay now I also need the matte plot Le because I want to visualize the image then I also need the torch Library now if you
want to see the version of any package you can execute this command keep show the package name it will show you the version okay now here I kept actually two kinds of diffusion model uh This is the model ID one and this is the model ID 2 you can see the first model this is the first model so the model name is uh dream like diffusion 1.0 so this is the model guys and it is available in the hugging fiz Hub okay see whatever models actually you can see in the model section this is called
actually hugging fiz Hub that means in the hub also you can upload your own model it is possible let's say if I click on the model now so inside this Model you can upload your own model let's see fine tuned one model and you want to share with the community you can also push your model here okay so that other people can download your model and they can use it okay I also tell you how you can push your model to the hugging pH Hub everything I'll try to show you even you can also upload
your own data set okay it is also possible here now this is the model guys you can read about the model and this is the few Results actually you can see they have given different different prompt and these these are the results they have generated okay this is one of the amazing model even they have also given the code s it how we can use it and all everything they have given now another model actually uh this one the model name is St diffusions Exel base 1.0 this is the model so this is the model
architecture high level architecture that means first of all you have to give The prompt and with the help of actually Transformer model uh what they will do they will just try to understand this prompt and they to try to perform the tokenization convert to the vector representation okay that means converting to the embedding then they'll be creating one latent Dimension 120x 120x and this latent Dimension will pass to the unit model if you don't know uh inside stable defion actually unit model is used so Unit Model will try to generate the image okay with respect
to the prompt user has given okay you can see last time getting the image output again they have already given the paper link and all if you're interested you can open it up and you can learn okay see they have also given the code example now let me show you how we can use this model so what I've done guys uh I'm loading the first model you can see uh St divion pipeline from preon I'm loading the First model and here you have to give some parameter okay that means TS uh data type t. float
64 and use uh save tensor is equal to two Okay you have to pass this two parameter now I'm loading this uh model inside my Cuda that means inside my GPU because if you load inside your CPU it will take lots of time right so that's why I'm loading everything inside my GPU now let me show you see it is downloading the model now here I have prepared one prompt guys you Can see this prompt dream like art a uh gri woman with uh rainbow hair traveling between Dimensions Dynamic poses happy soft eyes and narrow
CH chain uh extreme bouet Dy figure long hair straight down and tone uh qualy shot and baggy okay so this is my prompt actually I have prepared now you can give any kinds of prompt if you feel like okay you have to use this prompt you can give it here now this prompt actually I have to give my pipeline object that means inside my Model and model will give me the image Matrix okay so let me show you so it is generating this image right now I think currently in the chat GPT also it is
having one model called GPT 4.0 okay so this model also can generate uh actually different different image so see create an image for my presentation now if I click here now see it is utilizing something called GPT 4.0 see if I click here see GPT 4.0 they're using now it is asking uh just Try to give any kinds of prompt let's say if I give same prompt here let's see what happens see it is generating the image right now so they are also using these kinds of uh like diffusion model in the back endend U
multimodel model that means they're understanding The Prompt then they're generating the image okay see okay this is the output I got now let's see uh from my model actually The model actually I'm referring from the hugging face see if I uh plot this image right now you can see this is the results I got now see the chat GPT and see my results both are pretty good now you can also plot with the help of M plot Le it is also possible first of all I'm uh like uh printing The Prompt then I'm printing the
image okay now let me give another prompt a girl is sitting on a chair and she is accompanied by her tiger make Sure to keep it cinematic and color to be golden ID okay now let me see the output the same prompt you can give to the chat GPT as well let's see so here I think I got the results now let me plot the image see this is the image I got and see this is the image I got from my chat GPT so both are good guys now I think you got it how
we can use these kinds of actually multimodel as well okay if I want to let's say generate text to image Or image to text okay everything is possible now if I show you so if you just go to the hugging phas model section uh here if you click on the model now see here you are having different different task let's say you want to perform text to image gener generation so here is the task guys inside computer vision you can select it now see you are having different different model so here I was using diffusion
model you can also use any Other model okay see how that many of models are available around 34,000 model you can use any of the model okay any of the model and you can try so this should be your task guys I will uh leave it to you you can explore any kinds of model from The Hub itself okay only select this task text to image okay now let's see if you want to perform image to text that time select this one and you can see the different different even if you want to see the
example open it up they Have also given the lots of example here okay for snippit everything they have given you can learn from here so this is the best platform guys if you want to Deep dive inside hugging face fine now we'll be learning some uh parameters of the diffusion model like you can set some of the parameter okay uh let's say You Want U actually different dimensional image you want to let's add some more uh parameter in that image you can also do it for this I created a Function generate image so it will
take the pipeline object prompt as well as the parameter and whatever things actually will be assigning the parameter it will set inside Pipeline and it will render the image with the help of M plot l so let me execute see that many of actually parameter you can play with negative prompting n num inference St height weight okay number uh number of IM part prompt okay let to say this is another Prompt I have prepared as of now parameter I'm not giving anything it's empty now if I pause these are the thing inside my generate function
so it will give me the image so this is the prompt uh dream like beautiful girl playing the Festival of Color wrapped in traditional India uh at throwing uh colors okay this is my prompt now you can see based on the prompt it has given me one beautiful image okay see beautiful image it has Generated now you can give any kinds of prompt let's say number of inference step I have given 100 now let's see what kinds of input I get you can see the documentation of the like this one uh def diffusers you will
see different different um actually parameter we having there see this is another example now you can see both image are same I can't see any difference now let's see with any other actually Parameter let's say height I want to change actually different height uh height and weight so for this you can pass this height and weight parameter here now let me see so now you can see guys I got the results and in a different height and width right now so that's how you can play with different different parameter now let's see another parameter uh
number per image prompt that means if I want to let's say generate multiple Image that time I can give image number of image per prom two that means it will give me two image right now so here is the results now I'm getting two uh output from my image now let's say you want to get three you can give three here now let's also see the negative prompting so here I have added negative prompt ugly destroyed and low quality let's say this this is my negative prompt now let's see whether it is able to generate
or Not so right now you can see uh See this results and this results I think you can see the difference this is little bit uh low quality and the prompt I have given ugly destroyed and low quality so it is almost matching I think fine so yes that's how we can uh use this kinds of like multimodel and I can uh do the text to image generation fine now I already showed you we are having so many model guys just try to explore from your end some of the model and try to um like
Implement any kinds of projects fine so guys so far what we have learned so we saw like uh how we can use the existing model from the uh hugging pH that means let's say if you want to use any pre-rain model pre-train model you can directly use it so you have to use something called pipeline okay pipeline for this inside pipeline you have to mention the task you want to perform Okay let's say you want to perform sentiment analysis you want to perform let's say translation you want to perform let's say summarization you can mention
all else what you can do you can also do the fine tuning operation okay fine tuning of a pre-end model okay and I already showed you one fine tuning example like I did the tex summarization okay take summarization so there I trained on top of my custom data Now you can ask me when we have to use pretend model and when you have to do the fing operation so let's say whenever you are having a problem statement let's say uh let's say you want to do T summarization only take summarization only okay take summarization only
first of all try to use the existing model that means the pre-end model OKAY pre-end model let's say I showed you one model now Google Pegasus that CNN daily mail model try to use that model and try To see whether your model is able to give the correct let's say summary or not correct summary or not if it is able to give the correct summary then why you need to do fine tuning definitely not you can use this model as it is right but let's say if your data is little bit different let's say you
are using something called um let's say banking data okay banking data banking related let's say conversation you are having and this conversation is not working With that pre-end model that time what you can do you can collect your own data that means your custom data okay custom data and based on that you can f you one okay f t one pre model that means you are using one freend model on top of that you are adding some more knowledge then this model will be able to also work with the banking data as well this is
the idea only okay now I think you got it when you have to use the pre-end model and when you have to Do the F evening operation it can be for all kinds of let's say model it can be for the multi model it can be for the for the let's say LM model it can be for the LM model any kinds of model actually it can be applied okay so the model I showed you now right now this diffusion model this text image generation model let's say if it is not able to generate the
image the way actually you are generate giving the prompt that what you can do you can find tune this model it Is also possible you can also F tune the division model and how to fine tune again they have given all the let's say article here fine tuning everything they have given if you just go to their documentation you easily learn even I I will also show you in future okay how we can do the fine tuning of the large language model for this we'll be using another technique called p e f T okay that
means parameter efficient fine tuning what is PF I will tell you Parameter efficient fine tuning because what happens these kinds of large language model is very huge and I can't train this kinds of model actually on my uh this kinds of machine the Google collab actually I'm taking because it's a huge model huge parameter for this we have to follow this parameter equation fing process okay that time I'll discuss what is the use of that and how we can find tune those example on example and all okay everything I'll try to clarify As of now
I'm only the exploring like uh the service actually it is having so that you can get familiar with okay that are the things actually we can perform with this particular platform like how we can do text to speech generation with the uh llm okay uh with the help of this hugging face platform so again I have to install this Transformer library because inside that uh only I'm having all the let's say if I search for hugging fish so if I go to the model section so now Let's say we'll be working with the audios that
means text to speech now if I click on text T to speech see uh it is having actually different different model OKAY different different model actually it is having but the model actually I'm going to use this model called Sono bark small model so this is one of the Transformer Bas text to Audio model created by Sono okay now if you want to read about this you can go ahead and try to read about this particular Model even they also given the example even though they have also given the collab notebook how you can use
it see this the collab notebook they have given but so many things they have written so what I have done I just simplified this notebook and this notebook I created okay now let me show you how you can perform the text to speech Generation Now for this first of all I have to import the uh pipeline from the Transformer and here is the text guys so This text actually I'll be converting to the audio now see python is a high level uh uh general purpose programming language this is my text now in the pipeline I
want to perform text to speech I Have to Give and I have to give the model so here I'm giving my model ID this is my model ID bark small model then device is equal to CODA because I want to load my model inside my GPU now it will give you the output and output would be a uh numpy array so let me show You if I just execute this program see now if I print the output output would be a numpy array okay now I have to convert this numpy array to audio that me
audio frequency for this I'll be using one Library I python display audio this Library will be using and I have to give this array as well as the sampling rate okay so here you can see I'm passing my audio that means this array as well as the sampling rate now if I execute it will automatically convert This uh array to the audio now see if I play python is a high level general purpose programming language um see I think you uh hard of okay it is uh speaking that particular text now you can give any
kinds of text it will generate the speech for that fine now you can play with different different model guys I already showed you it is having different different model you can try with different different model OKAY U see I was using this model bar uh Small model you can also use any other model okay it's up to you so that's how guys you can explore this hugging face it's an amazing platform and it's like very powerful platform guys in the field of generative a or in the field of natural language processing if you don't know
about open a open is a platform for the generative a so with the help of open you can Implement any kinds of generative a application open provides lots of commercial large language model Like GPT series uh I think you already used hugging F right so this is the similar kinds of platform of the hugging F so hugging F provides all the model all the data set as free but uh provides uh actually everything as a paid that means you have to take their subscription so if you're taking their subscription that time actually you can use
their premium model apart from that openi provides some of the model for the fre access so that at least you can do Some experiment okay with this model but if you want to build let's say production grid application that time uh they are suggesting to take their premium model so guys through couple of video we'll be learning about this open and its platform so here we'll be learning all the services from the openai how we can use the entire openform platform so everything I'll show you here how we can generate open opena API key how
we can access like Different different model okay and how we can Implement different different application so let's open our open platform so guys if you want to open this open platform just search for openi login uh so you'll get the first website open platform so make sure you have one account if you don't have account you have to first of all create one account and whenever you are creating an account uh it will ask for your card so try to add your C so you will get initially Actually $5 credit here okay I think $5
credit you will get then once you use that $5 credits then what you have to do you have to pay for the services okay whatever Services you will be using whatever model you'll be accessing So based on that you have to pay now you can ask me how much let's say money it will take okay I'll tell you how much money actually it will charge um it will charge based on the token size as of now just try to see the like uh overview of This open then I'll tell you each and everything so here
you can see guys after login I'm getting to window is like chat GPT and is like API see chat GPT is also product of a open I think you already know right so this is my chat GPT and this is the product of open that means it is using GPT model you can see if I click here it is using GPT model OKAY GPT model so GPT has like different different model VAR like GPT 3 is there 3.5 is there four is there even Uh recently one model has public GPT 4 and this is another
like powerful model they have published apart from that it is also having some multimodel like uh uh it can generate text to image like Deli model is also there right then it can also let's say generate like text to text from a audio that means whp model they are using so it is having actually different different model I'll tell you what are the model it is available so first of all click on the API Part so now see this is the API interface but as of now I'll go to the playground see this is the
playground that means if you're opening this uh openi for the first time so what you can do you can test different different model here you can see but again I'm telling you guys you you have to first of all create your account and make sure this $5 is uh let's say credited in your let's say account okay otherwise you won't be able to use this uh playground Okay that means you want you won't be able to access this at the model so make sure your account is active Okay then after that you will be using
so whatever let's say chat operation usually perform with the chat GPT the same thing you can perform here okay so here you can select different different model you can see so by default it is GPT 3.5 turbo you can also take GPT 4 om mini GPT 3.5 turbo 16k okay that's how different different model it is available okay so here you Can give the prompt and here you can pass the system instruction that so I'll tell you about that what is system instruction so whenever let's say we are using GPT model we are having two
kinds of let's say API the chat completion API and other is like the completion API okay so both has the difference I'll tell you now here you can also perform the assistant related task let's say uh if you want to create your own assistant okay you can also use this service you Can also do the text to speech generation okay with the help of this openi you can also perform the uh completion operation that means sentence completion operations so you can write one sentence it will complete the entire sentence all its entire story okay so
all kinds of model it is available even it is also having Del model apart from that actually they are also having something called Del model Del is a text to image generation model if you are Giving any kinds of prompt it will generate image for you now it is in the chat GPT also they integrated this functionality let me show you so this is what I will select create an image now it is asking for a prompt okay it is asking from a prompt so let's give a prompt a dog is flying on the sky
let's see this is my prompt let's See now see it is generating the image so guys as you can see this is the result I got that means here I pass the text prompt and it is generating the image okay so like that they also have one model called Deli Deli to let me show you so Delhi Del 2 so this is one of the model uh I think recently they have also published Del 3 okay Del 3 is also there so this is actually text to image generation model you can give different different Prompt
based on that actually it will generate a image right so guys you can open this playground and you can test different different model okay as part your requirement now let's go to the dashboard so let's say you want to create uh like generate application without writing code so what you can do you can use there UI interface so from here actually you can create different different assistant system and all if you want to fine tune these are the Model that mean GPT model you can also fine tune from this these Services it is also possible
then it is also having batches Storage storage means uh it provides one vector database services so you can also use the vector database services to store your vector representation and no need to worry uh I will also discuss about this Vector database why it is required and why we'll be using Vector database with our large language model and if you click on The uses uh you will see like uh your uses limit that means let's say if you are have being uh $5 credit so how much you have used here so everything it will show
you here now you can also generate the API key I'll will tell you how to generate the API ke so with the help of this API key actually we will be interacting with our open a platform okay otherwise you can't use those are the model okay without this API see all the model actually they have hosted in Their server so you don't need to download this model in your system everything will be accessing through the API okay this is the best part here now this is the documentation guys you can see related all the task
they created the documentation let's say if you want to perform any kinds of task if you want to see the models like what the model it is having see these are the models actually it is having so GPT 40 GPT for mini gp355 Delhi I already told you About Delhi TTS that means text to speech whisper so you can see whisper is a model it general purpose speech recognization model it is trained on large data set of diverse audio and um is also multitask model that can perform multilingual speech recognization that means it's a speech
model actually even it can also work with different different langu languages okay even it is also having embedding model with the help of embedding model we can generate Vectors and why we have to generate the vectors I think I already told you now in the text pre-processing and text representation session so first of all what we have to do we have to uh let's say convert our uh text to uh embedding representation for our model let's say you want to F tune okay you want to F tune this openi model that time what you have
to do you have to use this embeddings okay to find you let's say you want to create any kinds of gener Application definitely you have to first of all generate the embeddings okay of that particular text whatever documents you are having so that's why embedding models are required we'll also learning how we can use this embedding model whenever we'll be doing the projects that time actually we'll be learning now it is having also some GPT based model you can see so these are the GPT based model okay this is called actually completion model uh we
can use Completion API only okay to use this model I'll tell you how to use that them and all see everything they have given okay everything they have given let's say if you want to do the fine tuning so there is a fine tuning guide they have also given okay so all information they have given in the documentation and this is the one of the best resources to learn this open if you want to let's say learn anything just go to the documentation they have the proper Explanation with the code sample as well okay great
now this is the API reference now if I click on the API reference you can see this is the API reference that means let's say if I want to use their uh let's say model and all how to call call with the help of API even how to call with the help of python C code okay everything they have given as an example you can see different different example they have given but you can see there there are so many things they have Written but I don't need all the let's say concept here so whatever
actually let's say concept you need to master this openi uh so I will show you each and everything that means we'll be learning about uh chat completion API completion API and we'll be learning another super important concept which is nothing but uh function calling okay open function calling so these are the thing we have to learn here so yes guys this is the overview and this is the Introduction of the openi uh I hope you cleared now in the next video we'll be generating One open API key okay then after that actually will be uh
doing some handson on this opena that means we'll be accessing different different model with the help of API key so to generate the opena API key so first of all just click on the dashboard and here left hand side you will see one option called API key just click here and now see I already created one API key Previously that's why it's showing but for you it would be completely empty now if you want to create a new API key just click on create a new secret API key now give the name let's say I
want to give let's say test one or anything you can give it's up to you now what you can do you can uh give the permission uh only read only permission or whether actually it's restricted okay you can give different different permission but I want to give permission uh for Everything let's say I want to read I want to write everything I want to do that's why I will be selecting all now I'll just simply create the secret key now see this is the secret key guys just try to copy and try to save it
somewhere and don't share the secret key with anyone otherwise they will uh able to access your account okay I remove it after the recording that's why I'm showing you now uh just click on done now see my your API has created that's Actually you can create different different API key and with the help of one API key you can also create multiple projects so there is no issue with that so guys I think you remember I was talking about this playground and I told you with the help of playground you can do different different task
and you can use different different model okay now see this model is having some of the parameter okay so these are the parameter is having so going forward Whenever we'll be accessing these are the model through python code we'll be also using these are the parameter right so the first parameter you can see called tempor so what is temperature parameter see temperature is a parameter if you decrease the temperature parameter close to zero that means you are telling your model just try to be stick okay with the let's say prompt whatever prompt user is giving
no need to take any kinds of risk And don't generate any kinds of random output if you're not sure try to give uh I'm not sure I'm not able to generate but don't give any kinds of random output that means you are creating you are giving the Restriction to the model okay it will only work uh with the true okay true generation okay true data generation okay this is what actually temperature parameter do that means you can decrease close to zero that means if you want to get a more stick output that Time you can
give close to zero and if it is close to one that means you are telling your model try to take risk okay whatever response you are generating it doesn't matter but try to be more creative here okay this is the idea here there is another parameter called maximum token maximum token means like you are setting the maximum tokens let's say let's say whenever you are giving any kinds of prom to the model and model is giving any uh let's model is giving Out output right and it is also a token okay it is also a
token and you can set the token limit like how how many tokens you want to get as an output so that time actually you can set this maximum token let's say maximum token is 256 it will give you 256 token output okay I'll discuss about these tokens so there is a token counter inside this openi with help of you can also count the token then there is another one called top P top p is Nothing but like how many let's say output you want to generate from the model how many response you want to generate
from the model model let's say top p is equal to one your model will give you only one response let's say top p is equal to two your model will give you two response okay and from the two response you can select which one is the best okay that's how you can play with these are the parameter so if you want to learn more about it you can simply Hover so they will give you the description of this parameter and all so these are the parameter actually we'll be using frequently that's why I already told
you here so guys uh yes this is all from this video so in the next video we'll be doing the environment setup like we'll be doing our local uh environment setup like what the tools and let's say service you need uh for this uh genv project implementation we'll be doing all the setup then after That we'll be performing the Hands-On okay of this open a platform uh let's say sometimes actually we'll be using cloud services sometimes actually we'll be using local setup okay so that's why for the local setup what are the tools and Technology
we need everything will be making the setup in this video so the first uh tools actually you need which is nothing but anaga okay anaga distribution so this will give you actually python interpreter so inside That actually we'll be uh creating the environment will be set uping the package okay and we'll be implementing our genbi application so if you have already anag in your system it's completely fine but if you don't have what you can do you can simply click on free download and you can download this particular anaga in your system so I already
install anonda so let me show you so this is my anaga guys so if you have installed successfully you will see These kinds of window like anaga prompt anaga Navigator if you're getting this kinds of window that means uh your installation is completed and the installation process is very simple only just click on next next next and install the tool okay in your system now the next thing you need which is nothing but visual code Studio why we need the V visual code Studio because uh see we'll be writing the code okay and to write
the code I need a code editor right that Means ID integrated development environment so either you can use pyam either you can use visual code Studio but personally I prefer visual code Studio that's why I'll be using visual code Studio in this particular course fine so again you can install with respect to your requirement whatever operating system you're using you can install this visual code Studio in your system now the next thing we need the git okay git is nothing but it's a let's Say client for the GitHub I think you know if you have
already familiar with GitHub so if you want to let's say comit any kinds of code in the GitHub what you need you need this git client okay and if you install the git actually will get one B terminal which is called actually git bash okay so sometimes actually we need this G bash because with the help of G bash will be executing different different Linux command because going forward we'll be implementing genv n2n Project so there I need this G bash to execute some of the Linux R command Okay so that's why make sure you
have this G bash install in your system fine and again you can um install from here just click on this button and it will uh download it then you can do the next next and you can install this G bash in your system so as of now these three tools actually I need for your local setup and once uh this setup is ready guys then I think we can start our Implementation so in the next video I'll show you the handson on this open a that means we'll be accessing different different model we'll be exploring
CH uh like completion API will be exploring completion API will be exploring function calling of the openi okay everything will be exploring one by one so as I already told you openi having two kinds of API that means if you want to use open model uh it is having two kinds of model one is like CH chat Completion model one is like completion model so we'll be exploring both of them so here you can see we are having different different model so these are the model actually related uh chat completion model you can see these
are the chat completion model and apart from that it is also having completion model so if you go to the base model section so I think there is a option called base model let me find so here's the base model guys zpt B model and this model is Nothing but it's a completion model okay it's a completion API now what is the difference between chat completion model and completion model I'll tell you so before that uh what I will do I'll just uh open up my visual code Studio and there I will uh do the
openi let's a package setup and we'll set our API key and we'll start our implementation so what I can do I can open up my local folder and inside that you can create a folder let's say I will create Open okay open demo inside that I'm going to open up my uh visual Cod studio so this is my visual code Studio guys let me Zoom then I will also open up my terminal you can also open up your terminal G bash okay whatever you are using you can open it here see I have opened up
my G bash you can also open up your anagon The Prompt whatever actually You are using whatever teral you are using you can open it here okay so I have integrated my anaga with my G Bash that's why you can see B has been activated okay you can also integrate anaga with your let's say G bash for this you can simply search on Google you will see the uh like process okay how we can set your anaga with your G bash otherwise you can also use your anaga directly okay it will also work okay anything
you can use I'm using my G bash Here fine now here the first thing uh we'll be set uping our open Package see if you want to use openi so you have to install one package actually so if you go to the documentation let me show you so this is the open a so if you go to the quick start section so first thing what you have to do you have to export your open IPI key that means you have to first of all collect your open IPI key and you have to add it there
okay so we have we have already collected our open IP I think you remember then second thing what you have to do you have to install one package called uh openi okay with the help of openi package you can access all the model that means you can access all the let's say chart completion API completion API everything you can access here okay now see you can access access different different model okay you only need to give the model name and you can access the model f so let me first of all install this package So
I'll go to my vs code and here I'll create a file called requirement okay requirements. txt so the first pack actually I'm going to install called openi so here I'll be installing one specific version of the openi that's 0.28 why 0.28 because this is the stable version so uh see openi has different different version even currently they have updated their package so in the updated package what happens actually so Sometimes actually we will get the error so that's why I'm using one specific version you can also install the latest version it will also work fine
now the next package actually I need pandas why pandas because I want to list down all the let's say open model through my code and I want to show as a data frame that's why I need this pandas library then another Library I need called python. whyb because I need to manage that secret key okay secret API key Whatever API key you have collected from our opening right dob so with the python. package you can easily manage this API that means we'll be creating a DOT EMV folder okay inside that we'll be writing that open
IPI key okay this is the idea so here let me paste my open IPI key guys so this is my open API key guys I have already pasted and here you have to give this particular key name and here you have to give this key name open API Key equal to you have to pass your API key here whatever API key you have created okay I think remember in the double quotation now let me save F now let's me install this requirement so guys before installing these are the package first of all you have to
create a virtual environment here so to create the virtual environment you can execute this command cond create ipenn give the name of the environment let's say I will give um open AI okay open a demo this is my environment name and uh I have to specify my python so here I'll be using Python 3 point 10 and hyen Y and why 3.10 because again 3.10 is the stable version of the Python okay that's why we are using because it supports all the library okay and hypen y means I want to create I'm giving the yes
permission now see if I execute the command it will create the 3.10 environment uh inside my anaga then after that we'll be Activating then we'll be installing all the requirements there this is the idea so guys as you can see my environment is created now I have to activate it so to activate this is the command cond activate open AI demo okay it should be activate sorry H now see my environment is activated now I'll be installing this package so I'll just write keep install ienr requirement. txt so it will install all the packets One
by one here so as you can see my installation is completed now let me clear my terminal now here first of all what I will do I'll create a jupyter notebook file I'll just create uh click on on new file and I'll create a file let's say open AI demo 1. ipy NB okay ipnb file is a jupyter notebook file and here you need to select the Kel I'll select my python environment so the environment I created Uh called open demo so let me refresh see open demo is there I'll select it now if you
want to test it just give the any kinds of let's say code here let's say print hello it should execute now see guys it needs this uh IPI Cal package to run this uh jupyter notebook so let me install I'll click on install so it will install all the necessary things uh it needs to run this jupyter notebook after that you will see that this code would be executed here so Let's wait so as you can see all the necessary package has installed and now I'm able to execute my code in my jupyter notebook fine
all right now we'll be importing some of the library here so let me import operating system then I need my openi then I also need so to import theb you have to import like that fromb import load EnV okay load. EnV now see if I execute it will import all the library now if you want to check any Version of any package simply you can write this command so P show let's say I want to see the open version I'll give open a here now see it will show you the summary of the openi openi
name version summary okay homepage author everything it will give give you okay now the first thing what I have to do I have to set my open API key if you check the documentation I showed you the documentation this one this documentation if you check the Documentation so the first thing what you have to do you have to set the Opia API key okay and if you're using python uh program so that time you can set the open key like that so just open call your openi package and inside openi package we have one attribute
called openi API key okay API key so equal to uh first of all what I have to do I have to load my API key so to load the API key and to load this uh API key from the EnV folder what you have to do you have To take the help from load EnV package we have imported right so just try to write load. EnV okay now after that just write this one o dot uh G en EnV Okay g EnV inside that you have to pass the key name so key name is nothing
but this is my key name open a API key I'll pass it here okay and it will give you open a API key I'll store it here now see if I simply print it now see it should load the API key now see it is loading my entire API Key fine now let me comment it as of now I don't need to show now this open AP ke I'll pass it here now see if I execute this code so it has set my open API key okay inside my environment okay you don't need to do
anything that means openi will automatically load this API key right now from the package itself okay now as I already told you open is having different different model so if I click on the model section you you will see different different models is there So if you want to list down all the models so what you can do you can use this code snippit so open AI dot sorry openi do model do list okay it will give you the Entre list of all the model now see if I execute now see it is giving me
all the list of the model it is having but I want to only see the I want to only see the data Okay so here I'll filter out The data now see these are the model now if I want to show as a data frame what I can do I can use pd. data frame I have to import pandas first of all so import pandas as PD now PD do data frame and inside that I'll pass this data whatever I'm getting fine now see if I execute it will give me as a data frame now
see all the model ID object created date as well as the Owned buy okay see everything it is showing now see you can see different different model now as I already told you there are two kinds of API you will get uh one is like uh chat completion API other is like completion API so here you can see we are having different different model so if I want to use these are the model first of all we have to know what are the models actually uh my uh chat completion model and what the models actually
only completion model That means when I have to use the chat completion API when I have to use the completion API so for this you can simply go to the model section now let's say you want to use this GPT 3.5 turbo if I click here now see this model is a chat completion API model okay if you want to use this model you have to use the chat completion API and if I go to the base model section so this is a completion API that means you have to use completion API for this model
okay Now let me show you some example of chat completion and completion API so first of all let's try to see the completion API demo okay let's say I want to use this GPT 3.5 tarbo model GPT 3.5 TBO model see so again if I go to the documentation so here is the model guys GPT 3.5 TBO I have to use chat completion API and how to use CH chat completion API if you click here that means you have to use uh chat. completion okay do create this Particular Cod cipit so let me show you
you can also write like that so here uh I'll just call uh open AI dot uh chat completion there is directly one function you will get called chat completion then you can call this create okay create one and inside that first of all you have to give the model name there's a model is equal to I want to use which model GPT 3.5 turbo so this is the model so you have to give the name Like that fine okay now the second parameter you have to pass the masses okay and message should be a list
here now see how to write the masses so they have already given the example that means the first thing what you have to give you have to give the role so Ro you are giving system okay system you are a helpful assistant that means you are giving a system prompt here I think you know like large language model is all About prompt okay you have to give the prompt here then you are giving the user prompt that means what whatever user want to ask so user will give it here so let me show you how
we can pass this prompt so the first thing what I have to do guys I already written so the first thing you have to give the role as a system that means you are so here the first role you have to give as a system that means you are giving the system prompt and what is the content of the System prompt you are a helpful assistant you can give any kinds of prompt here let's say you are a helpful assistant you are helpful translator you are a helpful let's say uh chat bot anything you can
give the prompt here so going forward we'll be also learning about prompt engineering there I'll try to clarify what are the prompts actually you are having okay different different prompt you can set here now the second role you are giving as a user that means Whatever user will ask okay whatever user will ask so if you open the chat GPT also in the chat GPT back end they have given the system prompt as well as the user prompt that means as the user we give our prompt in the chat gbt but in the back end
they have given the system prompt okay in the chat gbt that is the idea fine now let's give one prompt as a user here so let's say prompt is equal to prompt is equal to I'll give uh let's Say hello how are you so this prompt uh here it will come okay as a user prompt now it will give me one response and that response I'll print here simply okay now see if I execute the program so it will give me the response so it is giving invalid request at missing required parameter message you provide
message okay so let me check So it should be messages okay not messes now see it should work now see it is giving me the uh response right now and here is the response guys okay if I show you so here is the response hello I am here ready to assist you how I can help you today if I want to extract this content only that means this uh let's say response only what I can do so simply I can copy this example as it is and here let me paste so here you can see
first of all I have to go to the Choices okay because it's a dictionary I have to take the choices so let's take Choice first of all I'll take the choice now choice is nothing but it's a list so I will take the first let's say item of the list now I have to go go inside the masses now I need the content only now see it will give me the content only okay hello I'm just a computer program so I don't have any feelings but I'm here to ready to help you how I can
Assist you because every time whenever you will execute this uh program it will give you different different response okay it will generate different different uh response for you this is the idea fine now what is the main benefit to use this chat completion API that mean chat completion model see in the chat completion model you can pass multiple prompt okay here you can pass multiple prompt so how let me show you so here I have given one example so this Is the example guys see here I have created actually three prompt hello how are you
second prompt I am uh 25 years old I'm a programmer now third prom I've given tell me about me that means in the prompt itself I have given my data and I'm asking tell me about me now this the same way I have created my chat completion API I have initialized my model and in the message section right now you can see the first role actually have given the system role that means You are a helpful assistant second role I have given my first prompt third role I have given my second prompt and fourth role
I have given my prompt three okay that means you can pass multiple prompt here this is the main benefit to use this chat completion API that means in the chat GPT also so if I go to my chat GPT let's say here I give my data hey I am let's I'll give the same prompt here this is the prompt I want to give I'm 25 years old I a Programmer now you can see memory updated it it has memory updated right now if I give this prompt tell me about me now it is telling you
are 25 years old programmer who created courses and playlist okay see now it is giving me the answer the related my information because it it has remembered okay it has remembered my previous prompt so this is what actually they're using something called chat completion model that means It can understand multiple prompt okay not one prompt it can understand multiple prompting so here you can see I have given one demo so I'm giving the same thing hello how are you I'm 25 years I'm a programmer tell me about me now see if I execute the program
right now it will able to give me the answer see based on the information you provided you are 25 years old programmer is there anything specific you would like to know or discuss I hope it is Clear now what is the use of chat completion API and and what is the benefit to use the chat completion API model that means chat completion model got it great now we'll be tweaking some of the parameter here so the first parameter will be tweaking which is nothing but mix Max token okay so what I'll do I'll copy the
same example my previous example now instead of asking this question I'll write what is python let's Say this is my question okay now there is a parameter you can set called Max token maxcore token let's say I want to generate only 25 tokens okay okay 25 tokens now see if I execute the program right now it will only give me 25 tokens output now if you want to see that you can copy and you can go to the open AI Token okay tokenizer now go to the first website now here you can simply give it
this particular let's say uh response you have copied now you can see maximum token 25 that means whatever token limit you have set so it will give you that particular token output only now if you want to see the detailed information you can simply print the response instead of printing the content now see so now if I execute this program you'll see that my completion token 25 Fine now it will charge you based on the token limit so if can go to the openi pricing so in the pricing section they have given how it will
charge you so so guys as you can see this is the openi pricing page so here they have given the pricing as per the token limit you can see this is the input token this is the output token that means if your input token is that much that means 1 million and if your output token is let's say 1 million so you uh you will charge that Many of dollar okay that much of dollar so you can see this different different example so this is GPT 40 mini this is the embedding model you can see
if your input token limit is 1 million token and if your output token is and if your output token is also 1 million tokens so he will get 0.020 okay $2 only it's like very less amount now see different different model with respect to their different different price price okay so you can Check all the models see Audio model as well as the image model assistant API okay see whatever assistant API actually you will be using and Vector SS okay every model they have given the pricing okay every model they have given the pricing so
that means it will charge based on the token limits let's say whatever token it is giving you the output as well as the whatever tokens actually it is receiving as an input it will first of all do the count operation How it will do the count operation so with help of this tokenizer so they have one inbu tokenizer inside that it will automatically make the count now you can see I'm having total 141 character now you can see like how many character they are considering to calculate only one token I think four to five okay
four to five character they considering to count one token here okay this is the idea so here they have written you can see a helpful rule of thumb is is that one Token generally correspond to four characters okay uh whether it's a four character or sometimes it would be more than four character of text for the common English text okay this is called actually one token so they have given each and everything you can export here now let's get back to my code editor now I think this parameter is clear okay what is Max tokens
here now I think you remember in the playground also I showed you this parameter so if I go to the Playground see in the playground also I showed you this parameter Max Tok okay now it is I think it is clear now I'll show you the next parameter right now so the next parameter is nothing but our temperature parameter so you can see I have given the same example but uh one parameter I'm using called temperature and I already told you what is temperature parameter means uh it will give the randomness okay Randomness To the
response like how much Randomness you need let's say if it is close to zero your model would be more stick to the let's say prompt you have given and if it is close to one that means it will try to take some risk and it will generate some random output as well now let's say here I have given 0.6 that means I'm telling my model uh try to be actually balanced okay balance with the prompt user is giving try to give some creativity some somehow even U also try To be strict with the prompt user
has given now see if I execute the program now see now the response actually I'm getting this is the uh this little bit different response from the previous response you can see so here is the response I was getting python is a high level inter programar Simplicity and visibility it supp multiple paradigms okay now see this is the another response actually I'm getting okay that's how you can increase And decrease this parameter size but what I saw like people are using this parameter around 0.5 to 0.6 to 7 like that okay no need to decrease
and no need to increase much but if you sometimes need it you can increase otherwise you can decrease okay it's up to you now there is another one that n parameter uh n parameter means it will give you like how many response you need from the model let's say if n is equal to two it will give me two response now If I if I execute the program see it has given me two response so this is one response and this is another response now if it is three it will give you three response see
one response two response and if I open in a text editor two response uh this is one this is two and this three okay that's how you can get multiple responses if you want so this is called actually n parameter like number of response you want to get here Great so yes these are the parameter you can play with guys because these are the parameter we'll be using frequently now see uh with the help of chat completion model you can perform different different tasks so let me show you so you can give different different prompt
so let's say here I have given a prompt give me a sentiment of this sentence okay this movie is amazing now I think you know this sentiment would be positive now let's see whether my model Is giving me right answer or not now see this sentiment of uh of the sentence this movie is amazing it's a positive that means it's working fine nowart from that you can also give any other prompt let's say I'm given give me the Hindi translation of this sentence this movie is amazing now see it should give me the Hindi translation
see so whatever things we used to perform in the chat JP now okay that means we can perform multiple task I want to do language Translation uh then sentiment analysis anything I can perform here the same thing you can also do here okay because it is using the same model only okay and we able to access those model with the help of API key that means whenever you are uh sending this request uh it is first of all calling that API and API is actually hitting the model and model is giving me the response and
this response actually we able to see this is the idea only I hope You're clear now let me show you another prompt let's say detect the language of this sentence this movie is amazing so the language should be English see it is the English uh language okay now you can also generate code okay you can also generate code uh with these are the model so I've given a prompt give me a python code to add two numbers now let me see see this is the code I have given to add so here I got the
code to add two Numbers in Python okay so that's how actually we can create different different application with these at the large language model fine so yes guys this is all about our chat completion API now let's try to export the completion API as well so as I already told you so here is the definition of completion API guys you can see uh hooks you up with text comp comption from a single prompt that means it can only support single prompt you can only pass One prompt here in the other hand actually in the chat
completion you can give multiple prompt to keep the conversation uh flow intact okay as I already told you you can give multiple prompt and that prompt actually this model can remember but in the completion API you can only pass the one prompt here okay this is the idea in completion API you can pass only one prompt now if I show you some completion model model section and let's say this is the GPT Base so this is use actually completion API and these are the model are available related completion API okay now let me use this
Babs 002 model and let me show you one example see if you want to use completion API that time you have to call this completion open Dot completion okay open. completion. create not chat completion you have to call completion only and inside that you have to give the model First of all which model you want to use let say I want to use this Babas model that babz model okay and it will take the prompt so let me Define an prompt here let's say this is the prompt okay what is python this prompt I'll pass
it here now I'll print the response okay now if I execute now see it is giving you the response okay it is giving you the response so that's how actually you can Give uh actually only single prompt here and whatever let's say parameter you saw temperature then maximum token then n parameter you can also give the same parameter in the completion API as well okay it will also work but uh see we'll be using the chat completion API a lot we'll be using chat completion Model A lot going forward okay because I have seen like
people are using this chat completion Model A lot instead of completion API because here we can pass Multiple prompt okay this is the main advantage okay I hope it is clear so yes guys uh this is all about our chat completion API as well as the completion API and this is all about a Hands-On part on the open a platform okay because see going forward we'll be using these are the concept only to implement any kinds of project even I will also Implement some of the project I'll tell you how we can Implement different different
projects with the help of Openi that time also you will see I'll be using these are the concept only okay now in the next video we'll be learning about another super important concept inside openi called function calling okay again it's the very powerful concept inside openi so we'll be learning that function calling see first of all you have to understand what is function calling and what is the use of the function calling so for this I will go to my Blackboard and there I'll try To clarify okay see function calling helps the open a model
to interact with different third partyy API let's say here you are having one API let's say here you are having one API okay let's say you are using uh one databases okay databases so let's with the help of API you are hitting the database and database is giving you some kinds of response which is nothing but a just on response because most of the time Whenever you will use API right API to hit any kinds of website or database you will get the juston output let's this is a weather okay weather data database and with
the help of wether API you are hitting this website and it is giving you the Jon response that means wether related data now what you can do with the help of function calling okay with the help of function calling inside open a you can use open a API open AI API okay to interact with this model because let's say this is my large language model OKAY openi model now with the help of open API key what I can do I can interact with this model that means with help of function calling what it can perform
okay it can perform the Comm communication with this data communication with this data that means this model can directly communicate with this data that means whatever response you Are getting whatever response you are getting from this database or let's API you can directly communicate okay you can directly communicate communicate uh with the help of your large language model okay this is the main power actually this function calling will provides okay that means you don't need to manually extract this information and manually let's say feed inside your model you don't need to do like that you'll
be Using function calling and what your model will try to do it will directly let's say communicate with this particular response whatever just on whatever response you are getting it will directly do the communication okay so that's why with of function calling my large language model will be capable enough to communicate with any kinds of third party API okay third party API this is the main benefit to use this one and without this function calling uh What you have to do you have to manually extract the data and you have to manually fine tune this
model OKAY manually fine tune this model and again it's a very hetic task like because again it's a large language model and you can't do the fine tune okay easily okay and again it is having lots of cost involvement so that's why open uh this function calling helps us to do this communication that means instead of like feding the data in my model directly Function qu calling will try to uh make the communication between my response of like API response I'm getting and this particular model okay this model will try to communicate with my output
okay it can be any kinds of API it can be not only weather API it can be any kinds of API any heart party data okay so that's why openi supports actually different different uh third party actually integration you can U connect like different different platform you can Connect let's say slack you can connect let's say different different uh uh website you can connect different different database okay directly you can connect and it can directly to the communication with the help of function calling okay now let me show you one example I think then this
part would be more clear so what I'll do I'll go to my code editor and here I already created one file you can see now we'll be exploring the function calling here Again what I will do I'll import all of the lid then I'll select my cardal and I'll set my environment variable H now I can use the same example from here just to test whether everything is working fine or not I can copy see everything is working fine okay there is no issue instead of printing all the response I can only print the content
so as I already told you uh it Can actually communicate with any kinds of third party API so as a third partyy API what I will do I will use one API here so called rapid API okay rapid API inside rapid API you are having different different API whether it's a weather API okay and you API it is available so this is the link so let me open this link guys I will search on Google so this is the rapid open.com and make sure you have created One account so for me I already created One
account Okay for you also you have to create one account now here what you have to do you have to search for the API now simply left hand side you can click on view all categories now I need actually wether API so I'll simply search here wether now see different different weather API is coming but I need this AI weather by Metro Source okay this this one I'll click here now see initially if you're doing it for the first time you Will see one button here subscribe okay subscribe to the API you have to do
the subscription that means you have to take the basic plan that free plan okay you can also take that premium subscription but you can take that free plan free plan actually will give you some of the a request you can hit right now I already do the subscription that's why it's showing me test endpoint now simply what you need to do you need to copy this code snippit copy this code snippit And with the help of that I have written a function let me show you so this is the function I have written so the
function name is uh get current weather okay so it will take the location and it will return you the current weather in that particular location okay see this is the U that means the same code now here see the same code actually have copy pasted same code I have copy pasted you can see you can see the same code I have copy pasted And this is my API key rapid API key and this is my rapid API host and don't share with anyone otherwise they will also able to access your account okay I'll delete it
after the recording now with the help of request package I'm sending the request and it will give you the response and that response actually I'm doing the returning and as a Json format because I already told you now there any kinds of API will give you as a Json response and that particular Json Response I want to uh do the communication with my large language model okay this is the idea now I think yeah everything is fine now let me execute this function now if you want to test what I can do I can call
this function inside that I can give any location let's say I will give Delhi and it will give me response now see the place ID uh area India latitude longitude okay see all The information it is giving me okay all the information it is giving me uh related Deli you can give any kinds of place okay it's up to you you can give Bangalore Delhi Mumbai okay anything you can pass here now if I show you the response so this is the response okay this is the response I'm getting as it just on fine now
what I will do I'll just connect my large language model with this response okay with the help of function calling for this you need to Create a functions see this is the function format you have to follow it's a uh it's actually you can see again it's a kinds of Json format only so you can see I have written a list inside that I have mention a dictionary only okay so this is called actually function again if you go to the documentation if I go to the documentation let's say documentation so they have one function
calling you can also start here function calling see whatever things I explained They have written here also now see this is the format you have to follow okay this is the format you have to follow that's how you can write the function calling here so I already simplified this thing and I written a function okay so similar like format you can also follow now first thing you have to give the name so current get current weather so I have written my function name here you can see this the function name get current weather that Means
this will hit that particular function because this is my third party API okay you can also consider database here but as of now I'm considering the function now this is the description get the current weather in given location parameter object properties location string and the final thing required location that means it needs the location okay if I if it needs to give any kinds of output first of all it need the location because my function takes The location out input okay I hope it is clear now let me initialize my functions now I think remember
whenever I was using this chat completion API okay chat completion model that means I can give multiple prompt I can add multiple prompt okay in that particular let's say um message section so that concept actually will try to utilize here see here I have given an example let's say this is uh here I'm using chat completion model and I have to prepare The message function I think remember this message function inside that I can give multiple Ro right that is multiple prompt so what I'm doing here I'm appending one prompt here you can see user
masses masses. append because it's a list I can append roll user content user message okay now it will add this particular prompt inside my message now see if I execute now if I show you the now see if you want to see the output see this is The output hello how I can assist you now if you want to see the message only now see in the masses one prompt has been added okay this is the user prompt has been added that's how I can also add multiple prompt again what I can do I can
do the append operation see again I have taken another user message what is the temperature of Del now I'm appending in the message now roll user content user because this is a user okay user message only now again I'm adding the message and I'm also adding the function the function calling I have defined now here this particular function calling so with the help of this function it will try to understand okay I need to execute this get current weather okay this particular function and this particular function will return return the Json output that means the
data data related by that and it will automatically connect the large language model in that particular data see the Same diagram I showed you now here it will give you the response and my large language model will connect through function calling okay now let me show you so if I execute it now if I show you the masses now see two masses has been added two prompt has been added hi there now second prompt what is the temperature of Del you can also ignore this prompt I've just only given because I just wanted to show
you okay whether you can add any new prompt Or not okay this is the idea now if you see the completion that means the response see this is the response you got see as of now it is only calling the function you can see it is only calling the gate weather function okay and here you have given the location of Dilly you can see it is automatically extracting the location from your prompt okay you can see location D you have given okay now if you want to get the information what you can do now if
you want to get The actual information what you can do let me show you let's say say you want to see the masses you can also see the masses this is the massage it is calling my function and this is my location now this is the response okay this is the response now if you only want to extract let's say the function name what you can do you can extract like that so it is inside function called name okay and it is my get current weather function And if you want to see the location so
what you can do insert function called insert argument location you can see argument location I'm extracting the Delhi here see this is my location okay now this response actually I'm getting I need to pass to my large language model for this you can execute this code uh message. append response the response actually I'm getting I'm again adding to the message that means as a prompt okay as a prompt actually I'm adding I think Remember here now um here I showed you you can give multiple prompt this prompt this prompt that means I was providing the
data Based on data I was asking the question the similar things I'm doing the response actually I'm getting this response entire response I'm getting now this is I'm passing as a prompt okay as a prompt data and on top of this font I'll be asking the question okay with the help of function calling see the same thing I'm doing appending the Response message. append now I'm appending the role as a function name function content location okay this thing actually I'm passing one by one if I show you the entire masses see this is my entire
masses right now now I can initialize my gbt 3.5 tarbo model inside that I'm passing my message as well as the function calling now it will give you the accurate response see the prompt you have given um the current here is the prompt you have given I Think remember what is the temperature of Dilly now see this is the temperature of Dilly right now okay and it is coming like that because it's a 32 uh 32° cius so degree is a special character that's why it's coming like that okay you can convert to special uh
see you can convert this UN code to special character okay it is also possible now see here I have executed actually line by line that's why you can see you can Also execute as a snippet of the code it will give you the complete response now see instead of giving DHI I can give let's say Mumbai now let's see the Mumbai temp see uh it is not able to fetching the Mumbai temperature currently so what I can do I can give any other location let's say I give U as of now I can comment this
two line it's not required I'll give let's say Bangalore now see this is the current temperature of the Bangalore okay so that's how actually you can uh uh uh give any kinds of information any kinds of API you can connect with your large language model with the help of this function calling technique and it's like very powerful technique guys trust me it's like very powerful technique okay if you're uh interested learning more about it like what are the things you can connect you can go to their Documentation okay they have written each and everything what
are the things you can connect with it here right okay so yes guys this is all about from this video I I hope you liked it now in the next video we'll be implementing one projects whatever concept you have learned so far uh the project name is telegram bot okay telegram bot with the help of openi that means we'll be creating a chat bot and we'll be integrating with the telegram okay so All the fundamentals we have already discussed now it's time to implement one projects uh with the help of openi so in this video
I'll show you how we can Implement one telegram chatbot with the help of open that means uh here we'll be using one chat completion model that means one large language model we'll be using called GPT 3.5 uh and with the help of that actually we'll be implementing one chatboard and we'll be integrating this chatboard with the Telegram platform okay so make sure you have the telegram guys uh it can be a smartphone telegram application or it can be your desktop application as well anything you can keep both will work okay so here I'm going to
use my desktop telegram you can also use your smartphone telegram it will also work fine so before starting the implementation guys first of all let me show you the architecture diagram like how we'll be implementing the entire Projects then after that we'll be doing the setup and we'll be starting the implementation so guys let's say uh this is the user that means you so you will be asking one query okay you'll be asking one query and this query will go to the front end okay front end of our application so here front end wise we'll
be using something called telegram fine then what will happen uh this s will go to the back End back end means our openi openi API okay open a API then this API will access the large language model that means here will be using GPT 3.5 okay you can also use any other model like GPT GPT uh 4 you can also use GPT 4 or it's up to you okay I I I think I showed you how to like select different different model there is a model list so if you click there you will see the
model ID so that particular ID you have To write there okay and this model is I think you already know this is a chat completion model okay why that completion model because it can take the instruction that means multiple instruction you can give multiple prompt you can give okay you can also create with the help of completion model like the babz one okay I already showed you uh which will also work but what I feel like this chat completion model will be more powerful okay this is the idea here Now it will happen this uh
uh large language model will give the response so where so where it will give the response it will give the response to the backend API that means your openi API and openi API will send the uh let's say response to the front end okay front end means our telegram okay and from telegram actually user will again get the response response let's this is the response and this is the Question okay so that's how actually we'll be implementing the entire projects okay uh this is our actually high level architecture of our project now what I will
do uh first of all uh I just need to get my open API key and I already told you how to get the open API key so make sure you already created your open API key and you keep it with you now in my computer I'll just create a folder here and I'm going to name it as uh Telegram chatbot okay telegram chatbot and inside that I'm going to open up my visual code Studio H then I'm also going to open up my terminal here you can also open up your um command prompt then you
can also open up your anagon prom it's up to you now here the first thing what I have to do I have to create one virtual environment so to create the virtual environment just write cond create tyen in uh give the Name of the environment let's say this is my t Okay telegram so I'll just write tbot uh specify the python version let's say python is equal to 3 let's say 8 okay I'll use it 3.8 uh you can also use 3.9 but let's use 3.8 then H can bu um yeah now let's create the
environment so my environment is created now let me activate just write cond Activate teleport now let me clear now uh what I will do I'll just create a requirement. txt file requirements.txt so inside that I need to mention all the let's say library I need to implement this project so here you can see um front end wise actually uh I'll be using telegram okay and to integrate my chatbot with my telegram I need a python Package that means this package will communicate with my telegram so inside telegram will be generating one uh let's say API
token so with the help of API token we'll be communicating with our telegram app right so for this actually we need one python package so let me show you this package actually the package actually I'm going to use so guys as you can see this is the package actually we'll be using uh the package name is AOG so as you can see a is a Modern and fully ason chromos uh asnr framework for telegram B API and this is the complete documentation of this AI even they have also given the sample code like how we
can connect with theam and all okay they have discussed each and everything uh but you can see they have written actually too much line of code but uh that many line of code actually is not required so what I've done I just uh made it uh like simple and I created one Basics template for You if if you want to connect with your telegram okay so how you can connect for this uh what kinds of code you have use I already prepared one for snippet okay I'll share with you so first of all let me
mention this package name inside my requirements so so the first package I'm going to add called a fine now the second package I need open and I'm going to use the same version of the openi I already used in My openi Demo then I need another Package called python uhv why python. EnV because I already showed you if you want to manage the secret key so you can use this uh python. to load that secret key from the environment that means EnV file okay this is the idea now let me save now what I will
do simply I'll install everything in my uh environment so for this just write this command so P install tyen at requirement. txd so guys you can see my installation is completed now let me clear my Terminal and now first of all we'll be testing our uh this one telegram whether we are able to make the connection without telegram or not so for this I'm going to create a folder here called resarch inside that I'm going to create a file I'm going to name it as eobot eore bot so here we'll be creating one uh Eco
actually Eco functionality that means if you're sending any message okay to the let's say chat that means to your telegram chatbot it will give you the Same message okay this is called actually eobot that means whatever message actually will be giving as an input that message actually will be getting as an output okay so that means I just wanted to experiment whether I'm able to successfully connect with my telam or not okay this is the idea here but before that I just need to create this EnV file now inside that first of all you have
to uh write your open API key so Let me just write my open API key so this is my open API key I think you remember I already created this API key okay now let me save now just open up your telegram app so here I'm using my desktop telegram so you can see this is my desktop telegram to search for BFA so after searching BFA you will see different different B further but you have to take the verified one you can see this is the verified one now just try to open it up see
I already opened Now see uh previously I already created some of the bot with the help of Bot F that's why it's coming um that's why here you can see some of the uh like older masses but if you're opening for the first time you won't be seeing any kinds of masses okay you will see a start button you just click on the start button okay then it will give you the suggestion okay what you want to do here so here I want to create a okay I think my screen is Visible okay so here
I want to uh let's say create a new uh actually telegram bot so for this what you can do you can write this command new Slash new okay new bot you can see new bot just select this one now see now it will ask you all right a new bot how are we going to call okay please choose a name for your Bot now you can give any kinds of name here so let's say I will give U my live okay my live Bot let's say 24 let's say this is my bot name I want
to give now if I hit enter see it's telling good now let's choose the username for your Bot it must end with bot like uh this is example ter bot or terore bot okay so you have to give a username and that username should contain underscore bot okay at the last so I'll give the same name let's say my live bot okay 24ore bot now if I hit enter now see my Bot is created now it is telling congratulations on your new bot you will find your Bot here but before that what you have to
do you have to collect this user uh you can see uh use token so let me collect this use token I'll copy okay and I'll open up my visual code Studio inside environment what I will do so here I'm going to paste it as of now and you need to also give the key name that means this is my Telegram bot token okay now I'll be mentioning inside my string so here we have collected two secret key one is my open API key another one is telegram bot token so with the help of telegram bot
token we'll be authenticating with our bot actually we have created here you can see this is my bot if I click on the URL right now see this is my bot now if I click on the start now see there uh you won't be seeing anything because this is not connected yet now see if I Send any message here so you won't be getting any kinds of response because as of now we haven't made the connection so here first of all we have to make the connection and how to make the connection with the help
of this telegram bot token okay and this open API key we need to access our GPT model because this is my main large language model right with the help of that actually we'll be getting the actual response this is the idea here fine now In the eobot uh I'll be writing some of the code to connect with my telegram okay so as I already showed you so this is the AOG documentation and they have given the code snippet you can use to connect with the telegram right but I already told you this uh that much
of actually code is not required so I just created a Basics template let me show you so here first of all let me import some required Library so you can see I have imported some library but here it Is coming this yellow mark because I haven't selected my current environment right so right hand side below you can see there is one option so my base environment is selected now I have to select my own environment which is nothing but telegram B okay now see this arror would be disappeared now see these are the package you
have to import like login aam from aam you have to import bot dispatcher execute executor and types okay if you see the documentation Also they are importing the same thing see they're importing the same thing as well fine but as I told you they have written so many line of code I just simplified the code okay that's why um my code might be a little bit different um okay from the documentation this is the idea now I'm loading because with help ofb I'm I'm going to load by okay that means my secret okay inside uh
inside my environment now operating system I need because with the help of Operating system I'll be loading now O.G I think you remember previously you also did the same thing and why logging is required because if you see the aam uh if you see the aam they are also using login okay so you have to uh write the login okay login here see basic config loging so by default it will take the log because it will let's log all the information in the back end okay because see this is their implementation they're suggesting to use
these are the let's Say Cod cipit okay that's why I'm using it's not my implementation so whenever you are using any kinds of let's say framework or Library first of all try to check their documentation whatever things they are suggesting based on that you have to write the code okay no need to do any modification at the very first if you understand their implementation then you can do the modification but the core implementation you have to always keep as same okay this is the idea here Now the first thing what I have to do guys
I have to set my I have to set my I have to load my API key right so for this I can use this load EnV now simply o dog uh EnV inside that I can pass my key name let's say I want to load my telegram bot token I'll pass the key name here fine so it will return return me my telegram bot token now if I print this one my telegram bot token now see if I execute the file Right now it will print the bot token so let me show you so python
let's say my it is inside my resarch folder resarch I want to execute this eobot okay sorry uh eobot dop so guys uh you can see it is showing me one error it is telling from aam import bar dispatcher executor types import error cannot import the executor from a okay so it is uh throwing me one error so to solve this error what you have to do so you have to create Actually uh python uh let's say interpreter 3.7 that means python environment you need uh 3.7 so let's create one another environment so I'll just
write Conta create typen in uh let's give the name let's say mybot python is equal to 3. uh 7 actually I want to use typ and while okay now let me activate Honda activate mybot now let me clear and let me Install the requirements right now so P install ienr requirement. dxt see it has installed all the packages in my environment okay so I already installed that's why it's telling requirement is already satisfied but for you it will take some time okay so once it is done now let's clear my terminal and now if you
execute the command uh this command python research E.P that means if you execute this Eco bot right now now see it should work now See it is printing my uh API token okay it is printing my API token right now telegram B token fine that means it is working now now let me comment this line it's not required now what you have to do you have to first of all initialize the login so that's how actually you can initialize the login so you just need to write login. basic config Lael is equal to loging doino
okay because if you check the documentation they suggesting like that okay you have to configure the Loging like that okay after that you have to initialize the bot and dispatcher so this is the bot and dispatcher so inside that you have to pass the API token okay now see what will happen you can see from AIG I imported bot right because I want to Conn connect with my telegram bot the bot actually have created now if I want to connect what I need I need the API token tegram bot API token yes or no right
so here you can see I'm passing This bot token inside a token actually attribute so what will happen this bot will be initialized that means it will authenticate with my credential if the authentication is correct that means this uh statement would be true okay if it is true that means my dispatcher will be connected okay dispatcher means let's say the synchronization that means whatever message I want to pass okay so it will go to my telegram B okay this is called actually synchronization so it Will be connected okay it should be connected that means this
is the authentication code okay with your telegram I hope it's is clear now fine then after that I think you saw they they have written different different function with the help of asence and Aid keyword okay asence and Aid keyword so what is the use of asence and Aid uh see uh this is the concept of python if you don't know about python so what you can do you can check out our YouTube channel Okay okay so there actually I already uh let me show you so guys you can see this is our YouTube channel
so there I already created uh like python playlist so there I already covered each and every topic okay you need you need to master inside python so there I already explained what is asence and Aid okay so you can check those video so let me tell you as a high level what is Asin and AIT see what this Asin and AIT will do it will keep on sensing okay it will keep On sensing your update that means if you uh update anything okay if you update anything in the code base it will automatically get that
update okay that means what happens actually let's say whenever U you update anything in your code so you don't need to send the request okay you don't need to save and send the request it will automatically sense that it will automatically sense that and it will send the update okay so this is the work of asence and a and if You want to use AI you have to use this particular keyword okay ass and Abit with a function so I already created two function let me show you so I'll tell you what this function will
do so this is the first function guys so the function name I have given command start handle okay I think you saw whenever I started my telegram board okay whenever I started my telegram bot so there was a start button see I given SL start whenever I will give the SL start my bot Will start okay my bot will start see if I show you the bot father if I give SL start see bot father started and it has given me all the suggestion actually I can perform here so this kinds of functionality I want
to implement even and if I want to implement so for this I have to create a function and with the help of asence and a keyword I have to give here inside the function now you have to create a decorator uh your dispatcher decorator now inside that There is a function called message Handler inside that you have to pass the command like what kinds of command you want to pass let's see if I give SL help as as well in my bot father slel so it will give me the uh menu okay what are the
things I can perform with the help of Bot father that means if anyone using my bot for the first time they can get the information with the help of start and help okay SL so I I want to give two commands SL start and SL help if if you Are passing this one so what your Bot will do so what your Bot will do your Bot will reply hi I'm a eobot powered by AOG so this message actually it will give now here you can give any kinds of messes okay you can give any kinds
of message I have given this message now let me show you how it will work so for this what I have to do I have to start my bot so for this this is the syntax you have to write so inside this keyword you have to write executor start pulling Then you have to give the dispatcher object and as well as the you have to give one parameter skip updates is equal to true what is Skip update is equal to true I'll tell you later on as of now just try to remember this parameter you
have to give okay if you check the documentation they are also giving the same thing now let me save and now see if I execute my program my eobot see so it has connected now see it has start pulling now if I go to my bot Let's say this is my bot now see if I give let's say SL start here so it has given me hi I'm eobot Power by aam now if I give let's say slash help it will give you the same thing because you have mentioned two command there I think remember
you have mentioned two command never from that if you give any commment let's say slash hello it's not working if you give any other message also it's not working okay that means it will only work whenever it Will get this two command and here you can pass any sub masses okay I have given powered by aam or anything you can let's say powered by buy here you can give my name so once you changed anything just try to reexecute the code so I'll just stop the execution with the help of contrl C now if I
reexecute now again I will go to my Telegram and now if I give let's say slash help now see it will give me hi ma'am eobot powered by buy okay I hope it is clear Now but see it is not able to handle my casual response okay let's say if I'm giving any other message let's say hi hello okay hi hello whatever it's not giving me the response because I have to create another function for this let me show you so here I will create a function called Eco that means it will give me the
same message again and again okay so this is the E Eco function and see I'm using the same syntax only message hander now see here I'm not Providing any kind of command I'm not providing any kind of command because see whenever I'm giving my casual message this is not a command Okay so that's why you don't need to mention the command that means whatever message user will give that message actually it will send okay it will show you as an output you can see with the help of AIT now let me save and again I
have to re-execute let me stop the execution now reexecute my code now if I Go to my telegram now let's say if I just write High it will give me high if I write hello it will give me hello if I write let's say I am buy it will also give me I'm buy because it's a EOB bot we have created that means I'm successfully able to connect with my bot the bot actually I have created in my telegram that means telegram collection is completed okay telegram connection is completed there is no issue that means
we are able to communicate with the Front end now we have to make another functionality that means back end API that means we will be using open API uh to uh let's say process the response sorry process the question whatever user is asking we'll be taking the help from the llm llm will give the response and this response I'll will be sending to the front end again okay so now let's create our actual application so what I will do uh outside of resarch I'll create a file I'll name it as let's say main dot uh
Pi okay main. Pi inside that I'll be writing all the code so what I can do I can import all the necessary Library again you can see have imported all the library even I also imported here as well okay apart from that I'm also importing openi as well as the system package okay this is the idea now see this application I'll be creating little bit advanc because if I show you my chat jpt now so let's say this is my chat GPT Let's say if I give any message to the chat gbt let's say I
need a python code to add two numbers sorry I think my prompt is incorrect it should be add again I can give the prompt I need a python code to add two numbers okay now I think it is perfect now see it is giving me the code okay but see it is giving me with the function but I don't need the function what I can give I can give uh Without functions okay now see I just given without function it will automatically remember I'm asking this particular response okay it has remembered okay it has remembered
my previous input so this is actually using one memory okay temporary memory so I also want to implement this kinds of temporary memory inside my chart B so for this what I will do I'll just write a class here I'll just name it as reference okay Reference so inside that what what I will do I'll just write a Constructor and this will and it will return actually none okay so inside that I'll just write self dot response and it should be empty initially okay that means as of now let's consider this is my empty memory
okay this is my empty memory that means whatever question I'll be asking as a uh like as a temporary memory that question actually I'll be saving here so that if I want to ask any New question my previous question my model will try to understand okay it will try to remember now here you can also pass the doc string so let's say this is my uh class I have written to store previously response from the chat GPT API okay you can also give open API opena API okay this is the idea now let me uh
load my open API key as well as my uh um this uh telegram bot token so first of all I will load my open IP Key and I'm setting my open I key then I will also load the telegram bot token okay this is my telegram bot token fine then let's initialize this reference class so here I can initialize I'll just write reference is equal to my reference then I'm going to uh Define my model name let say which model I want to use so model underscore name is equal to let's say I want to
use GPT 3.5 tar model if you go to the openi Platform okay if you go to the openi openi models see here is the different different model and I want to use this uh 3.5 series actually CBO model okay this model actually I'm using right now fine now same way I have to initialize my dispatcher as well as the bot because it will authenticate uh with my telegram the same code guys from my previous uh let's say uh file and copy Pasting okay I think you remember so we are initializing our bot and dispatcher so
here the first thing I want to create a function so this function actually will do the welcome okay so if you give let's say start uh start command it will welcome you so this is the function I've written guys you can see simple function using using the same concept okay same concept so this is the function dispatcher message Handler and here is the command only Start okay if you give start command it will give you hi I'm a tbot created by buy how I can assist you okay this is the message actually it will show
you that's it now if you want to execute you can simply copy this line and here you can past it that's it okay now it's a dispatcher object DP previously WR as a short form but now what I did I just written full full name okay this is the idea now let me execute And let me show you so I'll open up my terminal let me stop the execution now let's say if I execute the file right now now I need to execute my main. Pi file so clear python main. by now see if I
open up my bot and if I give SL start right now so it is giving hi I a Teleport created by papy how I can assist you that means it is working fine now let me Stop the execution H great now I also want to create a help our function that means I showed you now if I go to my B father and if I give SL help so it will give me all the Manu actually I want to part from here so I also want to give one manual function in my bot so for
this I have created a function and I named it as a help see this is the function so it will take the help command after that it will show You this message hi there I'm a CH GPT telegram or I can write I a telegram board okay created byy please follow this command start to start the conversation clear to clear the past conversation and help to get the help menu okay I hope this helps now it will give you this uh this actually menu okay now see here I've written clear SL clear to pre like
clear the previous conversation now let's say uh if you're saving any conversation and if you want To clear it so you can use the clear function for this so what I will do I'll just write a function named clear so here I can write so it will just do the clear operation on the previous conversation see I think you remember we initialized our reference here so inside reference we're having one variable class variable called reference so here I'm doing the clear operation see reference. reference I'm making as empty that means whenever I I will save
something okay inside the variable and if I call the Clear C past function it will clear that particular response okay this is the idea if you go to the chat GPT as well and if you write forget my past prompt it will forget that okay it will remove that memory so same way I'm creating the same functionality here and to execute this clear past so I will pass one command so I'll just write clear this command clear so it will automatically do the clear Operations so for this I will be writing another function here so
function name is clear and it will take the clear command and it will call this particular function you can see clear past okay then it will give you message I have cleared the past conversation as the context okay this is the idea now I have to write my main code that means my uh main brain functionality of my application so for this I already prepared the open code See this is the openi code and this is nothing new guys this is nothing new just try to see here I'm using chat completion API okay now let
me check everything is fine or not now you can see guys everything is fine uh so I'm creating the chat completion API and role assistant content I'm giving because you can see reference. response first of all it will check this response okay this response if there is any response or not okay if not that means It will act like a you are using for the first time that means there is no previous context but if you are starting for the second time let's say I think remember here in the chat GPT I have given a
prompt I need a python code to add two numbers that means one response it will remember and when whenever I'm asking the second second question it will refer my previous let's say question I have asked okay the same thing actually happening here that means It will first of all check whether you are having previous response or not if not that means it's a completely new rule otherwise uh it will refer my previous let's say response okay now this is the user message then it will give you the response this response actually I'm uh only getting
the content and I'm printing in the terminal and I'm also printing in my user interface that means in my Telegram and it is also updating the response okay you can see This reference response is also getting updated okay now if you call this clear P it will also do the clear operation it's a simple code I have written now let me show you how it will work so I'll open up my terminal now let me execute my app now if I open my bot let's say if I just write uh uh Slash start I think
this command I already showed you now SL start SL help it is working F sorry SL Start okay now see if I give any casual message let's say hi see it is giving me hello how I can assist you let's say if I give hello how are you doing today let's say I'll ask what is python this is telling python is a high level programming language okay now I can also give the same promt let's say give me a Python code to add sorry code to add two numbers see this is the code I got
now I'll ask give me as a function now see that means it is remembering my previous context now if I give slash clear I have cleared all the past conversation okay now if I give the same message give me as a function now see here it is Giving me the function okay it is giving me the function but not these kinds of function that means my previous uh let's say question function okay that's why I uh created this clear okay clear functionality as well okay now you can ask any kinds of question okay whatever question
you can ask in your chat JB the same question you can also ask here okay there no issue so that means we have created one beautiful application now one thing I want to show you this is Keep update is equal to True let's say if it is true what will happen let's say if my application is offline right now let's say I'll stop the execution let's say it offline now if I give any message let's say hello now see it is offline okay it's not giving you any response but if it is online let's say
if I execute the program right now now let's say my application is online okay now see still it is not Giving me the response still it is not giving me the response because let's say sometimes it's possible your application will go offline and if user is giving any question when it is online again it should return the answer right but it's not giving so for this what I can do I can stop the execution and this parameter I can make it as false okay now let's save my application is offline let's say Hello now if
if I reexecute my program see it will automatically take the hello okay hello message whatever you asked during offline now see it is giving you the response okay so this is one of the amazing parameter you can play with if you want to let's say keep your application offline but if you want to reply it okay when you are online you can update this particular parameter fine so yes guys this is the um like chat bot we have implemented uh you can Also share this uh bot with your friends and family so what you need
to do you just only give uh need to give the username let's say this is your username of your Bot now if they startch with this username they will be able to access your Bot okay and if they're asking any question they will also get the response okay that's how just try to ask your friend to use your chat bot okay you have developed with the help of open so in this video actually I will Show you like how you can f tune your uh gpt3 model okay with your custom data so basically what I'm
going to do here so basically here we'll be funing add classifier okay to distinguish between two sports like basball and hockey so basically we are going to perform one classification task here okay so here I will uh like take one sports data and with the sport data okay I will uh train my uh gpt3 model which is nothing but Ada okay adaa classifier I think you Already know it has various model okay if you click on the model so ad is one of the one of them okay so uh let's install first of all open
AI and uh here you don't have to worry about your system configuration because here you are using uh open API okay so all the training and all the you can say computation everything will be performed on the uh on their website itself okay on their uh you can say engine itself okay so you don't have to worry about so You just need the API key and just try to open up your Google collab okay and try to start the training that's it so because see uh it has like millions of parameters so it's not possible
to train in our system okay so that's why they have proposed uh API okay gp3 API instead of giving the raw model to us okay then uh this command actually you can actually upgrade your openi so let's upgrade it also and here I will uh import my data So this data actually is already available inside psyit learn data set okay like 20 News Group data set the name of the data set is okay so here here you have two categories one is like baseball and hockey now if you want to see the example the first
example so this is the data basically it kinds of email okay and here you have the news sports data okay uh if you want to see the label Target so you have uh sports baseball and hocky Okay now if you want to see the first one see the first example I shown here first data set with respect to that I'm showing my um first Target okay which is nothing but it's a b baseball okay now if you want to also see the count so you have total example uh 1,197 uh baseball has 597 okay example
and hcky has 6 600 so basically this data is not imbalanced so it's fine now inside data preparation first of all we need to prepare the data so what kinds Of preparation we need to do here if you see uh if I show you the target so it has actually uh r. sport. baseball okay so this is the additional information I don't need okay so I need to suppress that so if you just run this code so basically what it will do it will take all the data okay and it will create two columns one
is like prompt and completion okay so why I'm creating prompt and completion because if you uh Go to documentation of U open a so here is a uh uh section called fine tunes okay just click on fine tunes so they have given like how you can find Tunes okay uh gpt3 model so here uh one thing they have mentioned data preparation I think uh here F tune guide and if you just go below prepare your data set okay so if you just read read it okay if you just read this uh okay entirely so you
will get to know you need prompt and Completion okay so that's why I have given this name prompt and completion so prompt is nothing but your data okay your input data and completion is nothing but your level okay so that's actually you need to prepare the data so once it is done now um you need to convert them uh a file format called jonl okay so this is the command just run it okay now let me show you what is jonl okay so if you just refresh here and let Me download it now if I
open it okay so this is the same data format okay like this is your prompt and at the last you have the completion see this is the completion okay and if you see this is the same format if I come here this is the same format okay so that that is why you need to uh you can say convert your entire data frame to jonl format okay is done now openai has a data preparation Tool Uh so if you just call open a and from Tool itself you just call prepared data okay so it will
actually uh uh I mean prepare your data first of all it will take this jonl okay see I'm just passing Sports to jonl hypen Q it will take this one and will do some analyzing okay if something is okay wrong with your data set it will throw error otherwise it will actually uh uh fix it okay so see I have run this command so it has generated two file which is nothing but Isort to prepare train Zess validation Zess okay so it is also it will also do like Trend split okay you don't have to
do it separately okay so this thing actually you need to run and here also they have written okay they have their own data preparation tool you need to execute so whenever it is done now uh this is the section actually you can uh execute okay to F tune your uh you can say model okay this is the training command actually so here you need to Change this API key because this is my older API key so I will what I will do I will change it uh with with the newer one you can also read
it from uh secret okay uh but I will uh give it here just for Simplicity now here API find Tunes create okay now here actually you need to give your data path okay so this is my train data path and this is my validation data path but here one issue Actually I was having so I was uh trying to actually uh execute this command because here if you see if I show you my model okay so find TS here I have like already trained to model models okay with the same data so here actually you
can't give the same ID again and again okay if you give this same name so it will create the same ID again and again okay and it will give you uh error like just give a new ID okay so that's why uh if you have done training one time okay You just change the data name otherwise it's fine okay if you're doing it for first time it's fine just keep it as it is otherwise what you can do just rename this name with a unique name so I'll just rename so I'll just give Sports uh
222 okay because this name is already present in my model so I that's why I'm changing the name sport 222 okay similar wise just change it also here Sports 22 this is your validation okay and Computation metrix uh okay everything you providing and at the last you're selecting the add model OKAY epox is not required so it will basically uh okay automatically observe whenever your loss is not decreasing or accur is not increasing it will stop the execution okay so now everything is prepared now just click on this one so it will take the data
and it will start you can say training your okay it's telling okay it's telling this data is Missing so let me do LS okay so here one mistake I have done here I have given one okay one won't be there so just remove the one and now I think everything is fine now let me execute again so guys uh actually uh training takes a lots of time so uh what I'm going to do I'm going to show you my previous notebook okay I did the training so basically whenever you will click on the command so
actually it will Start the training and it will start the epox okay see this is the epox actually it will be running okay and once uh your training is completed it will save your model okay it will save your model so where you will get the model just come here okay come here and here you need to refresh this page okay so if you just refresh this page now if you just click on the model and at the last there there would be a section called fine tunes okay inside Fine tunes actually you will get
your model okay see I train add a model okay and this is the ID okay it is the ID and basically this is the date actually when I train this model model okay so this model actually you have to use okay whenever you'll be doing the inferencing so now how you can do the inferencing so this is the code so basically you can also uh like uh see the evolution Matrix okay performance Matrix actually you can U see okay uh you can also save it as a Result. CSV okay and here you can also plot
this uh accuracy score okay this is the accuracy score and uh this is my data okay this data set actually I'm like taking one prompt okay and here actually I'm doing the generation okay basically I'm loading my model so why do you need to give your model so let me show you yeah so here actually you need to provide your model name okay the model name actually you'll be training okay Here see I I have took this model okay uh so this name actually you need to copy and you need to paste it here okay
and it will load the model from the opener itself okay and it will like uh do the inference see this is the score of your hockey and baseball and after that if you want to see okay this is the baseball and hockey so that's actually you can uh okay see see the example okay U like uh do the inferencing so yes guys I think you got It like how you can uh find tune your openi uh like gpt3 model okay with your custom data set and if you want to learn more about fine tuning I
will Su just try to explore this U documentation it is pretty good documentation just try to read everything not only like uh classific you can also do it from summarization and other task okay for that actually you need to prepare the data like you need to learn how to prepare the data okay so yes guys I Think you got it uhu this was all about from this video and yeah that's it actually you need to learn about open a and uh like gpt3 okay so once you are comfortable with this API I think you can
do anything okay so as I already showed you the opena platform even I already use different different model Even in our previous video we also implemented one projects called telegram chatbot so here we'll be exploring another actually variant of the model so The model name is whisper so I think I showed you the whisper let's say introduction what is this whisper model so let me show you again so if I go back to my uh let's say openi platform so here if you go to the model section and uh if you just go below this
is the whisper model guys so let me open it up so you can see whisper is a general purpose speech recognization model it is trained on a large data set of diverse audio and it is also multitask model That can perform multilingual speech recognization as well as the speech translation and language identification okay so this is actually a speech based model that means you can uh give any kinds of speech or audio so it will give you the transcript of that particular audio so here uh what we'll be doing let me show you the Entre
diagram see here we'll be implementing one audio transcript translation let's say uh here here I will pass one audio File okay it can be MP3 or any other format you can pass uh I'll will pass to the whisper model okay and Whisper model will return return uh transcript of this audio okay transcript that means it is a text the enter audio text actually it will return then what I will do I will use one um actually chat completion model that means here I can use any kinds of model let's say here you will be using
GPT okay gpt3 so I'll pass this Transcript to the gpt3 and here I will provide a prompt uh Translate okay translate so here as an input you will pass one language okay you'll pass one language and it will give you the um like translated text okay translated text okay so this is the entire actually high level diagram so for this I'll open up my uh local folder and here I'm going to open up my uh visual code Studio so this this is my visual code Studio then I'm also going to open up my terminal here
okay I think you remember previously we created one environment like openi demo so let me activate the environment instead of creating again and again so what I can do I can just write P activate open AI demo okay so this environment I'm going to use because inside that openi is already installed there Right fine now here I'm going to create a requirement file so here the first requirement you need the openi I think openi is already installed but here I'm adding because I want to share this uh material with you so later on whenever you
want to install it you can check the requirement. txt file then I also need python. EnV uh I think uh you know why python. because we'll be managing the credential right that means we'll be creating a EnV file And inside that I'm going to mention my uh open API key I think you remember we already created One open API key so this is my open API key okay let me save it here fine now see here I'm going to create a user interface that means see this audio file uh user can upload user can upload
this audio file for this I need a web interface okay web application so to create the web application I'm going to use one uh framework called flask so Flask is a framework inside python with the help of that you can create the web application okay so let me also install the flask here okay flask now simply what you have to do open up your terminal and write this command P install hyen requirement. dxt for me it is already satisfied but for you it will take some time to install all the packet fine now the first
thing uh I'll show you the demo Like how we can use the whispar model so first of all let me import openi and it is showing this warning because I have to select my environment I have created openi demo then I also need to import operating system because I want to load my EnV then EnV as well so from EnV import load EnV F first of all I load my EnV so here is my open API key w. gmv then I need to add this API key Inside my open API key so open. API key
is equal to my open API key fine H I think my authentication is completed now now what I will do I'll just uh open up one audio file so let me show you I have one audio file with me so this is the audio file guys let me play so this audio actually I have recorded python is a high level interpreted general purpose programming language it is very easy to learn and You can use Python Programming in data science fine so this is the audio file actually I have recorded so I just need to open
this file so to open this file I think you can use this U open and inside that you can pass this uh name so recording dot MP3 okay MP3 and again it's a binary file so you have to give the mode uh RB read binary fine so it will give you one uh object let's say audio underscore File okay so I have successfully uh loaded this file now if you want to get the transcripts what you can do you can use this open uh sorry open a whispar model so I'll just write open a do
audio this is Audio model I think I told you so you have to call one F function called Translate Okay translate and inside that first of all you have to mention the model name so I want to use whispar one model so it is having different different model I want to use Whar B model then you have to pass the audio file so audio file now here uh it will give me the output now let me print the output H now let me show you so guys here is the text I'm getting python is a
high level interpreted general purpose programming language it is very easy to learn and you can use Python Programming in data science that means uh whatever uh things you can uh actually hear in the audio the same Thing you can see as a text so it is like very powerful model it will give you the accurate result always okay you can try with different different audio I think you will get it now what I have to do I have to translate okay I have to translate this transcript uh in a different language okay for this I'll
be using uh gpt3 or GPT 4 model whatever you can use here okay and we'll be creating one user application so that user can upload this audio there okay This is what actually will be implementing so for this what I can do I can create another file here I'll just name it as app.py H now I'll copy paste the same code this code actually I'll just try to copy request here because here we are authenticating with our openi only fine now apart from that I also need to import another package called flask okay uh because
with the help of flask will be creating this uh user application Okay that is why I'm importing flask so you can see from flask I'm importing flask request so redirect URL this is not required as of now just un ify render template okay so with the help of that actually we'll be rendering our web webp for this you need a basic HTML file I'll tell you how we can create a HTML file you can also get different different template you can also use that fun fine now the next thing I'll be initializing my flask to
initialize the Flask just write this code this is the code guys flask u in the parenthesis you have to give underscore name now you can ask me how to initialize the flash go to the flash documentation you can see that okay you can see the code the basic template they have already given so first of all here I'll just add one configuration so have. config that means whatever audio actually user will upload so this will save so this will save actually first of all let me just write Upload unders folder this will save inside a
folder name static static okay now let's create this folder left hand side I'll create a folder name static okay inside that this audio will be saved fine this is the idea now let me create a route so I'll just give at theate app. Route so as you can see this is my route so it will look for actually two kinds of let's say request one is like get request and is Like post request okay if it is matching what will happen this will execute this function so here I'm going to create a function called main
okay so this will execute this main function so inside main function the first thing I have to check uh whether it's a post request or not so if uh request okay if request. method is equal to is equal to if it is post okay if it is a post request that means if you upload if you upload one audio file and if you click on the Button that that means it would be a post request okay that is the idea if it is a post request so what I will do first of all I will
get the language okay I'll get the language that means user will also pass the language like uh in which language he or she wants to translate this uh let's say transcript okay that's why I also need to take the language input so here I'll take the input I'll just write request do Form uh so from the form itself I'll take the language now how I'll get the language because for this I'm going to create a HTML file so let me create a folder here first first of all if you're using flask you need to create
this folder guys called templates okay templates inside that I will be creating a sorry it should be templates not spelling is not correct H now it is fine now inside that I'm going to create A HTML file called index do HTML okay now here I have written one very simple HTML code it will only uh accept the audio file see here I've given the title and this is like very basic uh CSS code I have added here just to like show my front end little bit actually beautiful that's why now see it will uh user
can upload one audio file okay you can see user can upload one audio file uh input type is file and user will also able to give the language input that Means in which language he or she wants to translate that transcript okay then there would be a button called upload so whenever uh they will upload so what will happen it will hit the post request okay it will hit the post request and that time my you can see method is equal to post that time it will hit this route okay it will hit this route
and I will then I'll be able to get the language and whatever audio actually is submitting everything I will get okay From the back end this is the idea only fine so this is a simple HTML code and now you can ask me why to get these kinds of HTML code there is a one beautiful website you can follow called bootstrap okay bootstrap so here you will get all kinds of FD HTML CSS template you can go to the example section and see different different actually template you are having you can uh get their HTML
and CSS code even you can also SE let's see you Need a file uploader button okay file upload see different different codes in if they have already given okay that's how actually you can also get the code you can also get the code okay so I just referred this website and I uh actually let's say collected this HTML and CSS code and I modified okay with respect to my requirement this is the idea now I think everything is fine now what I will do I'll go back to my app app.py and I'm getting the language
then I also need The file okay so to get the file I'll store inside a file and I'll just write uh request. form uh request dot not form do files actually because it's a file type input right so it should be a file you can see here I have mentioned it should be a file okay it should be a file type should be a file now once I get the file uh first of all I will check whether this file is available or not if file is equal to True okay if this is not False
that means user has uploaded the file that time what I will do I'll just simply save this file where I will save I'll save inside my static folder so this is the code I think you remember we said the configuration okay static okay upload file static so it will save the inside stratic folder and to save this file I need a file name so let's create a file name here I'll just write file name is equal to file do file name okay it will give you the file name now after That you can easily save
this file inside stratic folder then I will load my audio so to load the Audio I think you remember I was using this code snippit so I can use the same code here H now see this recording. MP3 would be available inside static folder so I need to also give the path of the static okay stratic / recording. MP3 now the same code I'll be writing here I'll copy and here I'm going to paste It so this will give me the output that means the transcript here I can mention it as transcript okay transcript now
what I have to do I have to translate this transcript to the another language for this I already told you here we'll be using something called gpt3 or four model okay so let me initialize my GPT model so I think remember how we can initialize it uh we can use chat completion API so here is the Cod s it guys so here I'm using chat completion API inside that I'm mentioning I want to use GPT for model OKAY GPT for model and this is the message I have given role you can see RO system content
uh you will be provided with a sentence in the English and your task is to translate is to language that means whatever language user is giving that in that particular language then role so let me bring it here I can see R user content transcript. text that means whatever text actually I'm getting after doing The let's say I mean audio to text I'm just passing it here and here is my prompt okay here is my prompt I have given to my large language model and this is the temperature parameter and this is the max tokens
okay I already explained this parameter okay why it is required now simply what I will do I'll just try to return return this response but I just try to return with the Jony why justy because whatever let's say response you Want to render in the HML P it should be Json format okay that's why I have to do the jsonify so inside that I will be passing my response okay and this on ify I already imported I think you remember from here then if it is not a post request okay if it is not a
post request that means if it is a get request what will happen it will only render okay it will only render my index.html okay this index.html that Means my uh web app okay my web app menu it will open in front of the user this is the idea and to render it I'm using render template function you can see I've already imported it here now if I want to execute my application so you can execute it here for this you have to give the app host and app Port see app.run host 00 that means I
want to run in my local host and debug is equal to r that means if you change anything it Will automatically update the changes and this is the Port Port number 8080 now let me execute and show you this application so I'll open up my terminal and now if I execute let's say python app.py now it is running on my Local Host now let's go to the Google and search for local host port number 8080 now see if I search so guys you can see this is my my interface okay interface of my application now
here you can upload any kinds of uh audio file now Let's upload so let's say I want to upload my this audio Now language let's say I want to translate in a Hindi okay Hindi now let's upload it now see guys this is the response I got and this is my response you can see this is the Hindi translation okay this is the Hindi translation whatever actually I had in my audio now you can Al give any other language let's say again I will pass my audio let's say I want to translate uh in the
let's say I Want to translate in Bangla okay Bengali now I'll upload see this is the Bengali translation I'm getting okay now you can also extract the content only uh instead of like let's say getting all the response but I have printed all the response I just wanted to show you each and everything whatever things it is returning okay I already showed you how we can extract out this content okay from my previous I think session I think Remember fine so yes guys that's how actually we can implement this project and we can use the
whispar model okay now what you can do you can improve this project you can add some more functionality let's say you want to again convert this text to audio you can also do it for this just go to the whisper documentation and try to check there okay so I hope guys you like this project so if you have like this project guys so please try to subscribe to the Channel and share this video with your friends and family so I already showed you openi is having different different actually large language model uh like different different
kinds of large language model like uh language model multi model OKAY image model like different different model actually it is having so we'll be using another model uh this is called actually Deli so Deli is having different different like version like Del 2 is there Del 3 is There so we can use openi API to access this model okay so let me show you this Deli model so guys if you go to the openi documentation so here you will see this Deli model let me show you so this is the deli model so as you
can see Deli is having different kinds of uh let's say version like Del 3 then Del 2 okay so different different version it is having and Deli is one of the model actually it can create actually realistic image from a prompt let's say Here you will be passing one uh like text prompt and with the help of this prompt actually it will generate the image okay the prompt you will be asking the same kinds of image actually it'll try to generate and this would be realistic image okay now let's see how we can use openi
API to access this Del model and let's build one uh image generator application so what I will do guys I have uh used the same template as I already used in my previous project Implementation I think remember we are using FL there so I use the same template and I just did little bit modification here so the modification I have done uh the entire HTML code okay I've done the modification uh let me show you the application like how it will look like let me clear now if I open up my app.py again so let
me go to Google and Local Host port number 8080 so guys you can See this is the interface of of my application so to build this interface I have actually collected this HTML and CSS code from the bootstrap website I think I showed you that bootstrap website right so you don't need to do anything only you can change the title and all okay everything you can keep it as default because you can use this uh HTML and CSS code uh as a template okay let's say you want to build any other application with the same
template you Can use this code as it is okay if you're not familiar with HTML CSS code it's completely fine you can copy paste from different different website okay now the second changes I have done I have created two route one is my default route that means if user is hitting my uh let's say default route that means port number 8080 so he will get this kinds of landing page okay and to render the landing page actually I'm using render template index.html okay and Whenever user is giving any kinds of prompt and there submitting okay
submitting button so it will hit this route and it will generate and it will execute this function and this function will take the prompt whatever prompt actually user is giving here I think you can see okay and this prompt will go to the open AI Del model you can see to use the open Del model you have to use this Cod spp it open. im. create okay inside that just try to mention the prompt Number of image you want to generate just give the number of image and the size of the image okay everything you
can set here now I'm printing the response as well as that and here you can see I'm also rendering on my user interface okay and again I'm running on Local Host and port number 8080 and debug is equal to True means if you're changing anything this will reflect automatically fine so this is the simple modification I have done guys otherwise Everything I kept default okay everything I kept default whatever things we have implemented in our previous project so yes guys I think you got it uh using openi is like very easy only you just need
to know how we can access different different model rest of the things you can take care okay let's say if you want to create any kinds of application you have to design how you'll be creating okay otherwise the fundamental will remain same now let me Show you the demo guys so what I can do I can give a prompt here let's say I'll give a dog is flying and smoking let's say this is my prompt I have given let's see whether it is able to generate the image or not you can give the same prompt
in the chat GPT also let me show you chat GPT is also using uh this kinds of image model recently they have published one model called GPT 40 okay GPT 40 so this model is able to actually create these kinds Of image okay uh from your prompt now let me pass this prompt a dog is flying and is smoking see it is uh generating two response and let me see my see this one actually it has generated okay this is a curon version even you can also generate the realistic one let me show you another
prompt so guys uh this is the prompt I have prepared you can see uh dream like a beautiful girl uh playing the Festival of Color drapped in The traditional Indian uh throwing the colors okay now let me send and let me see the response and if I go to the chat jbt see this is the CH jpt response fine see uh this is the output actually we got okay it's amazing you can see so you can give different different prom like that and you can generate image and you can also control like how many image
you need so here I mentioned I think you saw only five images that's why it's giving five images okay and you can also Mention the size everything you can mention so yes guys uh this is the application we have developed I hope you liked it and now uh what you can do you can improve this application you can add some more functionality see I think the previous application we created that uh whispar model what you can do you can integrate like both application in one place okay and you can create a complete platform okay that
thing I think you can perform okay this would be a good Project for you so yes guys this is all about from my side please subscribe to the channel and share this video with your friends and family and please support the channel guys uh if you support the channel so definitely uh we'll be bringing up these kinds of content okay for you and you can let us know what kinds of content you you want okay from my end I'll try to provide so I think as of now we have worked with the large language model
I have showed You the hugging face platform even we have also worked with the openi platform there we use different different large language model and whenever I was using these kinds of large language model I think you noticed I was passing the prompt okay uh again uh if I'm talking about chat GPT Google but whatever application you can see in the field of geni all the application takes the prompt as an input okay based on the prompt it will decide what kinds of Output it should generate okay that means prompt is everything inside large language
model okay whatever let's say large language model you are using whatever generative model you are using prompt is everything here okay you have to give a proper prompt to get a proper answer otherwise you will be getting random kinds of answer from your large language model okay so that's why there is a separate concept inside genbi called prompt engineering why prompt Engineering because whenever you are writing these kinds of prompt you have to know some uh technique okay how we can design this prompt efficiently so that I can get the output I can get the
effective output I can get the let's say useful output from my large language model okay otherwise what is the use of large language model if it is giving you a random kinds of output fine so that's why we'll be learning about this prompt engineering and make sure you are Watching this video till the end because in this video we'll be covering each and everything you need to master about prompt engineering so guys uh as you can see what is prompt engineering prompt engineering is nothing but so prompt engineering is nothing but prompt engineering is the
process of structuring uh an instruction that can be interpreted and understood by by a generative AI model that means I already Told you now in said generative prompt is everything and whatever prompt actually you are giving to the generative EI model or your large language model okay this should be well structured okay this should be well structured and well organized okay otherwise what will happen your model will give random kinds of output okay that is why here I have written the definition what is prompt engineering exactly okay I hope it is clear now now As
a generative engineer you should have some responsibility whenever you will be working with the large language model whenever you'll be preparing the prompt and all see here have listed down some of the points write uh refine and optimize prompts perfect the interaction between humans and AI that means you always need to make sure whether your prompt is perfect or not okay to communicate between human and AI that means you will be communicating with Your AI assistant let's say whatever let's say application you are having with the help of large language model okay there is a
a good interaction on not okay The Prompt actually you have given you have to always make sure continuously monitor those prompt you have to continuously monitor those prompt whether this prompt is perfect or not and is there any change in the model or not then maintain and up up toate the prompt that means if let's say model Model got updated let's say they again did the fine tune that particular model and if let's say previous prompt is not working better in that case what you can do you can maintain and update that particular promt for
that large language model so these are the key responsibility you should have as a generative engineer now let's try to understand why prompt engineering okay so for this I'm going to show you one demo I think this demo will give you the Clearcut idea why prompt engineering is super important if you want to work with the genbi so here you can see I have written a prompt uh you can see this is a prompt I have written correct my paragraph today was the great day in the world for me I went Disneyland with my mom
it could have been better if it wasn't raining you can see some of the spelling mistake is there that means grammatical mistakes are there in this paragraph I have written so I will do I'll give this prom to my let's say large language model so large language model wise here I'm going to use ch GPT okay you can also use Google b or any other actually let's say model it's completely fine but just to show you the demo actually I'm going to use the chat GPT platform later on whenever let's say we'll be implementing different
different let's say application with the help of large language model that time actually we'll be using this prompt with The help of python code okay as of now just to show you the demo I'm going to use the chat GPT application so guys as you can see I'm inside my chat GPT now let's say if I give this prompt uh to my chat GPT now let's say if I send it here see what will happen see it has given me straightforward results without let's say explaining this uh like paragraph I have given without let's say
asking me any kinds of question it has given me straightforward Answer so here is my uh paragraph and here is the correct okay correct of my paragraph correct version of my paragraph today was uh today was a great day for me I went to Disneyland with my mom and it could have been better uh if it wasn't raining okay you can see so here you can see this uh grammatical mistake I did it is resolved okay by my large language model now let's say you are learning about English language you are learning about grammar you
are Learning about let's say spelling and everything like that means the complete let's say English guidance you want okay from your Mentor but as a mentor wise what you want to use you want to use the chat GPT let's say your large language model that time instead of giving these kinds of prompt to the large language model what you can give so instead of giving the prompt like that you can design your own prompt in an effective way so let me show you another example So guys as you can see this is the next prompt
I have designed so here is the problem guys I want you to act as a spoken English teacher I will speak to you in English and you will also reply to me in English to practice my spoken English I want you to keep your reply NE uh limiting the reply to 100 words I want you to strictly correct my grammar mistakes and and typos I want you to ask me questions in your reply now let's start practicing you could ask me a Question first remember I want you to strictly correct my grammar and mistakes uh
typos and factual errors okay so this is the prompt I have designed okay see previously The Prompt I have given it was like very straightforward prompt I had given right but now what I did I just designed one very powerful prompt here now see what should be the response from my large language model right now so let me give this prompt to my large language model this is the prompt guys The same prompt I have given now let me send now see it is giving me great let's start with a question what is the most
interesting place you have ever visited and why did you uh did you find it uh so fascinating okay now let me give the answer let's say the most interesting place I visited India because I loved Indian okay Indian food Indian food okay now you can also Do some grammatical mistake let's say here I'll just write like that okay now let's say this is my response I have given to my uh model right now see my model will try to correct first of all you can see here is the correct version the most interesting place I
have visited in India because I love the Indian food now see the reply you have given so there actually lots of spelling mistakes were there there are lots of actually grammatical mistakes were there Now it has corrected everything now as a learn that you can easily get to know okay so that's how actually you have to uh write this particular sentence okay now you don't need any kinds of actual actually English teacher okay with the help of this kinds of large language model you can easily learn this kinds of syntax yes or no fine now
again it is you can see it is asking me one question can you tell me more about your favorite Indian dish that means now this Conversation is like more interactive it's not like straightforward answer it is giving me it is like more interactive the way actually human communicate with each other yes or no now yes I can give another response so here I can give um I loved CH uh V okay now let's see the output so here is the correct version I love CH V what do you like most about CH V okay so
here I can write the Puri now see this is the Correct version I like the Puri uh what do you enjoy about the Puri okay now see this conversation is more interactive now you can um just keep on replying uh to the question actually uh this model is asking that's actually you'll be learning about the correct let's say grammar correct let's say spelling and all yes or no now see with the help of just only one simple prom design I got like very interactive output from my large language model okay this is called Actually prompt
engineering and that's why it's useful okay that's why it is important because sometimes whenever you'll be creating any kinds of application any kinds of LM PR application that time you have to design The Prompt in such a way so that it can give the correct answer to the audience even it can it can be more interactive to the audience that means those who are using your application so they should enjoy your application yes or no so guys That is why it's like very interesting concept and it's like very important concept inside gener B you have
to master now see to master the prompt engineering the first thing you should know about the Linguistics okay what is linguistics Linguistics is nothing but it's a study of language that means whenever let's say you are uh studying any kinds of language let's say English Language inside English language you are having some kinds of uh let's say Linguistics okay what kinds of linguistics you can see uh phonetics is there so phonetics means the study of how speech sounds are produced and perceived okay then there is another one called phology the study of sounds pattern and
changes that means whenever we are speaking any kinds of let's say language like how these sounds look like okay how this sounds look like how you're pronouncing this particular word and all this is called actually Phonetics and phology got it then there is another one called morphology the study of word structure then there there is another one called syntax that means the study of the sentence structure that means whatever grammar you usually learn inside English then you have also having the semantics that means the study of the linguistic meaning then the next one is pragmatics
the study of how language is used in the context then historical the study of Language change then so on okay you can see these are the actually Linguistics are available to study about the language okay and if you have good knowledge if you have good knowledge on these kinds of let's say Linguistics definitely you can design a very good promt for the large language model so that's why I have written here Linguistics are the key to the prompt engineering that means if you are having good knowledge about linguistic Definitely you can design a good prompt
for the large language model now we have already studied about like large language model or our generative a model right so there are different kinds of generative AI models are available like GPT Bart then Lambda Palm Bloom Lama cloudy okay then uh Nemo llm okay generate so these are the actually apart from that like there are so many large language models are available I think I showed you one GitHub right I think you Remember open llm GitHub there I think you saw thousands of large language model were there and these are the provider and brander
okay like those who are created this these are the actually large language model see all kinds of large language model over the internet all the large language model accept this prompt okay that means you have to give the promt there if you want to use this kinds of large language model okay because this large language model is Already trained with huge amount of data huge amount of unstructured data even they also did something called supervised fine tuning they also applied something called reinforcement learning technique to improve the quality of the model that means this model
is also trained with the instruction lots of instruction that means they did the chat operation with this model I think I showed you one uh in one video one of my video I already explained how chat GPT Train okay so that's why whenever they train with this kinds of instruction data that means they are passing the prompt and whenever you want to use this particular model you also need to provide the prompt I think now it is clear why this prompt engineering is super important and whenever you want to use any kinds of large language
model why you have to use the prompt engineering technique that means why you have to give the correct prompt always To the large language model now let's learn about some best practices to design a prompt see if you want to design some best prompt for the large language model you have to keep some of the point in your mind let's say the first thing clear instruction definitely you have to give the clear instruction to the large language model then adopt a Persona then specify the format avoid leading the answer and limit the scope okay so
these are the thing actually you Have to always remember now let's open our CH gbt and let's try to see some of the best practices okay we can perform whenever we're giving any kind of prompt to the large language model so guys let's say if I give a prompt here let's say I'll give when is the election okay when is the election let's say this is my prompt okay now see if I pass this prompt to my large language model let's see the output it depends on Which election you are referring see election can uh
be vary by the country definitely because it's the correct one now you are asking what is the election but you are not clarifying okay you are not clarifying in which country you are referring okay that means you have to always pass a clear prompt okay clear instruction to the large language model now see I can give this prompt like that when is the next President election in USA okay now see this prompt is clear and it is like well organized for my large language model now see if I pass now see this is the answer
the next US president election election is scheduled for the November 5th uh 2024 okay now I'm getting the correct answer from my large language model but whenever you are not giving any kinds of clear let's say clear let's instruction to the model it will give you Unnecessary output and again it's a chargeable because whenever you are using Char GPT or let's say uh any kinds of let's say commercial large language model It is charging based on the token input and output token I already showed you okay how it will count the token and how it
will charge so whenever you are getting these kinds of unnecessary output that means unnecessary you are spending your money yes or no okay so that's why whenever you are using these Kinds of commercial large language model make sure you have the proper promp so that in just one prompt you will get the actual output you are looking for okay this is what actually you have to always remembered now let me give you another example I'll just write write a code to filter out the ages from data so let's say this is my prompt I have
given now See my llm model has given me this this is the output but let's say you are looking for Java code you are looking for Java code but it has given you the python code again what you have given you have given actually uncleared instruction to the model yes or no right so you have to give the clear instruction always now you can write this prompt like that uh write a Java code okay to filter out edges From the data now see now it is the clear instruction to my large language model now I
will get the Java code now let's see the next one like the add of the Persona okay like uh what is the Persona exactly whenever you are giving any kinds of let's instruction to the model uh what kinds of persona you have to add up there let's say this is one prompt I'm giving to my large language model write a poem for a sister's High School uh graduation that will be read Out to the uh to family and close friends okay let's say this is my instruction I have given to my large language model now
see uh here is the response I will get so here is the response I got that that means this is my poem now if you read this poem actually this is not actually much interesting the poem actually I was looking for now what you can do you can actually uh give the Persona okay you can give the Persona Whenever you are giving the prompt so now let me show you another prompt I have design for this so This Is The Prompt guys now here I'm writing write a poem as Helena Helena is a 25 years
old and amazing writer her writing style similar to the famous uh 21st century poet uh rupik cor so I think you know about rupik cor you can also search about rupik see she is the actually Canadian poet and illustrator okay you can read about her now see write as a Helena writing as a Helena write a poem for her uh 18 years old old sister to celebrate her sister's High School graduation this will be read out to the friends and family at the uh Gathering okay now see this is the prompt actually have given now
see if I send it uh if I this prompt right now now see the response actually I got here now if you read this poem actually it is too good it is too good than my previous response I got okay so that's how actually you Have to adapt the Persona always you have to adapt the Persona what kinds of things you are looking for now see this large language model is already know about rupic cor okay because it is already trained it is already trained uh let's say around 2022 data okay till 2022 data it
has already return and in that data actually rupik or poem was there okay that's why you that's why so that's why this model is already known about rupic cord and it is trying to get The similar kinds of let's say poetry okay whatever let's say poem actually rupik used to write okay the same actually syntax context actually is trying to Let's gain now the next thing you can also specify the format let's say what kinds of format you want as a output let's say what I will do I will give you one example for the
summarization let's copy one English story here so let's say this is one English story I'll just try to copy the Entire story and here I'll go to my chat GPT and I'll just write uh please summarize this story okay summarize this story now I can pass the entire story here now see I can send it now see it is giving me the summary but see this summary again it's a paragraph okay it's a paragraph now I can also specify the format let's say I'll give I want I think I can copy the same Instruction now
here just try to modify please summarize this story in bullet point okay bullet point now see I'm specifying the format now see if I pass this now see it will give me inside the bullet point okay now see it's like more readable now it's more readable it is giving me uh some actually important uh important actually summary of this entire story it is trying to return me okay so that's how you can provide any kinds of format You need let's say you need a data and you need a juston format data you can also provide
I need a juston format data okay you can pass the prompt and it will give you that so that's how actually whenever you are designing your prom you have to always keep these are the point in your mind okay the point actually I listed down in my presentation I think you saw that right now there are some types of prompt are available uh so the first type you can consider zero short Prompting and the second type actually few short prompting apart from that actually there are like some types of prompts are available like emotional prompting
and all right but this is not actually required but if you want to create any large language powered application so this two prompt actually very famous so I have seen like people are using this to prompt a lot like few shot prompting and zero shot prompting so let's explore this zero shot and few Shot prompting uh in the example so again I will go to my CH gbt now first of all let's write the zero shot prompting zero shot prompting means you are only asking the instruction because the see this Char GPT is already trained
with huge amount of data so this Char GPT has already like lots of knowledge about let's say the data over the whatever data they have trained now let's say I will ask one question when is the Christmas in America now see Christmas United States celebrated on December uh 25th each each year okay now it is giving me the answer so this is called actually zero shot prompting that means you have only give one prompt okay one prompt that means the instruction you have given the question you have given okay but let's say if I ask
these kinds of question now let's see if I ask this kind some question what is BU Favorite types of food okay let's say this is the question I have given now see it will give you I don't have the information about buy favorite types of food could you please provide more details clarify who is buy okay now see here actually uh zero short prompting is not working so here you have to pass the few short prompting that time okay now see here I'll give a prompt now I'll just write buies buy is a Data scientist
and his favorite types of food are or you can write uh food includes let's say burgers burgers Pizza okay and chicken now I can pass this prom to my large language model now it will learn about me okay because I have given a few short prompt here okay now see got it puppy uh s types of foods are Burger pizza chicken and if you need any more details or have a questions just let me know now here I can give another prompt What is the or I can write what restaurant now let's say this is
the prompt I have given what restaurant should I take buy uh to in Dubai this weekend okay now see if I pass now see it will suggest me some restaurant Dubai has fantastic range of restaurant that CS a vity of test since buy like barers piz and chicken here is the few recommendation so here is the restaurant now see this is called actually few short from that means you Are also giving some few short prom that means some some let's say information information about the question you are asking okay this is called actually F short
prompting so these are the thing guys you have to master so uh see this prompt engineering will come automatically whenever you will be writing these kinds of prompt so unless and until you are not writing you are not experimenting you won't be able to understand okay which prompt will give You what kinds of output so try to practice a lot okay you have the chat gbt you can give different different prompts and you can get the response from the chat gbt and the prompt actually you're giving to the CH the same prompt you can use
to any kinds of large language model OKAY in future you'll be using whether it's a let's G whether it's let's Mistral Falcon whatever let's say your model are using you can use it as it is fine now there Is another concept inside actually generative AI model called AI hallucination now what is an AI hallucination see AI Hallucination is when a large language model generates false information that means let's say you are giving a prompt and that model is giving you the false information okay so this is called actually AI hallucination that means the prompt you
are asking the information you're asking this information is not available to the Large language model okay so that time what we have to do if we are getting this kinds of alation so that time we can use something called rag concept that means retrieval augmented generation that means we can connect our external data source to my large language model okay and from that data sources my model can learn okay this is the information uh you are looking for and if rag is not working that time what we can perform we can perform something Called fine
tuning of a large language mod model I think I showed you now we can perform fine tuning with the help of hugging face okay so I'll also show you how we can fine tune different kinds of large language model on top of our custom data we can also perform F tune operation but fine tune is a costly task because here you need a good system like lots of lots of data okay so again it's like very costly task but rag application is a very easy kinds of task See here you don't need lots of cost
here you don't need lots of computational power okay so both we'll be exploring one by one like what is uh rag concept and what is let's say fine tuning concept will be exploring okay so yes guys this is all about our prompt engineering now I think you got it why prompt engineering is super important and how you can Master this kinds of prompt engineering technique okay it's nothing but it just a way of let's say Communicating with your large language model the input you are giving to the large language model this is called actually prompt
okay and the way you are designing this particular prompt this is called prompt engineering okay I hope it is clear now see if you want to learn if you want to master Genera VII if you want to work with the large language model so this is super important concept you have to first of all master and uh this this is called actually Vector Database because see whatever let's say data you will be using whatever documents you will be using let's say to work with your large language model so first of all what you have to
do you have to convert the documents to the embedding representation I think I already explained like what is embeddings right so my model can take the English input directly so for this I have to uh create some number okay that means I have to generate the embeddings I have to generate some vector and after generating these kinds of embedding and Vector uh we have to store these are the vector to a databases uh because I have to connect my large language model with these kinds of databases okay so this is called actually Vector databases okay
I hope it is clear now so no need to worry I will give you the entire idea of this Vector database like what Vector database is even I will also show you the different different practical uh With different different Vector databases so here we'll be covering almost all the famous Vector databases whatever let's say people are using broadly in the market we'll be covering all of them no need to worry and guys to show you this Vector database demo I'll be using one framework called langen so langen is a genbi framework with the help of
langen actually we can build different different genbi based application because inside langin Actually all kinds of vector database functionality are already available okay so that's why I'm going to use langen uh throughout the entire let's say series of this Vector database but if you're not familiar with langin no need to worry go through this tutorial and try to implement whatever things I'm implementing so after this Vector database series I will start the complete langen series as well so there actually we'll be uh starting from very Basics of the lch even we'll be covering till Advanced
part of the lch okay we'll be learning each and everything as of now let's say as a code wise just try to understand okay what are the functionality actually I'm using from the Lang chin and try to implement with me and make sure whenever I'll be installing any kinds of package on my collab notebook try to see the version okay try to see the version of that package because uh today actually I'm Creating the video I know that because this is a completely research field so some of the functionality they will change in their package
and then you will uh get some issue okay you might get some issue you might get some error so make sure whatever let's say version I'm installing on my notebook try to also install the same version and if I'm not mentioning the version okay with the package what you can do so whenever I will show you the installation uh Below In the console you will see the version okay like what is the L chain version what is the openi version what is the let's say Vector database version okay just try to see the version try
to just note it down and try to install the same version it will work because see it's a completely resource field every day they are updating their package and all so I know that if you're learning something today so tomorrow actually they will update that package and you might get Some issue okay so that's why I'm telling you guys always try to use one a specific version okay uh if you're using a specific version so you won't be having any kinds of issue because you know that this version will work fine okay the code you
have implemented so that's why just try to see the version whatever version I'm installing uh see the console console of that collab notebook there actually you will see all the version has been written okay for All the package so note it down and try to install the same version now let's try to see what the topic actually we be covering uh through this Vector database Series so guys as you can see uh here first of all we'll be learning about Vector database like the in detail explanation of the vector database what is Vector database how
it is work work and all everything we'll be learning then we'll see that why we need the vector database how Vector database Works use cases of the vector database then some weedly used Vector database we'll be also learning then at the last of the series actually we'll be also doing the Practical demo with the help of python and langen as I already told you we'll be using langen framework to work with the vector database okay now let's try to see what is Vector database is see Vector database nothing but a vector database is a database
used for storing High dimensional Vector such as Uh word embeddings or image embeddings so I think you already know that uh we are lots of unstructured data and higher dimensional data like documents image PDF okay then we are also having audios videos and so on right so this is called actually unstructured High dimensional data so what we have to do inside Vector database concept we just need to first of all convert desert the higher dimensional documents to the higher dimensional Vector you can see we are Converting this documents to a vector representation and this Vector
actually we restoring to the database and this is called actually Vector database okay that is why here I have written a vector database is a database used for storing High dimensional Vector such as word embeddings as well as the image embeddings because I think you already know that inside generative VI we are having textual data as well as the image kinds of data that means video data Audio data and so on right so that is why you can consider any kinds of data and whenever you're considering any kinds of data it will have some embeddings
and that particular embedding you have to save to the vector database so this is the introduction of vector database I think it is clear now you can ask me like why we need these kinds of vector database because we have lots of existing database are available in the market now uh to give you the answer Actually uh I want to show you one example so first of all let's see like uh why we need these kinds of vector database uh because if you see over like 80 to 85% data out there is unstructured data like
we have images we have videos we have text and we have audios okay so these are the data actually unstructured data so we can't uh easily store these kinds of unstructured data to the relational database or let's say our traditional database like we have my SQL Then we have postre SQL then we have uh sqlite so these are actually relational database and these are actually traditional database so we can directly store these are the data so we can directly store our unstructured data through these kinds of relational or traditional database okay why we can't store
I will uh show you one example it would be very much clear so guys to give you one example let's take one image data as an example so let's say I want To store uh these kinds of dog images to our relational or our traditional database so if I want to store it I can directly store these are the data to our uh relational database okay so if I want to store it so what I need to do I need to create some of the schema let's say I have some schema here I have some
table so in this table let's say I have one column called animal okay animal then I have another column called Color okay then I have another column called Tags so basically you need to manually label them uh to store these are the images okay if you want to identify a similar kinds of images if you want to apply the query on top of that then then you need to uh create this kinds of schema manually now here uh you just need to set those parameter manually let's say animal wise it's a dog okay it's a
dog then color-wise Let's say it's black or let's say I can and color-wise you can give any color let's say golden okay this is the golden uh dog golden dog then we have let's say black dog as well okay black dog you can give anything based on your image actually you are trying to store then you also need to assign some of the T let's say this is very cute um let's say this dog has black eyes Okay black eyes so these are the tags actually We'll be assigning uh manually so what happens actually so
whenever you assign these kinds of schema manually so what happens let's say you want to uh query uh the similar images okay from the database so you just give these are the parameters so it will return these are the image for you okay based on the tags actually you have written based on the schema you have prepared right but again it's a manual thing okay it's not a efficient way to uh store our data and It's not a efficient way to like query out our data because let's say I want similar kinds of image so
I need to assign these at the labels manually and it's like very uh hard task for us right so that's why actually we can't use our traditional or let's say relational database for these kinds of unstructured data we can easily use this kinds of relational database for the structured data let's say we have CSV data we have uh Excel data but for this kinds of Unstructured data it's like very much difficult to store these kinds of data right you can store them uh there is no issue but uh the thing is like whenever you will
be quing out uh from your database let's say you want similar kinds of dog images at the time it would be very much difficult because if I let's say convert this image to Bas 64 string so using B 64 string I can easily store the data to the relational database but I'm talking about whenever Let's say you are applying query on top of that let's say you want similar kinds of dog images from this database at that time it would be very much difficult for you to quering out because because it doesn't know like whether
this best 64 string is a dog image or let's say it's a cat image or let's say it's a horse image okay it doesn't know so there is only one option to uh filter out these kinds of images uh the thing is like you need to create one manual schema like These kinds of tags you need to generate you need to label out these are the images uh using that actually you can filter out these kinds of data but again it would be very hard approach uh as I already told you because you need to
apply manual observation here now guys I I think you already got the idea like why it is difficult to store these kinds of unstructured data to the uh relational database because as you can see image is nothing but it's a pixel And your relational database doesn't ever know like how to query out similar kinds of images using these kinds of pixel right so guys this is the one major problem uh with these kinds of relational database uh whenever we are using unstructured kinds of data so guys whenever I'm talking about unstructured data it is not
only image data so it can be video data it can be Text data and it can be audio data as well so guys to overcome this issue uh there is a Concept introduced called Vector embedding so let's say we have unstructured data let's say we have image we have a text and we have audios and it can be also videos so first of all what we need to do we need to apply something called embedding model okay so embedding model is nothing but it's a neural network based model it's a deep learning model so using
this embedding model first of all you will be generating some of the embeddings or Let's say vectors okay let's say this is my image we have applied embedding model embedding model will give you vectors okay like this is the represent ation of the image okay this is the numerical representation of this image it can be also applied to the text data so you see here I'm giving the text Data uh and this is my embedding model and it is returning these kinds of vector again uh you can also put audio data so it will convert
to the numerical representation And this is the representation of the audio so here if you see embedding model is nothing but it's a neural network based model here you can use any kinds of embedding model let's say we have what toake then we have Transformer based embedding model as well right so these are the things actually you can use so if you're already familiar with NLP that means natural language processing I think you have heard of like what to V then you have heard of Like Transformer embeddings okay and uh nowadays actually we have large
language model based embeddings like we have open a embeddings we have uh lots of Open Source llm based embeddings as well we have hugging face embeddings as well so using these kinds of embedding model we can easily generate these kinds of vectors and we can store them in a vector database so guys this is the entire idea uh on top of this unstructured data like how we can store These kinds of un structured data to the vector database so first of all uh what we are doing we are just trying to convert these are the
unstructured data uh to the vectors Vector representation then we are uh storing these are the vectors okay to the vector database so guys here is an example you can see so let's say we have a text unstructured data and here you can use any kinds of embedding model so in this case actually I have referred this uh open a embedding Model so it is already trained on GPT model okay gpt3 and four model and if you put these kinds of unstructured data to the embedding model it will you these kinds of vector okay this kinds
of vector representation of all the text actually you have in the documents then once you got this Vector actually this Vector embedding then you will be using some uh uh Vector database it can be anything any database you can use so here you can store one by one these are The data okay so this is the complete idea of this Vector database like uh instead of directly storing our Text data to the database first of all what we are doing we are taking help from the embedding model uh and using the embedding model we are
trying to generating these kinds of vector embeddings then restoring this kinds of vector embeddings to the vector database and guys this kinds of vector database has one beautiful function called Similarity SE so using this similarity sech functionality it can return return you similar kinds of vector represented in the vector database okay I will show you this similarity search as an example like how it is uh doing actually simil search now guys you got to know like what is embedding uh but you don't know like how these embeddings are generated so to give you one example
I will open my whiteboard and there I will try to make you understand so guys I'm inside My whiteboard and here first first of all what I will do uh I will take one pan and here I'll be using U unstructured data unst structure unstructured data as a text Data here I'll be using Text data to give you the example so so first of all what I will do uh let's create one table here so let me create one table uh so guys let's say I have taken one table here so here uh we are
using one Embedding model so let's say we are using one embedding model to convert our uh text to numbers so here you can use any kinds of embedding model you can use open a based you can use uh what to F you can use like uh large language model based you can use anything here okay now whenever uh let's say I have some wordss here so let's say I have something called King then I have something called Queen Then I have something called men and I have something called women then I have some uh let's
say monkey so let's say I have five words here now what this uh embedding model will do it will try to uh generate some of the features on top of the words okay let's say this is my feature column features column okay so what kinds of features actually uh it will generate let's say it can generate um it can generate anything let's say it Can generate gender it can generate wealth it can generate power because this is a neural network based model we can't say like what are the features actually uh it will generate because
U it it will generate these are the features based on the back propagation so it will calculate the loss and all based on that actually it will try to uh find out the best possible features okay for these kinds Of work so here I'm just giving my example that's why I'm taking these are the features okay but it might not like generate these are the features but to give you one example I will be using these are the features now let's say it will uh generate weight and it can also generate something called speak okay
now what happens actually based on the features actually it will assign some of the value let's say let's say here uh my First word is King okay now King has gender or not yeah King has gender and the gender is like male so it would be one and for Queen uh yeah Queen has also gender so it would be zero because Queen is a female so similar wise man man has a gender so it will have one and women also has gender so it would be zero because it's a female and monkey uh let's say
it's a male monkey so I'll assign at one okay now it will come to the next features called wealth okay now Here and here you can see King has wealth right because he's the king and he has lots of wealth so it would be let's say one I'll assign it as one uh Queen will also have wealth because Queen is also part of King and uh man will also have wealth but it would be less than king and queen so let's uh assign as 0.5 and women will have also wealth but less than man so
here I can give 0.3 monkey doesn't have any wealth so I'll Give as zero okay now King has power so I'll give one Queen has also power but less than King so I'll give 0.7 man has also power let's say man has power but it's less than King so I'll give as 0.5 women has also power but it's less than man so it would be 0.2 and monkey uh doesn't have any power so I'll give as zero now weight yeah King has also weight so I'll give 0.8 and let's say king is little bit obese
uh Queen has also weight so I'll Give as 0.5 because let's say queen is Slim and all man has also weight so I'll uh give it as 0.7 women has also we I'll give 0 point um uh 0.5 weight is fine I think monkey also has weight let's say 0.3 now King can speak so I'll give one Queen can also skip speak I'll give one man can also speak I will give one women can also skip uh speak I'll give one and monkey doesn't uh speak okay so I'll Give as zero okay now see guys
beautifully it has generated these are the vectors okay uh by generating these are the features now let's see if I want to represent King okay if I want to represent King so what would be my king Vector so here this is your king Vector so King Vector is nothing but 1 1 1 0.8 uh one so this is the king Vector now if I want to represent Queen so this is my queen Vector so Queen Vector would be 0 1 uh then 0.7 0.5 and one okay now if I want to represent man so this
is the man Vector so one 0.5 0.7 and 1 okay now let's say if I want to represent women okay I think you are getting it like how uh we have generated these are the vector now this is the vector of the women so 0o uh 0.3 0 0.3 and 0.2 0.5 and 1 okay now let's take one 2D Dimension okay and plot these are the vector and try to understand like what will happen so let's say here I will take one two dimensional uh space so let's say this is my two dimensional space okay
so this is my X and this is my y so if you plot all the vector here okay if you plot all the vector here you will see King And uh queen king and queen will appear uh to the similar so here let's say this is my king and this is my queen okay so what will happen uh whenever you will plot these are the vector to the two dimensional space or let's say three dimensional or any Dimension okay you will see king and queen will appear uh like very close together because if you see
here okay if you see here as per the features if you calculate the number if You calculate distance between these are the vector you will see this distance would be like very less okay that's why king and queen would be appearing very closely together and here man and women let's say this is my man man vector and this is my women okay this is my woman so these two Vector would be uh appearing closely and here if you see here uh there is another Vector called monkey okay monkey will appear uh completely Different from these
are the vector because monkey is a different different entity here so let's say this is my monkey okay now if I show you see so it has buil one cluster so this this is one cluster this is another cluster and this is another cluster okay now if you see here uh using this concept called Vector embeddings uh we are easily getting these are the similarity score okay similarity Vector so now if I want to extract any kind of Vector okay let let's say I want Queen okay I want Queen so it will go to the
this cluster and will return this vector and let's say not only Queen let's say it would be let's say I want something called Princess okay I want something called Princess Vector so what what will happen it will come in this cluster because if you calculate the vectors okay of the princess you will see it will have these are the features okay it will have these are the features related To the king and queen okay so now this princess will appear here so that's how actually this uh embedding model Works actually so it will first of
all assign some of the features based on that it will generate these are the vectors and it will plot these are the vectors to the H High dimensional space so here in this case I have taken two dimensional space but it might be 3 4 five and so on okay based on your dimension of the data so whenever I'm using unstructured data Whenever I'm using U real world data so Dimension might be vary okay it might be Millions Dimension it might be thousand Dimension we can't say like whatever Dimension it will have but as a
visualization we can only show 2D u space here okay so this is the complete idea guys I think you got it like how this embeddings mod model are working okay and how it is generating these kinds of uh Vector okay now here in this case actually you have lots of embedding Models so we have open AI open AI embedding model and this model is trained on GPT okay GPT model I think you have heard of GPT model then we have something called hugging face hugging face embedding as well okay hang face embeding so uh it
is already trained on let's say lots of Open Source open source llm okay that means large language model then uh we also Have uh something called uh LMA okay Lama to embedding I'll be discussing like what is Lama 2 and all okay uh as a open source model then we have something called Google uh Google Palm embedding so these are the open source embedding you can directly use so this is from Facebook Facebook research okay and this Is from Google research okay and this one is paid because if you are using open AI uh embedding
an open model so you need to pay this is completely FID and these are the things are uh free okay these are the things are open source and free but to get the Lama to access actually you need to uh like uh uh send some information then they will give you the access I'll tell you like how to take this Lama to access and all okay Everything I'll be discussing about don't worry about like just try to stay with the video I'll be discussing each and everything so guys I hope you got the entire idea
like how this uh embeddings models works okay and how it generate these kinds of embeddings as a vector right so guys here I kept uh another example uh if you just see this example uh your concept would be very much Clear see here I have uh this kind of work okay and we are using these Kinds of embedding model and we are generating these kinds of vectors and if we plot these are the vectors to the 2D Dimension so it will look like that so let's say we have this three word we have this three
word and we have this three word and we are getting different different vector and this is the cluster okay this is the cluster of each of the words if you see here King man uh women is appearing in the same cluster then apple banana orange is appearing uh to The another cluster and football Golf and Tennis it's appearing uh in the another cluster because uh this is related uh to person okay person and this is related to fruits and this is related to sports okay that's why this cluster is different now I think you got
it the vector embeddings like how this Vector embeddings looks like and how it is generating okay I have given you the entire idea but uh one issue actually uh with this Vector embeddings uh which is Nothing but uh whenever you are uh like getting any similar kinds of vector it is applying something called distance formula I think you have heard of like distance formula we have ukle distance okay we have Manhattan distance or let say we have something called cosine similarity score as well okay so using these are the equation actually it Tred to calculate
the similar kinds of vector but again if you see here it would be very much time-taking because uh it will Calculate uh with respect to one by one all the vector then it will give you uh the similar kinds of vector so uh the main problem is like the time if you're using uh this thing directly so it will take lots of time to give you the similar kinds of vector so to overcome this issue actually they propos something called indexes in the vector database okay like we call it as Vector indexes so what is
Vector indexes so let's say whenever we are having this Kinds of unstructured data we are applying this kinds of embedding model and it is generating this kinds of embedding okay so on top of this embedding uh where uh what is trying to do it is trying to add some kinds of indexes okay so this indexes is nothing but it's a data structure it's one kinds of data structure actually so what it will do it will find out similar kinds of vector and it will assign some of the indexes okay and it will make some Cluster
kinds of thing so when let's say you have any new data first of all it will see like which uh indexes actually lies this data so in that index only it will search that query so uh to understand this let's say this is my U example so first of all let's say it has assigned these kinds of indexes now you have one new data now first of all it will see like which cluster actually this this is lies okay so it will go to that cluster then it will return the Similar kinds of vector with
respect to the uh new data you are giving okay instead of searching to all of the Clusters so this is the idea of this indexes uh in this Vector database now uh using uh this indexes uh to our Vector database it's like very fast now whenever you are asking any kinds of query to the vector database and it is returning any kinds of vector so this process time would be very fast now guys I hope you got the entire idea like what Is uh Vector database and all about and what is embeddings and all about
now you can tell a vector database indexes and store Vector embeddings for first retrieval and similarity SE uh so we used to use our traditional database to directly store our data but uh in the vector database first of all we try to convert as Vector then we try to store them to the vector database okay this is the complete idea now guys let's uh discuss some use cases like U why this Vector database is useful like what are the use cases are there so as you can see uh it will help you the longterm memory
for the large language model because it is uh storing all of the data as a vector to the vector database so whenever you are connecting these kinds of large language model to the vector database so it can be uh used as a memory okay memory of your large language model like long long-term memory and it can also perform something Called semantic search I already told you how to perform the semantic search using the formula and using the indexes and all so s based on the meaning of the context let's say if you're giving something called
let's say king and queen so it will uh not only consider the text you are giving it will also consider the context like what is King what are the features of King what is the features of Queen based on that it will uh give you the S uh similar kinds Of vector okay as a output then it can also perform something called similarity S as a already told you like on the text data image data video data or let's say audio data let's say I have one image let's a dog image I will tell hey
just give me some dog images okay similar kinds of dog images so it will uh first of all try to see the like vector okay similar kinds of vector and it will return uh the similar kinds of like representation of the dog to you okay so We call it as similar uh similar SE then it can be also used uh in the field of recommendation system uh because as as you already saw like if you if you're having these kinds of vector together and it it can perform similar SE let's say I want to build
something called recommendation system so let's say I'm asking for like food related content so it will uh suggest me food related content only okay using these kinds of similar SE and All so guys this is the entire idea of this Vector database and this is are the use cases actually uh you can consider okay inside this Vector database so guys I think uh so far everything is clear now let's try to see some weedly used Vector database so you can see we are having chroma DB we8 then F so F has implemented by Facebook AI
then we are also having pine cone then Neo 4G neo4j is a graph based Vector database okay so uh these are actually very common Vector Database people are using in the market and it's like very famous Vector database okay in the market right now so here we'll be exploring all of them one by one so in the next video we'll try to see how we can use chroma DB and chroma DB is a local Vector database and this Pine con wave Neo 4G it's a cloudbase vector database okay and here is another Vector database you
can see f f is also a local Vector database that means if you want to let's say store your data as A locally so what you can do you can use chroma DB or F and if you want to store them in the cloud you can use pine con weate or NE 4J okay now in this video actually we'll start with uh one practical uh demo that means we'll be learning one amazing Vector database local Vector database called chroma DB you can see chroma DB is a local Vector database so they're also bringing the hosted
version of the Chad DB so you can see it is still under development so Once it is done you will be also able to let's say use this chroma DB in the Cloud Server as well so as of now if you want to store your let's say uh embeddings in the local okay that means if you want to set up everything in the local you can use chroma DB Vector database for this so this is the documentation of chroma DB guys you can see this is the official documentation so they have given each and every
guideline so you can follow this Documentation like how we can set up and all okay so what I will show you guys uh I will show you one uh complete practical that means we'll be uh experimenting this chroma DB on the collab notebook okay so guys let's open up my collab notebook and let's see how we can use this chrom ADB Vector database so guys as you can see my notebook is connected now if you want to test it whether it's working or not just write one message here so I'll just Write print uh okay
so it should uh yeah so it's working fine okay so now the first thing I'll be installing some of the uh like required packages like uh I need to install the chroma DB and all so there is the guide actually they have given like you need to use peep install the chroma DV and how to like uh uh set the client and all everything they have given okay so first of all what I will do I will just set up this chroma DB so To set up the chroma DB you will need some more required
package okay some some more dependency I'll tell you like what are things you need so just right peip uh hypen Q uh install uh first of all I need something called chroma DB so I'll copy the name from here proma DB then I need something called open AI because I'm going to use open Ai Embeddings and open AI like you can say LM model okay that means GPT 3.5 turbo model so that's why I need openi and I also need something called Lang chain because I told you I'll be using Lang Chen here Lang Chen
and there is another dependency unit something called tick token okay tick token so if we are using open AI embedding so you need this package actually Tik token now yeah I think these are the things actually I need now let me install it and guys you Need openi account for this video actually uh so just try to create an account here uh otherwise uh you won't be able to do it because you need the open API key okay to run this uh like you can say notebook so guys installation is done now if you want
to see let's say any uh version of any package so what you can do let's say I want to see the chroma DB version so I'll just copy the name and here you can just write Peep Show um pep show and just uh give the name of the package now if you execute it will show you the version and every uh information about this package see guys uh name is chroma and this is the specific version and uh this is the entire summary of your uh like chroma DB package okay so that's how you can
see for all the packages okay you have installed now uh here first of all I need to uh download uh some data uh Because here I'm I'm showing you Vector database example and there actually I need the data okay to show you like how to store and all so uh for this actually what I will do so there is a actually this Dropbox links actually you can see this is the Dropbox links and inside that actually you have some article data if you see Google article is there then you have some AI startup article Amazon
article then you have databas article chat jpt article so we have different Different articles here okay and this is a txt data txt format data now if I open any kinds of txt file see this is the article related to this uh default okay generative VI so they have written all the article and all okay so I'll be using this kinds of data um you can use any kinds of data it's up to you but this is the open source data actually I got uh from the internet and this is the link of the data
so to download this data first of all I will copy this link And here I'll be using something called W gate to download this data so I'll just write w w gate so Wate is a Linux command and here we are using Google collab and Google collab is running on the uh Linux operating system okay so it is running on the Linux kernel so that's why you can execute any kinds of Linux command here so here I'll just write w k um hypen q and here I'll just paste this link okay I'll paste this entire
link now once it will download uh let me show You how it will download so if I execute this cell left hand side you will see it will download that zip file okay see new article. zip file this ZIP file has been downloaded now I also need to unzip this ZIP file so I'll just write unzip hen q and this is the name I'll just copy the name I'll just copy the name and here I'll just uh tell the directory like where uh you need to um I mean unzip it So I'll give the same
name so this is the name new uh underscore articles okay so it will create one folder new UN articles and inside that it will uh unzip everything okay now if I show you it's done now if I refresh the page I think inside new articles uh now see guys all the articles are present okay now let's if you want to open it you can also open and see so this is the article and all about so now let me close this Thing so we have successfully uh downloaded our data now what I need to do
I need to uh first of all create one openi API key and uh I'll be setting up my environment okay uh so for this actually what I need to do I'll uh first of all open up my openi account so you just also open up your openi account just say uh just search like open.com and here just try to loging with your account account so let me quickly log with my account so guys as you can see This is my account now here I'll just click on this personal my U this uh this is my
profile I'll click here and here you'll get one option called view API Keys okay now I'll click here and here you will see uh API Keys option okay now see previously I created some of the API Keys that's why uh you can see here but for you it might be completely empty it's completely fine so you'll get one option called create a new secret key okay just click here and just give the Name so in this case I'm using um Vector DV so Vector DV I'll just name it okay you can give any name I'm
just giving Vector DV and create the keys now you need to solve one puzzle here uh it will verify whether it's uh whether you are a human or not okay so I'll just click here now it's telling just try to move this object uh with respect to this hand Direction so let me move it and submit I think uh yeah attemp is Complete now uh I will get my secret key yeah now I'll copy this key and here uh I'll paste it okay as of now I'll paste it and I'll just uh give a comment
here uh setting up um setting up environment and don't share this key with any one guys otherwise they will also able to access your uh like you can open AI account I will be removing this keys after this recording so that's why I'm showing now here first of all I will Import operating system package called W and here I'll set the environment so I'll just write o dot environment Environ okay and here you just need to give the keys like uh which name you need to save it so I'll just give uh open AI underscore
API underscore key and equal to I'll set my API key here so this is my API key I'll copy or I'll cut it and here I'll set it yeah That's it okay now I can remove this uh cell now let me execute done now I need to import some of the necessary Library so import so so first of all I need to import something called uh chroma okay so if you're using langen so langen already has this chroma API because I already told you uh the integration so it is the integration with the langin as
well as the LMA index so instead of uh Like importing directly like import uh import coma okay import chroma import chroma we have to uh import it uh like from The Lang chain itself because here I'm going to use langen framework okay to build my application that's why I need to call it from the langen but let's see if you're using it for any other uh like task parpose let's say you are not creating any generative VI application let's say you are doing some different thing okay Different application at that time you can directly import
it and all the functionality will remain same only all the code will remain same only you just need to change the import okay at that time so here I'll be using Lang change so I need to import it from the Lang itself so how to import it so just write from l do Vector store so there is a uh function you will get called vector vector Stores import so I'll import my chroma so I'll just write chroma then I also need to import uh open a embedding because I think you remember let me open my
board uh I think you remember first of all what I need to do let's say I have my unstructured data so let's say this is my data so the first place what I need to do I need to use something called uh embedding model Embedding model so this model might be anything so in this case I'm going to use uh open AI open a embedding model open a embedding model I'll be using so you can use any model here okay then after that it will return me uh my U vector embedding Vector embeddings okay and
that Vector embeddings I will store inside my Vector database okay Vector DB so in this case I'll be Using something called chroma chroma DB chromat okay so this is the complete idea so first of all I need to import this uh embedding model as well so here I'm I'm going to use opena embedding so I can directly import opena embedding so let me do it so here I'll just write from lanen do embeddings uh import open AI embeddings open eddings okay then I Also need to import open AI because I'm going to use openi large
language model so again I'll just write from Lang chain uh llms that means large language model I need open then uh I need to load my data as you can see this is my data and it is uh present inside a directory okay and again this data format is a txt data okay if you see it's a txt data so for this I again need two libraries so I'll Just write from Lang chain so guys I'm expecting you are already familiar with this Lang chain because if you check our YouTube channel so we have already
covered this Lang chain tutorial and all okay so you can go with uh that tutorial and you can learn the Lang chain so here from Lang chain U there is a uh class called document loader so just write document loaders okay so first of all I will be import something called directory loader Okay directory directory loaded because uh my data set is present inside the directory that's why I need this directory loaded then I also need to load my text loader because it's a txt data right so I'll import the same thing only uh sorry
it should be uh data loader I think uh document loader document it's a document loaded okay now it should be uh text loader text loader okay uh yes so as of now I need these are the libraries uh And if I need anything I will be importing later on so let me just quickly import them now we'll be loading the documents so load uh data so guys uh to load the data just write uh directory loader directory loader and inside that you just need to mention the directory so here this is the directory I'll just
copy uh the name copy path and I can give it here so this is the directory and inside that actually I have but txt data okay I have If you see I have txt data so I only want to load the txt file so if I let's say I have some many other file okay I will be ignoring so for this there is a parameter you you can give called Globe okay just write Globe equal to so it should be txt file so all the file just load is means all the file okay it should
be Dot dxt then you also need to define the loader class okay so just write loader loader class and and should be my Text loader because uh all the file are txt format okay that's why you need to give this uh text loader okay now it will load up your data and I'll be storing this data inside a variable called loader so this is my uh object of my data loader now let me uh execute it now to load the data you need to call one function called uh loaded. load okay if you call this
function it will load the data and This data will store it in a variable called document okay and see guys it will uh load all the data inside this document see guys this is the all of the text actually you can see here so it has loaded all the txt file one by one and it has has extracted a data and this is your entire data as you can see so let me show you see guys and this is a document object okay this is a document object now uh we have successfully Loaded our data
now guys before converting this data set to the embedding and storing uh it to my chroma DV first of all I need to apply something called text Splitter on top of it basically I need to convert my entire uh data as a chunks because as you can see this is my complete Corpus okay see this is my complete Corpus if I show you so if I show you the see this is the complete uh data okay this is the complete Data Corpus but this data is Like very huge okay I can't like you can say
directly convert it to Vector okay so if I'm converting so what will happen Okay let me explain so what is the use of this chunks okay so see so whenever you are using any llm model okay whenever you are using any llm model so it has one input size okay it has one input input size as well as each has one output size okay output Size so whenever you are giving something as a input to the llm okay you need to always take care about the input okay so I'll tell you like where you need
to check the input size and all so in this case let's say I'm using open AI open AI llm okay open LM so open LM wise I'll be using something called GPT 3.5 turbo so there there is a model you will get from openi called GPT 3.5 TBO TBO and it has actually one specific input length so that input length Actually you need to uh see so if you want to check the input L just uh search like open model open models so this is the link just open it up and here you will see
all the models are available inside open so we have gbt 4 gbt 3.5 GPT based Deli okay gpt3 Legacy so these are the model actually and and uh here I'm going to use uh this GPT 3.5 Series so I'll just click here now here if you see the first model is GPT 3.5 turbo and this is the description about the model OKAY like uh what are the things they are doing and all and this is the Max uh token length as you can see so it can take a input around 4,000 97 tokens okay at
a time and this is the training data okay like up to uh September 2021 they have trainer data okay so this is the input length Okay but if you check if you check your uh input here uh if I open my data if you See this input length actually will be more than this 4,000 token because here if you see I have loaded the entire documents I have loaded the entire documents all the articles I have loaded okay so there is a possibility uh this input size would be more than 4,000 tokens okay and if
I'm converting all the Corpus as my uh vector embeddings and I storing to the vector database so it will create issue okay so again uh during storing it will create issue and And whenever you will be giving this data to your llm model again it will be creating issue and you will also get some memory error and all okay so it's better to split our data as a chunks okay now what is Chunks so let me first of all tell you chunks is nothing but here I have already written chunks is a size of the
maximum number of the character that uh a chunks can contain okay and there is another concept called chunks overlap okay so chks overlap is Nothing but this is the number of character that should be overlap between two uh adjust chunks okay so let's say so to explain this one let's say I will take one text here so let's say this is my entire Corpus okay now first of all I need to convert it to chunks okay chunks means I'll be taking uh some of the Chun size okay let's say if I am writing chunk size
is equal to let's say 500 so it Will take uh let's say I'll take a new color so it will start from here and it will uh go and it will count like 500 character okay it will count 500 character so let's say this is uh this will end here okay this will end here and this should be one chunks okay this should be this should be one chunks okay so again it will start from here again it will count let's say 500 word uh sorry 500 character uh and we call it as a Token
okay so let's say this is uh here it will end so this should be another chance okay so let's say this is another chance this is another chance now again let's say it will start from here it will start from here again it will count let's say it will end here this is another chance let's say again it will start from here and it let's say it will end here okay I'm taking it randomly just try to consider so this is another chance and it will start from here it Will end here okay now how
many chunks we have got so one chunks two chunks three four and five okay so here I got five chunks okay five chunks five chunks of the data see now if if you count this input length okay now it should be less than it should be less than less than my 4,000 uh I think uh what is the number it should be 4,000 97 okay 4,997 okay now my input now my input uh size is less than 4,097 okay so this is the idea of creating this chunks okay instead of converting my entire Corpus to
the vector first of all I need to create as the chunks okay then I need to convert it to Vector embedding then I will be storing and that embedding I will feed my llm model okay and there is another concept called chunks overlap okay so what is Chunks overlap so if you set This chunks overlap let's say I will set this chunks overlap chunks uncore overlap overlap let's say I have set as 20 so what it will do so whenever uh first of all I will gra these are the chunks okay now let's create a
new chunks again so let's say whenever it will create a chunks let's say it will start from here and Chun size is 500 so let's say it will end here now you have mentioned chunks Overlap is equal to 20 now what you will do instead of starting from here it will go back and it will count 20 character okay it will count 20 character let's say this is my 20 character okay now it will create the chks form here now again it will count and let's say your 5 500 character will end here so this
is my another chunks okay so basically chunks overlap means like how much overlapping you are needed from your previous chunks okay so that that is called chunks Overlap so it helps you like to understand your context of that text okay U instead of using only Chang size you can also use chunks overlap so that it can also got to know like what was the previous context okay whenever it will let's say see I'm giving this chunks by chunks let's say this is my chunk one the Chun two and Chun three so okay so instead of
giving directly the chunks if I'm also mention s overlap what will happen your llm model llm Model automatically will get to know okay so this is the context actually so this this sentence okay this sentence and this sentence so after this sentence actually this sentence is coming okay so that's why this Chun SI this chunks overlaps actually helps us okay this Chun overlap actually help us to understand the context of the uh entire story okay so that's why we'll be using this two concept called Chun size and chunks overlap okay now I think this Thing
is clear now let me show you how to do it using Code so I'll come here now here first of all you need to to import uh something called recursive character text splitter okay so from langen from langen uh dot text splitter textor uh splitter input recursive so recursive character text splitter okay so this is the uh class actually we'll be using to convert my entire Corpus as a chunks okay now I'll just call this Class and inside that you need to mention two parameter I think I already explained one is the your chunk underscore
size okay CH size so let's define as 1,000 okay I am considering 1,000 character uh as my one chunk okay you can give any number so it's a hyper parameter tuning you can uh set any number here but I I saw like people are starting with 1,000 here okay and there is another parameter will give called Chunk underscore overlap so chunk overlap I will give let's say 200 okay so again it's a hyper tuning now guys I have created this object now I will save this object so I'll just uh create a variable called text
splitter now guys finally I will convert my entire Corpus to my chunks so I'll just call this object and here you need to call split dot Split _ documents and this is my Ed documents okay this is my raw documents so I'll give it here and I'll store in a document variable okay or I can store it I think in a text variable I think it should be fine so let's make it as separate now uh let me execute it's done now if I show you my text guys so you will able to see it
has converted to chunks and how many chunks it has converted I will also show you See guys uh so if I just go up so guys see this is my first chunks okay and it is ending here then again it is another chunks okay then it is ending here then again this is another chunks and it is ending here and from uh and if you see in each chunks actually you have thousand character okay th character and here if you see the chunks overlap so if you read this text carefully so you will see some
of the sentence are coming okay in this uh new chunks as well okay Because we have set this chunks overlap as uh 200 okay so see this is my entire chunks now if you want to see the length like how many chunks it has created so just print the length so I'll just read as my text so it has created 23 uh3 chunks okay all together now uh it's fine now if you want to see any chunks specific chunks you just give the index let's say I want to see the number one chunks this is
the number one chunks and I can also print number two now uh we have successfully converted our uh like Corpus to chunks now what I will do I will uh create my uh DB object okay like vector DB object so just write the comment here creating DB object so as I already told you chroma DB is nothing but it's a local database so first of all you need to create a Local uh like DB here okay local DB so you need to specify the name uh so here I already prepared uh this code so here
uh you need to first of all create one par directory and the name can be anything so here in this case I'm giving as DV that means database and uh you need to specify the embedding first of all okay like I think you know why we need the embedding because uh first of all I need to convert my text to embeddings okay that means Vector then I Will store this uh Vector okay to my Vector database which is nothing by my Comm so in this case I'm using something called uh open a embedding I think
you remember here we imported so open a embedding so I'll just uh call it it open a embeddings and by default if you execute uh this code so let me show you which embeddings model it is using so it is using something called text embedding add a model OKAY 002 this is the model now you can search also Here uh so if you check here so it has this model called text embedding um add 002 this is the model it has also another version text uh add a 01 okay now we have successfully uh loaded
our embedding model now what I need to do I need to create my Vector uh instance okay uh I need to like create my Vector uh database instance so to create it so this is the code for it it's a simple code only so first of all I have created one variable called Vector DB and here I'm just calling my chroma I think you remember we imported chroma from my Lang chain I'm just calling this chroma so inside chroma you have one function called from uh documents okay from documents now here you just need to
give the data like you just need to give your text like you just need to give your chunks like which uh chunks actually want to convert as a vector so in this case this is my chunks called text as you can see this is my text I'll just Copy the name and here I'll just name it text okay and it will also take the embedding model okay so this is the embedding model I want to use so in this case you can use any embedding model you just need to import it here okay so here
I'm using openi based embedding model so that's why I have initialized it here so here I'm giving the Ming model and here you also need to give the proc directory like where it will save after let's say Converting to vectors where it will save so it will save inside this DB folder okay so here I'm giving the directory so these three parameter users only need to give here okay that's it and if you're not giving any embedding model uh inside chroma it will automatically take Opia embedding model okay I think they have already uh written
here if you see here uh embeddings so they're automatically I think taking uh some embedding model using open API key and all okay so they Have already mentioned here like it will automatically load but it's better practice to mention okay always better practice to mention here that's it now if I execute it uh so first of all what you'll do I'll just remove this directory now see it is converting everything uh to the uh vectors okay all the text has been converted to vectors now I need to uh save it to my disk okay now
see it has uh converted everything to the Vector now I need to save it to The database okay now this is the code for it just write Vector DV persist and Vector DV uh is equal to none now if I execute it now you'll see it will create One DB Here Local DB now if I refresh see guys DV has been created now inside that it has saved all the vector now see it's a binary format you can't see the vector here so that is the issue with this local DB but whenever I'll will be
showing you like pine cone And wave you will able to see the vector like how it it has generated the vector okay how it has generated the embedding everything you can see in the cloud okay you can like see all the list of the vector there okay so that was the beauty of this Cloud uh uh Vector database but again it's a local one okay so that's why it's a binary presentation here so yes guys we are successfully able to save our Vector here okay now uh let's say you want to uh load it So
how to load it so this is the code for it so here you just need to mention chroma okay just call the chroma and here you just to mention the the directory you have your data so this is my database DB so here I'm giving the par directory which is nothing but DB in this case and here you also need to give the embedding okay so this is the embedding I'm using uh now here I'm giving my Pur directory so in this case It's DV okay now it will automatically load this vector and everything okay
from this DV itself because it's a binary presentation I already told you okay and as well as I'm also giving my embedding model now let me load it now I think you remember I told you something called similarity s okay now once we have loaded this uh Vector uh to my uh Vector database now you can apply similarity s on top of it okay we call it as uh retriever okay retriever so now Let's show you the demo of the retriever so I'll just comment it out make retriever okay so uh to retrieve any uh
specific information with respect to the question you are asking you need to set this parameter called Vector as R okay Vector DB so this is my object Vector DV as rer and this is my rer object now let me execute now here I'll just call this R object and uh here there is a parameter called Get retrieve documents okay that means that means I'm trying to tell my uh Vector database uh whatever question I'm asking try to uh give the similar kinds of uh you can say data okay with respect to the question I'm asking
so here I'll ask one question let's say here I have Microsoft articles as well I think if you see here I think Microsoft yeah Microsoft articles is also there so I can ask one question just like how much micro how much Money did uh Microsoft race okay so let's say this is my question now uh by default uh I'll store it in a docs so by default it will return for uh relevant uh answer okay four relevant answer if I print it see it will return return you four uh similarity answer okay for most similar
answer okay like whatever Vector are closed I think I was explaining in the theory so it is the Most uh relevant and most similar answer okay four most similar answer so if I show you the length so you can also set this parameter like how many relevant information you want see it is uh rning me four so let's say you only want let's say two relevant answer okay so in this case you can set this parameter Vector DB as retrieve search keyword as K2 okay K me like how many answer you want okay so let's
say I have given two now if I execute It okay done now if you want to see like search types so so it's a similarity Matrix I think they're using similarity equation they're using cosine similarity to calculate this uh answer okay like the question you are asking basically it will match with the vector like what are the closest Vector okay it will come so it will return you so they're considering this similarity uh coine similarity Matrix I think they are okay Qu similarity equation and this parameter will give you like number of um return actually
it will provide so uh it will only provide two uh answer okay two answer now if I again execute this code now see guys now length is two okay it is only returning two answer okay so using this parameter you can set like how many uh like answer or how many uh similar answer you want okay from your Uh database okay now see guys it has written two or let's say four similar answer but I want a specific output let's say the question I asked okay let's say if I print my document so it's a
different kinds of answer actually it will return okay but it's not understandable actually it's like U it is giving like lots of answer and all okay it's giving lots of text but I want like a relevant answer like I just asked like how much money did Microsoft raise So it should give me like that is the money Microsoft raise okay so it should give me that kinds of answer but it is returning this kinds of document okay these kinds of chunks so uh to get a specific output okay from your uh data you can take
a help from the L M so what you can do let me show you the diagram so let's say this is my this is my Vector DB Vector DB okay now we have stored our uh Vector embedding here Vector Embedding we have stored here okay now you are uh giving uh any kinds of query let's say this is your query so here in this case let's say you ask like how much money uh Microsoft raise right so this is your question so it is returning let's say four or let's say two output output documents okay
but this for output documents is not readable okay I am not getting my specific answer So what I can do I can integrate LM model so let's say I will initialize my large language model here which is nothing but GPT T point five turbo in this case Okay turbo so I'll give this answer to my llm model okay I'll give this answer to my llm model and llm model will analyze this answer it will also Analyze This quy okay it will also analyze this quy with respect to that it will give me the Authentic output
okay the correct output correct meaningful output okay that is the power of L&M I think you know why it is powerful and all okay because it is it has strained with huge amount of un instruction data so it can perform all kinds of task okay that's why uh we will be using llm okay to retrieve our specific answer with respect to the specific question I'm asking okay so this is the idea so now to do it actually uh first of all I need To make one chain okay so let me just comment it out I
think you know what is chain in uh Lang chain okay so I'm not going to explain because I think you already learned this uh chain in Lang chain okay why we use uh chain and all so first of all I need to something called retrieval QA chain okay so I'll just write from Lang chain from Lang chain do chains input Retrieval so whenever you are using uh Vector database okay it is returning retrieval output right like related output so you need to use this retrieval qway that means retrieval question answering chain okay so it will
help you to ask the question okay to the data actually you have stored in your DB okay so that's why this chain is required Ral question answering so I'll import this chain now here first of all you need to make one chain so I have already make Made one chain here so this is the chain so I'm just calling retrial Q chain and there is a function called from chain type so here you need to specify the llm model so here in this case I'm using open AI llm model okay so let's let me show
you how it will give you the model so if you just run it in a new cell now if you print this llm so this is nothing but your GPT 3.5 turbo yeah so it's not giving the name but if you see the documentation by Default whenever you call open a so it will give you something called gbt 3.5 turbo okay this model actually uh if I gbt 3.5 so this model actually it will return okay so I'll be using this llm and here I'm giving this llm model and chain type is tough okay I
think you already know what is chain type and all and retriever so you need need to give the retriever object here okay this is the retriever object so basically whatever question uh you are asking okay Whatever question you are asking uh it will return you from this retriever right because you have given this retriever here okay so that's why you need to give this R object here and you also need to give the return Source documents okay return Source documents means U it will return the source also like see we have lots of uh we
have lots of data here if I show you we have lots of article here okay now which article it is referring to give the answer okay It will also give you as a reference that article okay so this is the idea of this now if I execute now let's ask one question and print this response okay so here I'm calling my qn and this is the query same query I'm asking how much did Microsoft raise uh the money okay so this is the quy I'm giving my inside my chain and this is the response of
the llm now if I print this see guys this is the query now it's sing Around $ 10 billion okay U raas like this Microsoft and uh this is the source document okay but it's not readable so what you can do actually uh you can write one function here process LM output or process llm response so this is the function you can write so whatever response it will give it will process only like it will give you the result source and the answer okay now if I execute now if I call this function here Now
I'll just remove this response here okay now I think it should give me the uh yeah so now you can see this is the clear output now around $10 billion and these are the source actually it has referred this two file as a source now see guys uh we are able to successfully uh store our Vector to the vector database and we are also able to make the query on top of that and we we also saw that like how we can found a similar uh of you can output okay using this Retrieve okay using
this retrieve option okay so you can ask any different question also so let's say I will ask another question here so uh what is the news about Pand so there is again another Pand articles you will see inside the data now again I'm giving inside my chain and it is giving the response and I'm just processing the response now see Pand is a startup that has raised $3 billion uh in a series B Round and bringing its total raised okay so this is the complete answer and this is the source it is giving okay I
think I don't need to print this yeah so this is the source so yes guys that's how actually you can uh do it and one thing I also want to show you like how to let's say you want to delete this database so how to do it so deleting DB so it's like very easy first of all you can make a z file okay of your existing DB so that later on if You want to retri it you can retri it okay so first of all I'm making it a z file then I'm I'll be
deleting it so first of all I'll make a z file now see guys if I refresh so first of all it will make a z file of this DB okay now I'll be deleting so this is the deleting code so it will clean up everything now if I refresh now see uh database has been deleted okay now let's say uh you want to um get your previous DB okay like you had let's say specific Some specific or let's say important Vector there you want to uh get back so what you can do you can unzip
that zip file okay so this is the zip file you can unzip now again your database will come now what you need to do you need to restart the run time okay so just click here and there is option called restart on time if you restart on time again you can able to use your DV same wise okay like we already imported out DB right here so here that's how you can uh load Your DB again okay so this is the functionality actually chroma DB provides okay uh if you see the documentation they have written
everything even they have also given the stter notebook and all okay so guys I think as of now you got the entire idea about the chroma DB like how we can use chromat Vector database and how we can connect the large language model and how we can perform this uh let's say Vector sarce operation right so one thing I Just wanted to show you see if you click on the integration see this chroma DV is having like different different integration that means you can use chroma with openi Google gini CAD hugging face okay instructor hugging
face embedding server G roof flow okay Allama embeddings everything you can use okay everything you can use with this chrom RB now you can see some framework integration like it can also support Lang chin llama index then brain tast Open uh llm uh llm tree okay then uh stream lead highest stack open lead so these are the framework also you can connect okay that means it has actually all the integration uh if you're using different different framework different different let's say large language model different different platform everything it can support okay so as of now
we saw like how we can use with the help of Lang chain going forward we will be also learning how we can use with the help of Llama index and so on right so yes guys this is all about from this video in the next video we'll be learning another amazing uh Vector database called uh Pine con and pine con is a cloudbased vector database that means we can store our embeddings in the cloud platform as I already told you Pine con is a cloud-based Vector DB that means you can store your embeddings to the
cloud so if you want to use pine cor so search for actually pine cone in the Google so Pine Con so you will get the first website just try to open it up and make sure you create one account okay you just need to create one account inside this website so for this you just do the sign up operation so with the help of your Google account you can create one account so once you have the account just try to do the login so I already have the account so I'll just try to do the
login here so once you have logged in guys you will get this kinds of Dashboard so guys now let's try to see how we can use this Pine con Vector database to instore our embeddings so guys what I've done I already recorded one video previously on this Pine con that detailed video how to use this Pine con and all so I'll add this video here so this video will give you the entire idea how we can use this Pine con with the Lang Channel all so guys as you can see recently they have updated their
dashboard but previously uh it was Actually different kinds of dashboard but see functionality wise it is same you have to use as it is but somehow actually they have updated some kinds of actually dashboard you can see uh previously this index was like out that means there was a option called index directly create index option was there but they have actually created this index now inside databases okay see ins databases now there is a index option so guys this Pine con is not a free Vector Database for this actually you have to take the subscription
but if you creating let's say new account for the first time here you will get actually free actually some storage that means 2 GB storage you will get that means you can only create one index there that means one cluster you can create so if I click on the create index you will see I can only create one index and and whenever you are creating uh free index make sure you are selecting Google Cloud Okay not any other Cloud otherwise actually it will ask for the charge see I already occupied my let's say free plan
now if I click here see it will ask for the upgrade that means I have to upgrade with their let's say subscription I already utilized the free plan but for you actually will see the free plan option is there because you have created the new account but for me I already use this uh account okay that's why uh it is showing I don't have Any free plan right now fine so make sure guys you just try to create on new account and try to use this Pine con and to create the API key guys just
click on the API key and here you can create the API key okay and this API keyy will help you to connect with your uh this Pine con server okay from your python code now let's try to see how we can use this Pine con uh Vector database with our Lang Chen and guys one thing uh whenever you're installing Pine con client make Sure you're installing 2.2.4 okay this particular version because uh whenever uh I was using this pine cone that time actually this version was the stable version and try to use this version guys
because this version will help you a lot because it is like it is having like very easy syntax to store the embeddings to the Pine gon server so make sure you're installing this 2.2 four version so let's try to see how we can use this fine gun right now first of all you need To install uh some required packages so just let me write down the comments here so I'll installing these are the packages so I'll be installing something called langen then uh Pine con client okay so if you want to so if you want
to use this Pine con uh Vector DV so you need to install this package called Pine con client and this is the python package and it will help you to connect With this Pine con okay then I also need P PDF Li because uh here I'm going to load PDF data so previously I showed like how to load txt data now I will show you like how to load PDF data so I'll show you like all kinds of data like how to load and all okay so that's why I need uh to install these are
the packages so let me install so guys as you can see my installation is done now I'll take some new cell and here I also need open Ai And T token okay I think you already saw from my previous demo so openi is also needed because I'm going to use open embedding and as well as this tick token uh because I'm going to use open embedding so Tik token is the dependency there now installation is done now let's import some of the Liberties here so here I'm going to import these are the libraries and I
think you are already familiar with these are the libraries why I'm loading and all I also Need something called operating system so I need this Pi PDF reader loader because here I'm going to read like PDF data so for this actually I need to load this one from documents loader but previously I was loading like text loader okay so this is the difference then recursive character text splitter means like you want to make uh your data as a chunks because I told you why this chunks is required and all so that's why this is needed
and open embedding I will Be using and open llm model that means GP 3.5 turbo so for this actually I'm using this open llm then uh from uh Lang chin I'm also going U and here from langin I'm also importing like vector restore which is nothing but my Pine con but previously I was loading my chroma DB and uh retrieve retrieval QA means like you can also make some uh question answer okay on top of your uh data uh you have stored in the vector database I think I was discussing okay with the Help of
this chain then you can also import something called prompt template uh basically you can uh give your custom prompt here so this is also possible okay I'll show you one demo here so let's import this at the library so first of all here what I will do I will create one folder uh so let me first of all give one title like load data so first of all I'm going to create one folder here um called PDFs so inside That I will download some of the PDFs I have in my Google Drive so this is
the code to download anything from the Google Drive so here if you see in Google Drive you will see this ID okay of any kinds of file otherwise you can U download it and manually upload it inside okay it is also fine but I'm just downloading so I have one uh yellow V7 paper in my Google Drive and also I downloaded one uh resume okay like uh from Google itself I've downloaded one Resume uh so called so the name of the resume is like richel G CV okay so this is the resume I have downloaded
so this two file I'm going to download uh inside my PDF folder and with the help of this G down package you can download anything from the Google Drive now if I refresh and if I show you my PDF so see these are my data are available inside that okay now what I will do first of all I need to extract the text from the PDF so for This uh let me give one title here so how to extract the text from the PDF so this is the code you need to take the help from
P PDF uh directory loaded and here you just need to provide the path of the PDF file you have so this is my path like PDF so here I'm giving the path and once uh you have given the path you just need to call this load function and it will automatically load the data so let me show you so it has loaded my data now if I Print my data you will able to see my data see guys all of the data has been extracted okay now what I need to do I need to uh
convert it to chunks because I can't um directly convert it to my Vector because uh input size might be vary because I told you uh whenever you are using a large language model so it has one input size okay so you always need to take care the input size so that's why uh we need to create as a chunks so uh to create the Chun so this Is the code I think you remember so I'll be just calling this recursive character text splitter and here you need to give Chun size and chunks overlap okay I
I think I have already discussed what is Chun size and Chun overlap then uh simply I'll just uh execute it then I will apply on top of my data okay so this is my entire data and here I'm passing this in my text lior object okay now it will give me that text chunks now let me execute and show You so guys uh this is my text chunks okay see uh this is another chunks this is another chunks so let me see like uh how many chunks I got here so length takech chunks so I
got around 168 chunks here okay now if you want to see any specific Chun so you can also print let's say I want to see number one you can also print for other this is number two okay now here if you read it carefully you'll able to See like also some chunks overlap is also there like some uh text is coming from the previous chunks also okay now guys uh you can also print like third one as well now guys uh first of all I need to download the embedding so download the embedding first of
all you need to set up the environment uh you need to first of all what you need to do you need to set up your openi API key okay so I think you remember uh I can use my Previous notebook so in my previous notebook I already created this API key and I already set up my environment I'll just copy copy this code and here I will paste it okay now I'll set my OPI open API key so let me set it here now first of all I need to download the embeddings so this is
the code for it so it will give you the embeddings like open embeddings now first of all let me test one uh thing here so what I can do uh I'll just call My embedding like I just want to show you like uh what is the length of the vector it will generate okay so I'll just call embedding do embed embed query and here if you pass any word let's say I will give hello okay now if I execute uh you will able to see the vector okay see this is the representation of the hello
okay and this is the uh Vector Okay this is the vector representation numerical representation so I'll store it uh u in a variable called r okay now if I uh show you the length of this results so let me see the vector length yeah so uh by default actually if you're using open AI embedding so the each Vector length would be uh 1,5 36 let's say if I give any any word here let's say I'll give uh something called uh good okay let's say good if I Give good so you will see uh the length
would be same okay now if I give let's say anything like uh I can give hello or let's say how are you how are you so again the length would be same see length would be same so basically what is happening I think I was discussing uh this thing in my uh uh theoretical lecture so whenever it is uh like trying to generate this kinds of vectors okay so it was following this approach basically it was creating some Features okay so with the help of the features actually it was uh generating these kinds of vector
so basically what is happening so the sentence you have given so uh this sentence is representing uh in this Dimension actually 1,5 and3 uh 36 okay this is the dimension of the sentence okay and it is representing in that here I was referring this uh word by word but you can refer as a sentence by sentence so Basically this uh King is uh appearing in this Dimension actually 1 2 3 4 5 6 okay six feature we have cre so this was the six Dimension okay six Dimension uh this six uh this King was representing
there okay but whenever I'm uh using open AI so I have written like how are you there okay how are you is representing with the help of 1,590 um sorry 1,536 Vector okay that is the this is the dimension okay if you if I plot it okay so this should be the Dimension and this is the representation of the how are you okay I think you are getting like how it is generating this vector and all okay so if you give any sentence here so this should be the length so now now I need to
store uh these are the vectors okay like uh I will apply this vectors embedding on top of my data I have uh created the chunks then I will store it on uh in my Pine con Vector database so for this first of all I need to initialize the pine cone So let me initialize it so uh for this first of all what you need to do you need to collect uh uh some uh pine cone credential so you need to collect like Pine con API key and pine con environment okay so how to collect it
so just go to Pine con and this is my uh dashboard and here I already told you how to collect this API key just create a API key and try to just copy and here just replace your Value okay no need to use my value because I'll be removing my keys so you need to use uh your value okay so just try to replace this value okay then you need to uh also collect something called uh Pine con API environment okay so how to collect it so here if I go go to now just click
on index so here you just need to create one index here so first of all it would be empty and here you just only can create one index because it's a free uh one I am using okay now I'll just click on create index and need to give the name so here I'm using let's say this is my test uh or you can give any name I'm I'll give test test index and here you just need to specify the dimension of the vector okay now here I already showed you so this is the dimension of
my each Vector okay so this is the dimension I'll copy this value and here I'll just write it and you can also select the metrics like what kinds of metric you want to use to retrieve Your similar kinds of vector okay so by default it is cosine you can also use ukian and Dot product but uh cosine is a little bit fast okay because if it calculating cosine distance okay so uh uh it will be like little bit fast okay to calculate because I already I was already discussing right if I show you that uh
graph so see if I want to calculate cosine similarity within this two Vector so it will just create one angle here and it will give me the Cosine similarity score but if I'm using nule distance so it will just calculate with respect to all the data points I have here okay so that's why cosine is uh first of in this case okay so I'll be using cosine metric now here uh this is the default pod you need to select and this is the free plan I already told you now I'll just create the index now
see guys it is initializing so you need to wait for some times once uh it is done so it will give you this uh Green status okay now see guys has created now here you will see the environment name okay and this is the gcp provider so basically this uh uh cluster has been created in the gcp cloud and this is the region and this is the uh uh environment name so this parameter you can set if you're taking their paid subscription but by default if you're using three one so they will assign these kinds
of provider and region okay now I'll copy this Environment name and here I'll just need to replace okay so for me I have already replaced which is nothing but gcp starter okay now let me just quickly set this uh uh environment variables okay inside my environment now here I will initialize my pine cone so if you visit pine cone documentation so they have given this code like that's how you need to initialize Pine con so here you just need to provide this API key so here is my API key it will read from my Environment
and here is my Pine con API environment so this is the environment so it will automatically read it and it will initialize that okay and here you also need to provide the index name so this is my index index name we have created called test but if you creating any other name you can give the same name here now let let me just quickly initialize this pine cone yeah so it has initialized now what I need to do I need to create embeddings from my chunks so Let me first of all comment so to create
the embeddings from the Chun so this is the particular code you need to execute so first of all uh here whatever embedding I'll be generating okay so I'll store it my pine cone that's why I'm just writing writing pine cone from text okay and here I just written one list comprehension okay I think you know list comprehension in Python okay so basically the chunks uh text chunks I have so I'm just reading One by one then I'm just reading the page content okay then uh this is my embedding object I think you remember so this
is my embedding object and I'm also passing my embedding okay embedding objects and also I'm giving my index name okay that means this is the index name so this pine cone client will automatically take it and it will automatically uh convert your chunks to embedding with the help of this embedding objects and it will rest Inside this index name that means my this index okay inside this index it will store all of my Vector okay and it will give you one object called docar okay now let me just quickly uh execute and show you what
will happen see guys it is taking all of my chunks one by one and it has converted to vectors and it has stored inside the pine con uh cluster now if I go to my Pine con cluster and here if I refresh this Page so guys as you can see I refreshed my page and now I able to see all of the vector here if you see if you just click on browsers so here is the vector okay and first of all it will show you like top uh like 50 Vector here if you see
this is the representation so this is the text and on top of that actually this is my Vector okay I'm getting so you can also copy the vector and you can also see okay and the length of the vector is uh this one actually I think You remember um 1,536 okay this is the length and this is the beauty of this Cloud uh Vector DB you can see your vector here like how this Vector has been created and what is the text with respect to that and you can also see the scores okay like what
is the scores uh uh with respect to these vectors like this is the distance actually okay that's how it create the index okay I think I was also talking about like index in Vector database Right indexes in Vector database this is the score okay using this score actually it try to figure out the similar kinds of vector whenever you start any quity right so this is the beauty of this vector and I love this uh pine cone one because it looks look very uh easy and it's like very beautiful interface it provides okay and you
can also see the second page and you can see all kinds of vector here even you can also change the parameter let's say I want to see top 100 Vector here so if I just U search the query so it will give me all all the 100 okay 100 Vector here okay so yes guys and you can also see the Matrix like how many times you are raising the request and all so it will give you all kinds of log here like if any request is failed so it will give the logs and everything it
will give you okay so yes so this is the simple uh interface of my this pine cone and Vector counts is U 68 because if I show you uh if I Show you my uh chunks okay my chunk size was 168 so that's why you you are able to see this uh Vector count is 68 because I have 168 chunks so with respect to that this Vector has been created okay so yes case this is all about my pine cone one now we are able to successfully store our uh data here and this is the
cosine and this is the dimension and all kinds of host and provider you can see here now let me uh show you like how to query out on top of Your vector and all so for this uh what you can do you can use something called similarity sear but before that let's see if you have already one class T here okay um so what you can do you can load this class T directly instead of storing that uh Vector again and again so this is the code for it so let's say I have already one
cluster so you just need to give the index name which is nothing but let's say test and here you also need to give the embedding like here I'm using Open embedding so here I'm giving the embedding objects now if you execute it will automatically load this uh index okay it will automatically load these are the vector here now see it has loaded this Vector now on top of that you can perform similarity score uh operation now here let's say I will ask one query let's say I have one yolow V7 paper here I think you
saw that so here I'm raising one query like YOLO V7 outp Part from which model OKAY like we have lots of YOLO model right so from them actually YOLO V7 performs actually which model so this is the query I'm asking so now this query I will give it to my uh docar this doers this is the object okay basically it will it will have all the like you can say Vector objects and all okay so here I'm going U so here actually I can directly ask the quy so it has one uh function called similarity
SS okay so inside that you just need to Pass the query and you need to set the K okay I think you know what is K so K means like how many output you want to get okay from your uh search results okay so it will give me three results okay now if I execute and show you uh query is not defined because I haven't executed now let me show you see so it will give me a top uh similar uh three output see this is the three output it has given but again it's not
readable so what you can do you can Take the help from llm so llm if you give this query and this output to the llm llm will try to understand this query and the output and it will give you the best uh uh answer okay from them so here let's ask uh this query to my llm so I'll first of all initialize my llm and here I'll be using retrieval question answer from langen I think you remember I was also using and CH type is stuff and here I'm just giving this ret Objects as my
doc s okay and as ret okay okay you need to give it like that then it will consider you are uh it will you want to ask any kinds of question okay and you want to get the answer now let me initialize this uh question answer uh object now you can ask the query here so I'll ask the same query so this is my query and I will ask it to my uh llm model so I'll just initialize keyway run and if I give the query so it Will give me the answer see guys YOLO
V7 per outperforms YOLO V5 uh L6 YOLO X YOLO ex okay so these are the model actually uh it outperforms okay see it is giving the correct answer now I'm not getting these kinds of answer okay I'm getting my relevant answer now I have one uh uh resume I think you know okay I have one resume also uh Richard uh G uh CV so let's ask some query from this uh resume also so here is one Question I prepared called Richard green uh experience okay like what is what is our experience okay so uh see
it will uh automatically give you that experience as well this is the power of llm see it's giving richel green has a PHD in English from the University of uh uh it is giving like some more information about uh richel green but is pretty good right um yeah so you can ask any kinds of questions so you can um also create one while loop here so let Me show you how to create while loop so this is the while loop I have created and here I'm taking the user uh uh prompt so here you need
to import one Library called CIS basically whenever you are uh you want to exit so you can exit it also so this is like uh input I'm taking from the user and if user is giving like exit so it will exit Okay this code would be exit and if it is not then it will ask take The query and it will return you the answer okay now let me show you so if I ask like what is uh YOLO V7 now if I press enter so it will give me the answer see yellow V7 is
a real time object detection which surpassed all known object detection in Bon speed and accuracy okay it's giving the correct answer now let's ask any query uh Tell me about tell me about uh recel green yeah I think recel spelling is not correct I think it should be now let me ask see uh it's giving the correct answer okay now if I press let's say exit now it should exit the yeah okay guys I think you saw the entire uh like demo how we can use the Pine con and you got the clearcut idea now
in the next video we'll be learning another amazing Vector database called webat webit is also a like you can say cloud-based Vector database there also you can create a cluster and you can store your embeddings so if you search web in Google uh you will get this documentation so here they have already mentioned like how to connect this we8 with the Lang CH and all everything they have given all the integration and all How to install okay how to configure everything they have given so guys what I've done I prepared one notebook this notebook will
give you the entire understanding like how we can use this Webb Vector database now this is the official website of the webat web iio so here you need to create one account first of all make sure you have created the account and after creating the account actually you will get uh free credits with the free credits you can Only create one cluster the same way you would do it for the pine con as well right and again it's a paid one guys you have to pay if you want to use there let's say Advance let's
say I mean functionality you have to pay for that so here we'll be only exploring the let's say free one the free Ty one because free TI one is enough for landing fine later on if you need it you can also upgrade your subscription so if you want to create the account just Click on try now button so you will see uh they will ask for the email and password with that actually you can create your account and you can log with this uh web so guys now let's try to see the Practical how we
can use this web at Vector database so here first of all you need to install some of the libraries so these are the libraries are common so here so guys here now I'm using something called wave so this is the like python package like wave client so You need to install this package to connect with your O Okay o um cloud data database now let me just quickly set up them uh now guys what to do uh you need to collect some of the credential so these are the credential are required so first of all
I need my openi API key then I need my wave API key and wave at cluster okay cluster URL so this three thing actually I need so I think we already created my open API key just try to copy from my previous notebook so Here is my API key so I'll just copy and here I will just try to paste it okay now let's see like how we can collect this W API key so for this just uh visit your U so for this just visit your uh weat dashboard and here just try to create
one cluster just click on create cluster okay you need to give the cluster name so let's give any name so I'll give a test okay so you can give any name so everything will remain uh Same because this is a free free one and it will expire in 21 days because this is the free version free tire I'm using okay but if you want to use for your production grade application so at the time you can take the subscription okay now just try to Cate the cluster so it will take some time once it is
ready so it will give you the status message and guys if you see web also support these kinds of client like python client Java client CLI uh typescript then go so it has actually multiple client support so like those who are working in Java and all so you can also use it now guys see my web at cluster is ready now if I click on detail now here it should give me that uh detail of my cluster now here first of all I need to uh copy this uh uh cluster URL okay just copy this
clust URL here from here you can also copy now let's go back to my wave and here you need to paste It oh sorry uh this is my API so I need to paste it here this is my cluster URL okay now I also need to copy the API so here is the API option just click here and this is the API key just try to copy and don't share uh I'll delete it after this this recording so that's why I'm sharing and I'll just uh past it here okay now I got my three credential
now let me execute them okay so first of all I need to read my data so let me read the Data so here first of all uh so here first of all I will create One Directory called uh MK DRS data okay now data folder has been created now inside that I will upload one uh so inside that I will upload one PDF file I have here so this is the Yello paper actually YOLO related content so you can download any kinds of PDF and you can upload here no issue with that you can either
use text Data Okay doc data it's up to you uh so guys as you can see uh my PDF has been uploaded here now what I will do uh I'll just try to load this PDF okay and uh I'll just extract that text from it so let me show you like how this PDF will look like so guys see this is the yellow uh like presentation okay so I have downloaded as PDF I'll just execute this particular code so here I'm giving the data directory and I'm telling just load the PDF file okay then I'm
loading The data now let me load so it's giving one error it's telling unstructured package not found uh please install it with people install on instruct okay so what is happening actually so if you see here I have downloaded this uh Google uh like you can say uh slide Okay Google slide as PDF that's why it is considering as unstructured PDF okay so that that's why we need to install uh one package called unstructured unstructured PDF so let me Install it so this is the command to install but if you're having like like uh PDF
like very simple PDF so you don't need to install it okay so this is for like unstructured PDF so if you have any complex PDF so you need to install this package so previously I was using like uh ppad and all okay that's why it was a uh simple uh PDF but now I'm using like Google slide PDF that's why it's giving this ad and it's pretty good like it will Tell you like what are the package you need to install and all so it will give you the message as well so guys it's done
now it's telling just restart the run time so let's restart our run time okay now let me uh execute Discord now it is extracting all the text from this PDF you can also upload multiple PDF here uh it's up to you but I have already added one PDF I just want to to Show you the quick demo done now let me show you the data so guys this is the entire data I got from my PDF okay now I need to apply something called text splitting I think you remember I need to convert it to
chunks so let's convert it so Rec character text is spitter okay I'm importing from langin and this is the Chun size and chunks overlap you can give any number here so let me just do do it Quickly now I got my chunks okay see this is my chunks and also let me see the length see all the things are common only we are just changing the data uh database client okay like I'm using chroma DB M code and weave so just client is changing but steps are same I think you are getting now I need
to convert to embeddings so let me just comment it out so here I'm going to use open a Embedding P embedding and here you need to pass the op API key uh if you're not passing it's fine you need to add inside the environment variable so previously we added if you see here environment variable but here I'm not adding environment variable if you're not adding inside environment variable you can directly pass them okay that will also work okay it's giving error open a is not defined because we have restarted The run time so I need
to execute this cell again H now I think it should work now here I'll be uh initializing my we Vector database so if you visit we documentation uh there's the documentation and here they have given the installation guideline then configuration then what is the schema libraries everything they have explained okay so you can visit the documentation and uh you can get all the information But here I already uh gone through the documentation and I uh prepared this notebook so this is the code actually you need to write to initialize your web actually uh that's why
actually I like Pine con one because pine cone looks easy uh okay to me because what I feel like Pine con is like more easy and it's like more uh easy to understand and uh like more clear but if you're using web so you need to write some extra Cod it okay uh but it's completely fine okay no Issue you can use it I I will share my notebook you can use this notebook as it is so here I'm just first of all importing weat from langen you can also import directly it is also fine
now here you just uh so first of all you just need to connect with your wave cluster okay this cluster we have created here I think you remember this is the cluster to connect with the cluster you need to give the API key W API key and I think you know we already collected the oi o API key here and we stored in the video able okay then I also need collected the cluster URL okay both I have collected now cluster URL you also need to give then you need to initialize the client and inside
that you need to give the cluster URL and as well as the open API key also because uh using the open API key it will use the open embedding okay whenever it will convert your um data to Vector embedding so it will use that open a embedding okay so that's why you Need to give the open API key here now I also need to give something called uh authentication configuration so this is the Authentication configuration which is nothing but my API key wave API key and this is the startup period these are some default parameter
you need to give and let's execute okay now if you want to check it whether this client is ready or not so this is the code for it so is ready so now it is true that means my client is Ready now it is connected with my wave client now wave is telling you need to uh prepare something called schema okay here uh they have given uh one client schema so if you visit this client libraries Okay so so here if you see example client schema so you Al also need to Define some schema here
schem is nothing but it just a it kinds of configuration it will take okay as input like you can uh change the configuration with the help of code so that's why These are the things they are giving but if you're not able to get it okay it's completely fine so you just need to give this schema and it will common for every U experiment you'll be doing so this is the complete schema guys okay so client schema detail U first of all if it has any schema it will delete then it will add this schema
so here uh you can give any name okay so I have given chatbot here then U here you can give an description then uh what the vector Actually model you are using you can give the name here so in openi actually you have Adam model okay so here I'm using Adam model I think you remember if I print my embedding so here I was using something called Adam model okay as you can see here I was using something called text Adda embedding add model that's why you can also give the name and these are some
description you can give and after that You can create this schema and you can uh create your client okay now this is the code so you can execute and this thing is not required in Pine con so Pine con can handle these other the thing automatically but in weave you need to uh provide that's why I personally like like Pine con okay now once it is done you need to initialize the uh vectory store okay now this code will initialize the vector so what it will do basically it will take your Documents okay I think
you remember we have um we have uh created the documents okay uh we have created a cluster and from this cluster uh with the help of this embedding we are just trying to generating the vector and storing inside the wave okay now let me St uh execute and show you now see guys now if I go to my wave8 and refresh now it hasn't updated yet so after some times you will see it will Update like how many embeddings had generated okay then uh how many times you have uh you can say did the request
everything it will show you here okay so all the information actually it will give you and these are the and Guys these are the embedding model are available by default with this we so you can use any of them it's completely fine but we are using this Adda model uh open a generative model okay so yeah so guys usually it takes some time around uh Like uh 5 to 6 minutes after that you will able to see all the update here so in between what I can do I can show you the other step now
we have successfully stored our embeddings now now let me just do the similarity measurement that means similarity charts so here first of all I will ask one query called what is YOLO because this is the yellow related content and here I'm initializing this similarity SAR Okay because Vector store is it this is my object and on top of that I'm just writing similarity search here I'm giving the quy and top K that means how many output you want to get so here I'm given 20 you can give any number now it will return like 20
results okay 20 similar kinds of uh results see okay but um I can't actually take the answer so what I can give I can give one chain with the help of Lang chain so here uh there is another Parameter called question answering okay load QA CH you can also use this chain otherwise you can also use retrieve uh qn okay that that is also fine now here I'm going to use open LM now I'll Define my chain and here I need to pass my open API key and uh this is the CH typ and temperature
means like uh if temperature is zero so your model would be like more uh you can say stick to the U answer like it won't Be giving any Randomness and if you're changing the temperature to close to one so it will uh give some random output as well okay so this parameter you can change I think if you have learned like llm so I you know this parameter and all okay so these are like Basics thing okay in llm now let me Define my chain now here I will ask this query so chain. run input
documents so this is my documents this is my entire documents I'm giving okay the documents I have and uh see this is the documents that means I got all the answer right uh 20 answer and from this 20 answer I want my query that means the query I'm asking which is nothing but what is yellow so now with the help of U my llm I will get my authentic answer see see is an algorithm for object detection that is uh uh that is uh unified real time and has higher accuracy it is presented by shivang
Shing and this is the date of the publication okay so yes I'm getting My output and all so now you can ask any kinds of question about yolow and all okay I think you got it like I already showed you multiple time you can either create one while loop and you can ask many question here all right guys I think everything is clear now how we can use this weate uh like vector database so we have learned different different Vector database guys so far now we'll be using all of them to implement different different like
gni application if you Don't know about Lang chain Lang chain is a genv framework with the help of that we can Implement any kinds of generative AI based application with the help of large language model so I think you already used something called py and tensor flow whenever you used to learn deep learning right so inside deep learning we used to implement uh computer vision project NLP project so there we used to use these kinds of framework like P and tensorflow so both Are actually deep learning based framework so with the help of that you
can create a neural network right that means all the functionality this framework provides to implement any kinds of neural network to let's say work with your data to load your image everything you can perform there so similar wise if you want to work in the field of genbi so inside gen if you want to work with the large language model and all so you have to use this Lang Chain framework because Lang chain is having all kinds of integration all kinds of functionality okay so only you just need to learn how we can use this
Lang chain properly and with the help of that how we can access different different large language model and how we can build different different kinds of application I think you saw the chat GPT right chat GPT like it's a very powerful application so if you let's say give any kinds of prompt that particular Prompt actually it will try to remember Okay then if you are asking the next prompt it will also try to remember that prompt because it is using something called memory okay memory inside chat GPT and how this memory is utilizing so this
concept is already available inside langin langin is also having the memory functionality so if you use this langin to implement this kinds of project so you can also create a memory okay memory for that application so that if user is Giving any kinds of prompt this prompt would be also remembered by your application so not only that actually it is it provides actually lots of functionality guys unless and until you are not learning about langin so you won't be able to understand so what I will do guys uh let's uh quickly start our uh this
practical demo of langin because langin is all about practical here nothing theory is there so I already prepared one amazing notebook For you so there I will show you the entire Lang chain demo so guys as you can see this is the collab notebook I prepared so first of all let's connect this notebook and make sure you select GPU because going forward actually we'll be using uh some open source large language model from the hugging fa itself and to use that one you need a GPU based machine okay so make sure you selected the GPU
if you're selecting CPU it's Completely fine it will also work but the execution time would be little bit High okay that is the thing now let me cancel H now if you want to see the official GitHub of langen guys this is uh so you can see this is the official uh GitHub of the langen okay see it's like very active resarch the last comit was 3 hours ago only okay now you can see um they have given all the let's say uh tutorial how we can install and how we can access uh let's say
different Different Services of the Lang chain everything they have given apart from that see uh they are having actually a very huge community that means uh these are the communities supporting this Library a lot that means if anything is going wrong in this Library they uh let's instantly solving that uh let's issue even they're also upgrading this framework day by day now apart from that it is also having one beautiful documentation so let me show you guys This is the documentation link if you just search Lang chain documentation on Google you will see this documentation
now see this is the entire documentation they are having okay now see different different uh things they have developed they have also develop ecosystem like Lang speth Lang graph we'll be we'll be also discussing about Lang spe and Lang graph no need to worry first of all let's try to understand the fundamental of langen like how we can install the Langen how we can integrate with different different let's say platform like you can see it is having different different integration let me show you so if I open the third party integration see it supports anthropic
then AWS Google hugging face Microsoft op and many more okay many more see all the different different Cloud jni servic it is also support all the different different let's say lar large language model platform it is support even you Can also integrate with hugging face that means whatever hugging face model I think I showed you now I think remember all kinds of Open Source large language model is available inside hugging F you can also easily use these are the model with the help of langin and always try to remember guys langin doesn't have any kinds
of large language model OKAY langin is just a framework it's just a framework to work with the Genera VI based application with the help of large Language model large language model wise it will connect with different different platform let's say hugging face let's say open AI let's say AWS let's say you can see different different integation it is there let's say Microsoft okay so those who actually provides these kinds of large language model okay so it can connect with those platform this is the idea only got it so it's just a framework you can see
they have also written Lang chain is a framework for Developing application powered by large language model llm okay I hope it is clear now see uh they have written so many things so what I did guys I just prepared one amazing notebook so this notebook will give you the comprehensive idea how we can install and how we can use this langen I think this is enough for you okay to learn about langen now see this is the overview so these are the things actually be covering inside langin like Installation we'll be learning about how we
can like access different different large language model we'll be learning about prom template chains agents and Tool memory document loader and indexes okay so these are the thing we'll be learning apart from that we'll be learning some more advanc let's say technique like we'll be learning about ecosystem like Lang speed Lang graph okay these are the thing also we'll be trying to cover now let's see how we can Install the Lang chain so this is the command guys to install the Lang chin just right P install Lang chain and with that you need to install
another package called langin Community now let me install the langen okay so after installing I just need to set up some environment like I need to set up my open API key as well as the hugging face API token okay why because here I'll show you how we can uh use open okay open language model if you Want to use let's any kind of commercial model how we can access it even I will also show you how we can access any kinds of Open Source large language model from the hugging face uh let's say Hub
okay so for this I need to collect this authentication API I think you know we have already generated One open API key so see here I have already written my open API key so you can also write your open API key and you need to also give the hugging face API token and Where you will get the hugging face API token go to the hugging face platform click on the profile go to the settings now left hand side you will see access tokens now you need to create a token for me I already created the
token guys you can see Lang chain now if you want to create just create a token give the name let's say I will I'll give test okay after that give the read permission and create the token now copy this token and try to mention it here okay I hope It is clear now let me delete I think I already created so I don't need it I will delete it okay this token so my token name is langen you can give any kinds of name it's up to you now let me set these are the token
inside my environment okay it is giving one error because I have to initialize the operating system package first of all then let me set inside my environment variable okay now what will happen whenever you will using Lang chain now It will automatically load these are the API key from the environment itself okay you don't need to manually set that this is the idea now we'll be learning about how we can use any kinds of large language model with the help of Lang chin so I think we saw we are having like two kinds of large
language model one is like commercial model one is like open source large language model OKAY commercial wise we already saw the entire openi platform we are having gbt Based model and the open source wise we are having this hugging face model that means we are having let's say llama model we are having mral model falcon model like different different models are available okay in the hugging face Hub if you see okay there are thousands of like lar langage models are available I think I showed you one platform open llm okay it should be open llm
GitHub Now list of all the open llm see all kinds of large language model you can See and these are actually open source large language model fine now first of all let's try to see how we can access actually open large language model see whenever I think I showed you the open platform and I showed you the demo of the let's say model like if I want to access this model if I want to give any kinds of prompt so how we can do it see this open ey is already integrated inside Lang chin so
I think I showed you the integration now so this open source Integration just open it up now see open is already integrated with uh Lang chain so you don't need to manually so here you don't need to manually uh use the openi platform like the way we used previously like chat completion API completion API no you don't need to use like that see it's the very simple first of all install the open a then just import from langin large language model I want to use openi okay openi platform that means open large Language model just
try to import this particular class now here just try to set the temperature parameter okay temperature parameter now what is temperature parameter I think you remember temperature parameter it's just a creative actually parameter like how much creative you want to be your lar langage model you can see if it is zero close to zero that means temperature uh it means model is very safe and it is not taking any bits that Means it won't be taking any risk okay it will stick to the output and if it is close to one that means this will
be taking risk it might generate wrong output but it's like very creative okay I think I already explained this part now let me initialize my open a rer now see I have already initialized my open a raer now see this is my large language model so guys whenever you initialize this rapper by default actually it will utilize one model called GPT 3.5 turbo Model okay but if you want to use any other model you can give the model parameter here there is a model parameter with the help of that you can also give any other
model name okay later on I'll show you how we can change the model as of now let's take the default model only now let's say this is my prompt you can see what would be a good company name for a uh for a company that makes colorful socks okay this is the let's say my prompt I'm giving to my Large language model now let me initialize this prompt now see if I want to let's say generate output gener the response I need to call the predict function lm. predict inside that I need to give my
prompt now see it will give me the answer now rainbow trades are vibrant socks uh company okay so this is this is suggesting the name of the company okay that makes colorful socks it's great now you can also directly pass this text to the large language Model rapper so instead of calling the predict you can also give like that it will also work so these are some actually way we can uh give the input to the large language model now see it is also giving me the name now you can see uh uh now you
can see it has given me this name okay this is the response now I can also use something called invok function okay inside invok function also you can give the input it will also work see invok has given some more Detail output you can see it has suggested me different different actually company I can uh use this name uh that makes actually colorful socks okay it's pretty good now let me show you the second example so I have written the same code guys you can see now here I've given another prompt I want to open
a restaurant for a Chinese food suggest me a fancy name for this so now let me give so it is suest me Imperial Dragon Palace okay great now I can give The input like that also inside LM object now dragon fire dining okay great name now let me show you the hugging face okay hugging face model because see at the end openi is a chargeable that means if you're sending any request to the open open AI okay that is open AI model it will charge you based on the token so it's a at then there
is a cost involvement but let's say you don't want to spend the money you don't want to use this kinds of paid model you can also Use open source large language model with the help of uh Lang chain it is also possible for this what you have to do you have to use hugging face up okay because inside hugging face I showed you we are having lacks of model we are having lacks of model and all the models are open source okay all the models are open source and these are like Lar language model from
the different different organization you can see from meta from Google okay from open uh then We are also having see openi also having some kinds of actually free model they are published in the hugging face you can also use them m is also there different different model see that many of models are available inside hugging face it's like very hug model now even I also showed you different different task you can select based on that also you can also select the model it's up to you now let me show you how we can uh integrate
hugging face platform and how We can access hugging face large language model also for this you have to install one Library called hugging face Hub so let me install hugging face Hub because the models are available it is present inside hugging face Hub because I already told you now the model they have published it is called Hub okay Hub inside Hub actually this models are present now see I installed the hugging face up now let me import hugging face up from Lang chain now see uh this is The model actually I want to use so
I've already provided the link let me open so this is called actually flant T5 model this is from Google and this is one of the large language model guys you can see if you are interested you can go through the documentation okay what can this model can perform and all everything they have given they have also given the paper link of the model now if I go to my notebook now see here I'm using the large model see this is The extra large model and it's like very huge I don't want to load this model
I want to use some smaller model because here I I want to show you the quick demo that's why so I'm using this model actually T5 large model let me show you so this is the model the same model only but this is the small variant this is the small variant and this is the X large variant like this is like more bigger than this one okay I hope it is clear now see if I want to load the llm Object right now that means large language model idty to use hugging for from Lang chin
inside that I have to give the repo ID that means the model you want to use just copy the repo ID and pass it here and you have to give some arguments okay you have to give some model arguments that means the same parameter I think remember temperature what temperature will do I think I already showed you this is the creative parameter and the max length I think you Remember inside open I also used to pass the max length that means how many let's say output token you want to get from your large language model
so here I have given 64 okay you can increase and decrease as per your requirement you can also remove this one okay it's completely fine because at then it won't be charging you okay it's a PRD model so you can use as many of length you can okay now see this large language model we are using this is not like very good Large language model but somehow actually just to test actually I'm using it going forward we'll be using very powerful large language model like we'll be using Lama model mol model falcon model OKAY Jimma
model we'll be using different different model but this is like very basic SL language model actually I just wanted to show you as of now now this is the prompt I have given translate English to German this particular sentence okay now see how old Are you this sentence I want to do the translate operation English to German and for this I'm going to use this model okay now let me execute and let me show you the magic see this is the output I got this is the German translation of how are you and this model
actually it is trying to access with the help of hugging face API okay hugging face API that means the token you collected and anything you just set inside in enir variable with the help of that it is Sending the request to the hugging face hugging face is returning the answer okay we are accessing through API request right now going forward we'll be also learning how we can download the model as well and we'll be loading that model with the help of langen and we can create a llm rapper and we can do the application building
okay so I'll also show you this part no need to worry as of now I'm showing you the API access okay if I want only want to perform the Inference operation how we can perform it that is the idea now again I've given the second example the same example I want to open a Resturant for Chinese suest me a fancy name for this again I initialized my LM rapper with the help of my huging f model now if I give this prompt to my large language model see it is giving me Chinese Resturant so this
is not actually creative Rim because I told you I'm using very basic SL language model that's why okay so yes Guys this is the introduction of Lang Chen I think you got it what is uh Lang Chen is and how we can use Lang Chen exactly okay and it's like very easy to use very easy to use it will trust me guys it will make your task very easy if you're using Lang chain it will make your task very easy okay now in the next video we'll be learning about something called prompt template okay like
how we can create different different prompt template and how we can pass to the Let's say large language model okay with the help of langin so previously I think I showed you how we can use this uh langin to access different different large language model and whenever I was accessing this large language model that means uh whenever I was let's say sending my prompt to the large language model you can see I was giving the complete prompt okay this is the the complete promt actually I was to pass let's say this is the prompt I
want to Open a restaurant for Chinese food suest me a fancy name for this now let's say if user needs to change this food okay food let's say category let's say uh he wants to let's say put Indian food here so what he has to write again he he has to write the complete prompt yes or no that means if I show you let's I can copy and I can paste it here let's I will comment this line now let's say here instead of Chinese food I will give Indian food Okay Indian food now if
I execute see it is giving me Indian Resturant now let's say if I want to give any other food let's say I want to give Korean food I want to give let's say Pakistani food I want to give let's say I want to give let's say Italian food so I have to every time write the entire prompt okay but let's say you are creating a complete application you are creating a complete let's say web application or let's say stream late application or any Kinds of application there you don't want to pass this entire prompt again
and again because if user let's say needs to pass this prompt entirely again and again so it would be little bit actually let's say hard for the user yes or no now see instead of passing the entire promt you can see we can only change this uh food category okay the food category will only change otherwise see this promp will remain same now every time so that time actually I can Do the slight modification here that means I can create a template of the prompt and user will only select that part he needs to change
here okay this is the idea now let me show you how it can be done so for this actually be using something called prom template okay now inside Lang chain we are having uh inside Lang chain prompts we having something called prom template now here I've created a promp template and here I've given the input variables is equal To kuin now this is the same prompt I have written you can see I want to open a student for now you can see in the bracket I've written kuin okay now this is going to be the
input only and rest of the let's say prompt will remain same okay as per the prompt you have written previously so me a fancy F this okay now here is the prompt template I'm creating I'm just doing the format oper and kuin is equal to Italian that means user will only give the food category Right now let's say Italian Indian Pakistani okay or let's say Bangladeshi anything they can put here it's up to them now user don't have to write the entire prompt user will only change the food category here got it so this is
called actually promt template so whenever we'll be creating any kinds of let's say large language Prov application whenever will be creating any kind of GNA application we always need to take care this particular prompt Let's say I don't need to pass the entire prompt instead of that what I can give I can only give the prompt which is required to get my output okay this is the idea now see if I execute this line now see it is creating my complete prompt that means user is giving Italian food now see in the cuisine section it
will come the it now you can see in the cuisine section it will replace with the Italian now if I give let's say Indian now see here it uh Indian will Come okay I think you got it now there is another way you can uh Define this prom template now you can write uh prom template doore template now now this is another prompt I have given what is the good name for the company that makes now you can see product should be the input now here you can pass the product name like that prom. format
product is equal to colorful socks or any other name let's say cakes or let's say biscuits anything you can give here if I execute See in the product this uh input would be replaced okay so this is called actually prom template inside langin and this is like very powerful concept guys because going forward whenever you'll be implementing application this concept will help you a lot got it so yes this is what I just wanted to show you in the next video we'll be learning about chains inside uh uh langin and with the help of chain
actually we'll be inting this prom template as well as the large Language model and we'll be again doing the inference operation and what is the use of promp template I already explain now see if I want to use this prompt template with my large language model so what I have to do I have to combine my large language model as well as the prompt template I have created okay for this we'll be using something called chin you can see combine llms and prompt in a multi-steps workflows okay to create a multi-state workflows I have to
Create a change now let me show you one example I think then this part would be clear let's say first of all I will initialize my large langage model rapper and here I'm using open AI model now see previously I think you remember we learned this prom template that means if I want to let's say give any kinds of prompt but I don't need to uh pass the entire prompt only I want to pass some specific let's say U point of that particular prompt you can see here only I want to change the product name
so for this I created a prompt template and the product name I've given the colorful socks okay this is my entire prompt you can see now it will create my entire prompt so which is the good name for the company that makes colorful socks that means whatever input you are giving it will replace it here fine now as of now let me comment this line let's say this is the prompt I have created this is the entire prompt I have created I think you Would remember now if I want to use this prompt The Prompt
template I have created with my large language model I have to use something called LM Chen and how to import LM CH so from Lang Chen do CH import l M chain now the first parameter you have to give the large language model the large language model you have defined which is nothing but open open E large language model you can also use open source large language model you can use hugging face large Language model as well the way I showed you then you have to give the prompt template the second parameter you can see
prompt is equal to prompt that means the prompt template I have created this object actually I'm passing here then just try to call this function chain. run and inside that give the input let's say whatever input you used to give like that instead of giving you have to give chain. run and give the input let's say colorful soof now if I execute see it Will give me the output see Rainbow traits or chroma shocks I think okay chroma shocks actually it is giving me the output that means now we are able to use our prom
template as well as the uh let's say llm okay and for this we are using something called LM CH that means first of all it is creating the entire prom template first of all it is creating the entire prom template that means this entire prom template if I show You that means it is first of all creating the entire prom template and this prompt actually is passing to the large language model and how things are happening with the help of llm chain okay that's why here I have written um with multi-step workflows that means it
is creating a workflow in the back end it is creating a pipeline so first of all it will create the prompt then this prompt would be passed to the large language model and this things are Handling by the llm chain I think you know what is chain now chain is chain means like one let chain would be connected with another one okay that means complete pipeline complete workf flows sequence of you can say chains okay so I think now you got it what chain exactly inside uh Lang chain now let me show you another example
example two so again I'll initialize my large language model and see now here I'm using the second approach I think Remember we can also create the prom template like that so let's say here input variable is nothing but my cuin that means kin will only change and rest of the prompt will remain same okay now this is my prompt template now if I want to pass this prom template to my large language model what I will do I'll use the llm chain inside llm chain I'll first of all give the llm then I will pass
my prom template the prom template I have created now you can see I'm Calling chain. run and here I'm giving the input let's say Mexican I want to see the Mexican now here you can see I have given Mexican as an input that means I want to open a resturent for the Mexican food now if I execute so it should suggest me some fancy name for this see this is the name it has me L sober D Mexico okay the flavor of Mexico now if you want to see internally how it is like creating the
entire PR for this you can activate one parameter called Varos isal to true now see if I again execute it will show you internally first of all it will create the complete prom template that means here you have given Mexican so it will first of all create the complete prom template with the help of Mexican this prom template will be passed to my large language model and large language model will give you the output got it so this is the work of chains now this is called actually Single chains now let's see if you want
to create a multiple chain that means let's say you want to create a chain and this chain output you want to again pass to another chain okay for this you can use something called multiple chain okay multi chain and to use the multi chain you have to use something called sequ Simple sequence chain so there is another class inside Lang chain chain called Simple sequential chain we'll be using this one let me show you one Example let's say this is my large language model I have defined and this is the prompt I have created the
first prompt you can see the same prompt I want to open a Resturant for the kisin food suest me a fancy name for this so it will take the cuine input and this is the LM chain I have created okay I think you remember this is the LM chain that means this is my LM rapper and this is my prom template okay now what I want whatever let's say Resturant name Actually it will suggest okay whatever student name this prompt will suest that means this large language model will suggest this name again I want to
pass to another prompt template for this I created another prompt template prompt template now input variable is restorent name that means this prompt will give you the restorant name yes or no because here you have given kuin fo suggest me a fancy name for this okay now it will give you resturent name and this Resturent name I want to utilize and here I written another prompt s just me some menu item for this restaurant name that means let's say You have given Indian Indian let's say food now it will give you some Indian restaurant name
and this Indian resturent name here I will pass and it will give me some menu item for the Indian resturent name okay I think you got it how it is interconnected with each other this is called actually multiple chain okay now See here Ive created another chain now if I you want to use this both chain I have to use something called Simple sequential chin now there are some limitation inside simple sequential chin I'll tell you what is the limitation as of now let me show you the let's say example so you can see I
have initialized the simple sequential chain inside that chain is equal to first of all I will give my name chain you can see first of all I'll will give my name Chain that means it will give me the restaurant name okay then I will give my food item chain that means this particular chain that means whatever Resturant name I will be getting from this chain I want to pass to the next chain okay which is nothing but my suest me some menu items for the resturant name okay now uh here is the final code chain.
run here I've given the Indian that means first of all it will go in inside that and it will give the output And this output will go in the next next CH okay now it will give me the output let me show you now see I have given Indian now see this is the Indian menu I got that means what is happening first of all it is executing this promp template then it is executing this prompt template okay and how it is handling it is handling by the simple sequential prom template that means it is
following the sequence first of all this sequence will execute then This sequence will execute that's how you can create multiple prompt not only two prompt you can create 3 four 5 six and so on if you need any kinds of let's say application uh this is like interconnected with let's say previous prompt okay you can create these kinds of sequential prompt now I think you got the idea how CH GPT works okay how CH GPT Works let's say how uh let's say if if you giving any kinds of input how it is connected with the
previous prompt as Well okay they're also using something called this sequential chain in the back end okay chain method they're also using I think it is clear now now the limitation of simple sequential chain is it is it is only showing you the last output last last chain output you can see only the menu item but if I want to also see the previous chain output what is the output it is giving for this you have to use something called sequential chain only okay sequ sequential chain Not the simple sequential chain okay this is the
difference only now let me show you one example let's say I've given the same example so I want to open a stent who in name okay here have to pass and here creating the llm chain okay now this is the another chain I created guys you can see so whatever let's say uh Resturant name actually I will get from my first CH you can see output key it will return you the resturant name so it should be the Input for the next CH which is nothing but my uh menu okay menu of the resturant name
here You' given the input is equal to Resturant name and this is the template suggest me the menu of the item for Resturant name and this is my second actually chain I have created and what would the output of this chain the output would be menu item that means whatever menu actually it will suggest you now here I have initialized my sequential chain now inside sequential Chain I have to pass my chain first of all I have to give my name chain that means the first chain I have created then I have to give the
second chain now here you have to specify the input variable what is the input variable I think you saw the first input variable is nothing but kuin okay kin should be my input variable and what be the output variable output variable would be the resturant name as well as the menu item okay resturent name as well as the menu Item that means both it will show me now let me execute and let me show you see now I'll call my chain and here I'll give the input only which is nothing but Indian that means cuine
name now see it will give you the output see first of all it will give you the cuine then the resturent name as well as the menu item that means this prompt it is giving you the output this prompt it is giving you the output even the menu item it is also giving you the Output that means complete output you can see here so which is not available inside my simple sequential chin okay this is the difference only I think you got it right now got it so yes that's how actually we can use chin
to interconnect my prom template with my large language model and we can uh easily get any kinds of response from my llm okay now in the next video we'll be learning about something called agents and Tool inside uh langin so what is Agents and Tool exactly uh let me show you see agents and tools is a actually very powerful concept inside Lang chain so it will help you to let's say plug in with different kinds of let's say tool let's say if I give you one example um let's say I have traveled from Dubai to
Canada I type this in chat gbt now give me to flight option from Dubai to Canada on September 1 20124 now CH gbt will not be able to uh give the answer because it has actually knowledge till September 2021 that means after 2021 if you asking anything to the chat gbt it won't be able to give the answer because it doesn't have like the new information new information regarding uh 2024 but chat GPT plus has uh xandia uh plugin if we enable this plugin it will go to the xandia plugin and uh it will try
to pull the information about flights and it will show the information that means if it is not able to give the answer related 20124 what it will do it will Try to connect some third party plugins third partyy API okay and from there actually it will pull the information and it will show you let me show you one example let's say this is my chat GPT now here if I give this prompt what was the G uh GDP of Canada in 2024 now see if I give this to the chat gbt see chat gbt searing
Canada gbt see it is already searcing okay it is already searching in the third party API now see in 2024 now you can see it has given me the answer In 2024 what was the uh G GDP okay in that Canada that means it doesn't have the information about let's say 2024 uh what Wass the Canada JDP but it was searching okay it was searching in the third party API that me it has some plugins inside that okay so it is happening with the help of this agents only agents and Tool okay with the help
of agents and Tool they are doing these kinds of task now let me show you how you can also perform this agents and Tool operation inside langin let's you want to create an application and this application will also uh be able to give the answer from the newest actually let's say uh information okay over the Internet for this you can connect actually different different tools different different plugins you can connect SARP apic this is the SARP API SARP API is a Google search actually let's say API you can also directly search from the Google even
you can also Connect the Wikipedia okay let me show you one Wikipedia example even you can also connect different different actually third party plugins okay there are so many third partyy plugins if you search on Google you will see different different third party plugins they are using okay you can also connected weather okay weather API to get the let's say recent weather it is also possible now see first of all I will be installing Wikipedia just to show you The demo of the Wikipedia now you can see I'm importing this Agents from the Lang chin
I'm importing agents type then initialize agents and load tools and all then I'm also importing my openi that means I want to use openi large language model as of now now this is my let's say llm rapper okay now if you want to define the agents so you have to Define like that guys first of all call this load tools and inside that first of all mention which let's say plugins you want To use I want to use Wikipedia plugins that means whatever things actually I'll be searcing if it is not available inside my chart
GPT CH GPT let's say knowledge base it will go to the Wikipedia okay it will hit the Wikipedia and from Wikipedia actually it will get the response then it will also use something called llm math tool and why llm math tool because let's say if you are asking something related JDP or let's say uh any any kinds of number any Any kind of numerical representation so for this actually this llm math is required okay they're suggesting if you go to the documentation you will see that then I'm also passing my llm rapper this particular object
okay now it will give me the tool now here I have to initialize the agents first of all give the tool the tool you have created then give the llm the llm you have created as well as the agent type so this is the agent type you can give and verbos is Equal to True means it will also show you the output okay like what it is executing in the back end now just simply search agent. run what was the JDP of us in 2024 let's say this is my question 2024 JDP I want to
ask okay now see if I execute this program now see uh it has actually connected with my uh Wikipedia and it has fced the information and this is the final results as you can see the JDP of us in 2024 was 28 trillion uh 28269 trillion okay it's great that means it is also able to get the current information okay newest information over from the internet itself and this is like very powerful tool guys trust me if you want to implement any kinds of let's say LM powered application and you want to integrate this kinds
of functionality you can use this uh agents okay because Char gbt is also using agents in the back end it is clear okay now in the next video we'll be also learning about Memory let's say I told you not CH GPT can also remember the previous context let's say if I give one let's say message here give me a code to add two numbers in Python let this is the prompt I have given now see it has given me the code now see it has given me one function but I don't need the function I'll
just write without Function now see it has automatically remembered my previous uh prompt I have given and now see beautifully it has given me the second response that means it is it is having some memory okay buffer memory in the back end with the help of that actually it is trying to remember my previous context so we'll be learning this part in the memory part I think I already showed you one demo of the chat GPT uh chat GPT can remember my previous uh Let's say prompt that means whatever previous prompt you are giving it
can remember that context right so how they remembering the previous let's say prompt and context because they're using something called memory okay memory in their application now with the help of Lang you can create this memory so whenever you want to create any kinds of genbi application so definitely you should integrate the memory with that because people will be asking the Question and your application should remember that particular context okay this is the important things now let me show you one demo of the memory so here I've already written chatbot application like chat gbt uh
you will notice it will remember the past information and how they're remembering the past information they're using memory so let me first of all initialize my llm rapper I'm using the open llm here now first of all let me show you if you are not using like Memory okay memory let's say functionality from the Lin what will happen let's say here I created a pom template the same prom plate I want to open a restorant for the kuin food s me a fancy name for this now see if I create this prom template and if
I give this prom template what I have to do I have to use llm chain here I'm passing the LM and here I'm giving the prom template okay now this is the let's say input I've given Mexican so it will give Me the output the resturent name so guys you can see it has suggested me different different restaurant name now if I give Indian also see it has also given me Indian press random now if I show you the chain memory that means whether it has saved anything or not so I'll just write chain. memory
see it is non type that means it hasn't saved my previous information whatever let's say question I have asked it hasn't save any kinds of Information okay you can see it's a non- type completely it's a non- type now if I want to add the memory functionality what I can do I can use actually three kinds of memory inside Lang chain I'll discuss all of them one by one first of all let's try to see conversation buffer memory okay so just try to import Lang chain. memory import conversation buffer memory okay now see this memory
will try to remember the previous conversation now first of all create a memory object You can see conversation B memory memory object now same thing you have to initialize only another parameter you have to pass memory is equal to memory okay now again I'm giving Mexican food now I have given let's say Arabic food okay now if I show you my chain. memory. buffard you will see that it has remembered my previous context so human has given Mexican then AI has replied again human has given the let's say Arabic input then again my agent has
Replied that means it is trying to remember my previous conversation amazing right now one issue actually will get with this conversation B memory it can remember all the input it will be giving let's say whatever input it will be giving all the input actually it will try to remember so whenever it is trying to remember all the input that means uh it is occupying your memory space so I don't need to remember all the let's say context of the user I can only remember Let's say 5 to 10 let's say context it would be enough
for me because if you go to the chat jpt also it can only remember five to let's say six uh let's say context five to six conversation okay apart from that whatever things it will say previously it will delete okay so for this actually we can use another actually memory called conversation chain okay now here I have already written conversation buffer memory goes growing endlessly that means it can save All the information and if I want to set only let's say five conversation or let's say or 10 to 20 conversation that time I'll will be
using conversation chin now here is the example of conversation chin first of all you have to define the conversation CH and inside that you have to pass the your llm rapper now let me show you the template now see this is the template actually it will give you okay now you can see current conversation there is no Conversation that's why it's empty now let me do some conversation so here I have given the first conversation who own the first Cricket World Cup let's say this is my first conversation so this is the output I'm getting
you can see see the first Cricket World Cup own by the India in 2000 uh sorry in uh 1983 okay now here I will give another let's say uh prompt how much is 5 5 + 5 see this is the irrelevant question I'm asking now this 5 + 5 is equal to 10 now again I'm asking who was the captain of the winning team now it will refer my this this actually prompt okay this context now see the captain of the winning team in India was Kil Dev do you have any other question see amazingly
it has remembered my previous previous let's say conversation okay that's how also chgb is working I think the same example I have given you you can also ask this question like uh this add number code Question you will also see the same output okay now if I show you the uh my buffer memory right now so this was my input then this was my like uh model output again I asked this question again this was the output again I asked this question this is the output okay that's how actually you can say five to 10
conversation here okay five to 10 conversation here after that it will remove everything okay now you can also set the window size like how many Conversation you want to save you also have one parameter to set for this you have to use something called conversation buffer memory window sorry conversation buffer window memory now just try to define the memory inside that just mention this parameter K is equal to one that means it will only remember one conversation okay now just try to uh refine this conversation chain inside that mention your llm as well as the
memory now if I again do the Conversation who won the first Cricket World Cup now I'll again Ask how how much is 5 + 5 now it is giving me the output now again if I ask who was the captain of the winning team now see it is telling I don't have an information context answer this question okay because I have only given K is equal to one that means it is only saving my one information okay now if I give there's a k is equal to three now it will remember my three previous three
conversation let Me show you now again if I ask this question who was the captain of winning team now see guys it is giving me the output okay now if I show you the conversation now you can see it is remembering my previous context got it so that's how whenever you are creating any kinds of application guys try to use this uh memory okay memory inside your application this will make your application very powerful and now I think you are getting the logic behind Chat GPT how chat GPT is working it's just a application guys
always try to remember chat GPT is an application inside that they're using large language model which is nothing but GPT series okay and whatever functionality they have created this memory functionality then chain functionality prompt template functionality they're using these kinds of framework langen framework or Lama index framework there is another framework called Lama Index this is the Alternative framework of langen we'll be also discussing this part as well fine okay great now in the next video we'll be learning about documents loader let's say uh whenever you want to build any kinds of rag based application
retrieval augmented generation let's say you have some custom documents and if you want to load these documents and you want to connect with your let's say uh large language model that time you have to load your documents okay whatever Documents you are having whether it's a PDF documents documents documents or let a THD documents even you can also load the website also just on any kinds of file format you can open any kinds of website you can integrate here okay so L chain is very powerful guys you'll be loving a lot trust me guys so
in the next video we'll be learning how we can load the documents then after learning this Lang chain we'll be also implementing different different Projects so that this concept would be clear how we can load our custom documents how we can create a rag based application okay how we can create a chatboard different different things we'll be learning so if you go to the langen documentation you will see we can also load different different documents with the help of langen so you can load let's say custom documents let's say any any kinds of file format
CSV then file directory HTML Json markdown okay Microsoft Office PDF Tex 3 okay any kinds of documents you can load with the help of this Lang chin even you can also connect different different let's say platform you can also connect slack you can also connect let's say uh Discord and from there also you can extract the data it's like very powerful guys you can go through the documentation you will get each and every IDE even they have also given the cod in it okay how we can let's say connect uh with Different different let's say
document format and how we can load the documents fine see always try to remember whatever let's say data we are loading inside langen it would be considered as a document let me show you one example so here let's say I want to load one PDF uh documents let's say PDF I want to load let's say I'm having one PDF let me show you so this is my resarch paper guys I so this is my research paper guys I published uh some years back you can see I'm the author so this is the author okay I'm
the author now you can see this paper is about actually uh development of multiple combined regression method for the uh rainfall measurement okay and this paper is already published uh in the research gate let me show you if you go to my LinkedIn profile this is my LinkedIn profile you can connect me on LinkedIn guys you can follow me so if you have any question you can ask me there I'll try to reply now see if I Show you uh this particular publication so see this is this is what I've already added now if I
open this publication see this is the publication and this is in research gate okay even if you open the Google Scholar also you will get it there so this PDF actually I have downlo downloed this paper PDF I have downloaded now what I will do I'll just try to load the documents okay whatever let's information I'm having in this let's say uh PDF I want to extract these Are the information for this I'll be using langen so let me show you first of all I need to install one package called P PDF now let me
upload that PDF here so my paper I'll upload it here now see it is getting uploaded you can use any kinds of documents guys docs dxt csb just on anything and again visit the documentation you will get all the codes let's say you want to use Jon so this the Json okay Json loader you have to Use let's say you want to load csb you have to use CSV loader for this okay now I'll be loading something called PDF that's why I'll be using PDF loader okay now see this is updated now I'm importing this
PDF loader from Lang you can see document loader PDF loader now here you have to give the location of the file let's say this is my location okay now you have to call load loaded. load now see it will automatically extract all the Information and this uh data would be considered as a document let me show you always try to remember inside Lang chain inside geni all the data would be considered as a documents okay that's why I think you remember in the vector database session I was talking about documents okay chunking documents these are
the thing okay now I think you got it now see it has extracted all the pages now see this is considering as a document this is the Type of documents right now and all the information present inside my paper it now we can use this data to implement any kinds of rag based application that means with help of vector database I can store these are the data with help of embeddings and I can connect my large language model and I can ask some information there okay don't need to worry I'll be explaining this rag concept
and all okay why rag is important why we have to create the rag Even we'll be also learning about fine tuning okay how we can let's say take our custom data and we can perform the fine tuning as of now we have learned the fundamentals of langin and I think I covered almost everything about the L chin whatever things you need to work with the Lang chin but I will show you some more advanced concept of the Lang chin this will also help you okay so so far I think we have covered so many things
inside Lang chain we have uh Learned the entire fundamental concept of the Lang chain so one thing I wanted to tell you see Lang chain also can work with this one your pandas data frame that means directly pandas data frame you can pass to the large language model and you can ask some question there okay it is also possible for this uh you can use your large language model any kinds of large language model and with the help of langin framework you can perform these kinds of operation let me show you So first of all
I have to install some of the library so here you can see I'm installing langin and langin experimental why langin experimental because this Panda's data frame actually agents will be using now this is available inside Lang and experimental okay that's why I have to import all of them so let me import and I'm going to use open large language model that's why I'm using open here now I'm going to import some of the required liabilities I need now you can see from Lang chain experimental agents A toolkits I'm importing create pandas data frame agents okay
you can go to the documentation you can see pandas data frame documentation inside langin and they are also loading in that way okay and they are giving u a data set okay any kinds of data set they're loading with the help of pandas and they are performing lots of operation okay see you can use different different large Language model and you can perform chat operation okay with this uh data frame let me show you so let me import the library first of all I'll import the pandas as well I'll set my open API key then
this is the data I want to load let's say this is the csb data Titanic csb data it is available in this GitHub if I show you the live GitHub link so this so this is the data guys it is already available so let me first of all load this data with the help of Pandas so here is the data guys I think you already familiar with this Titanic data now first of all we'll be performing this chat operation with the single data frame so I'll initialize my large language model now create a agents to
create agents just call this create fond as data frame agents this class inside that give the llm data frame varos is equal to two means it will show you the entire let's say execution details and here you have to give Another parameter called allow dangerous code is equal to true now I'll create the agents now here I can ask the question right now I'll ask how many rows are there in the let's say data frame now see it will count and it will automatically give me the answer see around uh uh 891 okay rows are
available I will give another question how many people have more than 23 Ed see amazingly your model will try to um go through that data frame and it will give You the answer see for 468 now I can also work with multi dat Fame so for this what I can do I can copy this data uh in another variable called df1 see I'm copying this DF only and I'm creating another df1 here this is the data now what I will do I'll just fill the uh nonone value okay which is present inside H because H
is having some non value I'm just fing now again I'm creating the agenty you can see now here you can see I'm giving multi data Frame DF and df1 okay now let me create that agents now I'll ask one question how many rows in the ages columns are different okay now let me show you because I already filled the information now around 177 rows were have the N value now you can see rows have different values in the agge column I think you got it now I'll create another data frame called df2 and now I
will just multiply two with the age whatever age I'm having I'm just Multiplying two now this is the results now I'll create another agents and it will have three different data frame okay now let me create the agents now we ask one question are uh the three of The Columns same in the all the data frames are not definitely it should be same for all the data frame because I because I have just copy pasted the same data frame again and again three times so so here all the column name should be same if I
show you the output C uh so see Guys I'm getting the output agent stop due to the iteration limit or time limit because I'm executing multiple time now that's why I just uh uh use the limit okay that means my limit has been done uh so what you can do you can restart the run time again you can execute the entire code you will see that all the column should be same okay in three different data frame okay you'll get this kinds of output now you can ask any kinds of question okay about this data
Frame so this is called actually multi data frame agents inside Lang chain and this like again very powerful concept if you want to work with the data frame and all going forward we'll be also implementing some projects with that okay this concept will help you a lot so as of now I think I showed you lots of example lots of demo of the langen like what are the things we can perform even I also told you we can also let's say access different different uh open Source large language model from the hugging F itself okay
so this would be the dedicated video about this Lang chain integration with the hugging F so there I will see that how we can access different different model uh with two kinds of method like one method I will show you how we can access through the API request other method I will tell you how we can download the the model okay uh in our Google collab and how we can use this particular model okay this is The idea so first of all let me install some required package so you can see I'm installing Lang chain
Lang chain Community hugging face up Transformer why Transformer because I want to use hugging face platform I want to download the hugging face model that's why Transformer is required I think so I think I already taught you about the hugging face like how to use hugging face and what is Transformer Library everything I already discussed and these Are some actually dependency package you need to also install with the hugging face that means your Transformer now let me install all the one by one so guys as you can see my installation is completed now let's import
some of the library you can see I'm importing prom template hugging face up llm chain from the Lang chain itself now first thing what you have to do you have to setting up the environment for this you have to set the hugging face API token and how to get the hug API token I think I already explained go to the settings and left hand side you will see access token option so you can create a new token here I already showed you how to create a token okay so I'm not going to show you again
so now let me set my token now we'll be learning the first approach like access the hosted model on hugging face through API request okay now see here we'll be accessing one text to text Generation model so first of all let's create a prom template so here's the prom template I have prepared what is the good name for a company that makes product so product should be my input here now here I'm creating the LM chain I'm giving my large language model and you can see I'm using Hing face up and I'm giving this model
ID that means I will be using this model so this is the flan T5 model uh this is from Google actually so we'll be using this model Here so let me load this model and here is the model arguments that means temperature and max length parameter and here I'm giving the prompt okay the prom template I have created now let me create the chain object now here you can give the output let's say I give colorful socks let's see whether it is able to give the response or not see it has given me sock Mania
okay this should be the name now let me show you another prompt you can see I have written Another prompt can you tell me about famous footballer so it will take the famous footballer name and it will um tell you about him so let me show you so again I'll create the chin and see the same code actually I'm using now if I give Messi now you can see guys it is giving me the response Messi is a footballer who plays for Argentina great now you can give another prompt let's say uh can you tell
me food items for the cuisin Restaurant so kuin should be the input let's create and I'm using the same code snippit now let me initialize the chain and let me give the input let's say Indian now see it is giving me uh vegetable okay can you tell me the food items yeah vegetable is fine now if I again execute let's see okay vegetable it is giving because I told you now this model is not good now guys let's see how we can use any other model see again I have Prepared one prom template and again
I'm using the same thing can you please tell me about famous footballer now instead of this T5 model I'm using Falcon 7B so this is one of the large language model it is already available in the hugging P okay you can see so this is the organization they publish this Falcon and Falcon is having different different variant Falcon 7B 13B okay that's how is having different different variant and this is one of the like great uh large Language model you can use now let me show you one demo now let's load it and see I'm
using the same codit and I'm sending the request see this model actually I'm not downloading in my machine so it is using the API to access the model and it is giving me the response and that response actually I can see okay now see this is the response I got and it's like very detail response I got okay than your previous response now let's see the approach too Like how we can download this model locally and how we can create a pipeline that means that hugging face pipeline I think remember right for this let's import
hugging pH pipeline from Lang chin then I'm also importing some additional Library like Auto tokenizer auto auto model for casual LM then pipeline okay these are the thing I'm importing because I think you remember how to create the pipeline so let me import them now see I'm using this flant T5 model again so let's initialize the model and first of all I have to create the tokenizer if I want to download the model I have to create the tokenizer I think you know that right tokenizer will pre-process your input that means it will process input
it will create them Bings and then it will pass to the model now and now here I'm going to initialize my model you can see I'm giving the model ID then device map is equal Auto that means it will load the model inside My GPU and make sure whenever you are like downloading the model locally you are using GPU based machine okay good configuration machine otherwise execution time would be very like slow okay but if you're accessing through API you don't need a GPU based machine now let's create a pipeline you can see it's a
text text to text generation model so here I've given text to text generation model tokenizer and max length now let Me create the pipeline now I'm going to initialize my llm rapper so I think remember we called this hugging face pipeline from langin now so here you can see I have initialized my llm wrapper and pipeline the pipeline object I'm getting I'm passing inside this H pipeline now this is going to be my Raper now again I'm going to create the prom template and this is the example I have taken the same uh like a
company okay company Example now let me pass to the LM chain and as well as my prompt see I'm passing my large language model as well as my prompt and now see the model actually I'm passing the this model actually have downloaded okay llm local local LM not the API one okay so both way you can use this hugging face now let me show you the results see socks Mania now let me prepare another prompt can you tell me about famous footballer now let me initialize the chain now let me ask About Messi see Messi
is a footballer who place for the Argentina okay so that's why guys we can access uh hugging fish model open source langage model with the help of langen and going forward we'll be using this concept guys and with this concept we'll be implementing so many application so this project we'll be implementing uh with the help of whatever things we have learned so far let's say Lang chain large language Model Vector database okay even we'll be also implementing one front end uh application of this project that means uh user will get one user interface and their
uh us user will upload the data and they will get the results okay so these kinds of things actually will be also implementing that means I'm going to show you the complete project structure here that means we'll be doing modular coding here but before that I'll show you the notebook experiment uh like How uh we can implement this project through the jupyter notebook then I'll try to convert this jupyter notebook to the end to end manner okay so uh I think you saw the name interview question Creator I think you are already getting what is
the project about interview question Creator means let's say you are having one PDF documents or let's say any kinds of documents whether it's a doc uh docs documents PDF documents TT documents or whether it's a books okay So what you have to do you just need to upload that let's say PDF uh to this application and this application will automatically generate the questions okay question as well as the answer okay uh of the topic actually will be giving let's say you have uploaded one python PDF here so this application will generate some python questions interview
questions uh as well as the answer okay and that question and answer you can use let's say for your candidate or let's Say you are a teacher you are a teacher and you want to generate interview questions or let's say uh exam questions what you can do you can use this application to generate these kinds of question and answer so it would be very easy for you to generate these kinds of question otherwise you have to manually find the questions manually let's say prepare the answer so it will take time but if you're creating these
kinds of application it will help you a lot okay To automate your process again it seems like a simple project but whenever you will be implementing you will get the compx City and this is like very interesting project because uh this is the first project I will be implementing after learning Lin going forward will be also implementing some Advanced projects like we'll be implementing medical chatboard source code analysis okay so these are the thing we'll be also implementing so instead of talking too Much guys let's start the implementation but before that uh I will show
you the let's say the requirements actually I need to implement this project and I'll will also show you the project architecture so guys to implement this project uh I need some requ requirements okay I need some requirements so here we'll be using langin to implement the entire let's say uh project and why langin I think you know Because langin having all the functionality right let's say you want to integrate Vector database you want to integrate large language model okay everything is available here so that's why we'll be using the complete Lang chain here and large
language model wise we'll be using open model open we'll be using GPT 3.5 okay this model actually will be using going forward we'll be also learning how we can use open source Starge language model with the help of that I will also show you how we can create the application now third I need uh Vector database Vector DB so here we'll be using F okay so FS is another actually Vector database I think I haven't covered in my Vector database series uh so that's why I thought let's also show you this F in this video
so that you will be also learning about F so F has implemented by Facebook Ai and this is One of the vector database this is the local Vector database actually local Vector database so you have to set up everything in local machine going forward we'll be also using pine con we okay no need to worry then to implement the entire let's say front end part part of our application that means we'll be creating the user interface with the help of fast API fast API I think you know what is fast API fast API is a
python framework with the help of fast API we can create a uh user interface web application apart from first API we are having let's stream lead flask Jango okay Falcone so these are the thing you can also use we'll be exploring one by one all of them I think we saw the flask previously I think remember flask I think we I showed you we created some of the application with the help of open uh we'll be also using stream lit streamlit is very easy if you're using a stream lit you don't need to write it
in kinds Of HTML and CSS code so guys yes as of now these are the requirements uh are needed uh if I need it later on I'll tell you okay now let me show you the project uh diagram okay the project let's say structure uh that means the high level architecture so here uh first of all user will let me just draw it here first of all user will upload one PDF documents okay let's say this the PDF documents so what I have to do I have to Extract that data extract docs so why I
have written docs because I think you already know if I I'm using langin and if I'm let's say extracting uh any kinds of data from any kinds of PDF or any kinds of let's say um I mean file format it would be considered as a documents I think I already covered multiple time like Lang chain data loader even I also showed you inside Vector database session so guys uh Vector database session is very much Required here because inside Vector database session explained each and everything you need to know to implement any kinds of project okay
how to connect llm what is documents what is Chun what is Chun overlap everything I have discussed there so please go ahead and try to watch that uh let's say session first of all then you'll be able to understand okay what is happening here because here I'm not going to uh explain in detail everything we are implementing The project only and we are using our previous concept okay this is the idea so make sure you uh complete that Vector database series first of all fine so we are uh we'll be extracting the document from the
PDF then we have to create a chunks and what is Chunks I think I already told you why chuning is required because llm is having one kinds of input limit okay so that's why actually we have to create a chunks okay chunks then what we'll be Doing uh we'll be using one embedding model what is embedding model I think you already know I already explain inside my Vector database uh session embedding model will help you to generate the vector embedding okay because I have to convert this chunks to the vector embedding so this will give
me something called Vector embedding okay uh Vector embedding okay embedding and with this Vector embedding we'll be creating One centic S I already explain what is santic s santic search because uh I think you know the power of vector DB it will give you one CTIC s kinds of functionality now uh what I have to do I have to store this uh Vector to a vector database Vector database because if I want to perform the centic source operation I have to uh Use the vector database okay I think you already know now guys you can
see my knowledge base has been created so this is called actually knowledge base right now okay knowledge base because it is having the entire information about my data now I'll be connecting my large language model here so let's say um this is my large language model um llm that means that GPT 3.5 okay so this large language model I'll be connecting Here with my Vector database okay and here I will set a prompt um I'll just set a prompt here so in inside this prompt actually I'll be writing what kinds of let's say task it
has to perform so here I'll be writing you are a uh expert uh uh interview question Creator so what you have to do whatever let's say data user will give you you have to create a interview questions okay or let's say 10 questions 20 questions you have to create it okay This is the idea so this kinds of prompt actually I'll pass to the large language model so I think you already know prompt is everything if I if I'm using any kinds of llm right so with the help of prompt we can uh achieve our
task this is the main idea so this llm will go to the vector database that time and because Vector database having all the information about the PDF and it will create okay it will create what it will create you uh Interview questions so it will return interview questions interview questions okay fine now what I told you I'm not going to take only the interview question I will also um going to take the interview question answer as well from the large language model so again what I will do whatever questions actually I'm getting again I'll pass
okay again I'll pass to my large Language model again I'll pass to my large language model and I will tell you tell my large language model model also provide the answer the questions you have generated so it will give me the answer that time so it will give me questions answer okay as well so this is going to be my complete project architecture okay so based on this Architecture we'll be developing the entire projects fine so first of all Let's uh do everything in my jupyter notebook then once jupyter notebook is working fine then I'll
uh show you the uh let's say modular coding implementation as well so for for this guys what I will do I will open up my local folder and here I already created one folder here Mt folder inside that I'm going to open up my terminal okay so in case if you're using anaga G bash Whatever terminal you are using you can open it up now here I'm also going to open up my visual code Studio so let me open up my visual code Studio now here the first thing uh what I have to do I
have to create one virtual environment I think I already told you um you just need to create a virtual environment every time whenever you are creating different different projects so to create the virtual environment you have to execute some of the command so Let me share the command with you as well so here I'm going to create a file called readme MD so this is the markdown file inside that I can mention all the command and this file uh uh will help you to render actually these kinds of message in the GitHub so let's say
whenever you will be uploading this code to the GitHub that time you'll see that in the GitHub description that means your project description uh all the Details has been written okay let me show you one example so guys uh this is my GitHub profile you can follow me here also because here I have created lots of repository that might help you a lot so let me show you my repository guys one of the project actually I implemented previously so let's say uh this project okay um MLS production ready machine learning project so this is the
project actually I created now if you open this project and if you go to the read me Section you will see that the complete details about my project okay see the complete details about my projects like how we can set up what is the workflow what are the command you have to execute okay everything I have set here so how it is rendering it is rendering because of this rme file you can see readme.md file okay so that's why we are creating this rme file here because whatever things you will be writing here it will
render in the GitHub later on and this Is the idea yeah so here uh some of the common command you have to execute every time let me show you all the all the command so see guys these are the command you have to execute first of all you have to create the environment after that you have to activate it after that you have to install the requirements to install the requirements you have to execute this command okay be install hypen requirement. txt so let's create a file here uh I'll name it as Requirement. txl copy
and let me create the file as well okay now first of all create the environment so copy this command and open up your terminal and execute it here okay execute it here for me I already created the environment what I'll do I'll just try to activate it but for you you just need to create the environment so for me let me activate so this is the environment I created already interview you can see fine now inside interview I have to Install some of the required package the package actually I need I think I showed you
the Lang genen open AI then uh Vector database fast API okay everything I have to install here so I already listed down all the required package you need h Huh so these are the package guys you need okay you need to install so you can see lch lch Community langen code so these are the actually dependency package if you want to use langen so you Have to install it and again we are doing local setup guys that's why some additional package you can see otherwise what will happen actually you might get issue installation issue that's
why I have added everything now open i p pdfi p PDF because user will upload PDF documents here as of now we are considering PDF documents fine and Tik Tok and Tik Tok is the dependency of open then then you can see I files fast API uh uvon so these are the dependency Of fast API uh I'll tell you whenever we'll be using these are the package that time I'll explain okay as of now just consider these are the package actually I need to implement the entire projects okay now you can see we'll be using
F and to install the F you have to um execute this command F CPU then python. EnV because I have to manage my uh secret credential okay now let me save everything and you have to install it to install it this is the command Guys I already shared so copy the command and let's execute it here for me I already installed all the package that's why Stelling requirement is already satisfied but for you it will take some time okay now let me clear all right now first thing I'm going to create a folder here called
resource inside resarch I'm going to create a notebook I'm going to as experiment I'm going to name it as experiment do ipy NB because this is The Jupyter Notebook file I want to create okay that's why it's ipb file now let me select the cardal so the environment I created this in interview okay I'll select it up now here first of all let's import some Library I need the operating system then I also need EnV so fromb import load EnV now let me execute so it's working fine so I think remember I have to create
a EnV file here inside that I have to mention my open API key so let Me do it so guys as you can see I have created this EnV file inside that I mention my open a IPI key okay so for you you have to do the same thing now let's load this open II key so to load it I'll call this load EnV after that simply I'll call the key open API key and I'm going to use os. gmv inside that I'm going to pass my key okay so this will give you open a
key let me show you see okay it is printing The open a a ke now let me comment as of now I don't need to print it fine and now what I have to do I have to set this hop AP key as an environment variable so for this you have to execute this line of code so it will set this open a key as an environment variable so that open a package will load okay from the environment itself I think I already explained previously this part okay how to do it and all now the
first thing what I have to do guys I have to load The data but I don't have any data here so let me show you some of the PDF actually I'm having so guys as you can see in the data folder I'm having actually two PDF one is like HGD PDF one is like starts PDF let me show you let me open the PDF and let me show you so if I go to the data folder so this is the PDF guys I downloaded from the internet uh so the PDF name is sustainable development goals
okay so about this topic actually they have Written this PDF you can see what is sustainable let's say development and all they have written this particular let's a PDF so this is uh one PDF actually we'll be using to generate the interview questions let's say you want to generate interview question based on this topic okay so you can use this PDF not only this PDF guys you can use any kinds of PDF whether it's a python book any kinds of physics chemistry English anything you can use here it's up to you So this is the
PDF guys actually I'm having now the next PDF let me show you so this is the statistics PDF so this is the topic of importance and the use of correlation in statistics and again you can see correlation topic has been written so we'll be also generating some interview questions based on the statistics okay that means the correlation so this two PDF actually I just downloaded from the internet okay now let's load the data here so I'm Going to to open up my notebook so to load the data you have to I think remember you have
to uh import one let's say uh function from the langin so let's import it from langin dot documents loader okay documents loader input uh input Pi PDF uh Pi PDF loader I think yeah this one p PDF loader because it's a PDF document right now let me import it now one thing I want to show you so let's say if I show you my uh project working Directory right now so let's say PWD this is the command to check my project working directory now you can see that I'm inside my resarch folder okay you can
see I'm inside my resarch folder so because this experiment. ipnb is available inside my resarch folder but where is my data located data located outside of the research folder yes or no because outside of the resarch folder data is available okay now if I want to go back if I want to go back uh to my Root project directory that means Lang chain project what I have to do I have to execute one command uh this command is CD dot dot okay CD dot dot is the command so this will actually go back one folder
back let me show you so now see if I execute the command now if I again check the project working directory now you can see that I'm inside my root directory that means this directory okay Lang chain project now I Can easily access my data right now okay this is the idea so make sure you always check the path project working directory okay if you're working with let's say uh jupyter notebook uh in a specific folder okay this is the requirement now let me Define the file path file _ path is equal to so my
file is present inside my data folder okay data and let's say as of now I want to load this hd.pdf so H sorry it should be S DG okay sustainable uh Development Goal this PDF I want to Load PDF okay and to load it I'll be using pi PDF loader inside that I'm going to pass my file path okay and it will give you uh loader loader object now I'm going to load the data so I'll just write data is equal to loader loader do load that's it now see if I execute it should load
the data now if I show you the data see so the complete documents it uh so guys you can see it has extracted all the data from the PDF Itself and it is having around uh 24 pages you can see it has started from zero till three that means 24 pages actually it is having this PDF if I open this PDF you'll see that 24 pages it is having and all the pce content it has extracted and some of the places you can see p content is equal to empty because in this page actually there
is no content that's so it's empty and whatever actually documents I'm having it has extracted okay one by one Everything now if I want to uh show you the length you can see 24 pages are available okay in the PDF now what I have to do see it's a separate separate pH content now I'm having now I want to let's say uh store all the data in just one variable so for this what I will do uh I'll just take a variable here so let's say variable name is question undor gen you can give any
variable name and subtitute it should be initially empty String now I'll just write a fold Loop so for page in let's say data let's say the complete data I'm having so it will give me all the page one by one and I just need to extract the content so question gen uh plus and equal I'll just I'll be just adding okay in this variable okay this is the idea I'll just write P dopor content okay so it will give me uh all the content one by one and it will save Inside this question gen variable
now let me show you now if I print my question gen now see all the data I am extracted okay see now it is in one place only this is the idea now see this part actually have completed that means whatever PDF I uploaded I extracted the entire documents okay now what I have to do I have to um do the chunking operation that means I have to create a chunks um for my large language model uh because if I want to let's say generate The embeddings before that I have to do the chunking operation
I already explained this part I think remember so to perform the chunking we'll be using one package so it is available inside langen so len. text splitter so we'll be using uh one text Splitter Splitter here called token text splitter okay so previously I think we used something called character text splitter then recursive text splitter I think remember we use Character okay character text splitter okay splitter so with the help of that we can perform the chunking operation I think remember then we have also have something called recursive uh recursive okay text is splitter so
again this is another method to perform the chunking operation and now we'll be using something single token text spitter token text spitter now what is the difference between token Tex spitter and character and recursive see let's Say this my entire documents so I think remember we used to perform something called chunk size okay CH size let's say uh let's say uh five okay 500 and there is another parameter called s _ overlap let's say it's 100 okay now how it will the chunks I think remember first of all it will start from the first and
it will count 500 wat let's say 500 wats ends here so this is our first chunks okay and to start the Second chunk first of all it will apply the chunk overlap let's say 1001 back actually it will go so it will count uh 1001 let's say this is the 100 W start from here so it will start the second chunks from here and again it will count till 500 wat let's say 500 wat ends here okay so this is my second chunk okay so that's actually it will perform the chunking operation now you can
see uh there is a chunk overlap you got so this is called actually overlap okay this is Called overlap and with this overlap actually my uh model will understand after this first chance this Chance is coming because you can see it is getting overlap now this is the actually manual manual approach actually we can consider this is the manual approach we can consider to get the context context of my first SS and second chunks okay and so on like whatever chunks actually are defining here but uh this is actually what we do in the character
Tex spitter Or recursive text splitter but what about token text splitter see token text spitter what it will do instead of using Okay instead of using actually this manual approach it will use one large language model it will use one large language model in case actually we'll be using something called GPT 3.5 okay so this GPT 3.5 will decide okay uh like uh how to create the chunk how to perform the overlapping and it will automatically remember let's say After this first chunks this second chunks is coming after second chunks this third chunks is coming
after third chunks this fourth chunks is coming okay your large language model will try to remember this part okay and this is like more powerful method than your previous one Whatever let's say method you used previously because now LM is like remembering everything the context okay that's the idea so let's try to explore this uh one this token text is printer So for this I'm going to open up my notebook so I already imported now let me execute now see uh this is the synex to use the token text sper so I think you remember
previously we didn't use any kinds of model name whenever we used to perform the let's say chunking operation now you can see I'm using the model name I'm using GPT 3.5 turbo that means my model will try to decide okay how to remember the context now you can see this is the Chang size Chang size I'm expecting 10,000 okay 10,000 actually what would be consider as one chunks and chunks overlap should be 200 okay as of now let's say this is the things I'm just showing you just to show you okay how it will work
it just experiment okay I'm just showing you how we will try to create the chunk now let's define the splitter okay this is my splitter now inside that I'm going to pass my entire documents so you can see the entire questions and documents Actually I'm having I'm passing to my splitter okay splitter objects you can see speed text it will give me chunk question gen that means the uh chunking okay it will give you the chunking result let me show you so here is the chunk I got see this is the chunk I got now
how many chunk it has written for this you can use length function you can check see it has only given me one chunk because why the PDF you can see this PDF Is DG so if you count all the words it won't be actually 10,000 words okay it won't be 10,000 words it is less than 10,000 wordss that's why here I given chunk size is equal to 10,000 it is only considering uh one okay one chunk the entire documents because I told you if you count the all the let's say what okay in this do
let's say PDF it won't be 10,000 okay because it's a like hug size I have given here just to show you okay how chunking will happen now let's Perform our actual chunking for this what I have to do I have to first of all convert this uh this actually data to the document format because you can see now it's a list format okay if I show the type okay if I show you the type of this one CH question now it's a list inside that if I just extract the first element it would be a
string see it's a string but I have to convert to the documentary presentation So for this what I will do I'll import one uh class from lanin called doc documents import documents okay with the help of documents we can easily convert any kind of string to document representation okay now let me show you see now whatever let's say um data you got that means your uh this one chunk question gen so I'm applying on for Loop that means it will give me all the let's say data one by one and I'm converting to the documents
you can See okay and I'm storing in this particular variable now if I execute now see I got the documents and it is format of documents right now okay now if I show you the type type of document question in it should be a langin code document BAS document documents okay it's a now document format because if I want to apply the splitter so first of all you have to convert everything to the document this is the recommended way Okay if you're also giving the let's say string directly it's completely fine but this is the
recommended way first of all convert everything to the document format then try to use the splitter that means the chunking operation now I'll do the chunking again see again token Tex splitter again I'm using gbt 3.5 TBO model now I have given the Chun size is equal to 1,000 that means 1,000 mod I want to consider as one chunks and chunks overlap is equal to Uh 100 okay now let me execute now let me give my documents now you can see do document question gen okay I'm passing it to my uh splitter okay splitter object
right now now it will give me the chunk now let me show you now see I'll get multiple chunk here so I got here four chunks okay four chunks actually I got from the entire PDF now if I show you the length see four chunks I got fine I think it is clear now so guys as you can See we successfully uh created the chunks now the next thing what I have to do uh first of all I will generate the questions uh like whatever let's say chunk I have created I will pass to my
large language model and I will generate the questions okay interview questions based on the prompt actually I'll be providing I think remember I think you remember uh we'll be passing one prompt to the large language model okay so with the help of This prompt actually I'll tell my large language model um I need a uh interview questions okay let's say the documents I have given based on that it should provide provide me some interview questions that's what actually I need to perform so for this I don't need to convert this documents to the embeding representation
because as of now we are only passing the let's say uh documents to my large language model large language model will give me the Questions but later on whenever let's say I'll be doing the answer generation okay the complete answer generation that time actually I I need to pass the entire documents okay and it will create the vector embeddings and it will store to the vector database okay and uh Vector database I'll will connect my large language model and large language model will refer this interre inform to give me the answer as well okay this
is the idea so let's define my large Language model right now so here I already told you I'm going to use something called open uh large langage model that mean GPT 3.5 okay this model and to use it I can use this class actually chat openi you can also directly import openi it is completely fine but we can also import like that okay both way you can import and if I'm importing like that so here I can also give my model name that means which specific model I want to use whether I Want to use
let's say gbt 4 model 3.5 model you can use so let me show you so this is the Cod nibit so chat open inside that you can give the model name let's say I want to use GPT 3.5 TBO model and this is the temperature parameter you can also give max length parameter and so on so temperature parameter is a creativity parameter I already explained okay what is the temperature parameter now let me Define my Llm now here I already created one prompt guys promt template let me show you so this is the prompt I
have created guys you can see you are an expert at creating question based on the coding material and document mentation see uh whenever I designed this prompt actually I was uh like generating coding related questions and now I'm using any other documents you can see um that means the uh this one sustainable development goal and statistics one so if you're using Statistics one that time you need to give statistics material if you're doing uh sustainable development goal you have to give sustainable development goal here that means whatever subject you are referring you have to give
the prompt with respect to that because it's a changeable prompt is changeable okay you have to change every time based on your requirement now your is to prepare a coder for a programmer for their exams and their coding test you do this by Asking the questions about the text below that means whatever let's say data user will provide that means this chunks okay based on that it will create interview questions that will prepare for the coder um programmer for their exam test okay make sure not lose any important information okay this is the prompt actually
I have created and I already told you like how we can design a efficient prompt what is the art of let's say design a prompt I already took On session uh on the prompt engineering that uh session you can refer okay so that's how always you have to prepare a uh good prompt whenever you are preparing good prompt that means you will get a good response from the large language model this is the idea now let me Define The Prompt now to use this prompt I think I already told you one concept called prompt template
you have to use something called promp template from the langen so let me show you so This is the Cod cipit prom template okay so here I'm passing the prom template the prom template I've created and this is the input variable is text that means user will give this data okay this is the input variable I think it is clear now let me execute the prompt okay now this is my template right now uh we can do another actually uh very important let's say technique here uh see if you're not doing it it's completely fine
it will Work but I was going through some of the research paper research article and I got to know people are using something called second layer verification that means they are doing refine template refine prom template so what is refine prom template let me show you I created one prom template see this is the prom template I again created that means whatever questions actually your model will generate from the first prompt you have to give those question to the Second prompt that means it will refine that particular question let's say if there are some issue
with the question if let's say this question is irrelevant based on the topic you are asking again this refine prompt template will try to refine and again it will give you some new questions okay on top of it so here you can see I've written the same prompt again but one more additional information I have added we have received some practical questions to Certain context that means it will give me some questions okay this question actually I have to give to my second prompt and second prompt we try to check we have the option to
refine the existing questions or add new one only if necessary with some more context below that means if some issue is there if let's say it is irrelevant that time what it will do it'll try to rectify that question okay this is the idea give the new context refine original Questions in English if context is not helpful please provide the original questions this is the idea so this will actually refine from template you are refining the questions here that is the idea only and this will like very powerful method guys if you are using you
will see that your result would be pretty good if you're not using it it's completely fine no need to worry but I'm showing you uh because see we are creating in Advan application and in Advan application we have to take care these are the part okay this is important let's also Define this refine prom template again I'm going to use prom template inside that I'm going to pass the input variable now input variable is two one is the this existing answer the existing answer I will get and the text that means the again you have
to give the entire text that means the chunks okay so this two input now you have to give you can see text and Existing answer and it will give me the refine prom template fine now I'll generate the questions for this I need to integrate my large language model as well as the prom template for this I'm going to use one function from Lang chain called load summarize chain okay previously I think I showed you we can use chain okay only chain we can use now there is another chain actually called load summarized chain because
inside Lang chain you are Getting different different chains okay okay we are using this particular chain actually load summarize chain now we'll be creating my entire CH see this is the entire chain first of all you have to give the large language model OKAY the LM we have defined GPT 3.5 then chain type is equal to refine because we have reped one refine PR template okay that's why it's a refine varos true that means it will show you the execution um all the let's say logs then here I'm giving The question prompt first of all
the prompt I have defined at the very first okay this is the this is the question prompt okay because it's a chain it will execute after uh one one after another okay first of all it will execute and whatever question actually I'll getting I'll pass to the Define PR template okay you can see refine prom template okay now this is going to be my chain right now now simply I'm going to pass the document question that means that Means this one I have to pass this one that means the entire document okay this document I
have to pass right now to my large language model and it will give me the questions okay it will give me the questions and these questions I'm printing let me show you see it is generating and it is giving all the information because you have defined barbas is equal to true now see these are the questions I got I got 10 questions now if you got if you want To get 100 questions you can Define here you can Define inside the prompt I need 100 questions okay you have to generate 100 question it will generate
100 question by default it generating 10 questions now if you if I want to show you I will open in a text editor see all the 10 questions I got here but apart from that I'm getting some more additional information I don't need I only need the questions you can see 1 to 10 questions it is giving me what is the Main goal of the sustainable development goals method what are the some key achievements in the see that means if you read this questions it's pretty good now as a human if if I tell you
just prepare the question for me I think uh you'll be thinking in that way okay this is the idea that means my answer is pretty good guys and why I got the pretty good answer because I use the refine prom template okay so this is another powerful concept you can use Okay so now we'll be creating the next part that means we'll be using the embedding model we'll be generating the vector embedding and we'll be storing to the vector database okay because I also need to generate the answer right now fine so this is what
I have to perform so let's do it so for this first of all I have to Define one embedding model so here I'll be using opena embedding model so for this you can import from the Lang chain embeddings open I need open Embedding model okay so let me initialize so here I'm going to use open embedding model so if you go to the open website and if you go to the model section you'll see that open also provides lots of embedding model okay so you can use any of them now I also need to initialize
my Vector database so here I'm going to use f Vector database I already told you so it is available inside lanin lanin Vector restore F you can also use chrom RB Pine con web it's Up to you but I'm going to use the F because I need to show you the f as well okay and F is very like Fast um I mean than your chroma DB okay you'll see that it's like very fast quite fast now let's initialize my Vector store see that's how you have to initialize the vector store F from documents and
you have to give the documents here like which document you want to create the embedding I think remember we created a chunks this chunks I think Remember document and Sur this chunks I have to pass and I have to create the embedding okay and I have to give the embedding model this the embedding model I'm passing so this will give you the vector store and it will save locally guys okay it will save locally you won't be able to see that okay it will save in memory actually and if you want to see that you
can store in the cloud based uh I mean you can use the cloud Vector d in the Cloud you can sa I think I showed you the pine cone there you can see the visualization of your vector okay it's possible now let me again initialize my large language model you can see again I'm using gbd 3.5 turbo large language model to generate my answer because you can see I told you now the question I'm getting I'll again pass to my large language model large language model will give me the answer okay that's why I'm initializing
my llm now previously I um got some questions I think remember so this is the questions I got okay these are the question questions I got uh if I show you uh beautifully so what I can do I can split with the help of Slash in so it will give me line by line everything now this is the question list see so this 10 question I got now I have to generate the answer of this 10 question okay with the help of large language model now for this let's create a retrial QA okay question answer
let's A chain because inside chain we are having different different chains I think I told you now I'm using retriable Q CH this CH I already used in my Vector DV I think you remember let's create it now let's initialize my qn so here you have to give the large language model first of all now you have to give the chain type is equal to stop because it's a qn okay now you have to give the object that means your vector store that means you are connecting with your large Language model right now see llm
you are connecting with the vector store okay for this you have to write this code Vector store as reter that's it now this is going to give you one chain now we can give the let's say question and you can get the answer for this I written a full loop here see what I'm doing whatever question list actually I'm having I'm looping through it that means I'm taking all the question one by one okay you can see then I'm printing it Then I'm generating the answer you can see I'm calling this chain run question that
means I'm giving the question it will give me the answer this answer actually I'm also printing after that I'm opening a txt file inside that I'm just saving it okay inside that I'm just saving it that's it as of now inside a txt file I'm saving it going forward inside a CSV file I I have to save it okay this is the idea now let me execute see first of all this Is the first question it is generating the answer see this is the answer okay second question it is generating the answer third question answer
fourth question answer so that's how it will generate for all the questions let me show you see beautifully it is generating so we have created one amazing application guys now we have to convert it the modular coding there the idea now see all the answer I got and if you read the answer guys if you open Your text editor and if you read the answer it's pretty good it's pretty good if you see that question and answer is pretty good now you can use as it is as it is for any kinds of exam any
kinds of interview you can prepare it's amazing right okay so this is the entire actually let's say notebook experiment of our project now let's try to convert everything in a modular coding modular coding is I just need to write everything in a function that is the Idea only okay I'll refer the same notebook only I'll copy paste the same code code but I'll write everything in a function okay there is nothing new I'll be doing so let's start the modular coding so guys now let's start the modular coding but before that let me show you
this answer. now see inside answer. it has saved everything all the question and answer but again it's not readable so next time what I will do I'll save everything in a CSV file uh in One column I will add the question in another column I add the answer okay that would be more readable so what I'll do guys now let me close everything so left hand side I'm going to create a folder and I'm going to name it it SRC okay SRC because this is going to be my Source folder inside that I'm going to
write some helper related function that means if you open up any kinds of let's say um I mean real time implementation Of any kinds of package or any kinds of project inside python let's say if I open this one gas GitHub you will see that they have also created this kinds of folder structure see this is the kasas now kasas is also using see it is having one root folder caras inside that they're also having the source SRC and inside SRC they have created all the let's say utility related functionality that means whatever functionality they
need let's Activation function application back end call backs they're writing everything here and they're integrating in the endpoint okay it is also having some endpoint see API gen this is the endpoint and they're importing it here if you open it any kinds of endpoint they're importing it here okay this is called actually modular approach that means instead of writing everything in just one file we have to create a separate separate file separate separate Function and whenever I need it I'll just call it okay from that particular let's say uh helper function this is the idea
so inside SRC I'm going to create a file called underscore uncore init uncore dop okay this is the Constructor file because this folder would be considered as my local package right now see this is the modular concept guys this modular concept I already explained u in my Python session okay so I think you know so guys this is our YouTube channel so here if you go to the playlist section I already created complete python course so here I already explained about modular coding everything what is this Constructor file what is modular coding how we can
write an any function in any other file how we can import it in the endpoint everything I have explained just try to go ahead and watch that one because modular concept is required here Okay I'm expecting you are already familiar with this kind of python concept fine so you can uh also see the master op actually playlist so there also I have covered lots of advanced topic related python so that's why I'm creating this Constructor file because this folder would be considered as my local package right now okay and inside that I'm going to create
a uh file called helper do PI and I'm going to create Another file called prom. Pi okay prom. Pi so inside prom. Pi I'm going to mention my prompt so let me show you so the prompt actually I showed you in my jupyter notebook so here I'm going to just Define see this is my promp template first prom template and this is my defined promp template nothing new guys I'm just copy pasting from my notebook only see the same prompt I just copy pasted because because prompt is a changeable okay it's a changeable it's a
Static one people can change any time so that's why I have written inside prompt prompt let's say file so user will open the prom file and there actually they will change so that they they don't need to open the actual code code implementation and inside that they don't need to change okay I don't need to hard code these other the value that's why I kept this separate entity I I hope it is clear now now let me close it now inside help. Pi see whatever code I have written so far that means I loaded my
documents I perform the chunking operation then I created the questions okay these are the thing I'll be writing inside a function only so inside helper I created two function first of all you have to import all the required package you need I think remember these are the packes actually I need I loaded in my notebook then I need to load my what I need to load my open and I have To set as environment variable I think you know got it right now the first function I have written called file processing same thing guys if
I show you the same thing I'm performing here first of all I'm loading my documents I'm adding all the let's say um I mean content inside one variable I was performing this one um I mean chunking operation first of all I was doing with the help of 1,000 token then next time I was doing with the help of uh sorry First of all I was doing the 10,000 okay 10,000 chunk that means one uh chunk I want I want to consider as 10,000 then I was doing for the 1,000 that means the same experiment I
did here remember same experiment see same experiment I just written inside a function only that's it that is the difference only this so This is called actually modular approach you can also create a python class but as of now it's a very let's say a small implementation that's why python class Is not required but if you see the actually very big implementation any ml project or deep learning project you will see that there I written a lots of class let me show you so inside US Visa if I open the components let let's say data
in this you can see I have written a class that time okay so class is recorded because you can see lots of function I have to handle and whenever you are having lots of function lots of method that time you can write a class But here I don't I don't have actually lots of method I only have one to two method okay that that's why I've just written directly a function not a class okay this is the idea so that's how you have to design your own project whether your project size is very huge or
not if it is us try to maintain the class otherwise try to maintain the function this is the difference only so if you see any kinds of jni project so most of the time actually we create let's say Inference on top of large language model create a rag application on top of large language model okay that's why the code implementation is not big that much of but whenever actually I'm creating ml project DL project there actually we are training our custom model that time actually uh lots of method we have to handle very big code
base we have to handle okay that's how we need class concept this is the idea now you can see guys I will get my object okay object Related my chunking you can see it will gen give me document question gen that means after performing the chunking and document answers then okay both actually it will provide so questions in is nothing but uh you can see I'm converting to the documents this question whatever questions actually I was getting so this is the document question that means after uh converting to the documents whatever documents I'm getting this
is My document questions Z and after doing the chunking this is my document answerers gen so this question generation and this document I need for the answer generation okay the same thing I perform in by notebook I think remember see uh first of all I was using document uh question gen just to generate the question okay then I will was using document answer gen just to generate my answer okay I hope it is clear so this function I have created Guys now the next function I have created my LM pipeline that means I will be
using my large language model I'll be creating the vector embeddings okay so everything will happen inside this function right now llm pipeline first of all I'm calling the file uh file processing that means this function inside that whatever file user is giving this file would be process and it will give me document questions in and document answers in I'm creating my Large language model creating the prompt both prompt actually that means my prompt question as well as the refined prompt okay both I'm creating then I'm first of all creating load summarization I think remember load
summarization I was creating okay this one so it will give me the question it will give you the question and I was creating the vector embedding after that Vector embedding then I was uh creating the vector index You can see Vector index then I was uh I was creating the retal keyway that time okay it will generate you the answer that means question answer that means here you will get answer question generation and filtered question list okay both actually will get so this two function I prepared here now let me save now let me create
the endpoint so here what I will do I'll create a file I'll name it as app.py okay now this is going to be a Endo so Inside that I'm going to call this function actually this llm pipeline because inside llm pipeline I'm calling my file preprocessing that's why I don't need to call separately so let me open it up but before that let me initialize my first API related code so if you go to the first API documentation guys you will see this is the U start our code that means this is the demo code
they will also give you so these are the library you have to import one by one Okay then you have to initialize the fast API so how to initialize the fast API guys so that's how you can initialize the fast API okay see this is the code to install the first API now here you have to create two folder static and templates inside that you have to create the uh index.html file so inside template folder you have to create a index.html file because index.html file would be your homepage okay that means the user Interface so
you need some HTML code let me show you I already created so guys as you can see I already created one template folder inside that I created a uh HTML file okay now see inside that I have written already HTML code okay that means my front end part of my application and how I created again I referred that boot strapo side I think you remember I previously I also used the boot website if you don't know about HTML code it's completely fine you can Skip this part you can use let's say any other framework let's
say stream later on uh and see this HTML creation would be done by the front end developer as of now just to beautify this project I created HTML okay from my end that's the idea and there is another folder I created called startic inside instatic I created two folders called docs and output see whatever documents user will upload that PDF it will save inside documents folder and whatever output you Will get from the let's say application that means the question and answer in a csb format because you can see I imported the csb it will
save inside output folder and from output folder actually I will get that particular let's say csb output okay and inside documents it will store the PDF this the idea now this is the first API code guys first API will handle each and everything now I also need to import my this one that means my LM pipeline let's Import so it is available inside uh I'll call it from let's say SRC SRC uh dot let's say helper import LM pipeline see it is inside helper I need to import llm pipeline because inside helper I have this
LM pipeline okay now see if you I'm importing like that from SRC helper import LM pipeline that means SRC is now considering my local package and if I want to use SRC as my local package what I have to do I have to create one file called setup.py okay setup. Pi so this is the modular coding concept guys again inside setup. Pi you have to write some setup related code let me show you so this is the code guys setup tool first of all give the project name version author name author email package and install
requires that means this find package will try to find okay wherever this Constructor file is available this folder would be Considered as my local package now you can see Constructor file is only present inside SRC that means SRC would be considered as my local package now see if I show you pep list P list inside my environment see SRC is not mentioned SRC is not a package okay see SRC is not a package but whenever I will install it whenever I will set install this setup you will see that this SRC would be considered as
my local package that means there would be a package name Called genv project you can give any name I've given genv project now let me save and to install it in the requirements you have to add one more line called hypen space dot okay so this hypen space dot will look for the setup.py and it will install this package okay let me show you now if I again run let's say pep install henard requirement. txt see this would be installed as my local package Right now done okay now see uh some egg info file would
be created okay whenever this egg info file is created that means this setup is successful now if I again open up my environment there's a pep list now see uh there is a package it has created gen project okay now see it is my local package right now that means this SRC would be considered as my local package now I won't be getting any kinds of error right now if I'm importing like That okay so this is called actually modular approach I think you got it so here first of all user will upload one PDF
for this I have written a function with the help of first API so this is the function uh there is a route called upload user will upload one let's say file and uh this file would be saved inside my documents uh inside Statics documents okay it will save now there is another uh function I will create called get csb that means the complete results Okay I want to save as a CSV file so this uh function will help me to do that that means the file actually I'll be receiving here this file I'll give where
in the llm pipeline and LM pipeline what it will do it will pre-process it will create the let's say prompt and everything then it will give me the um that me chain entire chain okay and that chain actually I'll utilize here you can see so you can see I'm getting the entire chain and whatever question list Actually I'm having okay question list I'll pass the question inside my chain and it will give me all the answer and this answer actually I'll save inside my CSV file you can see right row okay CSV writer right row
and with for this actually I'm using CSV package you can see csb package very simple code guys only have written inside a function only you just need to go through this code you will able to understand okay then I need another function guys called Chat so this particular function what it will do it will load the output file that means the CSV file okay and it will show it will return you okay it will return that means you can also see the complete PDF okay in the UI let me show you how this application will
look like so now let me execute my first API code see this is the first API code even on run app this is the local host and this is the port number 8080 okay now let me save and now let me execute so I'll open Up my terminal so python app.py now see it is running let's me give the allow permission now I'll go to my Google Chrome and here I'll search for local host 080 see this is the user interface guys okay now here you can upload a file first of all let's say I'll
upload this file inside data let's say I will upload this uh sdg first of all this sustainable go Development and here I'll generate the question and answer let's see see it is generating this is the user interface we created basic user interface now see it's loading and how this uh I mean uh loading is happening animation is happening because I written everything inside HTML code so guys you can see I got the output now left hand side you can see it has rendered my entire PDF okay the PDF I uploaded and right side it has
given Me that uh CSV downloader now see if I click here it will download the CSV file see Q CSV now let's open the CSV uh how I can open let's open it here so I'll open it see this is the csb I don't have Excel right now guys in my system that's why I've open in a b code you can see this is the question this side and that side actually I have the answer you can open in Excel viewer you see that it's a beautiful format I have just stored okay so that's how
actually things are Happening okay now I think you got it why we have written Des are the function so this function will help you to render your PDF documents as well as the it will give you the download option okay to download this particular PDF now let's upload another PDF let's say I will upload this Statistics one right now now let's generate again so guys as you can see I got the results and this is my pdfi uploaded and let's download the Qs B well see this is The correlation answer and question I got fine
so yes guys this is the project actually I hope you liked it and this is like very interesting project guys if you develop it I will feel like it's like very interesting going forward we'll be also implementing lots of project no need to worry so in the next video we'll be implementing another Amazing Project very interesting project called uh chatbot for any website that means let's say I think you saw website Chatbot if you go go for any kinds of website nowadays they're having their chatbot integration with that okay so if you ask let's say
some question related about the said about their let's say product and service it will able to give the answer so that means they're replacing the human behind it previously they used to let's say use one human they used to receive the chat they used to give the reply but nowaday they have uh added the chat uh chatbot there Chatbot can handle all the answer if you go to any of the website you will see that so we'll be implementing these kinds of application that means uh the custom website chatboard it will only work based on
our website data not any other let's say uh I mean question and all fine this is what actually we'll be implementing and this is kinds of rag based application we'll be creating okay that means we are using our custom data on top of that we are adding about let's Say a large language model large language model will try to refer that data to provide you the answer fine so in this implementation also I'm going to use Lang chain large language model even we'll be also using something called Vector database okay with the help of that
actually we'll be implementing this entire application but see this application would be little bit Advanced because here we'll be creating something called rag based system okay like r Augmented generation this is the basic rag so here we'll be using our custom data so custom data wise actually we we're going to use our website data that means we'll be giving some website okay website URL so U my langen will automatically uh fix the information from the website itself and this data actually I'll give to my large language model that means my llm would be connected with
this data and if you're asking something related about this Website okay it will give you the answer related that okay so this is the application we be implementing so first of all let me show you the architecture like we are going to develop this project then I think this part will be clear so before showing you the architecture diagram guys let me show you one thing so let's say if I want to open any kinds of website let's say I'll open this open.com okay this one open ai.com So this is the open.com website now see
if I want to like see the if I want to see the let's say XML page of this website so what I can do I can give this particular let's say uh U route here so slash sitemap um sitemap.xml so if you open any website and if you give SL sitemap.xml it will show you the entire structure of the website let me show you so guys here you can see uh this is the page actually I got now see inside Opi.com okay inside open.com whatever let's say content you are having whatever content whatever sub Pages
you are having all the information is coming here you can see if you just see it carefully so it is having different different URL okay different different URL for the different different tab let's say research for research actually it is having one URL for GPT 4 40 it is having one URL for safy company see company I think I saw here so see this Is the company one okay this is the company one you can see okay company one so I think I showed you the company so see this is the company URL so that
means it is having all the information all the data whatever data whatever content you can see in this website okay everything is visible now in this XML page now what I have to do I have to use this concept to extract the data that means I'll be using Lang Chen data loader so inside Lang Chen actually we are having website loader with the help of that we can easily load these are the content so I'll tell you how we can extract the information so this is the thing actually will be doing now let me show
you the architecture diagram what we are going to perform here so let me open up my Blackboard see here first of all I told you we'll be using website URL okay website URL so from website URL what we'll do we'll just get this sit Map uh sitemap.xml okay this page and from this page actually we'll be getting different different sub Pages as well let's say page one then page two page three okay and so on that's a multiple page actually will get from here now what I have to do from each and every page I
have to extract the documents that means the Data so it should be docs one let's docs 2 then after getting all the documents we'll be using one embedding model I think you know why embedding model because I have to convert all the documents to the embedding representation okay so I'll pass all the data to the embedding model okay and embedding model will try to return uh embedding Vector embedding so you will get different different Vector embedding let's say 0.2 0.3 0.5 and so on let's Say these are the vector representation as of now just try
to consider then with all the embeddings what I have to do uh I have to create one uh cementing index okay cementing index I think you know build cement index okay build centic index then this uh centic index that means this embedding I will rest where I will to the vector database let's say this is my Vector database so in this case actually I'm going to use f you can use any other V Vector DB let's say this is Vector database okay I'll be using F you can use anything you can use let's say chroma
DB you can use find con I'll tell you how to use pine con chroma later on I will also tell you now what will happen so user will ask question okay user will ask some question about the website and this Vector DB will return Some rank results okay I think you know what is rank results ranked results okay rank results I'll get but what I have to do I have to initialize my large language model because with the help of llm only I'll able to get the correct response so what I will do I'll pass
this rank res to my large language model as well as this quy also okay and this llm will give you the actual response okay actual response so this is the complete idea okay this is the Complete architecture of our application okay and this is called actually rag okay rank that means retable augmented generation because here we are inting our customer from data okay and we're integrating our large language model with that uh data that means this is called actually knowledge base okay knowledge B right now okay in knowledge base so now let's open up our
collab notebook and try to install everything and let's implement this project so guys As you can see I open up my collab notebook so first thing just try to change the Run Tye so here run time just keep it as CPU only because here we'll be using open a model but if you want to use let's say any other open source Source model that time you can use GPU okay that time so as of now let's keep it CPU only so later on I will also show you how we can use a new open source
large language model okay now let me connect this notebook so here first of All you have to install some package so you can see I'm installing Lang chain langen Community Pi PDF then sentence Transformer okay openi Tik Tok and these are the things so these are my dependency package actually I need so let me install all of them so some of the package actually you don't need I just written additionally because later on this project I will show you how we can Implement with the help of Open Source large language model Okay that time actually
this sentence Transformer is required because if I want to use any let's say open source embedding model that time this is required so as of now let's install everything then apart from that I also need to install some more package so tick token fire CPU and unstructured let's say if I want to load the website now website data that time I need this package unstructured because it's unstructured data and again I'm going to Use f Vector database okay that's why F CPU I'm installing so let me install all of them then I also need to
install NPI and nltk okay so let me install because these are some dependency package if I want to extract the information from the website so Lang needs these are the dependency okay so that is why actually we are installing everything now see uh after installing this nay ltk it will tell you restart The session so let me restart the session so it will restart the session then after that we'll be importing all the necessary Library we need you can see I'm loading unstructured URL loader from langin so with the help of unstructured URL loader we'll
be loading this data okay here we'll be giving the URL website URL it will automatically extract all the information and we'll be performing the chunking for this we are using character text spitter open Embedding we'll be using just to convert my uh data to the embedding representation we'll be using openi large language model GPT 3.5 turbo that's why we are using CH openi F this the vector DB rable QA with Source CH that means it will create the QA object that means the chain object okay and hugging face embedding as of now I don't need
I'll just try to remove it okay later on I'll tell you how we can use the hugging p as well now let me import All of the package so it's working fine there is no error now let me uh download this uh two thing like p n KT and you have to download another thing a percept perception tagger okay so this one you have to download from the nltk so let me download then after that I will set my open API key because here I'm going to use open large language model definitely you need the
key now I'm going to pass different Different website URL now you can see I have just collected different different website URL let me open all of them let me show you so these are the website actually I'm going to refer so this is one uh blog website uh medium website you can see and the content about paper review Lama 2 open foundation fine tune chat model okay now if you just read it you will see lots of content related Lama to okay now there is another website datab I'm using introducing MPT 7B a new standard
for the open source Community usable llm so this is another large language model now you can see uh the information is written about this large language model that means we'll be extracting these other the data and we'll be creating one chatbot it will uh provide and that chatbot actually will be able to give the answer related these are the topic okay the topic actually you have feed let's say Lama 2 then MPT 7B then uh stability. a okay so this is Another website then uh you can see vuna so this is another website okay so
these are the information actually I want to see so I have given all the website link so here you can pass any any kinds of website link you can pass any kinds of website link let's say you have one website your own website you can give the website link here so this will automatically extract your website information and you can create your website chatbot okay on top of it this Is amazing now let me initialize the URL so it suppos actually multiple website URL it's not like that it only supports one URL no multiple URL
you can give here now let me uh extract the documents with the help of unstructured URL loader inside that you have to pass all the URL and it will automatically extract all the information let me show you now if I print the data all the information it has Extracted from all the website see okay amazing now if I show you the length you'll see that four documents it has returned why four documents because I have given four website 1 2 3 4 that means four website information I'm having right now and this is in the
document format got it because I already told if you're loading anything with the help of lch that should be document format always so now I think this part is clear so we have extracted the Content successfully now what I have to do I have to create the chunks okay I have to create the chunks different different chunks you can see these are different different chunks so I think I forget to mention this chunking here so it should create a chunk okay chunk okay all the chunks that means it will create all the chunks this chunks
actually I'll give to the embedding model okay now let me create the chunk here so to create the chunks actually I'll be using character text splitter see these are some repetitive code again again I'm using previously i al already taught you what is character Tex bitter what is CH size what is CH overlap everything I taught you even in the vector database session so this is very much important session guys just try to complete the session first so you can see character text ler separator is equal to SL in because you can see Slash in
is there okay sometimes so slash Would be the separator so here SL n would be the separator it will separate by slash now CH size 1,000 chks overlap 200 okay now let me create my text splitter now inside that I'm going to provide my data now it will give you chunks now let me see how many chunks I got so here I got 86 chunks if you want to see them simply just uh print like that let's I want to see the first chunks this is the first Chunks let's I want to see the second
Chun as as well I can also see that see second chunks as well and you can see it is also giving me the source URL like from which URL it is giving you the data okay now we'll be initializing the embedding model because you can see the architecture now I have to download the embedding model then I have to convert everything to the embedding representation so let's use the open embedding now let's test whether it is Able to convert to the embedding representation or not so here I'm calling the embedding model and I'm passing one
sentence hello world so it should give me the embeding results let me show you and the vector Dimension is uh 1 1536 okay this is the dimension now if I show you the numerical representation of this sentence so this is the numerical representation that means this is my Vector embedding okay so we'll be Following the same concept now let me convert my um text to the embeddings but for this actually I'm going to use f Vector database because whatever embeddings I'll be generating I'll store in the F okay F Vector database and F is a
local Vector database I think you know so here I'm giving my text chunks and my embedding okay embedding model now guys see this is my centic Index right now okay this is my centic index that means my Vector Index right now okay now what I have to do I have to connect my large language model so let's initialize my large language model so here I'm going to use open a large language model by default it will use gbt 3.5 turbo now let's test our large language model whether it is able to generate or not so
here I've given a prompt please provide a concise summary of Harry Potter book okay now if I execute it should give me the summary of Harry Potter book see this is the summary of Harry Potter book Okay it has given me great that means it is working fine now with the help of llm as well as my Vector database I have to create my chain so for this I'll be using retal keyway with Source chain okay I think I already imported from the Lang chain and inside that I'm passing my large language model as well
as my Vector database okay and this will give me the chain now inside chain actually I can ask any kinds of question right now see question is equal to how good is Vuna so here I'm asking related vuna okay this particular vuna so vuna data I have already given to my large language model I think remember it is already available inside my knowledge base now it should give me the answer okay let's see okay now if I want to see the answer so I'll just call result inside that I I want to only extract the
answer okay now let me show you see vuna has shown competitive performance compared to the others model like Stanford Alpha okay That means if you read it okay if you read this documentation if you read this documentation you will see that uh that is what actually they're telling that means amazingly it is able to give me the answer right now great now let's ask any other question let's say the next question I'll be asking about Lama 2 Lama 2 how does Lama 2 outperforms other model okay let's ask this question and whatever answer I will
get I'll also print it Here now see L 2 model out for from others models in various categories including MPT models and similar size and so on okay now see it is referring the L 2 right now uh yeah Lama 2 great now let me ask another question so I'll be asking about MPT uh this one MPT 7B model and whatever result actually I will get I'll just showcase it here see again I got the amazing response okay now see it is referring my website data that means my custom data Not any other data okay
this is the main thing here we have developed now if you're asking any other question let's say tell me about India it won't be giving the answer because here we have created the custom chatbot it will only refer my uh data the custom data I have given now here I can create a while loop and I can continuously ask the question okay so you can see I'm taking the prompt from the user and if it is exit I'm I'll do the exit operation otherwise I'll continue and I'll will keep on asking the question and it
will generate the answer okay let me show you see it is asking the question so what I can give I can give the same prompt again copy and here I can pass and let me press enter now what you can do guys you can convert this project to the web interface that means web application I think I showed you how we can use different different let's say python package like flas then Fast API so this should be your task guys try to create one web application at least basic web application that user will give give
that prompt and it will provide the answer okay that's it now see it has given me the answer okay now again it is asking another question now let's say if I give exit it will exit the application sorry it should be exit only but I have given exit parenthesis now see if I if I'm giving this prompt it is telling that text provided does Not seems to be relevant to the questions now see this question actually doesn't have any context whatever knowledge base it is referring that's why I told you it it it won't be
able to give you any other information now if I give exit right now it should exit the application see okay so that's how we can create any kinds of custom website with the help of our custom data now what you can do you can open up any kinds of website and you Can implement this project guys okay and later on I'll also tell you how we can use the open source language model okay and with that how we can create this application so with that guys thank you so much uh for watching this video and
try to complete that task try to convert this application to the web interface as well so guys as of now we have worked with open AI based large language model which is nothing but uh GPD okay so we have seen like how to use GPT series uh To build these kinds of Genera VI application and one thing I think you have seen there like GPT is not free so uh you need to pay for it so whenever you are using open a API uh you need to pay some money okay for that uh but let's
say if you don't want to spend uh this much of money here so what you can do you can use something called open source large language model so in the market actually we have lots of Open Source large language models are Available so so if you are using it so you don't need to pay any money for this so you can directly use them and it is completely open source and for commercial use cases as well you can either do research on top of it you can either uh create a commercial application on top of
it it's completely free for you okay so uh in this video actually uh I'll be discussing about uh like open source large language model like what is open source large language Model okay and what are the open source large language model we have and from them actually I'll be discussing some of them uh like most popular uh used open source large language model and it is like very powerful okay and I will also show you how we can uh like build gener VI application with the help of these kinds of Open Source large language model
okay so it would be completely amazing video so make sure you are watching this video till the end don't Skip any part of this video otherwise uh you might get confused for sure right so so yes guys uh let's start with our uh video so guys here we'll be learning like most used and most popular very powerful large language model uh here you can see we have meta Lama 2 so this is the uh like model from meta AI I think you know uh like we also call them Facebook so they have introduced this model
called Lama 2 it has another version called Lama 1 but uh uh Lama 1 Is for like resarch purpose they have given the permission but uh you can't use Lama one for the commercial use cases so for this actually they have open source this Lama 2 model so you can either uh do research on top of it either you can create commercial application it's completely fine then from Google side we have another large language model called um Google Palm 2 so this is uh one of the very powerful large language model okay and it is
Completely open source and free then we have another like most used large language model called Falcon so Falcon had lots of variant I'll show you like whenever I'll be discussing like Falcon I will tell you like what the variant it has actually so it has different different uh parameter variant okay even meta Lama 2 has different different variant okay I will tell you whenever I will discussing about this Lama 2 and all okay so these are the most used and Most popular large language model I'll be discussing like how we can use it okay and
how we can create gener application on top of it I will be discussing each and everything so apart from that we have lots of large language model are available in the market and it is completely open source so guys I found one GitHub repository so there actually that guy collected lots of Open Source large language model list and uh he has created one uh read me file there So I'll show you that one like what are the open source large language models are available in the market so if you also want to explore that one
you can also go ahead and check it out okay so if you learn like how to use meta Lama to Google Pam to Falcon okay you will also able to do that okay so guys as you can see this is the repository and the repository name is like open llms and here if you see uh we have lots of opens large language model are available you Can see T5 E2 then pathia Dolly okay then here you you will see like Bloom is there then F chat then mp uh mptp uh 7B this is the 7
billion parameter then Falcon I was uh I will be discussing about Falcon so as you already saw like in my uh topic I will be discussing about this Falcone okay so uh even that actually they have also shared this paper blog and everything okay if you want to learn you can open it and all the steps actually You will be getting then then lamb 2 is also there okay then mrl 7B also there then Google Palm is also there okay so uh here actually we'll get all kinds of open um Source large language model okay
and if you see here it was committed 4 days ago so it is like very update repository so here you can check it out uh if any large language model is coming in the market so here actually you will get all the list and all and apart from that you can also search from your site Uh let's say if you are exploring any other topic and all so you can search and you can learn about okay so yes guys uh this is all about our large language model open source language model I think you got
it okay so guys we don't only have the option using uh like you can say open AI GPT model apart from that we have these kinds of Open Source L language model we can utilize okay uh so I personally prefer this kinds of Open Source large language model because this Is like very powerful so it it can also work like your uh GPT 3.5 turbo okay so it's like very powerful model so using these are the model actually you can solve very complex problem statement even you can create a very beautiful Genera application so why
you need to pay right but if you also want to pay because see GPT is like very good model I know that but again U it's like completely I mean hosted like API so you can directly call the API you can hit The model you can get the response so this is the beautification they have created in their website but apart from that let's say you don't have money and you don't want to spend money on top of these kinds of model so you can use this kinds of llm model and you can create your
application on top of that even you can also fine-tune them okay so this is the beautification of these kinds of large language model so in future I will also show you how we can fine tune this Kinds of large language model on top of our custom data on top of our custom use cases and how how we can use it okay so fine tuning is not required unless and until your data is like very complex and it is like very uh private data so that time you can find tune them but most of the use
cases can be solved using these are the model okay so yes guys now let's go back and try to see like what are the things actually we'll be learning from this video so guys uh in this video Actually I'll be discussing like the introduction to Lama 2 like what is Lama 2 and all like uh who has published this Lama 2 like what the variant actually Lama 2 has and all then I will show you like how we can run this Lama 2 okay I'll show you like some difficulty level like how why we can't
use the actual Lama 2 model OKAY hosted from meta AI so which model actually I'm going to use it so how to execute this L to okay I'll be discussing about on Google collab Because you can also do it in your local machine as well but I will on Google collab because Google collab would be pretty much good environment for me because there I will already have lots of predefined Library setup right then I will U show you like how we can use this lambat to with langin because nowadays actually people uses this langin framework
okay to create generative application so I'll also show you how we can use the langen okay with this kinds Of Lama to open source large language model and at the end of the video I'll also show you how we can build generative AI projects using Lama 2 so there actually we'll be implementing one projects very uh interesting projects okay okay uh with the help of this Lama too even I will be using Lang chin there also okay then uh we'll see like how we can integrate open source model with these kinds of framework Lang chain
then how we can create this kinds of Generative AI projects so it would be completely end to end video about this open source lar language model okay you'll be learning each and everything about open source language model like you will be knowing each and every part of this Lama to and all so make sure you are watching this video till the end uh so this should be my request guys and also try to practice with me okay in Live or whatever things I'm writing you just also try to write with me even all The code
and everything will share in the resources section from there actually you can get it so first of all let's discuss about Lama 2 and in the next video I'll be discussing about Google Pam and fcon so guys if you search on Google like meta Lama 2 so you will able to see this website the first website okay so this is the website of Lama 2 so here actually they have published this Lama 2 documentation and all like how we can use it and all and Uh this documentation will give you the entire idea about Lama
to like uh what is Lama 2 and U uh how many variants actually it has okay even The Benchmark also you will able to see so guys here if you see uh Lama 2 is nothing but it's a Next Generation open source large language model and Lama 2 is available for free uh research and commercial use cases as I was discussing about and if you want to access this Lamar to so they have given the link here so here Actually you need to fill some information like uh your name uh email address then country and
organization so these are the basics form you need to feel and you need to submit so after some time actually they will give you the permission because see you can't directly use this Lama 2 model so for this you need to send some information of you uh then after some time actually they will give you the permission okay I also applied this uh request form so uh After let's say 30 minutes or let's say 1 hour you will get the access for sure okay so uh I I will also show you one Al alternative so
if you want to do it right now so one guy actually cloned this uh complete uh lambat 2 model OKAY in his uh you can say hugging face repository and it is completely public okay so you you can use that model uh if you're not getting permission and once you got the permission you can uh use official version of the uh Lama 2 model Okay so I'll show you like both option how we can do it so for me I already got the access from this meta so they have given me the access I'll tell
you like how to like uh uh apply for the access and all okay so as of now uh the experiment actually I'm going to do like how we can execute this Lama to model I'll be using something called quantized Model Because the actual like L model you can't execute on Google collab on free Google collab so for this actually You need uh good instance okay like good instance means like you need powerful machine you need more RAM okay you need more memory there but uh uh if you are using free version of Google collab and
if you're using your local system so there actually you can't load this model so it would be very hard for you so better we be using some quantized model I think you know what is the quantized model so quantized model is nothing but it's a quantized version of the actual Model so they have just uh compressed the model okay with the uh integer encoding and they have proposed this model and this model size is a little bit less so we can load this model in our memory okay in very easily I will tell you like
how to use quantz model and all okay so guys here if you see Lama 2 was train on 40% more data than LMA 1 okay because I told you uh there is another model called LMA 1 and LMA 2 train U and here LMA 2 trained actually More than 40% data than lama lama lama 1 and it has actually double context length basically the context length is like double okay then you Lama one and see these are the LMA 2 variant actually Lama 2 has 7 billion parameter and another variant has like 13 billion parameter
and another variant has 70 billion parameter and uh this is the pre-training token you can say so they have used true trillion token to train this model and here if you see this is The context length so it's 4,096 but uh whenever I was talking about Lama 1 so it has actually 2,000 something context length but it is like double okay double than your Lama 1 and uh this size is around 4,096 and here if you see uh the training strategy so they also use supervised fine tuning technique on top of it then they also
use something called human and preferences I think you know like how we can train these kinds Of large language model okay what is the idea and all about uh basically this is the strategy we follow in every large language model okay then they have applied something called reinforcement learning to understand understand the human human input don't understand the human uh you can say prompt okay so they they have used this strategy to train their entire model and Lama 2 Preen models are trained on two trillion tokens here I already discussed and uh Have double context
length than Lama one it's fine tune models have been uh trained over 1 million human annotation okay so 1 million human annotation means they have applied this supervised fine tuning technique so uh they have created one chat use cases so basically see whenever I'm giving any input let's say just just fix this code for me or let's say generate a you can say python code for me so these are the chat actually I'm giving so with respect to that Actually what is human annotation human annotation means like see whenever you are giving these kinds of
prompt so one human will be replying the actual thing here okay that's how your model will be learning let's see if you're giving any random message let's say Hi how are you fine okay and all so one human will chat with you like manually they will chat with you so we call them human annotation and with the help of this data actually we train our actual model And we create these kinds of uh very powerful large language model right and this is The Benchmark as you can see so here uh we have lots of model
uh we have MPT 7B then Falcon 7B and we also have our Lama 2 7B 13B and we also have 60 5B and 70b okay so here if you see these are some Benchmark data and this is the performance of the model as you can see MTP uh MPT uh 7 billion uh this was the accuracy 26% accuracy and here if you're using Falcon so 26.2% accuracy and if You're using Lama 2 7B here so see accuracy is like very much increased here it's around 45 .3% of accuracy and if you're using 7B that uh
sorry and here if you're using 13B that means 13 billion parameter version of the model so here this is the accuracy 54.8% of the accuracy and here if you're using m m PT 30 billion so this is the accuracy then I mean it's like less than your lamb 2 and Falcon 40 40b actually will give you 55.4% and if you're using Lama 65 billion so 63 uh 63.4% and Lama 7B uh like very good accuracy here 67.9% of accuracy okay so with respect to that you can see this is again another data so this is
like mm Lu data so this is trivia QA data this is natural questions data so you have lots of Benchmark data with respect to that these are the model has been trained and this was calculated the acury okay and this is the and this is the complete Benchmark Benchmark table here you will see all the performance of the model okay and guys uh if you want to test this Lama to a model so here U there is one UI has been published here APC on top of the Lama 2 model so if you just write
ww. LMA 2. a so this is the chatbot actually you able to see so this is kinds of chat GPT kinds of interface they providing and in the back end actually they're using LMA 2 model here ifpc so you can select Lama 2 model Different different model so by default it is selected LMA to 7 billion so let's select this one LMA to 7B 7 billion parameter this model and you can also give the uh your system prompt here so let's take this prompt only and the temperature you can also select let's say if this
temperature is close to zero that means this model would be more strict to the output so it won't give you any Randomness and uh if you set this temp as close to one that means Your model will be like more random so it will give you like more random output okay and if you want to make your model like more uh generalized to Randomness so you can increase this parameter to close to one and this is the max length like how many output you want from your model and all so you can set this parameter
and all and this is the top probability score and all okay so here you can write your custom prompt as well let's say here in this case I'll be Doing uh chat chat operation with this Lama 2 here if you see chat with Lama 2 you can also set this prompt as summarization let's say code generation anything you can do okay so you need to just change this prompt I think you know what is prompt and all so this thing is very common okay in llm and all so you can give the prompt here so
let's take this LMA 7B parameter this model and let's uh do some chat with this model so here first of all let's say I just write Hey uh or let's say hi so here if you see this is uh giving me the response okay now here I can ask like uh what is the so what is the library uh we can use to build jna application I'm just giving any random question let's see whether it is able to uh answer or not so here if you see it's giving Different different answer okay with respect to
the question I have asked now let's say here I can also give any error of my code and if I tell uh it actually just try to solve this error so let's say I was trying one OCR code here so this is the error actually I got so let me show you the error so guys uh this is the error actually I got here so could not find a version satisfied the padal paddle okay this is the error I got so this error actually I will ask to this Uh LMA to and let's see whether
it is able to f F or not here I will just write fix it okay if I just write fix it now let's see what happens so guys see it is giving me the response and uh it's telling like U install this version of this paddle paddle okay and you can also use some third party GitHub uh to build this uh setup tool and all so it's giving all kinds of possibilities like you can try you can get this suggestion and you can Try and you can solve this error okay so Lama 2 is telling
actually this model is uh like more powerful than GPT 3.5 turbo model okay because see I already use this lamama 2 model and what I feel like yeah this model is pretty good than your GPT 3.5 turbo if you're using GPT 3.5 turbo you saw like lots of task actually don't be able to solve in a good way but if you're using Lama in this case Lama 2 in this case actually it can solve like various kinds of task okay so this is The playground you can play with this LMA 2 model you can select
different different version of the model and you can play with this model all kinds of question you can ask and you can realize like how powerful this modalities okay so yes guys this is all about our Lama um you can say introduction and all I think you got it like what is LMA to and all okay the basics overview you got it now I will show you like how we can use this how we can run this Lama to model On the Google collab so guys for this actually I already prepared one Google collab notebook
for you as you can see this is the Google collab notebook and here we'll be learning like how we can execute this Lama to model on the Google collab so as I already told you we can directly load this Lama to official Lama to model from the official Lama you can see this is the this Lama 2 is available on this hugging face so from hugging face itself actually you Can see this uh meta Lama okay and this is the Lama 2 model this is the original Lama 2 model official Lama 2 model it is
uh hosted on Hing pH from here actually you can load this model but this model actually I can't load in my free Google collab for this actually you need good instance good uh memory good GPU ANL but here I'm using free cup that's why I can't load and you can also load this thing in your local machine okay so for this actually what you need to do you Need to use something of quantized version of the model so again hugging phas also provides like quantization model so uh some of the actually organization have already applied
this quantization technique and they compress these are the actual model and they hosted on the Hing face community so here if you see we have like various kinds of quantized model here if you see so these are the Quant models so this is the so here guys if you see this is the GG ml format of quantized version of the model and if you see like lots of model are available okay so from this model actually I'm going to use one particular model so this is the model link I have given just let me open
it so the model name is uh um Lama 2 13 billion chat GG ml okay gml is the format of quantization and this is the organization so this organization has quantized this model okay and uh it is published here now this model is like More uh like uh s small model OKAY than your ual Lama 2 model which is published on hugging face uh this model actually you can easily load in your 3D Google collab okay or if you're using your local system as well you can also load there okay and this model has lots
of uh uh variant here if you see b means this is the model okay this is the bin format of the model that means binary model so from these are the model I'll be using one particular model because these are Different different quantization uh you can say format so I'll be using one particular quantization here and this is the size of the model as you can see this is the the size of the model and after applied quantization the size size has been reduced because what is quantization quantization means all the weights you have in
the model uh by default it would be floating number okay but if you apply quantization on top of it it would be converted to integer and Your size would be reduced okay your model side would be reduced automatically so this is the like a idea of quantization model so here your accuracy might decrease but again um uh you are uh like getting the first term model here you are getting the smaller model here so this is the idea okay and if you want to uh use actual this meta Lama or that means this Lama 2
model so here actually you need to get the permission you can directly access this Model so let's say if I open this link in a different or let's say incognito window so you will see it will ask for the permission actually because I'm using my account there so that's why you can't see the permission now see here if you see access to uh access Lama 2 on hugging P so here first of all you need to login with this website okay if I click on this log loing so it will ask for just logging with
your hugging face so here you need to first of all loging With your hugging face let's say I will loging with my hugging face here okay now see here I have already logged in that's why it's telling like uh permission granted but for you actually it will show like you need to apply for the access okay so whenever you will apply for the access so it will give you that form I think you remember it will give you this kinds of form so you need to fill this form okay and need to select all the
information so you can Select like which model you want to use so you can select both of them and and you can uh just send a request okay so after some times you will receive one mail like that so after some times actually you will see one mail like that uh let's say I applied for this access so this is the mail I got so hi puy this is let you know that your request to the access model metal Lama 2 on the Hing face has been accepted by the author okay so once you got
the access then you Will able to see this kinds of window here uh this kinds of window here GED model you have the granted access to this model okay but initially whenever you will be applying so it will take some time at least one to two hours then it will give you the access but if you uh if you want to try Okay this model so again I will tell you another option because this model is available on the public repository as well okay because see in hugging F we have two kinds of Repository one
is like uh organization repository like that means private repository if you want to use private repository so you need to like take the permission from the author and one like you have like public repository so in public repository actually you will have the access uh to the model you don't need to take any permission from the author they will by default provide the access and all so you can directly use that so one guy actually cloned this Entire model and he has published from his account here and I will be using that model okay same
model only but this is The Unofficial and that is the official one okay I'll tell you like whenever I'll uh show you like how we can use this uh Lama model with the help of langen at that time I will show you like how we can use it but as of now let's try to run this quantied version of the model as I already told you I'll be running this quantied version of the Model which is nothing but this model if I show you this is the quanti of the model okay and guys maai actually
also provides like different version of the LMA 2 model as you can see so we have 7 billion we have 13 billion we have uh 70 billion so in this case actually I'm running this 13 billion so you can't execute this 13 billion in your free collab that's why uh we are using quantized version of the model but in our next experiment so whenever I'll be Using Lang Chen over there so I'll be using this 7 billi parameter model and this model actually you can easily load in the Google collab or your local system as
well okay so for this model I I'll take the permission and see guys these are the model are hosted if you see these are the model are hosted from the Lama 2 as you can see chat model HF model I'll tell you like whenever I will be using these are the model I'll tell you so yes guys this is all about now Let me just quickly show you like how we can execute so see to execute like quantied model you need one Library called Lama CPP okay so this is the library actually uh you need
to use Lama CPP is a library actually this Library helps you to make the quantized of the model okay and if you're using quantized version of the model you need to install this Library so first of all I will connect this notebook now make sure you have selected Runtime as gpus so just click here and click on manage GPU uh sorry not here uh just click on the runtime and select uh here you will get one option called change runtime type and I have already selected GPU here so now if you want to check whether
you've got GPU or not so you you can execute this command and here you will see I'm I got Tesla T4 GPU and it has actually uh uh 15 GB of uh like vram now here uh these are the requirements actually you Need to install so first of all I'm uh installing this one uh Lama CPP Library okay and this is the command to install everything and uh here I'm also installing hugging face Hub because from the hugging face itself I want to access this model because see this is this model is hosted on hugging
face and if you want to access any hugging F model so you need to install hugging Hub and here I'm also installing Lama to CPP python okay so these two command Actually you need to execute to install Lama uh Lama CPP libr then I also need numai with this specific version now let me execute so guys as you can see my installation is done now uh here you just need to give the name of the model like which model you are using so here this is the model I'm using just copy this name here and
here you just need to paste it okay so I have uh just uh given this name here now here you also need to Specify the base model name like which particular model you you need to use so here I'm using Q5 1bin model so if I go here so it has lots of variant I already told you so here I'm using this key5 uh uh this model actually I think let me just see q51 dobin q51 dobin um yeah so this is the particular model actually I'm using so you can use any any model no
issue with that so I'm using this model and This is the size of the model so I'll be executing so you need to give the same name just copy paste from here and just give it here now once it is done now so here first of all I'll be importing one uh class called HF hub download that means hugging face model download so this uh class will help you to download any particular model from HF first of all let me import then I also need to import uh this Lama from Lama CPP library because we
have already uh Install this Library so let me import it and if you're not using quantized version of the model okay uh you don't need to install this Lama CPP so we can directly download the model from hugging F okay I'll tell you uh whenever I'll be using this model actually because I already told you Lama to provide Slots of version of the model whenever I'll be using smallest version of the model I'll be directly downloading the model from the Hing face okay at that time I don't Need this Lama CPP Library okay but here
I'm using quantied version of the model that's why this needed now first of all you need to download the model to download the model first of all you need to give the repo ID repo ID is nothing but your model name so this is the URL I'm giving okay and you also need to give the file name like which particular file you you need to download so here you need to give this Bas name here now it will automatically go to the Repository and it will automatically download for you see guys it's downloading and this
is around um 9. uh 76 GB so let's wait it will take some time I'm able to load this model because here if you see my vram uh GPU vram I got uh I got 15 GB that's why I can load this model very easily uh that's why actually I'm using this contage version of the model but if you're using actual uh this model if I show you Uh actual uh 13 billion parameter Lama 2 model okay uh without U you can say quantized model so uh it is more than your 9gb and you can't
load it okay that is the issue actually so I'm following this approach see this model has been downloaded now if you want to see the path of the model like what it has downloaded so it has downloaded in this folder actually in my machine now first of all I need to load my model to load the model actually first of all I'll be Defining one class Lama actually you have seen like we have imported the Lama and here you just need to give the model path so this is my model path here I have given
and these are the parameter you need to keep it okay these are the default parameter you need to keep it now let me execute and load the model so guys as you can see my model has been loaded now here you can also check like the number of GPU layer you have in your GPU so we have 32 layers And and uh now first of all I'll be creating one prompt template because because we don't directly uh give the prompt to our llm uh we also create a promt template okay so we won't be directly
asking the question because let's say what kinds of question actually you want to ask to your llm you need to specify using the prompt template I think you are already familiar with what is prompt template and all so here I have created one Prompt template so this is my prompt this is the question of the users just write amist image classification code in kasas and here is the promt template so I'm telling my llm so here I'm telling to my llm as a system system means uh it will consider I'm telling to my llm and
here I'm telling you are a helpful and respectful and honest assistant always answer as helpfully okay this is my prompt template and here I'm giving one example so user will ask this kinds of Question and you need to give the answer as a assistant assist assistant should be the response from my llm okay so this The Prompt template I have created now let me execute now here I'm initializing my llm so here I'm giving first of all my from template then I'm also initializing the max tokens Max token means like how many output you wants
from your model you can if you're using CPU so you can decrease this length if you're using GPU you can increase this Length Okay temperature I think you already know if it is close to zero that means you are uh you are taking like more streak output from your model and if you are increasing to one that means uh you are taking the randomness from your model but here I have set 0.5 so it's middle position and this is the top probability score and uh these are some default parameter you need to give okay now
let me execute it so it will take the prompt and it Will understand the prompt template and it will give you the answer so here I have written like uh write a amist classification code in kasas so it will uh give you the code with respect to that and this model is pretty huge that's why the response time is little bit High uh so you need to wait for some time so guys execution done now let me print the response I got so this is the Response but it's not readable so from this response I
only need to see the text uh from my model so let me execute see guys so uh so here I got the answer so here if you see um this is my system that means this is my prompt I have given to my model and this is the question user has asked and this is the response I got from my assistant so first of all it's telling hello I would be help uh happy to help you your Mist classification projects using kasas uh Before we uh begin may I ask what specific aspect of the projects
you need to assistant uh assistant with do do you any pyro experience with the Deep learning and caras additional uh so it's giving some uh output so what you can do you can again execute this code I think it will give you uh that because if you're using for the first time I think it will give you that uh response now let me execute for the second time so guys now you can see uh it has giving Me the code for the amist MS classification now let me give another uh prompt so here I'll just
write uh wrer um linear regression code wrer linear regression code let me let's see like whether it is able to give or not so again execute now guys I got the response and guys this is the code for linear regression okay so it's amazing guys so We are getting our output and uh like uh yeah so that's how actually we can use uh our LMA 2 model OKAY Lama model we can run and this is the Quant version of the model again U some accuracy might be uh less okay you you might get some wrong
output because of the Quant version of the model okay but again it's a pretty good uh you will get like all of the response okay whatever question you are asking now uh what I will do uh this was like how we can execute like qu PR version of the model because I was using uh because here I was using lamb to 13 billion parameter this model but next actually I'll will show you like how we can execute using Lang Chen okay how we can use using Lang Chen so there actually I'm going to use this
model called 7 billion so s 7 billion parameter this model actually I'm going to use okay and it this model has actually lots of variant okay I'll tell you so we'll be using some HF version of The model okay then I'll show you like how we can load this model so yes guys uh this is all of about this execution of this Lama 2 model on Google cab now let's explore like how we can use this Lama to model with the help of Lang chin so guys now let's see like how we can use Lama
2 model with the help of Lang chain framework and uh we have already seen like how we can execute this Lama 2 model on Google collab so there actually I used like one particular model uh so 13 billion parameter model but there actually I used something called quantized Model because I can't load this 30 billion model okay in my uh free Google collab I think I was already discussing about okay but uh in this uh example I'll be using this 7 billion parameters so this model would be a little bit uh uh small for me
to load in my memory so I can load this model okay in my fre Google collab there is no issue with that but but again uh to load This model uh you need to take the permission from meta AI okay uh this model is available on the hugging face website I will tell you like where where you can get this model so if you search on Google like Lama Facebook research so you will get this GitHub repository and there actually they have mentioned everything about Lama 2 and all and uh they will also give you
the download link and all so if you click on this download link so it will redirect to This page and here you just need to fill out this information okay so I think I already showed you this thing and they have already written like how we can access this model from the hugging face also see uh this model is also available on the hugging face website so you can visit hugging face and uh see uh it has this three variant uh like 7B 13B and 70b I think I was already discussing about and again if
you see here uh these are the model has lots of sub variant Like Lama 27b chat model Lama 213b HF model model okay so these are the like sub bant of the model so this is nothing but like I mean some of the model would be bigger some of the model would be smaller okay so that's how they have trained different different model with respect to these are the uh variant of the model okay now here if you see here in the GitHub they have written one thing this Lama to comes with two different model
one is like pretin model And another is like fine tune chat model so what is Preen model and what is this fine tune chat model so pretin model is nothing but these model are not fine tune for the chat or question answer they should be prompted so that uh expected answer is the natural continuation of the prompt so basically what they're telling actually with this pre-end model you can't do actually chat operation or you can't do actually question answering operation right like The way we usually perform with our llm model we ask any question we
do the chat operation with that so with this pretend model you can do let's see if you want to uh generate text or if you want to let's say generate uh something okay from the model you can use these kinds of PR model okay and what is this fine tune chat model so this F chat model we trained on dialogue applications so to get the expected features from uh and performance for them the specific format Define chat completion so basically what they're telling so with this fine tune chat model actually you can do the chat
operation let's say if you want to do any question answering if you want to perform in chat operation you can use this fine tune chat model so in this example I'll be using this fine tune chat model because I want to perform um chat operation with the llm okay I want to ask some queries and all so I'll be using this model but let's see if you Want to generate text and if you want to generate something so you can use this pretin model okay so now if you visit here in this metal Lama to
on hugging face so they have different different model as I already told you so it has chat model also and it has without chat model also so now you can choose any of them okay uh so guys from these are the model itself I'm I'll be using one particular model so I'm going to use this particular model called Lama 27b Chat H model so here is the model description and all about you can read it out okay so with this model actually you can do chat operation and all okay and this is 7 billion parameters
so I can easily load this model in my uh free Google collab okay so there is no issue with that I don't need to use any U you can say quantized model okay uh from them so uh now uh what I will do before uh see before loading this model you need to take the permission from The Meta AI so what you need to do here so you will see one uh like option here just apply for the permission uh see for me I have already applied for a permission and and they have already given
me the access that's why it's telling you have been granted to this this model okay uh so I already have the access to this model but if you're doing it for the first time make sure you have applied for the permission okay so once you get the permission so uh you will Get these kinds of message here if you see here uh you will get this kinds of email like you have granted to this model and all so they will mail you with respect to that and once you click here so you will uh redirect
to these kinds of form so here you just need to these are the form and submit your information so after some times actually they will give you the permission but if you're doing it right now and if you're waiting for The permission so what you can do in between I will show you another option okay another another alternative I will show you so that alternative you can follow and guys one particular thing you will always need to remember so whenever you are uh submitting the request okay whenever you are submitting the request for this model
make sure the email address you are using the same email address you are used in your hugging face as well okay so the same uh hugging Face email address okay you have created your account so same email address add you need to submit here okay as you can see you need to submit your email address here so make sure you have selected the correct email address with respect to your hugging F account so that you will get the quick access there so guys I already created one notebook for you so here I will show you
the demo how we can use this Lama 2 model with the help of Lang Chen and I think you Already know what is langen langen is nothing but it's a framework for developing application powered by language model or you can say we can also uh uh develop generative VI application with the help of langen and this is the langen documentation and G already given here you can go go and check and if you and if you don't know like Lang chain so we have a tutorial in our YouTube channel just go ahead and try to
uh finish that lecture otherwise Uh it would be a little bit difficult for you to understand like what I'm trying to say right so guys here I'm already expecting you are familiar with this langen framework uh like what is chain what is prom template and all about okay so this should be the prerequisite for this video now guys uh the first thing what I need to do I need to connect my notebook and make sure you have selected runtime at GPU because uh you need the GPU to run this model so change run type to
GPU so for me I have already selected and let me connect the notebook so guys uh notebook is connected now first of all let's see whether I got GPU or not so this is the command to check the GPU so here I got Tesla T4 and here I got 15gb vram now what I need to do guys uh I need to install some of the libraries see guys uh if you're using any open source large language model okay open source llm model which is available in Hugging pH so for this actually you need to install
this Transformer Library okay so using this Transformer Library we can actually uh access our hugging face model okay so that's why this Transformer Library we need to install then uh I need to also install these are the two libraries accelerate and uh enops then I I need also Lang chain because here I'm going to use langin okay and uh this uh bense bytes okay so this uh these are the packet actually I Need to install so these are the actually dependency so langen I need because I will be running these at the LM Model H
LM model with the help of Lin and I also need Transformer because I will be using my Lamar to model from my Hing fa itself okay that's why this Transformer is required now let me install uh so guys as you can see installation is done then uh first of all what I need to do I need to loging with the HF account okay so to loging With the HF account uh uh you need to use hugging face CLI to login okay so here you just need to give the uh API key okay hugging face API
key so there is a secret key actually you will get from your huging face account so let me open with my huging face account so guys this is my huging face account if you don't have any huging face account just try to create one account okay now if you click on the profile and go to the settings and here you will get one Option called access token okay just click on the access token so here previously I already created one access token here so I'll be creating a new one so you can also create a
new access token you can give any name so let's say I will give as Lama and uh you can get uh read and write permission uh I will be giving read permission because I only want to read the model from here then I'll just generate a token so this is my token guys now what I will do I'll just Copy this token and guys don't share this token with anyone okay otherwise they will also able to access your account so I'll remove this uh token after this recording okay that's why I'm showing now let me
execute this uh command and it will ask for the token so just click here and paste your token and just press enter so if you do it so now it will tell add token as G credential so what I will do I'll just give uh yes and press enter now guys here if you see My logging is successful now what I will do I will import uh then I need to import uh this HF pipeline from langin len. llms because with this uh HF pipeline so with the help of this HF pipeline class I will
be loading my large language model which is nothing but my Lama 2 okay so I need to import this Library then I also need to uh import tokenizer so what is tokenizer guys tokenizer is nothing but uh it is responsible for uh pre-processing the Text into Aras of number as uh input to the model because so whenever I will be giving any input to my model it would be completely a text input right but my model can't understand that uh text uh raw text actually I'll be giving so for for this actually what I need
to do I need to convert to numbers right right even I also need to perform some pre-processing so everything can be perform using this tokenizer okay autot tokenizer class that's why I need to Import it then I also importing uh Transformers then torch and this warnings okay now uh let's load our L 2 model from this huging face so as I already told you so you will get this kinds of access once you got the access and all okay so I have already attached some screenshot here and once it is done it will tell you
this message right now here guys you need to define the model actually you want to use so in this case actually I'm using this model as I Already told you I just copied this name and just paste it here if you see this is the complete name I have given and this is the model from meta because this is the official model actually I'm going to use and this model I need the permission I think I already told you this model I need the permission but let's say if it is taking for you to uh
take the permission and all so what you can do you can use this model you just comment this line and uncommon this line So this is the same model but this is this is present in the public repository and this access is given to everyone okay so you can easily use this model and this is the same same model guys okay this is the same version of the model so what this guy has done actually so he has cloned the official version of this LMA to 7B CH HF model and he has published in his
uh you can say uh repository here and this is completely and this is uh completely Public Access So you don't need to take any access uh okay from from this author so you can easily uh use this model so let's say if uh you have applied for the access and it is taking time and you want to do it right now with me so what you can do you can uncomment this line and you can load this model okay but for me I have already access to my official model so I'll be using official model
and I will comment it out okay so it's up to you which model you're going to use now guys First of all what I need to do I need to load my tokenizer so to load the tokenizer you need to give the same model name okay so with respect to that it will load the tokenizer now here uh what you need to do you need to create one pipeline objects and here you need to call this Transformer pipeline okay and here you need to uh give the pipeline name so here I have given Tex generation
because if I go to this model I'm using so this is a Tex generation Model as as you can see this is a tech generation model so that is why I have given Tex generation here now I I also need to give the model um like which model I'm going to use so this is the model I have given here okay then I'm also need to give the tokenizer so this is the tokenizer I have given and these are the parameters just keep it as default and this is the maximum input length so I've given
1,000 you can also increase and decrease the size okay it's Up to you now guys what I will do I'll just quickly uh load my pipeline so basically it will load the entire pipeline it will load your model it will load your tokenizer everything it will load as a pipeline so what is the use of this pipeline line means like first of all whenever you are giving any input to the model first of all it will apply the tokenizer and it will convert that text to numbers okay it will preprocess the text then it will
apply uh this data to The model okay so that's why this pipeline handles each and everything you don't need to write any custom code for it and now guys here if you see uh it is downloading the model okay and I'm using uh free collab and I got 15gb vram there so I think you can see uh RAM is uh is not utilized till now because it is downloading the mod so once it will load the model it will utilize my Ram so guys my model has loaded now what I need to do I need
to call this hugging Face pipeline I think you remember we load this hugging face pipeline from langin okay here if you see from langin LM we load this Hing face pipeline now I need to give this pipeline object to this uh huging face pipeline from my langin uh so here uh you will get to parameter called pipeline so here you just need to provide the pipeline okay this pipeline we have created and another uh keyword you need to give the temperature value so what is Temperature value so see uh I think you already know like
what is temperature value so here I've already written so temperature value like uh so if it is zero that means the model is very safe and it is not taking any Bits And if you are uh like uh giving this uh and if you're setting this value to close to one so it will be uh taking risk okay and it might generate wrong output but it is very creative so this is the parameter actually you can change uh Let's say if you want some stick output from your model okay you can set this parameter as
close to zero and if you want let's say very creative results and uh you are not woring about the wrong output or let's say random output so you can uh give this parameter as one okay so this is the parameter you need to give so here I want streak output from my model that's why I set this parameter as zero okay so this is the parameter I have given now here I'm defining as llm So this is my large language model okay but I think you remember initially we I was loading I was using something
called open large language model which is nothing but GPT 3.5 turbo you can use anything gpt3 GPT 4 anything you can use but there actually I I was using open API and it was taking charge for me right but this is completely free now see with the help of these kinds of Open Source large language model uh and it is also available inside hugging P we have Easily loaded this model okay and we have created our llm object here now this is one prompt actually I have prepared so what would be the good name for
a company that makes colorful socks so this let's say this is my uh prompt this is the user prompt this is the user input and it is giving this input to the llm and now let's see like what llm will predict okay so here I will uh give this prom to my llm and let me execute you can also set your custom template promp Template I will tell you like how to set it but first of all Let's uh give my prompt without any template because uh by default your LM will have some C like
you can say default template it will use that template so guys here is the response I got from my Lama 2 so I would suggest the following names for the company that makes colorful socks so Soul mats Hue and cry tall tally socks okay it's great actually even it is giving us the Description with respect to the name it is suggesting now you can give any kinds of prompt you can check it out so let's give I I uh another prompt so here I have given I want to open a restaurant for a Chinese food
suggest me a fence name for this okay so you can also give let's say Indian food okay I'll give Indian food now let's see whether it is giving me some name or not so I'll execute my Llm now guys as you can see it has given me the name so restaurant I would suggest the following uh names for your Italian uh sorry uh for your Indian for restaurant so tandur kns then spice R Mumbai bazer K Leaf okay so great I am getting very good name let's say if you want to open any Resturant so
you can refer these are the name and these are the name is like very unique okay and this is the description now uh also you can create your custom prompt let's say Uh see this is the better practice to create our custom prompt always uh let's say whatever task you want to perform uh with the help of your llm so you can give the custom prompt okay instead of using the default prompt so to give your custom prompt you need to import prompt template from Lang Chen first of all so let me import and I
also need to import something called llm chain okay from Lang chain now this is the first example I have given so this is my prom template One so here I I I've given the same prompt I I think you remember and uh I've just written I want to open a Resturant for uh here I've given kins cuin should be the input okay here I've given this should be the input variable could suggest me a fancy name for this now I'll be using this prom template and I will be giving to my llm model okay so
it won't be using that uh default prompt template after giving it now let's say how it will work so uh this is the Technique it will work so basically it will apply this format on top of it and whenever it will get this cuins actually it will replace this uh input okay the input I have given let's say I have given Indian now if I print this input prompt if you see I want to a resturent for the Indian food search me a fancy for this okay so this is the input I have prepared with
the help of this code right now let's take take the second example so this is the another prompt I Have created prompt two and here I'm using another prompt provide me a coze summary for this book and here I'm taking book name as input from the user so how it will work so whenever user will give an input let's say I have given an alchemist book book okay so it will take as input and it will make the complete prompt see it will make the complete prompt okay now let's use uh the first one first
of all so I'll be using this prom template one so here Just let me write now I'm going to use this prompt one the very first let's use prompt one and here I have created llm chain and here I have given my llm okay so llm model I have given so which is nothing but uh the llm we have created I think you remember this is my llm this is my llm okay and here I have given the llm and here I'm giving the prompt as my prompt template and here I'm also giving varos is
equal to true that means it will also Print the log like what is doing actually so if it is if you're making it false so you can't see the output log and if you are making it two so you can you can see the output log okay now here uh you need to call chain. run and here you just need to give the input okay so here let's say I'm running this prompt one and this is for the kin so here what I will give I'll give Indian kin Indian kin okay and this is the
response I will get now let me execute and show you see First of all it is uh uh entering to new uh llm chain uh prompt after formatting so this is the prompt it has prepared so it has taken my input Indian and this is the prompt actually it has created now it is applying LM on top of it and it will uh predict now let's see now guys uh chain has been finished and this is the response I got aurent name uh is is a crucial aspect of the any business okay and these are
the name actually it is suggesting now what I Will do uh I'll give this prompt template two that means my book uh book prompt and here now we'll ask any book name so let's say I'll give uh Harry Potter Harry Potter book I want the summary of Harry Potter book let's see whether it is able to give or not and this model is pretty good guys just try to explore this model uh I am loving this model A Lot okay um as I opens model I can say like this is one of the best model
you can use from meta AI so see guys uh this is giving me the summary again if you mention the book with a specific name let's say it is giving the summary of philosopher stone okay you can also give any uh specific book it will give you the uh summary in that summary it will give you okay so great guys we are able to use this lama2 model with the help of Lang chain and we have seen like how we can load this model with the help of Lang chin and all how we can create
the template okay how We can create the LM CH and everything we have seen right now guys it's time to make one projects with the help of this Lama 2 now what I will do I will uh create one uh chatbot projects with the help of this Lama 2 and Lang so basically uh we'll be creating one chatbot so that can chat with any kinds of website so let's say if you have any website okay so you can integrate that website to your chatbot so whatever question user will ask with respect to The website it
will give you each and every answer let's say uh this is one website Lama meta website okay so let's say I want to ask any questions okay from this website let's say I want to ask like what this uh website provides okay what are the services it provides uh give me the contact information and all so that chatbot will able to uh give me everything with respect to the uh website I have so these kinds of chatbot actually will be implementing with the Help of this Lama to and Lang chin okay and I will also
utilize like vector database uh with that uh projects okay because I have already explained like vector database so I already created one dedicated uh session on Vector database just go ahead and check it out if you haven't checked it out that session so guys we'll be using everything like vector database this Lama 2 and Lang Chen with the help of that actually we'll be implementing this entire Projects okay and this is going to be one very interesting projects okay you will be enjoying a lot so now let's start with the projects and let's see like
how we can develop these kinds of generative VI projects so guys now let's uh build one generative VI projects one uh like generate API application with the help of large language model so here I'm going to use uh lamp to and here I'm also going to use some of the additional tool let's say I'll be using Vector Database then I will be using Lang and framework okay then we'll be implementing uh this uh uh amazing application so the application actually I'm going to implement called uh website bot so what is website bot I think you
have probably seen like uh uh if you visit any kinds of website so some of them are already created some bot so if you ask some question in the website like uh what this website and all about like what the services is provide so Everything it will give you the answer right whenever we visit any website let's say we visit open.com so if I visit this website so if you want to see this website structure like how many pages actually it has uh and uh like what is the layout of this website what is what
the URL of that page actually everything you can see okay if you visit this uh link called sitemap Sitemap uh sitemap.xml okay if you just write this one sitemap.xml and if you press enter so it will give you all the details about this website let's say this is the website and it has actually different different pages if you see if I go to again open again if I go to openi so if I let's say click on research and click on let's say overview so it will open a different tab for me like let's say
this is the different page for this website So that's actually you have different different website and with with respect to that it has the different different links as well as you can see different different links actually it will also provide okay let's say uh this is another page okay this is another page inside open AI so let's say I'll copy this URL and if I paste it here so it will open that particular uh see it is opening particular that particular page okay so that's how actually you can Export uh with any kinds of website
uh so I already collected some of the website URL so what I will do uh I will use that URL and I will build this application so here if you see I have already prepared one notebook here and here I already uh collected some of the uh website link so if I open it so see this is one blog website link so paper review Lama to open foundation finding chat model so it has some content okay it has some content so We'll be chatting through this content like what are the things actually uh this content
is providing okay everything we can ask in our bot whenever we'll be complete completing this thing not only this thing I have uh attached multiple uh URL as you can see this is another link okay this is another link this is another website so again some content is inside that again it has different different pages as you can see it has different different pages Uh it has different different pages if you just visit that site uh map. XML you will see different different pages link you will see there then I also have another one so
this is the website of stability. and again actually uh it has some content so you can give any kinds of website link here so I have collected these are the website link so you can use any of them so here here is another website okay so we we'll be using this uh four website URL and we'll be Implementing one website bought on top of it so let's say if you're asking anything okay if you're asking anything regarding these are the four website like what are the content everything like what are the things actually it is
providing what is this uh let's say if you want to ask something about Lama to open foundation fine tune chat model you can ask so it will give you the answer with respect to the content actually it is giving let's if you want to ask like M PT 7B okay this is another uh open open source large language model if you want to ask something so here you can ask so it will give you the answer with respect to this content so any any website you can refer let's say if you have created your custom
website as well so you want uh to create one bot on top of it so let's say I am I'm the user so I will come to your website first of all I will ask to your Bot like what are Services actually provide what are the Things actually provide I can get all the information from your bot then uh I don't need to visit your website okay again and again so I can ask everything from the B itself so guys this is going to be one Amazing Project and uh what you feel like just try
to comment in the chat okay so uh whether it's helpful or not and make sure you are just watching this video entirely don't skip any part otherwise you might get some issue so guys now what I will do I will first of All give you the projects idea I'll be just uh giving you the projects architecture I'll be just writing the project architecture like what are the steps actually we are going to do uh throughout this entire projects like what are the component will be implementing okay so we'll be seeing the project architecture at the
very first then once you understand this architecture we'll be moving to the coding part so let's uh go to our Whiteboard and try to understand the project architecture so guys I'm inside my whiteboard so now first of all I will write just write the Project's name so it is uh website website bot using uh lama lama 2 okay so here let me give you the projects uh architecture like how we are going to do the projects so first of all what I will Do I need to uh collect those uh website uh uh sitemap.xml basically
I need to collect all the content okay from the website actually I'm referring so first of all we'll be uh taking all the content uh with the help of this sitemap sitemap.xml okay XML and you don't need to worry about uh manually you don't need to write these are the thing actually if you're using Lang chain so with the help of Lang Chen actually you can easily extract the content from any unstructured website okay I'll tell you like how to do it so what it will return return it will return return uh so it will
return you page okay page let's say one page two and page three so after getting the page actually uh I need to get the documents okay Present in the page so so it will return me the documents let's say docs okay docs one docs [Music] 2 docs 3 okay docs 4 and so on so docs 4 okay now it will return me that documentation like all the page uh content so once I get all the content okay uh basically here um let me write out here we are just Extracting extracting text okay excting text and
uh after getting all the text uh what I will do I will use my embedding API that means embedding model so in this case I'm going to use hugging face embedding embedding model I think you already know what is embedding model and all I already discussed this thing in my Vector database uh session so if you haven't uh watched that session you can Go ahead and watch it so everything would be clear there so I I'll be using embedding model so with the help of embedding model actually I'll try to convert this text to Vector
okay Vector representation now um this will return me the vectors so here I will get lots of vectors then once I get the vector uh I will build one build semantic semantic Index okay I think you will know what is cementing index uh cementing index means you will be uh combining all the vector together okay then we'll be building the semantic index and you will be storing the semantic index to the knowledge base okay so what is knowledge base knowledge Bas is nothing but your vector database so let's say this is my knowledge base okay
this is my uh knowledge base or let's say Vector DB Vector DB so these are the vector Will come here these are vector will come here and it will store inside my Vector DB okay so once I'm able to store these are the uh vectors okay in my Vector DB now uh what will happen let's say this is the user let's say this is the user so user will H some uh queries so let's say this is the question from the user so the first thing what I need to Do I need to convert this
questions to the uh vectors because again it's it's going to be the raw text okay so again I'll be applying something called embedding model embedding model okay then I'll convert this text to Vector so here I will get something called quy embedding or quy vector so this quy Vector I will apply Something called semantic semantic search I think you already know what is centic search okay so centic search on top of my vector DB okay so I'll do the semantic search so once you do the santic search it will uh return you some ranked result
okay rank results now I will have my rank results now I will take the help from my llm I'll be using large language model Which is nothing but my uh Lama 2 okay here I'm going to use LMA 2 so with the help of this large language model I will get my actual uh response okay the the query user has asked so it will give me the actual response okay so this is the complete idea and this is nothing but this is my user this is my user okay so this is the complete uh you
can say diagram of our projects as you can see first of all uh we'll be using these are the website With the help of sitemap.xml we'll be extracting all the pages available in the website then we'll be uh extracting ing the documents that means extracting the text after that uh we are just converting let me write here converting text to Vector vectors okay here we'll be converting text to vectors then we'll get our Vector embedding that Vector embedding we be building one centic Index and we'll be storing into the vector DB so in this case
actually I'm going to use something called Pine con Pine con Vector DB okay you can use any any Vector DB where at you can use chroma DB you can also use face okay from Facebook anything you can use but I will be using pine con then uh What uh will happen user will ask some query I will be applying again embedding model and I'll be getting quum beddings and with the help of this qu quum beddings I'll be do semantic SS on top of my Vector DV it will give me rank results and with the
help of llm I'll uh extract my correct output I'm I was expecting okay and it will give me the correct response uh to the user so this is the complete idea of this projects now what I will do and here I already uh prepared one notebook uh for this projects so I'll just explain like whatever things I have done so first of all I need to connect my notebook make sure you have Selected runtime as GPU so here uh just click change run type type and select GPU so for me I've already selected now here
I need to install some of the package so here I'll be installing langen then uh by byes accelerate Transformers why we need it I think you remember I think uh previous experiment I was showing how to use Lama with the help of um uh langin okay and I was reading that L Lama model from my hugging Fist and for hugging fist Actually I need to install these are the libraries then uh these are the actually dependency you can install okay then uh PF should be another dependency and also you need to install sentent Transformer now
let me install them so installation is done so so here I'm not going to use open AI so let me just delete this thing then I also need to install this library unstructured because uh the website will be loading so it has lots of unstructured format so To overcome this thing actually you need this package called unstructured so let me install it I'm not using F uh f is again a another Vector database so you can remove this thing I'll be using something called pine cone so this is done then I also need to install
tokenizers then X forers as well uh now I will be also installing uh Pine con client because here I'm going To use pine con Vector database and guys if you haven't checked my Vector database session you can check it out okay it is available otherwise uh you might get some confusion so it's better to try to understand this vctor DV okay then you will get the entire idea now uh all the things I have uh installed now I'll be installing all the requirements libraries actually I'm going to use and whatever libraries actually I'm going to
use I will tell you okay which Library Actually I'm using for which purpose actually I will tell you at that time so here you can see some additional Library also I'm importing but um again I will discuss like what are things actually I need here now I also need to install punctuation mantk H so the first thing I need to collect some of the website URL the website URL actually I want to use for my bot so here I already showed you I I'm using this four website okay I'm using this Four website you can
uh use any of the website and you can uh use it so these are the four website link I have provided inside one variable called URLs okay now let me execute yeah now I think you already saw uh we have imported one uh class called unstructured URL loader from Lang here if you see un structure URL loader so this uh class will help you to uh extract all the data from this other the URL okay the I think uh I was explaining This project architecture like it will be using something called sitemap.xml then it will
extract the pages okay from the pages it will extract the documentation so everything can be done with the help of this uh function actually we have imported here okay unstructured data loader and here you just need to give the URL and it is not only working with the single URL it will also work with the multiple URL as well okay so now if I execute and show you so It will easily extract the data and it will go through all the pages actually this website has and it will extract all the data for me see
uh it has extracted now if I print my data uh see guys all the data you can see here it is amazing right so that is the power of Lang chain so that's why we use this Lang chain framework to build these kinds of generative a application okay now if I show you my data length so it should be four because I'm using four Website so it will give you four website data okay here now uh this data should be in a document format I think if you see the format of this data it should
be a document format okay and uh now I need to apply something called uh text split on top of the uh entire Corpus I have because I need to convert my data to chunks okay why I need to convert my data to chunks because uh see I think I already discussed this thing in my Vector database uh session why this Chunks is important because whenever you open any kinds of model okay let's say if I open um let's say open a model here open a model open a or let's say if I open my Lama
2 model so here if you see uh it has context length as 4,000 uh 96 okay like it can take like 496 uh uh tokens okay at a time but if it is more than that actually at the time your uh model can take this and it will give you addor okay but the data actually we have Extracted it has like more than 4,096 uh you can say tokens okay so that's why I can't directly pass this data I can directly uh pass this uh Corpus to my model so that's why I need to convert
to chunks chunks means I will be converting uh different different chunks okay different different chunks mean different different paragraph okay and that paragraph I will be converting to like embedding Vector okay then I'll be giving that data to my model okay I Already discussed this thing in my Vector database session just try to go ahead and check it out out this should be very much clear now here uh I'll be converting my enre Corpus to the chunks so this is the character splitter uh function I have imported from lch here if you see character is
spitter from uh lch I'm importing now with the help of this one I will be creating the entire chunks now here you need to provide two things the Chun size and the chunks Overlap Chun size means like how many tokens you need in a chunks and chunks overlap like how many um tokens actually you want to consider from your previous chunks okay so these two parameter you give here now let me execute now my chunks object is C uh created now I'll give my data I'll give my entire data I got from my URL and
it will convert convert it to chunks okay now if you want to see the Chun size so it has created 90 chunks so If I print my text chunks okay so now let me show you see this is another chunks then again uh this is another chunks okay then again this is another chunks that's how it has created 90 chunks okay and if you want to see individually you can also see this is the Chun zero chunks one and if you just read it carefully you will see some of the uh tokens are coming okay
from your previous chunks as well okay so this is The Chun overlap so basically it helps us to understand the context of the sentence actually you have okay now we have successfully converted our uh entire Corpus to chunks now I'll be downloading the Hing Bing because here I was discussing I got my uh all the documents okay and I have converted to chunks okay if you see we have converted to chunks dox one docs 2 docs 3 and docs so on now I need to use H Model H embedding model to convert this chunks Into
embedding okay Vector embedding so that's why I'll be downloading first of all HF embedding so to download The Hang embedding I think I already imported this Library here uh here hugging F embedding I think somewhere yeah so from Lang Chain embeddings Hang embeddings so it is available inside langas Lang chain now here I'm importing okay now if I execute it will automatically download the default model so hang has lots of you can say Embedding model so this is the one of them okay so this is the name I have already given you can open it
up so this is the model for uh embedding okay so you can read about this model I have already downloaded this model and now let me see the model which model I got here so here if you see I got something called sentence Transformer all MP net base uh V2 this is the model I got okay this is the embedding model now if I let's say uh if I want to Convert any text to embeddings okay and I want to see like what kinds of vector and what is the length of the vector so I
can test it so let's say I have given hello world and from the embedding I'm just doing qu embedding now see guys the length should be 768 now this is the vector representation of of this sentence okay this is the vector representation of the sentence so you can give any here so let's say I'll give Hello or let's say how are you how are you and I have explained this thing detail in my Vector database session so that's why I'm again and again telling you just go ahead and try to check it out that video
it should be very much Clear see again I'm getting the same length like 6 768 and this is the uh how you Vector representation okay now what I will do I will initialize my uh Pine con U Vector DB so for this actually I Need to collect two things one is like Pine con API key and is like Pine con API environment so for this I need to visit first of all pine con so just write Pine con. iio and here make sure you have the account so I have already uh account so I'll just
log with my account so guys if you logging with your accounts you'll able to see these kinds of interface so first of all you need to get the API key just click on the API key and here is my API key I have Already created so what you can do um see this is my previous API key I will just copy this API key and if you don't have API key just create it here okay just click on create API key give the name and create the API key will get your API key then I'll
just paste this API key here so this is my API key and don't share this API key guys I'll remove this after this uh recording that's why I'm showing and I also need like Pine con uh API environment so for This actually you need to create one index just click on the indexes and create an index index and you need to give the name of the index so let's give uh Lama projects Lama or you need let's give Lama only give Lama okay and here you need to give the dimension of the vector so this
is the dimension actually uh so here I'm using hang embedding so hang embedding this is the dimension 7 68 so this Dimension to give it 768 and I will take this cosine metric okay so you have any other metric as well so I'll use this cosine metric and because I here I'm using free plan then I will create the index so it will take some time to ready your uh indexes so let's wait so once it is done it will give you this green icon now here uh you will get this environment name called gcp
starter okay I will copy this name and here I need to give it Okay done now let me execute so I have set my environment so API key and API enironment now I'll initialize my uh Pine con so here instead of giving inside your ways you can give it directly it is also fine so let me just give it directly so that's how I can provide now let me execute and now I'll will give this credential and I will just initialize my Pine con and here you need to give the index Name you have created
so this is my index name which is nothing but Lama I'll will copy and here I'll just give the name okay and this particular line will convert your uh entire uh like chunks to the embedding Vector embedding andore inside the pine cone see this is the code basically I have my text chunks and one by one I'm just reading the content and I'm using my embedding okay and I'm converting to vectors and index I'm Giving also like this is the index like this is the index I have created so in this index actually your vector
will store now let me execute and show you done now if I go to my Pine con and refresh the page you will see like uh all the vectors actually okay see uh this is my text with respect to that this is my vectors okay this is the vector representation and this is the index okay and now see we have successfully stored our U data We have successfully stored our vectors to the vector database okay and you can see all the vectors here okay you can also query out everything you can do and uh this
thing you can learn from my Vector DV tutorial so there actually I've showed you like how to use this Vector DV and all so here I'm not going to explain in depth okay because they already explained and this thing now see guys I already stored my vectors inside my Vector DV now what I will do I will Uh first of all loging with my uh uh logging with my this one uh my hanging face because I want to use that Lama 2 model from hanging face for this actually I need to login so let me
log in so if you want to log in you need this uh um access token I think you remember I was collecting the access token so this is my access token I'll just copy you can create your access token okay I I think I already showed you how to create it now here I just Need to give the token and login see logging successful now here you need to specify the model name which model you want to use I already told you I'll be using this Lama 27 B chat HF model so this is the
model name I have given here and you can uh use that alternative also like let's say if you don't have the permission you can uh use that model that open source model I think previous experiment I was showing okay because I already have the access So I'll be using this model now first of all I'll load my tokenizer and model I already discussed like what is tokenizer and model okay why loading because I want to create the pipeline now let me load it see it is downloading the model so it will take some time let's
wait so guys uh we have already uh downloaded the model now we need to create the pipeline I think I already showed you like what is Pipeline and all Okay why we need it so now let me initialize the pipeline okay so here if I show you my diagram so we have compl completed till this point actually we have stored the data now we are initializing our llm okay then we'll be handling the user now pipeline has been initialized now I will be calling this huging face Pipeline and inside that I will give my pipeline
objects and I will also give this uh argument called temperature I already told you what is Temperature now let me initialize my large language model okay now let's first of all test it whether it is initialized successfully or not so here I'm giving one prompt please provide a concise summary of the book of Harry Potter so it should give me the summary uh so guys as you can see we are uh getting the output that means my llm has been initialized and it is working fine now let's do some question answer with my uh llm
uh the data we have given Like the website we have given okay so from the website we can ask any kinds of question now first of all I need to import this retrieval keyway okay this function now here this is my query like I'm just asking how good this uh uh vuna okay so there is another uh website actually I think you remember this is all about vuna so I want to ask something about vuna okay from this uh website uh so it will give me the response okay respect to that now this Is my
question now this is my query uh if I perform something called similarity SE uh on top of my Vector DB so it will give you uh three different uh results as you can see I think I already discussed this thing like similarity sech how to perform and now let's say if you want to get the actual results okay leses with respect to your questions you can take the help from the llm so that's why so that's why you need to uh call this Retal QA and here you need to initialize the llm CH type and
retal you need to give your vector restore object here okay and as rer it will so basically it will refer your vector database and it will also take the query and it will understand the quy with respect to that it will also understand the um question you have asked okay the documents uh response you got then it will give you the authentic results okay now let me initialize this one okay yeah now I will Ask the same query like how good this vuna now if I ask this query to my llm see it will give
you the answer so guys as you can see it has given me the answer viona is a chat B that uh uses uh stable diffusion 2.1 generate the response while is quality is high okay it is giving this response now I'll ask another question like how does Lama 2 outperforms the other model let's see because if you see here I have another website of LMA 2 so this is the LMA 2 Okay so I'm asking uh the question with respect to that okay so this is the power of this llm and we can create this
kinds of generative application with with the help of this LMA to Pine con and langin see guys Lama to out performance model uh Benchmark okay so it is giving uh like GPT 4 p 2 okay uh basically yeah this is the response it is giving now let's another uh now let's ask another query like what is St stable LM okay I Think again uh if you see like I am also passing one question with respect to other website see here you can also add your custom website you can ask any kinds of question with respect
to that see this is the response I got now we can create one while loop um like ask the question repetitively one by one so now it will ask for the user input now here I will ask one query like what is uh MPT 7B so I think you remember we have one website of MPT 7B okay now let's ask one qu on top of it so yes guys as you can see I got the answer MPT 7B is a 7 billion parameter language model developed by llm foundary okay it is giving all the answer
now you can ask any kind of question you can give see again it is asking for the prompt so I won't be asking I just write exit so it will exit my code see here I have written the logic so yes guys uh I Hope you got it like how we can Implement these kinds of beautiful generative application with the help of Lama to Pine con and langen okay so this project actually you can also convert to uh stream lad app in future I will also show you how we can uh convert it to streamlit
app okay and whenever I will show you like end project implementation I will show you that part so yes guys I think uh you have liked this particular session and uh you have learned Everything about Lama 2 like open source large language model uh how to run it and how we can also implement this kind of gen application so if you have liked this particular video just try to share with with your friends and family and uh if you need anything from my side you can let me know in the comment okay so guys as
of now we have seen like various kinds of Open Source large language model we have seen like how to use LMA 2 how to use Google Pam 2 so in this video I'm going to discuss another U open source very powerful large language model which is nothing but Falcon so guys this is another open source large language models uh even Falcon has lots of variant like uh so recently Falcon has published this 180 B billion par met this model but it has some other variant as well here if you see Falcon has actually 1.3 billion
7.5 billion 40 billion and this is the recent one which is nothing but 180 billion parameter so Uh actually I can't load this 180 billion parameter with this Google collab and all so if you have good instance so you can load it process would be same but I will take this model guys uh 7.5 billion this model 7 7 billion model and I will show you one demo like how we can use this uh Falcon open source uh large language model to build our generative by application so first of all I will show you like
how we can uh uh integrate this Falcon 7B uh Model with our Lang chain so we'll be seeing like how we can perform inferencing on top of the model then I will also show you how we can create generative AI application on top of that so guys here if you see Falcon is nothing but it's a open source large language generative AI model and uh if you see here Falcon helps us to create a very Advanced application and it has actually lots of version like uh one 180 billion 40 billion and 7 billion and um
1.3 billion parameter and it is trained up with high quality like data set so that uh actually it can perform all kinds of job okay whatever you will be giving as a prompt so we'll see like how we can do it and it has actually these are the variant as I already told you 14b and even they have already given you the download guideline and all so uh to download this model actually I'll be using huging face because Hing F has already these kinds of model hosted here If you see this is the fcon 7B
instruct model so this is the 7 billion parameter model so I'm going to use this model so you can uh use any other model as well let's say if you want to use uh this 14 billion model okay just uh try to search here you will get the model so in this example I will show this model okay I will use this model and I will show you the demo so for this actually I have already prepared one notebook as you can see this is The Notebook I prepared so First of all let's see like
how we can perform the inference on top of this Falcon 7B model okay with the help of Lang chain so for this first of all uh make sure you have connected your uh notebook and you have selected uh GPU as a run time so I already selected so uh as of now whatever things we have learned like Lama then Google p all the steps are same only okay but only you just need to change the model there so this is the idea here so if you're using Any kinds of large language model not only Lama
Google pal or let's say Falcon if you're using any kinds of large language model I already showed you right like I already showed you like we have different different large language models so all the process will remain same you need to call the same method and all okay if you're using Lang chain and all and the way I imported my model here you can easily load it okay and you can refer some documentation because Whenever let's say you are referring any kinds of large language model they will have their documentation you can go through the
documentation you will get the idea and guys throughout this entire session actually I showed you three large language model like Lama 2 Google pom and this Falcon one and rest of the large language model you can explore all the process will remain same okay only you just need to do some exploration on top of it so guys here if you see I Already connected my notebook now if you want to check the GPU whether you got GPU or not this is the command to execute then you will see the GPU then first of all I
need to import some of the libraries because here I uh I'm using like hiding face model so for this actually I need to install Transformers and these are the libraries okay then I will be also using langen so I also need to install Lang Chen on top of it okay so let me install Them so guys as you can see installtion is done now I will be loading the model so guys to load the model if you're using Hing face model so first of all you need to call this Hing face pipeline from langen as
well as you also need to call the tokenizer okay and the pipeline objects now here is the name you need to Define so this is the name I've just copied from here and I pasted here okay this is the name and if you're using some other model you can change the name Here so it will automatically download the model first of all you need to define the tokenizer tokenizer will help you to convert your text to numbers okay and it will also do the pre-processing steps before feding the data to the model then uh I
will initialize in the pipeline so it's a Tex generation model that's why I have given the Tex generation it will take the model it will also take the tokenizer and these are the parameter you need to keep it as Default okay now if you execute this particular code so you will able to see like it will first of all download the model so guys this is the model size around 9.95 GB so let's wait it will take some time so guys as you can see my model has downloaded now I need to load this model
so I need to load this pipeline object actually so to load this pipeline object first of all I need to initialize this Hing face pipeline I I think I already Imported you saw that Hest pipeline from lch uh then here I need to give my Pipeline and here one parameter I need to set called temperature so temperature I think you already know what is temperature temperature means like if you're setting it to zero that means your model would be like uh very strict and uh it won't be taking any risk uh whenever it will give
give you the response but if you're setting it to one so it will uh it will take risk and it Will give you some random output that's why uh you need to think about like whether you need very creative results or random results from your model or not okay if if not then try to keep this value close to zero otherwise otherwise you can keep this value close to one so here I've have given zero because I want District output from my model so let me initialize my llm okay now here first of all I
need to initialize one promt template okay then I'm also initializing Uh llm CH okay so let me import them then this is my template custom template I have defined so you are an intelligent chatbot help the following questions with the brilliant answer so here you will get this kinds of question you need to return return these kinds of answer okay so this is the custom uh prompt I have created now this prompt I will initialize with the help of promp template it will take the template and it will also take the input variable so In
this case input variable is my question now let me initialize my template now first of all I will uh initialize my llm chain to initialize the LM chain first of all you need to give the prompt and as well as your uh llm okay then it will give you the llm Chen now this is my llm Chen so here is a question I'm giving explain what is artificial intelligence as nurser RS so uh it should give me that uh explanation okay so here I'm just writing llm chain. Run and here I'm giving my question now
let me execute and show you and if you see here guys my memory is getting full because I loaded this 7 billion parameter model so that is why I was telling like you can't load this uh 180 billion model because it's like very huge model for this you need good system you should have good GPU and good memory there but all the uh process would be remaining same okay only just need to change the model name there and Everything will remain same so guys here is the answer I got a child may be small but
with a AI can do all computers and machine learn faster and faster allowing the child to Great Heights so this is the response I got from my model and with respect to this uh nursary rims and you can ask any kinds of question like let's ask another question so I'll copy this one and here uh I can ask uh give me a code For adding two numbers so here if you see it it can also generate code so any kinds of prompt you can give any kinds of task you can uh do with this llm
let's say you want to do summarization generate text generate poem generate code anything you can perform with this model okay and is like very powerful model guys even uh you'll be loving a lot after learning this one okay I think you got the Idea now guys u in future I will also show you how we can fine tune this llm okay like we have learned Lama then Google pom then Falcon we can also F tune them with respect to our custom task let's say you have some complex data and this task uh your model can't
perform so at that time you can also F tune them so in future I will also show you like how we can f tune them on top of our custom task okay but guys all kinds of task actually these are the Model can perform so you don't need to even F tune them because fine tune is not easy so it like more costly so I'll always suggest just try to use this are the model okay these are the trained model so and you can perform your job here and you can also give the custom data
like I already showed you like how to put the custom data on top of that like how to perform these kinds of operation how we can uh do the information ret operation how we can Perform the summarization everything I have showed even in future I will also create some of the videos like we'll be implementing some projects and we'll see there like how we can create end to end Genera VI application with the help of these are the llm model now guys this is the inference on top of this Falcon 7B now let's uh create
one projects with the help of Lang chin so here I'll be using chroma DB and we'll be creating one M multidock retrial system so Basically here we'll be giving some documentation uh in the documentation we'll be doing the information retrial operation I think we already saw in our Google palom one okay there actually I was using PDF but here we'll be using the textual data so for this actually first of all I need to install these are the libraries so let me install them so I need to first of all terminate my previous instance done
now let me install Them so here I'm using chroma DV so you can use any kinds of vector database you can use pine con wave anything you can use and I'm also using langen so we are installing Lang chain Transformers we are also installing because we are using HF and we also need Cent trans Transformers okay so everything we need to install here so if you have already watched that Google Pam video so this uh notebook will be mostly familiar with you because I'm referring the same code only but only here I'm using Falcone model
and I'm also changing the database here okay only this part I'm changing so installation is done now I'll be importing these at the libraries so I'm importing chroma DB then recursive character splitter then retrial question text loader and directory loader so first of all I need to download the data so this data is available in this URL so I'm first of All downloading this Z file then I'm performing unziping operation on top of it see if I show you the data so guys this is the stock market data so it has lots of stock uh
uh detail like Nvidia stock then Tesla stock so these are the txt file data as you can see these are the stock data so on top of that we'll be performing this uh information retrieval operation with the help of this llm now first of all I need to load this data then I'll be extracting that Text so this is the code for it then uh if you want to see the documents like uh the final data you got so these are the data I got now what I need to do I need to perform this
text Splitter on top of it basically I want to convert my entire Corpus to chunks okay because I already told you what is Chunks and all so uh my model has like one particular input size if you see the documentation so if you're using this one Falcon 7 instruct so it has some particular U you can say Input size uh input token size so you need to give that particular size so it's better to uh convert to chunks so that uh these kinds of problem you w be facing okay whenever we'll be fitting the data
so let's convert to chunks so that many of chunks I got 49 now if you want to see some chunks so this is the code for it okay so see guys all codes are same as I already used in my previous experiment as well okay that's why I'm just going little bit fast Because I think you got the idea like how we are performing these are the operation right now here first of all I will initialize my file instruct model the same way I'll be using I showed you in that previous notebook right yeah so
it will first of all download the model from the Hang P so it will take some time let's wait so my model has downloaded now let's import my highing face Pipeline and Hing face embedding because see guys this Falcon model doesn't have any embedding model that's why I need to use highing face embedding and this is the model I'll be using for my embedding basically I need to convert my Tex chunks to Vector embedding so that's why I need this one now I'm initializing my llm and I'm also initializing my embedding model so see guys
it is first of all downloading the medic model do done now I will initialize my chroma DB database uh and I already Showed you like how to initialize chroma DB database in my Vector database session just go ahead and check it out so here is the DV name I'm giving and I'm also passing my text entire Corpus okay and I'm also giving my embedding embedding this is the embedding model as you can see uh so it should be embedding model so this is the name and I'm also uh giving my directory like this is my
DV directory now let me I Store the data in my uh chroma DV so it will convert all the text to vector embeddings and it will store inside your chroma now it has done now first of all I will load this uh so using this code actually we load this uh uh Vector emings okay so this code will load the vector Bings let me load so it should be HF HF is my embedding then here I'm initializing this retriever as my Vector DB then I'm Creating one uh qn R qn and here I'm giving my
llm now this is the process function I have just written so whatever response I will get from my llm so it will process the output and it will give me that one now let me initialize the qn uh now if you give any kind of query and execute this code you will able to see the response okay as you can see you will able to see the response I think I already showed you like how to to do it okay so that's how we can use this Falcon 7B um model okay with our custom task
as well like we can create this kinds of generative application now guys I want to give one particular uh task to everyone uh basically see you have learned lots of concept right now in the field of Genera VI like how to use these kinds of Open Source llm model now I have already shown you like three very powerful open source large language model now one thing actually you can explore from your side there is another Large language model called mral 7B so this is another very good model and this is the website of Mr 7B
and this model is also available inside hugging face so Mr 7B huging face so this is the model guys Mr 7B uh version 0 01 so this model you can U load inside your fre Google collab there is no issue with that so you can follow the same technique to load the model and you can perform these kinds of job here and also try to share this notebook in the comment section I Will happy to see that like whatever you have done from your site so it would be pretty much good learning for you right
so yes guys this is all about our open source large language model we have seen like how we can use them and how we can create generative application on top of it now uh in the next video what I will show you I will uh create one end to endend uh Genera VI projects okay with the help of this Lang chain then Vector database and these kinds of llm I'll Will be also using a streamlit package to create the web app and all so guys this is going to be very much interesting project just try
to uh Butch that particular video then everything would be cleared like as of now we have done the projects in the Google collab but uh now we'll be converting that Google collab to like uh end to end projects we'll be creating app and all I think you have heard of rag inside genbi right because this is the most used uh Actually topic inside gen VI so the full form of rag is like retrial augmented generation so in this video we'll try to understand what is rag exactly and how rag works and why rag is so
important okay let's say whenever we'll be working with genbi whenever we'll be implementing any kinds of let's say application so why we have to follow this rag concept there fine so guys I think you already know inside generative we use something called large language Model that means llms so LMS actually have some limitation you can see the first limitation it is having so the first limitation you can see llm can't answer correctly to the private data let's if we having any kinds of private data so that large language model won't be able to give you
the answer related that so if I give you one example let's say we are having one personal website let's say I have one website called code Commander okay so code Commander is my Website now if you open up your chat GPT and if you ask related code Commander your chat GPT won't be able to give the answer because chat GPT trained with older data that means I think CH jbt trained around 2021 to 2022 data okay around uh this year actually they trained this model after that actually I created this channel okay so this data
is not available inside my chat GPT chat GPT knowledge base that means that time actually this data was not available in The internet okay so you can consider it's a private data or you can consider any kinds of private data your private uh let's say website data private let's say your company data anything so if you ask any question related to the private data your model won't be giving you any kind of respon to that right then you can see the second limitation llm can't provide the most current information let's say if you're asking uh
any question which is like more current okay More current let's say information your model won't be giving you the answer let's say if you ask the question related this Olympic okay the current Olympic it won't be able to give the answer because it hasn't trained with the current information okay so these are the actually limitation we are having with the llms but I want to create an application that application will be also able to give the answer related to the private data as well as The current information so how it can be performed okay we
can do it with the help of rag concept that means retrial augmented generation now let's see how rag works so guys here you can see rag allows LMS to use external sources for the better response that means let's see if we having any private data let's say any external documents so what you can do you can give that documents to the large language model how for this you just need to use one embedding model you Just need to use one embedding model with the help of this embedding model you'll just try to generate the vector
embedding of the data the data actually will be giving and this Vector embedding you will be storing to the knowledge base that means to the vector database okay I think we have already learned this thing previously previously we already implemented these kinds of application but I think you didn't know okay this is a rag application I just Given some high level overview like uh how we can let's say use our external documents I think we already created some of the project right so yes this this is called actually rag that means we using our external
documents we are creating our knowledge base and to the knowledge base actually we are inting our large language model okay now whenever user is asking any kinds of question okay whenever user is asking any kind of question to the private data What it will do it will go to the knowledge base okay it will go to the knowledge base knowledge base will give the response okay the question you are asking the relevant question you are asking and your llm will process that answer as well as the query and it will give you the correct response
okay so this is the complete architecture of rag that means Ral augmented generation okay so with the help of this concept actually we can easily use our private Data sources or let's say external documents to get the information okay I hope it is clear that means if you're using rag concept it would be more accurate answer on the private data stay updated with the new information let's see if you're having new new information new data what you can do again you can uh give that data to the knowledge base that means you can uh use
embedding model you can convert uh those data to the embedding representation you can Store to the knowledge base and you can connect your large language model there and if you're asking any question related that information it will give you the answer okay your model will give you the answer so this is the concept of retrial augmented generation or rag so guys now let's try to see how rag works so as I already told you uh whenever you are using rag that means you are connecting your external data that means your private data let's say you
have Connected your private data let's say you have connected your website okay you have connected your website that means from the website itself it will extract all the information let's say we are running one e-commerce site so user has has asked one question what are your business hours okay now how it will give the response uh by using your information so first of all what it will do it will try to use something called retriever so with the help of retriever It will try to fetch all the information from your website okay all the information
it will fetch from the website itself then it will perform something called augmentation so what augmentation will do it will just try to augment the query that means whatever question user has asked okay and the data actually you are getting after retriever so it would be actually augmented that means it will uh add them together okay let's say the query as Well as the data then it will pass to the generation that means to the large language model large language model will try to understand the entire data as well as the quid user has asked
okay and based on that actually it will provide the answer so the answer is Monday to Friday from 9:30 a.m. to 6:30 p.m. okay so that's how actually rag works that means it will use actually three component one is like retrieval other is like augmentation and this like Generation okay that is why it is called retrial augmented generation okay I hope it is clear now so guys this was the introduction of the rag and I think you got it how rag works and all and why rag is important now let's try to see one practical
example of rag so guys here you can see I already prepared one code example uh of the rag demo so first of all we'll try to understand this uh Jupiter notebook file then I will also show you how we can create a user Application that means app let's say streamlit app how we can create it okay so for this what you have to do guys first of all you have to install these are the requirements and this code would be shared guys in the resources section from there you can download and I already told you
how to create virtual environment so to create the virtual environment you have to execute this command so cond create hpen in you have to give the name Let's say here I have given open demo you can give any kinds of name let's say I will give test then python is equal to 3.10 okay 3.10 then hypen y so this is the command you have to execute and make sure you using python 3.10 okay so for me I already created I'm not going to create again then after that you have to activate it that means cond
activate your environment name okay then you have install this requirements just WR pep install okay henr requirement. txt fine Now if I see press enter it will install all the requirements I'm having in the requirements file you can see inside requirements file I'm installing langen langen Community langen openi python. stream lead okay so these are the actually dependency packets you have to install one by one so for me I already installed each and everything you can see then after that just open this notebook rag demo notebook and try to select the environment so let's say
I Have created this environment open demo I'll select it now see now if I execute the first line see it is working fine there is no error okay that means it is working fine so guys in this experiment I'm going to use uh one website okay that means uh as a private data as a let's say external data we'll be using website so from the website itself we'll be collecting the data on top of that actually uh we'll be implementing this rag application so you can see this is The website URL now let me open
this URL so this is the URL of the website so this is the entire website guys you can see so they have given different different information so now I just collected some of the more page let me show you so this is another page this is the index page and this is the contact page okay now let me open them see this is the index page this is the contact page so you can see guys this is the index one And this is the contact one okay so that's how actually you can use any kinds
of website I'm using this website for you you can use your let's say uh personal website you can use your let's say private website your official website anything you can use even I will Al show you how we can use any other format let's say how we can use PDF documents okay these are the things I'll also tell you so this is the website data actually I'll be using now to load This website I'll be using something called unistructure URL loader and this is available inside langen okay you can see I'm importing this un unstructured
URL loader now this is the URL list so if you're having multiple URL you can create a list and this list you can pass to this uh function and this function will give you one loader object now you can call load loaded. load and it will extract the data let me show you so if I execute this Line now see it is extracting all the information from the website itself so extraction is done now if I show you the data now you can see it has extracted all the information now why three documents is coming
because here I was using three URL okay three different uh actually pays okay that's why it is coming three documents okay now what I have to do guys I think you remember we have to perform something called chunking operation so for this we can Use recursive character text splitter either we can use character text splitter either you can also use token text splitter okay so here I'm using recursive character text splitter so here Chun SI is equal to I have given 1,000 that means 1,000's word would be considered as my one chunks so let me
create the chunks and let me show you the uh chunks size okay you can see 11 chunks it has created okay now if you want to see the first one so you can see This is the first chance and this is the URL it is referring now I think you remember after extracting your data that means documents you have to use one embedding model and you have to generate Vector embedding and that Vector embedding will be stored into the vector database okay any kinds of vector database you can use so I have already showed you
like multiple Vector database like we can use chroma DB Pine con web okay F you can use any of the vector Database so here I'm going to use something called chroma Vector database okay chroma is a so chroma is a local Vector database fine so guys you can see I'm importing chroma from the Lang chain itself because inside Lang chain this chroma DB is available then I'm importing one embedding model so here I'm using open embedding model okay open embedding model you can also use any open source embedding model I'll also tell you okay how
we can do it then I'm Importing openi that means I'm going to use open large language model so let me import all of the package and now if I want to use openi what I have to do I have to set my environment okay environment variable so open API key you have to pass your API key so just try to create one API key and try to pass here then let's load this API key with help of ladb package then after that I'm going to initialize my chroma DB Vector database okay so you can see
I'm going To pass my documents here the documents I'm getting even I'm also passing my embedding model so it will give me the vector restore interre Vector restore that means this is going to be my knowledge base okay see this is my knowledge base right now now here you can perform something called centic SCE operation okay similarity SCE operation so let me show you one example so Vector store as retriever I want to perform similarity sarce operation sarce Keywords that means how many let's say result it will return so here I've given three that means
after searching it will give me three relevant answer okay let me show you so let me create the retri object now here I'm asking one question what kinds of service they provide okay this website let's say I'm asking one question now if I perform the invok operation see it will give you three results okay three results actually it will give You so let me show you see it has given you three relevant information and from where it is referring it has also given the URL and you can see the three relevant information but what I
need I need the actual response okay this question I have asked what what kinds of service they provide I need the actual response I don't need this kinds of response okay for this what I have to do I have to connect my large language model I think remember so here I have to Connect my large language model okay then user will ask the query so it will process the query as well as the relevant let's say answer will be getting from the vector database then your LM will give the XEL response okay so now let's
try to see how we can perform it now if you want to see the pce content okay the let's say result you are G getting if you want to see the contents what you can do you can call this one pce content okay I want to see The first let's say answer pH content so this is the entire Bas content you can see fine now here I'm initializing my large language model I think you know what is temperature parameter what is Max token parameter now here you can see I have created one chain okay create
retable chain so this will give me the chain object that means I can connect my large language model as well as the vector database okay then I'm also going to pass one prompt okay system prompt And for this I'm going to use uh prom template okay I think remember we we can also create the promp template so this is the prompt I have created guys you can see you are an assistant for the question answering task uh use the following piece of uh uh retri constuct to answer the question if you don't know uh the
answer say uh that you don't know use three sentence maximum to keep the answer concise you can also change the prompt it's up to you but I will use This prompt okay it will take the context and it will give you the answer now for this I'm going to create a promp template first of all so here I've created The Prompt template this is going to be my system prompt and user will give the input okay whatever input actually user will ask let's say this is the input user will ask okay this input actually it
will come here okay and it will create the complete prompt that time now let me initialize the prompt Now here I'm going to create the chain so I'm using Create stop document chain so this is the function actually I'm using because I already told you inide Lang chain actually we are having different different chains right now here I'm passing my LM as well as the prompt then again what you have to do you have to call another function create retable CH okay so inside that you have to give your vector database okay the vector database
you have created that Means your knowledge base and you have to give the this object your question answer CH okay this object actually you have to give now it will give you the entire rag genen now here you can ask any kinds of questions so here I'm calling this ragen invok and this is the input what kinds of service they provide and I'm only extracting the answer now see if I show you the answer right now see this should be the concise answer answer that means the accurate Answer actually it will try to provide now
see Victoria on move provides uh efficient and careful relocation services for the apartment Village householders okay see blah blah blah that means it is giving me the correct information right now okay so this is called actually rag system now it is referring my private data my external data on top of that actually it is giving me the answer and if you ask this question to the chat GPT CH GPT won't be Able to give the answer because CH GPT hasn't trained with this data okay I think getting my point now let's try to see how
we can create a user app okay so what I've done I've created another file called app.py and whatever code I have written okay s by sell I just copy pasted and I just created like that in this particular dot file you can see I've imported all the library I'm loading my let's say secret uh keys that means open API Keys then I'm giving the URL I'm extracting the data I'm creating that that means chunks okay then I'm creating the vector store I'm initializing my large language model then I'm using stream L you can see St
then I'm initializing my prompt so you can see this is the prompt and now if user is asking any quy this quy actually I'm passing here and it is giving me the response that response actually I'm showing on the stream L application okay now let me execute and let me show you How this app will look like so for this you can execute one command called stream lead run app.py okay now see if I execute the command so it will open up my application now see this is the title I think you saw that st.
title we have written uh here um see rag demo and this title will show in the search bar you can see this the search bar because here I'm taking chat input now so you can see this the chat Mode here it will show the message now you can change the message as per your requirement now let me ask one query so I'll ask the same query so I'll copy this quy and here I can ask you can go to the website and you can U ask any kinds of query it's up to you so guys
as you can see it has given me the response victorion move provides range of services including apartment moving Villa moving okay and so on that means It is working fine right now you can ask any kinds of question guys any kinds of question you can ask uh whatever actually information you are having in this website fine so yes guys this is the rag demo and that's how we can create a rag going forward we'll be also learning some Advanced rag we be let's say creating hybrid drag and all okay going forward we'll be learning these
are some Advanced rag application you can also create so in the next video What we'll do we'll just try to create another rag application okay with the help of Google jini pro model okay this is the open source model we'll be using with the help of that we can also create a rag application in this video we'll try to understand what is the difference between rag versus fine tuning because many people has this question when we have to perform the rag when we have to perform the fine tuning operation and what is the difference between
them so See both technique actually we used to extend the abilities of the large language model okay see when you have to use the fine tune approach when you have to use the rag approach let's say you are working in a very specific specific domain with some specific data okay and with some specific task that time actually you can use something called fine tune approach because whatever pre large language model you will be using it won be having like these are the Information let's say you are working in the field of medical okay medical domain
and they actually you are creating one bot okay you are creating one medical let's say information bot so this will give you let's say medical related information and here you are using very specific data okay so this data actually was not used during trading of this large language model so that time what you can do at the very first time you can do the fine tune operation that Means you can fine tune one pree model and you can create this uh application so let's say this is the application we created now user has given one
question what are the latest treatment for the type two diabetics so whenever you train this model so you provided actually these are the information so the latest treatment for the type two diabetes include meditation like like M forming then Lifestyle Changes such as diet exercise and in some cases actually Insulin therapy so these the information actually you have given ring fine tune these are the model now let's say there is a new treatment came now if you are asking the same question again to your let's say uh system it will give you the same answer
again and again because it is already trained with your older data yes or no so that new information actually it doesn't have so it won't be able to give the new new actually let's say technique for the let's say Treatment of diabetics 2 now in case what you can do you can use this new information and you can create a rag based application so what you have to do you have to to use the same llm model only same llm model only but you will be creating one additional knowledge base I think I showed you
how to create the rag so with the help of this data you'll be creating one additional knowledge base and with that knowledge base you'll be connecting your medical bot then your Medical bot will be able to give the response okay from the latest let say technique from the latest information yes or no okay now I think you got it when we have to use the rag technique when we have to use the find uning technique now see here we are using rag technique now if you're asking the same question now here you will see that
it will give you the recent treatment okay for the type two diabetics okay the recent treatment whatever actually it Came in the market okay now I think this is clear when we have to use the fine tuning technique when we have to use the rag Bas technique and again fine tuning is like more costly so for this you have you need huge amount of data you need a good computational machine that means you you should have GPU good configuration GPU good configuration Ram okay if you're not having these kinds of gpus so you can't do
the fine tuning operation that means uh in fine tuning You are retraining that model you are updating the weights right which is not involved inside rag so inside rag actually we are not training any kind of model instead of that we are creating an additional knowledge base and with the knowledge base actually we are connecting our large language model okay this is the difference only fine now I think guys you got the clearcut idea what is the difference between rag versus fine tuning and when we have to Use it so even I will also show
you how we can perform the fine tuning operation okay let's say if I want to create any specific task how we can perform the F tuning operation not of our custom data so guys as of now we have seen the rag demo like uh I already showed you what is Rag and what is the use of rag and why we have to use the rag okay even I also showed you what is the difference between Rag and fine tuning so in this video we'll be implementing one rag Based project so the project name is QA
system okay we'll be using here jini pro model and langen so if you don't know jini is a large language model by developed by Google okay so we'll be using this model and this model is actually free so we can use actually Google uh API to use this model okay I'll tell you how we can collect the API key and with help of that how we can use this large language model now see this is the entire diagram of our project That means here we'll be creating this rag application so we are having some external
documents so in this case actually I'm going to use PDF documents I'm having some PDF documents okay we'll be using this data so we'll be first of all extracting the documents then after that we will be part the chunking operation then we'll be building one santic index okay that means we'll be creating one knowledge base that means whatever let's say data we'll be getting We'll be trying to uh using embedding model to generate the vector embedding then we'll be storing this Vector embedding to the vector database then user will ask some query this query will
go to the knowledge Bas knowledge base will return some of the relevant answer and with the help of large language model we'll just try to process the answer as well as the question and it will give give you the correct answer okay so this is what actually we'll be Implementing here so let me show you how we can develop this particular application so I already prepared one example so let's open up so guys as you can see this is the demo uh so for this what you have to do again you have to create one
virtual environment I already showed you how to create the virtual environment for this you can use this command cond create typen in give the environment name then specify the python version let's Say 3.10 then hypen y okay this is the command to create the virtual environment after that just try to activate with help of cond activate command then install the requirement so keep install henr requirement. [Music] txt so for me it is already installed see that's why it is telling requirement is already satisfied so let me show you the requirements so these are the Requirement
you have to install guys so same requirements only as part your previous project the only thing I have added this langin Google J jni so the help of this package actually we'll be accessing the J model okay so here Vector database wise I'm going to use chroma DB again you can also use f fine cone okay weit any kinds of vector DV you can use here so let me open this first of all notebook and let me explain okay uh what are the things we'll be Doing here so guys you can see I'm having one
PDF document let me show you this PDF so this is the PDF guys so this is my resarch paper actually so I published this resarch paper so you can see I'm the author here so I'm the author here and the research paper is about development of multiple combined regression method for reain uh rainfall measurement okay you can read the paper like what are the things actually we have proposed here so we'll be asking Some question okay from this paper itself that means this is my private data this is my external data on top of this
data actually I'll be performing the chat operation so here I'll be asking some questions related uh this particular data okay now let's let me show you how we can do it for this first of all we have to load the PDF for this we can use I PDF loader from the langen now let me load the data so it will you have to give the path of Your let's say data so here is my path my pdf.pdf now see it will give you enter extracted documents that means you can see it is having around 15
pages okay 15 pages in the research paper if you open it up then what you have to do you have to perform the chunking guys I think you know what is chunking so your Chun size is equal to have given 1,000 that means 1,000 what would be considered as my one chunks now see if I execute here I will get around 44 chunks Okay now if I want to show you so this is my first document chunks now I have to get the Google API key so for this you can open this link so this
is available inside uh Google AI studio so just click here Google AI studio now from here you have to collect one API key now see uh it will give you this kinds of interface now you can create API key for me I already created I won't be creating again but for you you can create here See just select the project if you don't have project just try to create a projects and try to create the API Keys now see if I click here create API key in existing project it will create the API key okay
now after creating the API key you will get these kinds of API key okay this is your Google API key now just try to give the key Google API key is equal to past your key here make sure you are writing like that otherwise it w be working fine now let me save now if I Open up my notebook now guys what I have to do I think remember we have to use one embedding model OKAY embedding model just to convert my data to the vector embedding now for this we'll be using one open source
embedding model from the Google so you can see this is the embedding model embedding 001 and this is available inside this Google generate VI embedding okay nowart from that you are having so many embedding model let me show you let me open this link so you Can see you are having a ai2 AI Labs then LF Alpha okay so these are the actually provider so it is also having different different embedding model you can use them as well so let me load this embedding for this I'm going to uh import this Library I'm going to
load the API key then after that I'm loading this embedding model you can see then after loading the embedding I'm just doing one test operation I'm just writing hello world so it should give me The Vector representation let me show you so this hello word is represented with this vector and I'm only printing five actually Vector you can also print the complete Vector okay now let me create the knowledge base for this I'm going to use chroma I already told you inside that I'm going to pass my documents as well as the embedding model you
can see embedding model now it will give me the knowledge base then again I can perform similarity Source operation That mean stic Source operation for this you can execute this line of code so it will give me actually 10 relevant answer and this is the question I'm asking what is the new in development of multiple combined regression methods in rainall measurement paper this is my question now see if I execute it will give me 10 different response now if I want to show you so 10 different relevant response actually it will give me now if
you want to see the pce content you can also see That see this is the pce content okay now I will be integrating my large language model so for this let's initialize my jini model you can see so here you can see I'm initializing my jini model for this I'm using chat Google genbi inside that you have to give the model name jini 1.5 Pro this is the model so J having different different model you can search J Min model list you will get different different model list so from there Actually I'm using 1.5 pro
model and temperature max token parameter I think you already know okay I already explained this part now again I have to create the chain I think you remember the same chain actually we are creating the same we have created our previous project as well after that we are preparing the prompt okay this is the same prompt I'm using this is my prompt template right now so let me execute then here I'm creating the complete Chain I'm giving my llm object as well as the promp and this one I'm passing to my create Ral CH okay
you can see I'm giving my Vector database as well as my this object now this will give me the complete CH now here I'm asking the question again the same question what is the new development of multiple combined regression methods for the rfor measurement paper now see if I ask the question it should give me the answer from the paper so guys here you can see I got the response the paper of development multiple combined regression methods for rfor measurement introduce a Noel approach to predicting rainall uh quantity the author utiliz 10 supervised regation machine
learning models to predict the rful based on the historical data historical weather data their goal is to identify the most accurate regression techniques for the rful uh prediction see uh it is giving me the correct response now if you see the Paper now if you read the paper you will see that we have proposed the same thing here that means it is working fine okay that's how also you can use different different data sources uh that means if you're having any private data okay if you're having any custom data you can create the rag based
application like that so guys uh this was The Notebook experiment now let me show you how we can create a uh user app so for this I'm going to Again use streamlit and whatever code I have written here I just copy pasted inside this app dop you can see all the import operation I'm loading then uh my secret key that means my uh Google API key okay here I'm loading it then I'm loading my documents performing the chunking operation creating the vector store initializing the large language model this is my prompt and this is the
uh chat input I'm taking that means I'm creating one chat input box with the Help of stream lit so this is my prompt then after that if uh user has given any query so it will uh do the generation and it will show the it will show the on top of my stream L application now let me show you how it will works so for this you can run this command stream lit run f.p so guys you can see this is the title rag application using J Pro you can also change it here okay anything
You can give here now here I got the input box now I can ask any question let's say I will ask the same question you can ask any question from my paper just go through the paper and ask any kinds of question so here we're using J model uh through the API request and it is like free API okay you don't need to pay for anything now see guys this is the answer I got now if you see the answer it's Pretty good okay that means the correct information it has given me so yes guys
this is the project actually the Q project we have developed with the help of J Pro length chain and Vector database uh so far we have learned about like rag Ral augmented generation and there I showed you what is the use of rag and how we can create a rag okay even I already uh I think discussed what is the difference between rag versus fine tuning that means if you want to Work in a domain which is like completely new domain and you are working with very specific task that time what you can do instead
of implementing the rag you can f tune one LM okay and to find tune the LM what you have to do guys you have to use one Preen model on top of prean model you have to train your custom data so let me open up my Blackboard and let me discuss there what is fine tuning exactly so guys inside fine Tuning uh you will be using one pre-tin model OKAY pre-rain large language model okay I think I already showed you there are so many large language models are available as open source for this you can
go to the hugging pH so here you can see there are thousands of large language models are available now you can choose any kinds of model as for your requirement let's say you want to use something called Llama meta llama you can simply search Llama here so you'll get see all the Llama model OKAY llama 2 llama 3 all kinds of model actually will see that see okay and this is from meta AI now what you can do you can download this model on top of that you can fine tune your custom data but before
fine tuning uh what you can do see first of all let's say whatever let's say project you want to implement let's see you want to implement one chat board let's say you Want to implement one medical chat board okay medical chat board first of all use one pre-end large language model and try to perform the inference operation okay try to perform the inference operation inference on top of the model if this model is able to give the correct answer okay if it is able to give the correct answer try to select this model okay otherwise
what you can do just try to build one rag rable Augmented generation okay that means create a external knowledge base exter null knowledge base then try to connect your one pre-end large language model and try to again perform the inference operation okay and try to see the accuracy of the metal try to see like whether your model is giving able to give the correct response or not if it is giving try to select this model okay otherwise if this boat approach is Not working that time what you can do you can perform the fine tuning
operation okay fine tuning operation so there what you can do you can take one pre-end large language model from hugging pH from any kinds of let's say platform and on top of that you can train your custom data the data set you are using let's say you are using one very specific medical okay medical data that time you can use this data and you can train one Custom model on to of it okay this is called actually fine tuning and that should be my suggestion guys I mean directly don't do the fine tuning before that
try to check whether pre0 model is working or not if Preen is not working try to build the rag B system if it is not working then try to do the fine tuning okay because fine tuning is a more costly task at the end because here you need like huge amount of data even you need lots of computational power you Need you need actually higher configuration machine okay you should have good GPU instance in your system otherwise you can't do the fine tuning operation but if you want to do the fine tuning operation there is
some way actually we can do it because all kinds of large language model you can see all kinds of large language model you can see it is having actually huge parameter size you can see Lama 2 7B that means it is Having 7 billion parameter it is having 8 billion parameter that's how it is having 70 billion parameter just try to consider okay it's a huge huge size model so we can't use any kinds of low configuration machine here to fine tune these are the model so for this what we can do we can use
something called PF Technique we can use something called p e f t okay what is the full form of PFT parameter efficient fine tuning okay this is the full form of PFT so inside This PFT we are having some method we can use uh so the first one you can use something called l or you can use something called Q okay so this L and Q Lo will help you to do to perform this parameter efficient F tuning that means uh it apply something called quantization technique quantization technique okay that means it will load your
model in a lower memory instance that means it will load Your model in a 4bit Precision 8bit Precision okay that's how actually it will perform the parameter efficient fine tuning process now you can ask me what is quantization then okay let me show you what is quantization see quantization is nothing but quantization it just a way of converting from higher format uh sorry higher memory format to a lower memory format so this Is called actually quantization now I think you already know about neural network right let's say this is one neural network I'm having so
inside neural network what what we have we have lots of Weights yes or no right we have W 1 W2 okay and W3 and so on so this is this is what actually we are having that means we are having lots of weight weight and biases let's say B1 B2 and so on right now you can represent these at the weight as a Matrix because if you if you know about artificial neural network so there I think you know how we can represent these are the weights okay how we can represent our input data and
all okay let's say this is one 3 cross 3 Matrix I'm having and inside that we're having having lots of Weights let's say W1 W2 W3 and so on okay this is the weights and this weight is nothing but it's a floating number okay it's a floating number it's A floating value if you see any kinds of weight it would be like that let's say 7.23 like that okay so this is called actually floating value these are the floating value and if you see this floating value stores in a FP 302 bit okay ap32 bits
now what is the full form of AP AP means full Precision okay full prec okay full Precision you can also call it as single Precision okay single Precision got it so whenever we are doing the quantization that means what we are doing we are just trying to convert this higher memory representation to lower memory representation that means it is now saving these are the weights in the FP 32 bits now what we'll be doing after doing the ization we'll be converting this 32 bits to let's say 8 Bits okay 8 Bits or let's say four
bits four bits that means what will Happen let's say you are having one floating number like that let's say 1.70 8 9 0 now if you want to convert it to 8 Bits what you will be doing you will just remove these are the numbers actually these are the numbers actually will remove and you will take this this number only so that's how actually you can reduce the memory size that means you can convert any kinds of higher higher memory higher memory actually format to lower memory format so this is Called actually quantization okay I
hope you it is cleared and the technique actually we usually use let's say Loa and Q Loa okay Q Lo so this techniques help us to perform this quantization that means it will it will load the original model as a quantized model that means in a lower pre okay let's say I want to load this model in a 4bit Precision 4bit Precision okay so let's say previously this model was In 32 bit Precision but I want to load in 4bit Precision so how it will happen with the help of Q Lo or L you can
do it sorry with the help of L and Q L you can perform it we are having already some python libraries okay so we are having one Library called PFT okay this called parameter efficient find training so with the help of that you can perform Lura and QA technique so with the help of that you can load any kinds of actual model so these are the Model okay in a 4bit Precision in 8 bit Precision okay that means you can load as a quantized model so that you can perform the fine tuning operation because as
I already told you this actual model you can't load because it is having huge amount of size huge amount of let's say weight size okay and it is uh storing those are the weights in a 32-bit precision and to load this at the model we need a good configuration machine good configuration GPU memory and so on but we don't have that much of resources we are having let's say Google collab free Google collab or let's say you are having 30 60 GPU that time what you can do that time you have to load these are
the model with the help of this these are the technique actually the technique actually I showed you uh this one quantization that means you will be using L and Q Lo with the help of that you will be loading the model and we'll Perform the finding operation now you can ask me sir if I'm doing that one so definitely I will lose some information definitely will lose some information because you are loading the model in a 8bit and 4bit precision from the 32bits that means here you are getting rid of some of the numbers so
definitely you are losing some information but again L and K takes care this part actually so they try to optimize your let's say model weights as Much uh they can okay so you don't have to worry about if you're using these at the library everything would be happen in a optimized way so we'll be learning as a practical also how we can find un one large language model so in the next video we'll be learning how we can find tune one model called Lama to metal Lama 2 okay we'll be fine tuning this model on
top of our custom data and here we'll be using this PFT technique parameter efficient fine tuning technique got it Because uh I'll be using free collab here if you want to do any your local machine if you are having let's say any basic zpu you can also do it but I'll try to suggest try to train on google collab if if you're not having any kinds of GPU okay and not having any kinds of conf good configuration machine you can perform on Google collab fine so yes guys this is uh the introduction of fine tuning
and I think you got to know how we can perform from the fine tuning Operation and what is this PFT okay PFT technique exactly and what is L and K because I know that many people will have this question what is Kora laa and what is PFT and all okay so I think now everything is clear okay how things are working here fine so in the next video we'll see the Practical demo of the fine tuning so we'll be fine tuning one model large language model called metal L 2 so here we'll be fine tuning
one large language model called meta Lama 2 I Think you know Lama 2 right Lama 2 is a large language model so if you go to the hugging face and if you search for meta if you search for Meta Meta is having different different large language model if I show you let's say if I go to the meta LMC it is having different different large language model like meta Lama 3.1 llama 2 is also there if I expand all all the things see this is the Lama 3 family this is the Lama 2 family so
the way actually I'm doing the Let's say fine tuning of llama 2 family you can also perform llama 3 as well why I'm taking Lama 2 because it would be a little bit uh I mean small size model that's why I'm taking Lama 2 you can also take Lama 3 as well okay the process will remain same and how to get the permission from the Lama I think I already showed you in my open source large language model session so you have to apply for the permission first of all and they will give the permission
then After that you can load it and if you're not getting the permission as of now what you can do I can also show you one alternative approach you can also follow this one fine I think I already showed you one alternative approach like you can use any other repository to load the model it is also fine then what you can do after fine tuning this model you can also push this model to the hugging is Hub let's say if I show you my Hub now so if I go to my profile see I already
Published one model this fine tune Lama model after fine tuning it I publish this model in my um hugging face Hub now people can use my model so they will be just copying this name and they can use my model you can see it is already downloaded uh 65 times okay 65 times this model got downloaded by the audience right so that's how actually you can also publish your model if you want you can also publish your model to the hugging P sub got it so that people Can also use your model got it so
now let me show you how we can perform the fine tuning for this I have prepared one beautiful notebook so in this notebook actually I have written each and every step you want to follow so first of all make sure you selected runtime as GPU because you are doing the finding operation and definitely you need a GPU so I'll be selecting T4 GPU because here I'm using free Google collab you can also take the premium Subscription of the Google collab if you're taking the premium one so you'll see that your training would be a little
bit faster than this free one fine now let me connect this notebook so guys as you can see my notebook is connected now the first thing what you have to do you have to install some of the required package so here I have listed down all the package actually you need like let's say accelerate then PFT PFT is the parameter Efficient F tuning that package accelerate you need if you're using Transformer Library uh and if you want to let's say use the GPU that time actually you need to install this accelerate as well fine then
you can see bits and bytes is also requirement of the Transformer and if you want to set up everything in your local machine that means if you're having local GPU but if you don't know how to set up the GPU don't need to worry I have created one Dedicated video Let Me Show for you so if you search my YouTube channel DS with buy so here you will see that I have created one video in the video section set up Nvidia GPU for the Deep learning so here I showed you how you can install the
Cuda and Cuda toolkit so definitely first of all you have to set up your GPU in a proper way then you'll be able to set up everything that mean you will be able to utilize your GPU so make sure if you're doing local setup you can try to Watch this video okay otherwise if you're doing on Google cab try to do the same thing the way actually I'm doing here so let me install all the packet so guys as you can see installation is completed now the second thing you have to import all the necessary
Library so you can see I have imported all the necessary Library like Auto model for casual LM Auto tokenizer okay I think you already know these are the thing uh the hugging face actually Library okay Why we need it then you can see I'm importing something called PFT that means parameter efficient fine tuning so from PFT I'm importing something called L you can also use Q Lo but here I'm using L okay so both Technique we can use it's up to you so let me import as of now so whenever I will be using it
I'll tell you okay what are the package actually uh you are using and for which task so see guys execution is completed that means everything is working fine Now let me show you the data set first of all see this is the original data so let me open the data see this is the original data so the data set name is open assistant so the data set name is open Assistant goano I think this is the data set name and you can see this is the data that means it is having some conversation between human
and assistant you can see so conversation between human and assistant you can see human and assistant so this is the human uh Let's say conversation and this is the assistant conversation okay so that's why actually they have collected this data and it is having actually you can see uh how many example in the training you are having 9. 85 K rows F almost 10K example you are having now see it is also using different different language see different different language actually they have collected the data not only English it is also uh they're using see
Chinese language also they're Using different different language that means it is a multilanguage data and it is a conversation data so we'll be training one uh chat model that means llama to chat model okay I think I already showed you what is the difference between Lama to chat model and find un model let's say if you want to only perform the T generation that time you can use pretin model okay and if you want to perform let's say chat operation chat operation and let's say All kinds of task the way actually perform with the chat
GPT that time you can use chat model now if you open the hugging face and if you go to the actually llama llama model you'll see that it is having different different model let me show you so if I search for llama see if I go to the uh llama 2 Series actually Lama 2 family you can see it is having Lama 2 7B model HF model and it is also having something called chat model you can see uh See Chat model chat model means you can perform the conversation chat operation the way actually perform
with the chat jpt right so we'll be fine tuning one chat model so that is why I'm using this chat data set conversation data set okay instruction so these are the instruction um I think this one see this is the instruction human is giving assistant is replying so this kinds of data set actually we'll be using now you can use any kinds of data you just try to Prepare your data okay in that format you can also create a CSV file so inside CSV file uh in First Column you can write the human let's say
conversation in second column you can write the assistant conversation that's how also you can collect the data okay it can be any kinds of format so as of now we are using the data from the hugging F you can also use your custom data and how to upload your custom data to the hug f for this you can see hugging F documentation They will also give you the let's say entire process like how I can let's say publish my data to the hugging face up and from there I can load my data it is also
possible but most of the data actually will see from the hugging face itself you can use it from there so see this is the Lama to uh supported data that means whenever you will perform the fine uning now so you have to convert this data in that format see this is the format so first of all you have to give The uh s token you can see s token inside that you have to give the inst instruction and what is the instruction instruction is nothing but it's a human conversation whatever conversation human is doing now
this is called instruction see if I open this example see this is the human see this is the human conversation okay see till this token actually that means slash NST that means instruction this is the human conversation after that whatever text You can see this is the assistant conversation okay this is the assistant conversation got it so that's actually you have to prep your data and at the last your s token will end again okay that means this is the complete system token right now the complete prompt that means you are creating the prompt here
because my large language model supports prompting right so that that means we are creating the prompt here so that my model can understand whenever we are Doing the inference whenever we are let's say sending any kinds of prompt to the large language model it can automatically understand we sending the prompt okay so that's why I'm first of teaching my model this is instruction and after instruction whatever actually message you can see this is the is the your output that means your response assistant response okay this is the format now if you open any kinds of
data you'll see that it is using the same Format only okay same format only so we have to convert our data this format to this format okay now you can ask me how to convert this format to this format for this we can use one notebook see this is The Notebook guys even I already attached this notebook link inside uh This original notebook let me show you see this is The Notebook okay so in this notebook actually I already WR one function transform conversion so this one actually will take the human Conversation as well as
the assistant conversation and will add the other token okay like that the way actually I showed you here okay I think it is clear now you can use this function as it is and you can convert any kinds of let's say these kinds of conversation data to um uh your uh like llm format that means your Lama to format now there are so many large language model it takes actually different different kinds of format if you go to the documentation of The hugging face you will see that what kinds of format they supports based on
that actually you can U actually write the converter also you can get this converter from the hugging face itself hugging face also provide these kinds of converter you can use this converter to convert your data to a specific format okay so most of the large language model will follow this format only okay you'll see that most of the large language model will follow the same format the Format actually I'm using here fine so that's why you can use this converter as it is now see here I'm loading the data that means my entire data that
means this data this uh open Assistant data you can see I'm giving the name of the open Assistant then see instead of taking all the data all the let's say 10K example I'm only taking 1,000 okay 1K actually example because here I'm using free collab and finding will take time that's why I'm only taking 1,000 Example to perform the finding operation and I'm doing the let's say conversion operation then after that I'm just pushing this uh let's a data here so so this is the data now see the same data but here actually this guy
is taking actually 1K example and he has already converted this data to the required format okay okay so this is uh so this data is already available you can see this guy has already converted this data to this format so directly you can use This data but if you want to do it so this is The Notebook you can use okay the same notebook you can use got it and if you want to use the entire data guys what you can do so here I already given the complete reform data set link you can open
it up so see it is having actually 10K example only see around 9.85 K rows now see it is already uh formated okay it is already formated you can also use this data if you using good configuration PC you can use this data Okay for the fing operation but I'm going to use this data actually 1K example data okay only 1K example I'll be taking here so guys I think now you got to know how how we can reformat our data for the Llama 2 model and see here we are using free collab and free
collab actually offers 15gb of graphics card if you see here it will give you 15 GB of graphics card see okay and here you will get uh uh dis around 11 12 GB and RAM actually give 12gb okay so this is the Configuration of our fre cab now here you can see uh it is having actually limit resources barely enough to store llama to 7B vs that means we can only use 7 7 billion Lama to model OKAY 7 billion Lama to model if you're using 70 B 30 B 13 billion that time actually you
will get difficulties okay because it is a free Google collab but if you're using um premium Google collab that time actually you can also use this model okay it's completely fine now here you Can see full fine tuning is not possible here we need to perform the parameter efficient fine tuning technique like we be using L and Q because the full model we can't load here we have to use quantization technique that means we'll be using L and Q to perform the parameter efficient fine tuning got it and to drastically reduce the vram we must
fine tune the model in 4bit position which is why we'll be using uh sorry we'll be using Lo here not Q loer We'll be using Lo here fine that means we'll be loading our model in a 4bit position I think I already explain here what is 4bit precision right so initially your weight would be in the full Precision okay 32 bit Precision this is called Full precision and we'll be loading the model in the for precision with the help of Lura technique fine I think it is clear now now let's load the model guys now
you can load the official L 2 model you can See you can also load the official Lama 2 model so Lama 2 is having also official model so this one so see this is the official Lama to model you can load you can copy and you can just paste it here but if you if you didn't get the permission what you can do you can also load the model from any other repository so see this repository has having this model also this lama lama 27b chat HF model so this repository has already published and here
you don't need to Take the permission you can directly download the model so if you if you haven't get the permission you can use this model either you can use the official model OKAY official model let me also copy here and let me mention it here let's say I can comment here just for your reference you can copy the name and you can paste it here okay later on now this is the data set guys I'll be using I think you remember so this onek example will be using so copy the name And try to
mention it here okay and after fine tuning what is the name you want to assign I want to assign this name Lama 2 7 B CH find you and these are the lower parameter you can see and low parameter wise everything just keep it default because you can see I'm loading everything in a 4bit Precision okay 4bit Precision so no need to change anything keep it everything as default see I won't be changing anything only the change you have to do the epoc size Number of epoch you want to train so I think Epoch I
have mentioned here somewhere see number of epoch training epoch so as of now I'm showing you one one let's say Epoch training because one Epoch training will take time if you're using free collab and if you're doing the actual training that time just try to increase the EO size now everything keep it default no need to change anything now let me initialize all the parameter so this is my lower parameter Guys for the parameter efficient fine tuning fine now we'll be loading the data set I want to use the training data set because I want
to do the fine tuning and here I'm loading everything in a 4bit position you can see okay with the help of bits and bytes actually configuration we are loading everything a 4bit position so this is the way actually we can Define our L and how I got this code if you go to the hugging pH documentation you will see that okay They have already given these kinds of codes in a bit so you just copy paste and try to use it as it is no need to change anything fine now see here I'm loading the
model in a 4bit position so here you can see I'm loading the base model as well as the tokenizer okay tokenizer I'm also loading so guys here you can see I'm setting all the loader configuration after that here is the training arguments I think you know what is training arguments uh in the hugging PH you have to set your directory the number of epoch your Optimizer everything you can set here then with the help of sft trainer I'm initializing the training process you can see model. train now see if I execute this code it will
start the training but see training will take time actually around 30 minutes actually it will take I was training previously I know that it will take around 30 minute okay 30 minute you have to wait because here we are using Free Google collab so as you can see it is downloading the model after that you will see that it will start the training see it's downloading the model so see I won't be waiting 30 minutes what I'll show you uh the next process actually you can follow like after training what you can do because see
I have written all the code only you just need to execute if you're doing it because uh here I'm recording currently so I can't wait actually 30 minutes um to train the Model let me show you what you can perform here so after training the model you have to save the model you can see you have to save the model trainer model save pretend then you have to give the new model name I think new model name I think you remember we have given a new model name here whenever we Define the all the configuration
um this is the new model okay so it will save with the help of this model fine so it will save the Model in the here in this dis actually you'll see that it will create output folder inside that it will save the model then after that you can also visualize the tensorboard logs see this tensorboard will show you the losses then the accuracy okay of your model everything it actually it will show you that that means the complete metrics it will show you the training losses validation losses everything it will show here then after
that what I was Doing I'm I was loading the original model you can see I'm loading the original model not the train model I have train I'm loading the original of the Lama model and then I'm testing with a prompt what is large language model I just wanted to see what would be the output from my original large language model because that means I want to compar this result with my train model the model I have trained right now okay that's why I'm loading the base model That means my original model and I'm I'm only
want to see the output okay so here I've given a prompt what is the large language model and this is the actually output I got okay from my model then after that what I did actually I deleted all the model Pipeline and everything just to uh empty my vram because in Ram actually it will occupy lots of space so you have to execute this code after executing this line actually so no need to execute this line of code just try to Skip as of now first of all execute this line of code this line of
code what it will do it will try to load your base model as well as your train model okay that means new model then it will Mars that means if you want to save the model right now if you're using parameter efficient finding first of all you have to load your base model as well as your new model okay both you have to load then after that you have to Marge after marging what you can do you can also Load your tokenizer then you can push this model to the hugging pH H for this just
execute this line of code and you have to perform the hugging pH C login okay huging face AI login and it will ask for one authentication token see this is the token it will ask now what you have to do go to the hugging face uh go to the settings I think you know how to generate the tokens go to the access tokens now create a new tokens let's say I'll give the token name is equal to Test and now you have to give the write access because you want to write something you want to
push something to the hugging face up that's say you have to give the right access now create the token let's create the token now copy the token and pass the token inside that inside that you have to pass the token then give uh y then press enter after that it will automatically authenticate with your hugging face then it will be pushing your model to the hugging face Itself now here you have to give your user username see this is my my username entb so make sure you are giving your username here okay not my username
otherwise it won't be working okay so if you execute this line of code it will uh push your model as as well as the tokenizer as well now after executing this line of code go to the hugging face click on the profile go to the let's say your profile now you'll see that in the model section this model would be Published okay and this is the model and this is your tokenizer that means the same model you have to use for the tokenization as well I think I already showed right now after that let's test
this model OKAY whether it is working or not so for this I created another notebook let me show you so this is The Notebook I created guys fine tune model test now again just try to install Transformer accelerate that means all the required packet then after that just Try to load your model right now the model actually you have trained see this model just try to copy the name and just try to past it here okay then again I'm giving the same from what is the large language model I'm loading the tokenizer first of all
so inside this pipeline I'm giving my model then after that you can see I'm creating the complete pipeline then it is predicting and it is giving me the results now see this is the results actually I'm getting see Although I have trained only one Epoch but if you see the results it's not actually bad uh somehow actually it has given the good output then again I have given another actually prompt now see this is the results I'm executing my model and this is the result actually I'm getting so the prompt is actually explain to me
in a simple U simple to understand way what is the equation of finding the uh an triangle number is is and how it can be provided by using the High school level math please please give each step of the of a proof of using latex see this is the latex actually output I'm getting that means all the let's say mathematical equation and everything it has given me as a latex now you can convert this latex to the mathematical format and you can see that particular output okay so that's how actually we can also test our
model and this is the entire process of fine tuning guys entire process of fine Tuning and see still training is going on you can see still training is going on and it has created the result folder and inside result inside runs it will save the weights okay it will save the weights that means the artifacts fine so yes guys that's how actually you can perform the F tuning operation with the help of parameter efficient fine tuning you can use either L either you can use Q L fine so now what you have to do guys
just try to perform this fine tuning and If you're having any error if you having any issue you can let me know in the comment I'll try to help you with that okay to fix that particular eror now apart from that just go to the hugging face try to pick any other model guys go to the hugging face and pick any other model not this Lama 2 model you can also take Lama 3 Model you can also take let's say mral model Falcone model any kinds of model you can take just go to the model
section just find a model and Try to perform the fine tuning process okay you can use the same Tech technique to perform the fine tuning process fine so in future I'll create some more video there actually I'll show you how we can f tune any other model as well let's see you want to F tune Jimma model how we can do it and all okay we'll be creating some more video but before that first of all try to uh I mean explore by yourself the way actually I showed you the entire notebook I shared with
you try to pick Up any kinds of model and try to follow the same okay technique to perform the fine tuning operation so L index is nothing but it's an alternative framework of Lang chin so Lama index gives you the ability uh to build a very powerful generative application with the help of large language model so whatever things actually we have learned with the Lang chain like we saw like how we can use Lang chain framework to build generative application the same things Can be performed with the help of Lama index as well so this
Lama index and Lang chain is like very popular in the market nowadays so people are using Lama index and Lang chain uh very broadly in their gener application now it's up to you like which framework actually you'll be choosing whenever you are developing any kinds of generative a application right so we have learned like Lang chain like how we can develop generative application with the help of Lang chain Now it's time to also explore like this lend x one so in this video actually I will give you like how we can build genbi application with
the help of lam andex so this is going to be uh one amazing video alog together just try to watch this video till the end and don't skip any part of this video otherwise you might get some issue and if you have some query you can let me know in the comment I will happy to help you so guys let's start with our video so first of All I will take you through the Lama index documentation then I will start with our project implementation so guys this is the Lama index documentation and uh Lama index
is nothing but it's a simple flexible data framework for connecting custom data sources to the large language model so basically Lama index is nothing but it helps you to build these kinds of generative application with your custom data as well as it will also give you the Ability to connect with these kinds of large language model we have seen right so it is the similar kinds of framework of your langen so whatever functionality langen provides your Lama index will also provide that one but some of the syntax would be a little bit different okay so
I will discuss this thing whenever I will take you through the project demo okay whenever I will be building uh projects with the help of lendx at that time I will show you like Where uh what is the syntax change actually you need to consider now guys uh you can go through the documentation they have written everything okay so here it is telling we can easily build like lots of generative by application with the help of this lendex uh even you can connect uh with these kinds of unstructured data structured data and semi structured data
so unstructured data means uh text Data PDF data video data and it can be also image data uh Structured data means you can also connect with Excel data SQL data and so on and semi structure data means like you can also connect with the API then you can also with this slack then Salesforce notion so these kinds of I mean platform you can also integrate to collect the data with the help of this lamb andex okay so this is one of the very uh popular framework in the market and it is like very powerful so
you can uh use this one also to build your gener Application so it's fine uh you if you know like both framework like Lang chain and Lama index so it will help you to build like very powerful gener application right so um I personally like this Lama index a lot okay U I usually use langin to build my gener application but I also like this Lama index sometimes I also use Lama index to create my genbi application and guys uh this is the documentation of this Lama index so there actually they have given Then how
to set up this lamb index and what are the things actually it provides see it has also this agents uh then chatbot structured data extraction then uh indexing everything like whatever you have seen inside your Lang and everything it has uh if you see here all the things actually they are providing so you can go with the documentation so they have uh given each and everything now I'll uh quickly show you uh like how we can set up this lamb index on our Google collab and how we can build like one Basics gener projects with
the help of this lb index okay then it would be very much clear for you so first of all what I will do uh I will open my Google collab so let me open my Google collab so here I will create a new notebook here you can give the name so I'll just give uh Lama index demo and here make sure you have selected runtime at GPU so let me select Runtime as GPU and uh let me The Notebook so uh in between what I can give you I will give you the project architecture like
uh actually what I'm going to do exactly with the help of this L index so for this actually I'll open my um Blackboard and here I already kept this project uh architecture as you can see so basically here we'll be building one uh projects uh basically here we'll have lots of like documents So basically we'll be chatting with the documents okay okay so chat with multiple multiple documents okay so this is the project name so it can be any kinds of documents so it can be PDF it can be let's say Doc it can be
txt okay it can be image anything you can give so we'll be extracting the data all together okay then we'll be chatting with these kinds of documents so this is Nothing but our custom source okay so this is nothing but our Custom Custom data okay our custom data source you can talk about okay and here first of all what I need to do uh so user will give me these kinds of like data custom data so I need to extract the data and U after extracting I also need to convert them to chunks because I
already told you what is this chunks and why we need it because here I'm going to use large language model okay llm I I'm Going to use large language model so LM wise I'll be using something called Google Pam uh Google Pam 2 I already discussed like Google Pam 2 and if you already know Google Pam 2 has one specific input size which is nothing but 8,90 8,096 tokens okay so uh if you're extracting the data alog together so it might be more than 8,96 token so that is why it's better to create a chunks
so instead of giving all the Corpus together I will provide as a Chunks to chunks so that my input size uh it can handle very easily right so that why chunks is important so after creating the chunks from my entire Corpus I also need to convert this text to text to Vector okay I'll be converting text to Vector so for this actually I'll be using some embedding model so in this case actually I'm going to use hugging face embedding model hugging face uh hugging face embedding model you can also use Pam to embedding Model it's
completely fine I already showed you how to use Pam to embedding model you can also use hang embedding model then uh after creating this uh Vector store I also need to store these are the vector okay in a knowledge base so by default actually uh this lamb index has one functionality so it will create a knowledge base in your local I will show you like how to created then after that U we'll be handling the user so user will raise some query so we'll First of all convert that query to querium buding then we'll be
performing something called CTIC search so it will give me some rank results and we'll be applying generative U model that means large language model to get our actual response okay so this is the complete idea of our project uh architecture right so now let's Implement with the help of this uh lendex as of now we have seen like how to implement with the Lang chain we have seen like how we can uh Develop end to end uh Genera bi projects with the help of Lang chain and Vector database and so on right so same thing
you can do with the help of lamb index you can also add your vector DB here you can use pine cone you can use uh webat you can also use chroma DB anything you can use use but here I'm not going to use any kinds of vector DB I'm going to use default knowledge base providing your uh lamb index okay so it will store your vector in a local I'll show you Like how to do it okay you can also connect with the vector DV so that can be also done I already taken lots of
session on it so you can go ahead and watch it so guys now let's uh see like how we can set up this one so to set up Lama index uh there is a python package actually will get uh called L index okay so just write P install P install ipen Q Lama index okay Lama hypen index so this is the package name you need to Install if you visit their documentation and if you visit their installation guideline okay if you visit their installation guideline see this is the L index package they are using okay
now I also need to install something called uh Pi PDF because here I I'm going to use PDF data so let me install install Pi PDF Pi Pi PDF then I'm also going to install something called uh doc to text because I'm going to use also doc type data so I'll just write P install uh doc docs to text dxt okay this package I need then uh I already told you I'll will be using uh Google Pam to as my large language model so for this actually I need to install something called Google generated VII
so P install uh Google generative AI package okay now I will be using something called hiding Face embedding so for this I need trans uh Transformer so I'll install Transformer so pep install Transformers okay now let me install them together e uh so guys installation is done now uh I'll import some of the libraries so let me import so first of all I will import something called uh uh simple directory reader and my Vector restore from my Lama index I'll tell you like whenever uh I'll be using it So this two uh function I need
so this two function actually I need called Simple directory reader and my Vector store index I'll tell you like whenever I'll be using so what is the the use of it then I also need something called uh pal model so it is already available inside lamb andex so as you can see L index. llm spam and I'm importing this pum and uh this is the similar things actually with the Lang chain as well so I think you remember we are also Importing pal 2 model from Lang chain so from Lang chain. llms then I was
also importing this p p model right so if you have already used Lang chain so uh using this L index would be very much easy for you because lots of functionality will remain same uh will look like similar to you so some of the functionality and syntax might different but rest of the things are I think similar okay what I feel like then I also need something called uh service context from lendex I'll tell you like what is service context then I also need something called storage context and this load index from Storage okay these
are the thing then I also need something called operating system package now let me import them okay done now uh I will load my data so here let me comment out load data so to load the data first of all here what I will do I will create One Directory So here I'll just write one command mkd and I will name this directory as data now if I refresh I think huh directory has created now here I will upload some of the uh data that means documents so I can upload see I here I have
PDF as well and this docs data as well so let's upload uh these docs first of all so this is the like L index documentation as you can see so it is present inside my docs file now here I Will also upload this uh this one yellow okay yellow um PDF so with the help of this simple directory reader we can load everything any kinds of file format you can load okay so that is the power of this Lama index but if you used I think Lang CH you remember we used to use something called
PDF uh loader then txt loader separate separate file but if you're using L index so you can directly load them together okay so these kinds of Some functionality with will might vary and will uh look new to you so it's completely fine but functionality are same so guys let me show you like uh my data once like what are the things I have so in this L index docs I have some content related Lama index like what is Lama index and all okay so here is the content I prepared and it is present inside this
documents then uh in the PDF actually you can see this is yellow related PDF so what is yolow and yolow Architecture so these are the content are available so you can ask any kinds of query any kinds of question from this documents okay you have submitted so we'll be uh doing that with the help of this uh uh large language model and with the help of this uh lamb index okay uh so guys as you can see uh my data has been uploaded now what I can do I'll first of all load the data so
I'll extract the content uh from these documents and PDF so for this I'll be Using something called uh this simple directory reader so I'll call it and inside that you just need to provide the directory so this is the data directory inside that I have this data and here is one function called uh load uncore data you need to call and I will store it inside a variable called documents okay now let me load them yeah now if you want to see the data so this is the data it has Extracted okay and this is
in document format as you can see okay and it will also give you the reference ID and if you're using any embedding or not so it's none by default because we haven't used any embedding and this is the content it has extracted okay now what I need to do guys U we have successfully extracted the data now I need to perform something called chunking operation on top of it because this is my entire carpus and I also need to convert them To chunks so for this let me just first of all comment so we'll be
performing split text into small chunks so uh before that first of all I need to set my uh Google API key Okay Google AP API key I need to set so the uh using this Google API key we'll be able to access our P palom model OKAY from this uh Google palom API so if you open this uh Google palom website so here they're telling just try to create one API I Already showed you how to create the API just click on this pal API and here you just need to get your API key and
here you can generate a new key okay so for me I already created one key so this is my key I already created so if you don't have any key you can create from here so guys I already created the key so this is my key and don't share this key with anyone I will remove it after the recording so that is why I'm showing now let me set this key as my Environment variable done then I need to initialize my llm so llm wise I am using something called this PM model so it will
automatically uh authenticate using this uh API key okay so pal is not initialized please Google generi uh I think we have already installed this package why it is giving okay let me install it okay I need to restart the run time So that is why why I think it was not working so guys we have restarted the run time now let me import them again H now I think my documents is already there uh okay so I need to again load the documents now this is my data now I'll set this IPI key now I
load my pom model okay it's loaded now uh see guys uh initially whenever I was using uh Lang chain so we used to create this chunks In a different way we used to use something called text uh uh character splitter uh function to create this chunks but here uh this uh things is little bit different so here you need to use something called this service context okay so here is a service context uh function you need to use this function so here you just need to initialize it so it h it has one method called
U prom uh from documents so from no sorry From default so from default from defaults and here you just need to pass uh the llm at the very first so llm wise so Palm is my llm then uh I also need to give the Chang size chunk _ size I think you already know what is chunk size so it will uh like uh split your data with respect to that and I also need to define something called chunk uh overlap overlap okay So overlap so let's give 20 so chunks overlap helps us to understand the
context of my entire Corpus because it will convert to chunks and to get your previous context I need the chunks overlap I already explained this thing in my Vector database session now here you can also give like embedding which embedding you want to use and if you're not giving anything so by default it will use that hugging face embedding I'll show you so Service uh context uh this is my service context I got now if I execute this code you will see uh it will first of all uh okay it's giving one error no API
key found for open uh okay it's giving the error because I need to again install this uh Transformer I think so let me let me install the Transformers uh so guys as you can see it's giving one error it's telling uh no API key found for open a so here by Default if you're not giving any embeding model so it will try to take the open a embedding model but here in this case we haven't provided any op open a API key so that's why it's throwing error so if you visit this URL so they're
telling if you want to use some other embedding so you need to first of all import it so let me import this hanging embedding so I told you I'll be using hang EMB bding so first of all let me import this hang EMB Bing Here you can also use Google Pam embedding the way I showed you in my previous project you can do it but let's use the sface embedding now I need to create one object called embed model so here I can create now here I'll be using this model uh this is the model
available in the Hing face this is the embedding model so I need to install something called s sentence transformer for this so this is the command Let me install it here done now let me import it now guys see first of all it will download that embedding model from Hing fist this then it will load that so done now I need to give one parameter here called embed model is equal to embed model now let me execute it h so my service context is prepared right now now what I will do now what I need
to do I need to convert my Documents to chunks and that chunks I need to convert to something called Vector embedding okay so for this uh you need to use something called uh uh vectory store okay so vectory store I think we have already imported this thing above so this is the vector store index so this uh class will help me to do that so let me import it here and it has one function called from documents okay so here you just need to give you a document So in this case I have my documents
here I think you remember so this is my documents this is my entire Corpus then I also need to give my service context so service context equal to my service context this is the service context so it will take the model and everything from here and all the configure I have set here Str SI chks overl everything I have given here now I will get my something called index here so in Lama Index uh the vector you get we call them index okay so this is my index now let me uh convert them so it
is taking some time because it is taking all the documents and converting to chunks and converting to vectors okay now we have successfully converted them now I need to store them in my local so let me comment it here so to store it first of all call this object called index and there is a uh method Called storage Storage storage uh storage uh context dot persist persist okay now if I execute this one you will see it will save everything as your local okay so there is a uh folder it will create called stor is
inside that see all the vector it has saved okay now we can open it anything you can see that okay see this is the uh data it has Saved so it it follows one particular format to save your documents so it might not understandable to you but this is my see this is my Vector okay this is my final Vector of my entire data now guys we have successfully stored our indexes now let's see if you have some existing stories or existing index already so instead of creating them you can directly load it so this
is the code so this is the code load the index So here you need to call Storage context I think you remember we imported this thing storage context so using storage context there is a method called from default and here you just need to give the purchase directory which is nothing by storage and it will load that and to load it you need to use load index from Storage okay we imported I think I you remember load index from stories then it will load that index for you okay it will load that index for you
so you can Execute this code okay I already created my index object that's why I will comment it out okay I'll comment it out you can uh if you have already so you can load it out and here you can also use uh like vector DB to store these other data to your uh Vector DB as well okay this is also possible but LX provide this functionality so that's why I I'm showing this functionality to you okay how to store them in a uh local machine now we can perform our keyway Operation keyn operation so
let's do q q operation so here uh first of all you need to Define one quy engine so this is my index so it has one uh parameter as quy asore quy engine and this is my query engine now uh I'll call my query engine and here I will ask the query so this is my query so here first of all I will ask like what Is lamb andex because I have one lendex related documentation here right so that why I'll ask what is LAM andex and it will give you the response and as response
I will print it here uh so guys this is my response uh again it's not understandable so if you want to extract the actual uh response if you want to see the actual text so for this uh you can use this uh package because we are using Google collapse so there is a package called I IPython Display markdown and display so these two package I'll will be uh importing then I'll be using something called display. markdown and here I will give my response okay the response I'm getting now if I execute it so it should
me uh so it should give me the response Lama index is a data framework for building application it provides a comprehensive toolkit for integration management and Query of your external data so that you can use with your llm app great now you can ask any other query so let's say I have one YOLO documentation so I can ask anything about yolow so what is yolow now if I execute again see Yow is a regression algorithm uh it takes image input and output building bounding boxes and level uh images okay great now uh let's see if
I'm asking anything uh out of this context out of this data so let's see what happens so I'll ask Uh uh who is uh B let's say I will ask this question see it's telling B is a random name and it's not mentioned in the context so basically uh we have created this projects and basically it is only relying Upon Our Uh custom data we have given okay apart from that if you asking anything so it w be giving the answer so you can ask anything uh like related your documents so in this case you
can use this question anything You can ask like whatever question actually you have so it will give you the answer with respect to that yeah and uh this project actually you can uh convert to uh stream lit app I already showed you how to convert them to streamlit app and all so you can try this thing from your site okay I think you can do it so yes guys this was all about lendex I hope you get get it like uh how we can use this lendex as our alternative framework of our Lang chin And
this is one of like good framework I can say so uh you can visit the documentation and you can learn so many things about lendex okay now try to explore from your site so you'll be getting each and everything I think you already know about Lama index I already explained what is the Lama index Lama index is nothing but it's a alternative framework of langen that means whatever let's say things you can perform with the langin the same thing you can also Perform with the Lama index so both are actually Genera VI framework with the
help of that you can create Genera VI based application so in our PR video we saw like the entire introduction of the Lama index so there actually we already performed one practical example so in this video I'll show you how we can use open source large language model with the help of L index so as an open source model wise I'll be using something called Mistral okay so if you don't know Mistral is an open source actually large language model and it is available inside hugging face now if you search here let's say mral if
I search here see mral is having actually different different model let me show you so let me go to the mral AI see it is having actually 17 different model so expand all the 17 different models see Mr is having 7B model that means this is the 7 billion model 22 billion model okay that's actually it is having different Different model so from this model guys I'm going to use one model called Mr sa me instruct V 01 so this model actually will'll be using with the Lama index so this model we be downloading from
the hugging face Hub and we'll be using with the help of Lama index okay so here I'm going to create a simple system so let me show you the diagram what I'm going to perform here see here uh I will upload one actually PDF document then I will extract the data Extract the documents extract docs after that I'll perform chunking operation I think you know what is chunking okay and why it is required then I'll be using my embedding model here I'll be using one open source embedding model fine embedding model to generate the vector
embedding so it will give me something called Vector Embedding and this Vector embedding I'll be saving to a knowledge base that means Vector database and if you're using Lama index guys I think you already know Lama index has one vector database service so we can call it as Vector store okay Vector store so inside Vector restore we can uh save our uh embedding so whatever M actually we'll be generating and we'll be using llama index okay llama index vectory store okay L mind is vectory store and This is going to be my knowledge base then
after that uh user will ask one query let's this is the user user will uh send one query to the knowledge base and knowledge base will return return some relevant answer okay it will give give you some rank results rank results then what I have to do I have to use one large language model llm so here we'll be using something called Mistral okay Mistral so we'll pass this rank results to this large language model as well as this quy as well so this quy and this rank results will process your large language model and
it will give you the actual response okay it will give you the response so this is the system actually will be implementing fine with help of Lama index as well as the open source large language model because this system we already saw how we can Implement with The help of langin now let's try to see how we can use the Lama index as well so let's get back so here I already created one notebook guys you can see mral with Lama index so first of all let's connect this notebook so resources would be shared in
the description section so from there you can download and you can um execute all the code so it is connected now let me check the GPU which GPU actually I got here so I'm using collab Pro guys so that's why I'm using A 100 GPU so if I go to the run time change run time type c I'm having a 100 GPU so if you're using cab Pro so you can access this GPU otherwise you can select T4 GPU okay it's completely fine now let's install the pi PDF because I already told you I'm going
to use PDF data here and I have to upload one PDF documents here okay but before that let me complete the installation then I'm going to install the Transformer then I'm also going to install some required Uh package for for the Transformer okay so let me install everything why Transformer because I want to access this Mr model and this model is available in the hugging ph up and then I need to install something called sentence Transformer why because I already told you we'll be using open source embedding model and for this I need this sentence
Transformer so let me install this as well now I'm going to install the Llama index so I'm Installing one specific version uh why specific version because I already told you if you're installing any specific version uh that will help you a lot in future because uh let's say today you have created one projects with uh the latest llama index what will happen after some days actually they will upgrate their package and again you will get issue right so that's why it's always better practice to use when specific version fine now let me import All the
required Library so you can see I'm importing from Lama index Vector store index simple directory Reader Service context okay then hugging face llm then simple input from so this is the vector restore guys that means this is the vector database Service uh Lama index provides so uh here actually we can create a knowledge base and we can instore our embedding and this is the local uh local DB okay it will save everything in local then simple Directory reader will help you to read your content that means whatever data actually will be reading right then hugging
face LM will help you to load that mral model then with the help of simple prompt you can create a prompt template fine so now let me import all of them later on I'll tell you whenever I'll be using them now here I'm going to create a directory called Data okay because inside data directory I'm going to Upload my data now see if I refresh here data is created now let me upload one PDF here so guys I'm having one statistics PDF so I'm going to upload it here so let me show you the PDF
so this is the PDF document guys importance and use of code relation in statistics so this PDF actually I'm going to upload in my Google collab and on top of that actually we'll be performing this operation now here you can use any kinds Of PDF any kinds of documents okay whatever you have now see uh it is uploaded now let me read the content so inside simple directory reader you have to give the data location so you can see I've given the data folder it will automatically load the PDF and it will extract the information
let me show you see if I show you the documents right now see all the document is has extracted fine now here first of all let's create a so here You can see I have created a uh system prompt you are an QA assistant your goal is to answer the question accurately as as possible based on the instruction and the context provided now this system prompt actually I want to use as a default prompt inside my large language model for this you have to create one query rapper prompt for this you can use Simple input
prompt okay and inside that you can pass these are the keyword fine now let me execute now you have to do The hugging face C login because this model is available inside hugging face and to authenticate with hugging face you have to give the token now here you have to pass the token and how to get the token guys I think you remember go to the hugging face click on the profile go to the settings and left hand side you will see access token uh you can create a new token here so let me give
the token name let's say Test I want to give the read permission as of now let's create the token I'll copy and let's mention it here if I paste it and hit enter it will automat atically authenticate now here it is asking uh token as great credential I'll give yes now see login successful okay now you can see this is the model link I have already given so this model actually will be using now let's load the model and to load the model I'm Going to use hugging fist llm inside that you can pass context
with the max token I think you already know what is Max token so what is context window context window means this is the input size of this llm that means it can take 4 496 token U as input okay as the input that means this is the input size of your large language model M can take four 496 token okay now you can give this temperature parameter and all and inside that just pass your system prompt Whatever prompt actually you have defined here I think remember and you have to also pass the query rapper prompt
that means the default prompt you want to set and here you have to give the model name like which tokenizer you want to use I want to use the same tokenizer as per my model see this is the model okay I'm giving the model ID here now you you need to also give the model name that means the same name then after that Auto map is equal to that Means I want to use my GPU here I want to load this model in my GPU because it's a large language model it is it is having
actually lots of parameter right so we can't U let's say run this model on our CPU machine it's not possible if you're running U actually it will take lots of time even your memory will also get crash so that's why we have to load everything in the GPU based machine now let's load this model see first of all it will download the model from the Internet and it will create one llm raer see it's around 9.9 94 GB just try to consider guys how much big this model is so that's why actually uh whenever you
are using any kinds of Open Source large language model uh to create any kinds of llm powered application so this is the issue actually will be getting that means you should have good configuration machine otherwise you won't be able to use that model so we'll be learning something called llm ops so Inside L LM Ops actually I will show you how we can uh create a efficient llm powered application how we can deploy it also on the cloud platform we'll be discussing each and everything because of that only LM Ops SC in the market there
are so many PL platform actually we can use like Cloud generative we can use let's say Bedrock is there vertex a is there asure open is there we can use these are the service okay to overcome this issue so this model has downloaded Successfully now what I have to do I have to download the embedding model as well so to download this embedding model I'm importing hug face embedding from langin so because langin is having one embedding functionality so from there we can easily download the embedding model so let's try to download this embedding model
so we have to install R Lang chain Community for this so let me install Len Community now let me execute this code so see it has downloaded Successfully now let's create the now let's create the service context that means my vectory store that means I will be creating the knowledge base for this we'll be using service context from default inside that you have to provide the Chun size that means first of all it will perform the chunking then it will create the let's say Vector embedding and it will store that Vector embedding to the knowledge
base okay now inside that you have to pass the LM as well as The embedding model now let's create the service context so here actually some of the syntax are different okay with respect to this langin uh if you use the langin I think Lin synex is little bit different Lama index synex would be little bit different but uh I mean framework wise both are same okay both can perform the same thing now let me create the knowledge base so here you are so here I'm using Vector store index from documents here I'm giving the
Documents as well as the service context whatever service context we have provided okay now it will create the entire knowledge base now I can perform the qu operation okay that means this operation okay this operation so it will go to the vector restore that means knowledge base and it will return the rank results and your llm will process and it will give you the correct response okay so here I'm given one question what is the correlation now Let's see what is the response I get here so inside Lama index you need to write Le less
than have code but whenever you saw the Lang chain actually we have to write uh actually our Vector store actually manually that means we have to write some more additional code right but here everything is automated everything it will handle that means we don't need to perform the chunking separately chunking will automatically Perform here okay with the help of the service context see although I'm using using GPU based machine but still the inference timing is high here now see it's done now if I show you the response now see this is the response I got
corelation is is a statistical measures that Express the extent uh to which the two variables are linearly related that means it has given me the correct response okay so yes that's how actually we can use any Kinds of Open Source large language model with the help of L index now you can go to the hugging pH and you can use any of the let's say um large language model and you can use it with the help of Lama index fine so uh I'm not going to implement this projects from scratch because I have already discussed
the like process how we develop like one in 10 projects okay so that is why I already implemented one particular projects as you can see here so this is Like a Financial stock analysis project I have developed with the help of this lendex so first of all I will show you the demo of this projects okay like like what is the projects and all about so those who are interested in the field of financial or let's say in the domain of financial so uh they will be loving this project because here you can do like
a stock analysis okay so we have some stock data we have some articles actually as you can see so these are the Articles I have uh downloaded from the internet and this is in the HML format so we'll be processing these are the data we'll be collecting the content okay if you see this is the completely HTML data so here we have lots of like Financial stock data like we have Google data then we have Nvidia we have meta okay then we have u ms uh uh here you see msft okay as you can see
so we have different different like you can say Companies uh stock data here so I already uh given the link of the data in my GitHub so you can download from here I will show you like how to download the data so that is script I have already written so first of all let me execute the projects so I will first of all execute this projects and show you like how it will work so see guys this is the stream late app I have developed and uh here you can do two kinds of analysis one
is like you Can do a single stock analysis another is a competitive uh analysis okay like competitor analysis I'll show you both of you okay so here first of all let's do the single uh stock analysis so I'll just ask what is the stock of here if you see so they have given the short name for each and every company if you come here so I've already listed down so let's ask uh for the Nvidia so I'll copy the Nvidia and here I will ask So guys as you can see I have asked like what
is the stock of Nvidia so it is giving me the answer okay you can see it is giving it is giving like 2023 to 2027 okay it's giving the stock as you can see now and you can ask for any kinds of stock let's ask for the Google so I'll copy Google and here if I paste and enter so now it will give me made the stock of the Google so that's how actually you can ask different different stock Related question okay um if you're uh interested in uh these kinds of stock and all so
you can ask lots of question and you can get the response with respect to that okay as you can see this is the Google stock uh answer it is giving now you can also perform like competitive uh stock analysis so just select the competitive one now it will give you the interface of the competitive analysis so guys as you can see this is the Competitive analysis one so now here you need to provide stock one uh symbol and stock two symbol so let's do like uh msft and I will also do for my Nvidia let's
see the competitive analysis between them uh so see guys uh this is the competitive analysis between them as you can see you can read it out okay so not only that you can uh like see like competitive analysis between all the company here is the listed okay and this Is the symbol of the stock like uh you can't directly pass the company name so you need to give the symbol okay because this is your custom data you have given like that so yes guys this is the projects actually uh I have developed with the help
of this Lama index and this is very much uh interesting projects like in the field of financial domain and you can see Implement lots of thing with the help of this lamb index I think I already showed you like lots of Like functionality and all and I already showed you lots of things okay uh with the help of Open Source model and now you can Implement anything okay whatever you can think in your mind you can Implement anything just you need to try from your site okay now guys U let me show you like how
I have developed this projects okay so this is the projects guys I have already developed and uh let me show you uh this is the code for the projects okay so I'll explain each and Everything like whatever things actually doing and I have already given the idea about Lama index and this stream lit and all so I think you are already familiar with these are the thing okay I'm expecting so that's why I'll be explaining the projects okay instead of writing all the line from scratch so first of all let me stop the execution then
uh I will remove this article folder because this is my generated Folder then I will also remove this stories okay stories I think you remember stories is nothing but it's a vector store we created with the help of lam Index right so both I have deleted now here is the rme file I have already shared like what are the steps you need to perform so here first of all you need to clone the repository so this repository will find from the description section so from there actually you can download the projects Then you need to
create one environment okay python environment uh using python 3.8 then you need to activate the environment then you need to install the requirements so requirement wise I'm using these are the requirements as you can see so you are using L index then stream lead python. EnV okay and apart from that I'm also using openi and some U you can say dependency package I'm also installing here so this thing you need to install I Already showed you how to install them and all how to create the environment in my previous project implementation then after that uh
you need to uh execute this app.py okay but before running the app.py you should have your data and you should have your vector store presence okay otherwise it won't be working so for this actually in the SRC folder I have written some of the script as you can see first of all we need to fetch the data so this is the link guys so in This link actually this articles is available I have already given this link and this is uh this is hosted on my GitHub so from here you can download those data set
okay download those articles and after that see I have written one function with the help of this function I'm downloading so basically it will take this URL data URL and it will download that uh Z file okay from my GitHub and it will and it will unzip it okay it will unzip it here and That zip file it will remove okay so this is the code I have written now let me show you how it is working so I already created the environment so now I'll execute this particular file so I'll just write python uh
inside SRC inside SRC I have one file called 01 fetch data okay I will execute this file now see guys it will download that uh data from my URL so see guys it has downloaded all the HTML file and now see if I open my Articles all the data are presents now what I need to do I need to create one vector I think you already know like what is Vector I I already explained these are the thing in my notebook experience ment okay what are the steps I perform same thing I'm doing here also
okay so to create the vory store I created another file called index news. Pi so here I'm using this GPT Vector restore to create my Vector embedding so here you can also use Google p embedding You can also use hugging face embedding anything you can use just try to use open source embedding U because if you if you don't want to use like open AI so if you don't want to spend money so you can go go with like open source model otherwise uh if you have like open a account you can also use open
AP key so here if you see I have already set my open IPA key so by default if you're not giving any embedding models it will take that open a embedding model okay so this GPT uh Vector store index already has like GPT based embedding model it will take that model and it will convert your data to Vector embedding okay so guys this should be your task just try to replace this code with your open source model I think I showed you how we can use hugging face embedding model in this notebook uh see guys
here huging Quest embedding model I was using so you also need to use Hing Quest embedding model here okay in this case I'm using open so You need to replace with the H embeding model okay I think I already showed each and everything so you can do it now let me first of all generate these are the embeddings so I'll clear then I will run this uh second file so it's 02 so it will take time because it will take all the Articles it will it will convert to chunks then it will uh like convert
them to Vector embedding okay then it it will create the stories here So here you can also integrate like vector database so you can use pine con we yet and you can also use chroma DB but I'm using this default like vector store index so L index provide this thing so it will store your embedding local okay it will create one storage folder and it will store here so you can also use Vector database and you can store them in the cloud as well okay I already showed you just try to integrate here uh so
guys as you can see my stor Has been created now now this is my Vector uh so see these are my embedding so these are my Vector okay so now we are successfully uh able to store our Vector embedding now see uh now this is the third uh script I have written so if you want to test on top of your uh like stories so I here I'm loading the stories and I'm just asking one query like what is uh tell me about Google's new super computer okay so this is the query I'm asking so
it will give you the Response so you can try out okay so let me execute and show you how it is work working so I'll just write python then inside SRC I have this 03 yeah so see guys uh this is the um response I got okay so yeah that means my U Storage has been created successfully and we are also able to ask the query on top of it now we'll be converting these are the thing okay uh As our app so basically I will uh give some of the input okay uh user will
ask the query and they will also select like they they want to perform like single stock analysis or let's say multiple stock analysis so this kind of functionality I also add so for this I I have already created one app.py as you can see this is the app. pi here if you see my index is already created so that's why I'm importing the storage context and load index from the storage Because I want to load it then I'm also importing like wise openi because I told you I'll will be using openi like model here instead
of using open source so here you just need to replace this uh openi model with your open source model you can also use Palm 2 then you can also use lamb 2 anything you can use I already showed you this thing in my this notbook experiment okay in my L index session just try to replace that these are the model here then model wise see I Have initialized zpt 3.5 TBO then here I'm just defining my llm predictor and I think you remember we had we had created one service context okay uh so in lendex
we create something called service context and this will take your llm so once it's done I'm also loading my Storage storage that means this uh Vector embedding okay as you can see I'm loading I'm giving the path once it is done I'm creating this quy engine okay so once quy engine is ready now we'll be Creating the front end part so here basically if you see here we are uh creating this uh title and all like this is the title Financial stock analysis so let me execute and show you how this thing will look like
so see guys uh this is the title and this is the report subtitle so these two I have created now here I'm creating one select box so what type of uh like report you want to you want so this is the select box guys let me show you this Is the select box so if you want to create any kind of Select box you need to select uh select box from stream L and here is the message I've given so what type of uh report you want to do and single and competitive so here if
you see this two thing I'm giving here so single and competitive then if user is selecting single one okay so first of all we'll be handling the single so here is the uh prompt I have written let me show you so I'm just using my engine Query query engine so as you can see query engine we have already created and this is the like prompt I'm just giving so write a report on uh like on the outlook for symbols okay symbol user will give me that okay user will give me that symbol because as you
can see we are uh receiving the symbol so uh as a text input text basically here is the symbol AS input text we're receiving and that thing we're passing here okay as a F string then once it is done this is my Prompt stock from I'm just uh mentioning the year I need this stock from this year and uh I'm also telling like be sure include the potential risk and the uh head winth okay so this is my prompt I have given to my llm so my llm will give me some response and that that
response actually I need to show I need to Showcase okay so I'm just using s. write and I'm just passing my like uh this response and it it will show here so once it is uh done so I'm also Handling the competitive analysis so if user is selecting competitive analysis so first of all I'm taking like two competitive as you can see if I select the competitive one so here if you see I'm receiving the stock one and stock two and this thing actually I'm storing symbol one symbol two and this is the my second
prompt okay so I'm just uh writing write a report on the competitive between my symbol one and symbol two uh stock okay then it will Give me the response and that the response I'm showing in my front end so this is the simple logic actually I have written uh behind this projects okay and this is like very easy now you just need to uh like convert them to open source model and if you have any idea on top of it you can also create it okay I think you got the overall idea and all the
resources actually will get from the description section so I think you already know what is medical chatbot Let's say if you have any medical related query let's say you got one disease so you are asking for the diagnosis you are asking for the medicine so this bot will able to give the response with respect to that that means it will give you the suggestion it will give you the medicine suggestion it will give you the diagnosis suggestion so these kinds of information this bot will provide so we'll be creating the complete system Here that means
here first of all we'll be using our custom data the entire let's say medical information custom data we'll be using first of all we'll be teaching our model we'll be creating the entire knowledge base then I'm going to connect my large language model there that means whatever information you'll be asking related the medical let's say any kinds of disease any kinds of medicine any kinds of let's say diagnosis so everything your chatbot Will be able to give you the response okay so this is the system we'll be implementing and this is not going to be
guys uh like notebook implementation so here we'll be creating the complete pipeline that means we'll be creating the front end part of our application that means we'll be creating the beautiful user interface so that user can put the quy and they can get the response from the chat B fine and here we'll be using something called modular Coding in Python so it's not like scripting programming we'll be writing we'll be using modular concept here to implement the entire project fine and also we'll be uh using the git and GitHub to let's say do the version
controlling so uh that means we'll be following the entire development pipeline here what are the things you need even I will also show you how we can deploy this project okay in a cloud platform so everything guys we'll be Implementing in this project so make sure we watching this video till the end and if you have any question you can feel free to ask me okay in the comment section so first of all what I will show you guys I will show you the entire project architecture like what are the things we are going to
LEL up then I will also show you the tools and Technology we'll be using here then we'll start the implementation fine so let's open up our Blackboard and try to Understand the project architecture so guys as I already told you we'll be using our custom data here okay we'll be using custom data custom Medical Data I can write medical data so let me show you I'm having one entire actually medical book so guys you can see this is the book the book name is the Gil Encyclopedia of medicine okay second edition so this is the
book you can also search on Google you will get This book and it is having uh around 637 Pages it's like very big book and it is having all kinds of information regarding the medical so see if you open this book actually you will see all kinds of content let's say all kinds of disease even with respect to that disease what kinds of medicine actually you need what kinds of diagnosis technique you need everything this book has uh covered okay so it is having different different um different Different treatment different different uh actually medicine different
different disease even the disease image is also visible here all kinds of disease actually it is having all kinds of disease actually it is having even with respect to that they have also given the diagnosis even the treatment okay how let's say you can do the diagnosis how you can do the treatment what kinds of medicine you have to let's say take everything this book has uh given the Information so here we'll be using this book actually the entire medical information and we'll be creating the knowledge base that means here we'll be using pine cone
Vector database okay I think you know Pine con is a cloud-based Vector database that means there we can store the embedding Vector embedding that means first of all we'll be extracting all the information from this book then we'll be creating the chunks and that chunks actually with the help Of embedding model we'll be creating the vector embedding and that Vector embedding will be storing to the Pine con Vector DB and that Pine con Vector DB is going to be my knowledge base that means the entire knowledge base H why I'm not using the local Vector
database here because my data size is huge if you see 637 Pages information I'm having so if you just create the chunks just try to consider how many chunks you will get and if you're storing in the local DV it Won't be efficient so that's why we'll be using cloud-based Vector DV here fine and I'll show you how we can use the pine cone properly it's one of the amazing Vector DB I personally prefer so there actually you can also see your vector visualization okay everything you can see see this is another actually disease see
this is the disease actually even with respect to that they have Al given the diagnosis as well as the medicine now I'll share this PDF with You guys you can uh see The Complete Book if you want to see different different medicine and different different let's say diagnosis okay you can check it out so here what we'll be doing guys we'll be using this medical book so let's say this is one PDF documents actually I'm having or here I can write medical okay medical book so from the medical book what we'll do we'll just Extract
all the information extract all docks okay then after that we'll be creating different different chunks and I think you know what is chunking is required right because here we'll be using large language model and large language model is having input uh actually size one specific input size and we have to follow that input size I think previously we learned about about Lama llama is having 4,096 okay this is the input length uh that means this is the maximum input length it can take the data if you're using open model Also let's say GPT okay GPT
is also having one specific input lend if you want to see that just open the open.com and go to the model section you will see that GPT input length so whatever actually large language model you using first of all try to see the input length Okay of that model like what would be the maximum token size Based on that try try to perform the chunking operation okay that should be my suu now it will give me different different chunks so let's say I will get chunk one a chunk one we'll be getting Chun two we'll
be getting Chun three and so on then after getting the chunks what I have to do guys I'll be using one embedding model embedding model okay with this embedding model Will generate the vector embedding okay it will give me different different Vector embedding you'll be getting Vector embedding I can copy and I can paste it here so we'll be getting different different Vector embedding and with this Vector embedding guys what we'll do we'll just try to build one centic index that means the knowledge base right build centic index so this is going to be my
Knowledge base that means we'll be storing this Vector to the vector database so I can write knowledge base okay knowledge base and here we'll be using something called Pine con Vector store okay so let me just write Vector DB okay DB Vector DB and guys uh this is going to be my complete backend component component okay so here I can write this is going to be my backend component Okay component now I have to work on the front end component that means user will uh give some query and with respect to the query actually uh
I have to uh give the proper response okay let's say this is the user this is the user so user will send one query let's say this is the query user has asked so first of all it will convert to the query EMB bding okay querium bding that means this is going to be a English Text right and that should also converted to the Vector embedding this Vector embedding will go to the knowledge base okay it will go to the knowledge base and knowledge base will return return some rank results okay rank result or similarity
let's say result it will return return okay rank results then I'll be using one large language model so here we'll be using one large language model Llm so here I'm going to use openi large language model you can use any kinds of large language model you can use open source large language model okay you can also use the gini you can also use uh let's say mistal anything you can use here so here I'm going to use open large language model because here I'm going to create a production ready application so this application will be
also deploying in the cloud platform okay that's why uh I'll be using openi model okay why openi Model because openi model is already hosted in the openi server we can access this model through API request but if I'm using any open source L language model we have to download this model in our local machine and local machine doesn't have like uh good configuration let's say instance let's say you don't have good configuration GPU you don't have good configuration actually Ram CP that time actually you can't execute this model in your system okay it it Will
be very much difficult for you and you will see that inference time would be also very high that means uh this model will uh keep processing the input it will take lots of time to give you the response so that's why we'll be using openi model and going forward I'll be discussing about llm Ops and there I'll tell you how we can also use like different different open source large language model let's say if I want to use llama okay llama if I want to let's Say use mistal Falcon okay cloudy any kinds of model
if I want to use how I can use it even how we can use it use it as a efficient way even how we can deploy these are the model in the cloud platform so for this we'll be using something called llm ops okay llm Ops we'll be using that means we'll be using different different platform we'll be running about Bedrock okay AWS bedrock we'll be learning about let's say gcp vertex AI verx so these are the llm Ops platform okay so we'll be uh using this lmos platform to access these are the model and
on top of that we'll be creating the application fine so this is what actually we'll be explaining later on guys so first of all try to see how we can implement this production ready project with the help of openi model and there is another reason actually we are using open model because if I uh let's say if I'm using open source large Language model you will see that open source large language model size would be very huge let's say around 10 GB 20 GB and this model you can't execute in your local uh machine that
means your loow configuration machine and if you're deploying this project to the cloud platform there you have to take good instance that means you have to take GPU based instance with higher Ram higher CPU higher GPU and just try to consider about the cost like how much how much Cost actually it will take that cloud platform you can go to any kinds of cloud platform you can see their instance actually type if you're taking GPU based instance type with higher Ram higher let say CPU you'll see that per hour it will cost you like more
amount of money okay so again you are increasing the cost here whenever you are using open source large language model okay without any kinds of LM Ops tool llm Ops platform you will get the Difficulty for sh right so that's why we'll be using here openi model although it will charge you but it will charge you less than uh the open source model actually you'll be hosting in future okay this is the idea here so now what will happen this uh rank results will go to the llm even this quy also will go to the
llm and llm will process L will try to understand the query as well as the rank results whatever results actually your Knowledge Bas is returning based on that it will give you the correct response okay correct response or you can talk about answer correct answer let's say you are asking one question let's say I got fber what can I do so it will first of all refer the knowledge base it will try to find out the let's say fever diagnosis fever medicine then with respect to that it will give you the um response got it
so this is the entire system actually will be implementing and This is what actually our front end so this is what actually our front end many people also ask me like sir uh just try to create some application with the help of good front end development as well so that's why I also kept this front end development here we'll be creating one beautiful user interface so that user can give the quid here I think you use the chat GPT chat GPT is having one beautiful user interface okay so we'll be also creating this kinds of
Chatbot kinds of interface so that user can give any kinds of query and they can get the response fine so this is the entire architecture of our project guys okay I hope it is clear now now technology wise I already told you here we'll be using uh large language model openi okay open LM we'll be using then here we'll be using Lang chin as a Genera VI framework to develop the entire project then here we'll be using Vector database why is Pine con okay Pine con is a cloud uh cloud-based Vector database okay so there
actually you can create a um instance and you can store your vector and here to create the user interface we'll be using something called flask okay flask is a python framework you can create web application okay with that got it uh yeah so I hope these are the actually technology we'll be using as of now and if I need it anything I'll tell you okay later on now guys what I have to do guys uh first of All let's create one uh yeah so let me also write we'll be also using GitHub okay GitHub to
do the version controlling of our entire application fine and later on we'll be also deploying this project to the AWS Cloud okay AWS Cloud so we'll be learning about the simple deployment as well as the cicd deployment okay both will be learning so guys this is the entire agenda so now let's open up our GitHub and try to Create one GitHub repository first of all with our project then we'll be starting with the implementation fine so I'll go to my GitHub guys so this is my GitHub you can also follow me here I have created
lots of repository that might help you a lot so just go to the repository and here uh create one new repository and give the name let's to create a med I'll just write end to end okay medical chatbot generate so this is myo name you Can give any repo name as for your requirement now I'll be keeping as a public repository because I want to uh share this code with you so everyone can see that I'll add the readme file as well as I will add the dog ignore file so here I'll be using Python
Programming I'll be selecting the python let's take the license you can take any license I'll be taking Amit license now everything is fine now let's create the repository okay now let's SC the Repository I'll copy the link address make sure you selected HTTP 1 Now open up your local folder so let me open up my local folder and here I'll open up my terminal you can open up your let's say G bash you can open up your anacon anything and just write the command G clog and press the link here so it will clone that
repository in your local machine now see it has clone now let me go inside this folder you can see currently I'm inside Lang chain project Okay now I want to go inside this folder so I'll just write CD CD means change directory I want to go inside end to end medical chatboard gen this folder okay now if I now if I hit enter you will see that I'm inside this particular folder right now now I'm inside this folder now inside this folder I'm going to open up my vs code so let's open up my vs
code so this is my vs code guys okay now let me Zoom now the first thing guys what I have to do I have to create uh one Virtual environment here because I already told you if you are creating any kinds of end to end project first of all create a virtual environment then you have to set up all the requirements you need here so here let me add all the step you need to perform M so here I have added all the stem in the readme file uh first of all you have to clone
the repository if you're using my repository first of all clone it then try to create the environment with this Command so make sure you using python 3.10 okay so I'll copy the command you can give any name I have given LM app you can give any name here fine so what I can do I can I think use medical medibot okay I can use medbot you can use any name now let's create the environment so guys as you can see my environment is created now let's activate for this you have to execute this command all
the command I have shared in the RM file Just try to copy paste and I will also push this code in my GitHub so that you can see the changes now see medical Bo has been activated fine now the next thing we'll be creating one file called requirement. txt file and inside that I'll be mentioning all the requirements I need and we'll be installing all the requirements here so guys here are the requirement list so these are the requirements actually I need to implement this uh project we'll be using Sentence Transformer because here we'll be
using open source embedding model to generate the vector embedding so that's why we'll be using sentence Transformer and sentence Transformer uses actually Transformer library that means it will use hugging F platform hugging fist H to download that model open source model and with the help of that model actually we can generate the vector embedding and here I'm installing one specific version you can also install the latest version But this is the stable version I'm installing so far then Lang I you know Lang chain then flask we need for the user interface then P PDF here
we'll be using PDF document that's why then python. ENB why because here we'll be managing one EnV file okay EnV file so inside EnV file we'll be mentioning our openi credential as well as the we'll be also mentioning our Pine con credential because Pine con you also need to use one API key to authenticate with your Account fine then you can see I'm installing Pine con and if you want to install Pine con you need to install this to library Pine con gr PC okay then you you need to also install Lenore Pine con then
langen community langen openi and langen experimental okay so these are the dependency packet you also need to install so let me install all of them I'll just try to save I'll open up my uh readme file I'll copy the command and here I can execute now see It will install all the package one by one so it may take some time guys let's wait once installation is completed I will come back so guys as you can see my installation is completed there is no error that means congratulation you have installed successfully everything now let me
minimize the screen and now let me save everything yeah now see whenever you are implementing any kinds of end to end projects the first thing you have to create the Project okay project folder structure struct structure okay this is very much important guys that means you have to create the entire project folder structure like in which folder in which file you have to write which function okay that's how you have to create the entire project folders structure and again this is your uh actually let's say design pipeline okay it's not like that you have to
follow my pipeline always You have to follow my project folder structure always if you feel like okay this folder structure is good for you you can use it just try to design with respect to your project requirement with respect to your let's say uh I mean Choice okay no need to follow my one but if you want to follow my one you can also follow I usually prefer one project folder structure okay so I'll be showing you how we can implement this particular project folder structure that means Automatically we be creating the project fold structure
here so here we'll be uh creating one file so the file name is template template. Pi okay this is going to be a python file so this will create the entire project template for me so here only I need to give my one time effort that means I'll be writing the entire logic to create the structure going forward whenever you want to create any kinds of product for the structure you can execute this file it Will automatically create otherwise what will happen you have to manually create it let's say you want to create a folder
you'll be clicking here you will give the folder name let's say test okay then I want to create another file inside folder again you'll be opening this folder inside that you'll be creating another file let's say test. Pi okay so this is like very time-taking task let's say you want to create 100 of folders 100 of files just try to consider how Much time it will take even you have to manually check everything whether this file is present inside this folder or not so it's not a efficient way to create the folder structure okay you
can create it but I won't be suggesting you because it will take lots of time so what you can do you can create one template file and a template file you can write one very basic code and this code will automatically create the project folder structure for me let me Show you how we can create it so for this I'm going to use some package I'll using operating system package then I also need something called path Li okay so let me UT so from it is available path Li input okay path I need the path
why I need the path I'll tell you later on as of now let me import all of them then I also need something called login okay so login is inbuilt package inside python so here I'll will also log the information in my terminal fine so first Of first of all you have to create an logging stream okay so basic loging configuration you have to create so that's how you can create a basic loging configuration see okay this is the basic login configuration so here you have to write login. basic config inside that you have to
mention the log level first of all so here I to create a information label log and this is the format of my log that means first of all it will save the uh asky time that means the current In uh time stamp like let's say whenever you are executing your code it will save that time stamp that means the time date so it will save everything then after that it will also save the log message like like which message you want to show okay it will also save that so whenever I'll execute the code you
will get it okay what I have written here as of now just try to consider you have to mention this loging string here fine okay now you have to give a list of the file you Want to create so let's say left hand side I want to create some list of the folders and file so for this I have created a python list you can see so the name of the list is list of the files inside that I mentioned I want to create a folder you can see folder name is SRC okay so it
will create a folder here called SRC inside SRC I want to create a file called init.py that means underscore init.py and this is nothing but my Constructor file if you already Familiar with OP concept I think you know what is Constructor so Constructor is nothing but it's a special method it's a let's say the magic function it can execute automatically inside op but you can also create this is uh this actually Constructor as a python file okay so if you open up any kinds of n2n project now let me show you so if you go
to any kinds of end repository let's say if I open tensorflow tensorflow uh GitHub it's a open source Project so I can also see the their code implementation see they have also uh follow the modular approach see if I open any kinds of folder let's say if I open Java folder or let's say python folder now if you just go below you'll see this Constructor file is present here okay why because they have created so many folders you can see so many folders so many folders and inside that they have created different different python file
so this folder Would be considered as the local package that means they can also import some functionality from this folder let's say I want to UT uh let's say this one this particular function let's say this function input graph dep what I need to do I need to write so what I have to write I have to write from tensor FL dot python okay then dot this file and import I want to this I want to import this function okay that's how I need to import this file That means instead of writing like all the
function in just one file what I'm doing I'm creating a different different folder okay different different let's say component inside that I'm keeping those functionality okay and whenever I need it I'll be using it later on inside my endpoint okay this is called actually modular approach and to uh whenever you are uh like following this modular approach whenever you are creating this kinds of folder to keep your function in A file make sure you are creating this Constructor file because this folder would be considered as your local package okay because later on you have to
import it let's say from tensorflow do python import this function so whenever you will will be importing uh make sure your python interpreter will consider this is my local environment okay that's that means I can import something from that folder this is the IDE okay so that's why we are creating This Constructor file because SRC is my Lo folder inside that I'll be creating one file called helper. Pi okay and helper. Pi inside helper. Pi I'll be writing all the functionality let's say I want to inest the data I want to let's say extract the
information I want to download the hugging face model all the functionality I'll be writing inside helper. Pi okay this is the idea now apart from helper. PI I need another file so inside SRC I'll be creating another file called prompt okay prompt. Pi inside that I'm I'm going to write the prompt okay prompt whatever prompt actually I need the prompt let let's say whatever system prompt I will be using inside that I can write it here now I want to create a EnV file then I want to create a requirement. txt file oh sorry requirement
is already created I can remove it as of now I don't need it I want to get up setup.py file why setup. P file because I want to install these are the uh folder as my local package for this I need setup. Pi okay previously I think I already explained this part then app.py then I want to create a folder called research inside that I want to create a jupyter notebook file that means trials. i1b so inside that first of all we'll be doing the jupyter notebook experiment that means the entire project will be implementing
through jupit notebook then we'll try to Convert everything as our modular coding okay this is the idea so this is my list of the files and folder I need now I have written a simple python logic let me show you guys it's not a very fancy code code it's a simple python code I have written so this code it will Loop through this list one by one that means it will Loop through this list one by one see I'm looking through the list it will give you the file path the entire file path then you
are giving this path To the path function why you are giving to the path function because see here we're using Windows machine and windows by default use backwards slash I think you know see if I if I click here see windows by default use backward slash but which slash actually we're using here we are using forward slash okay but in Linux Mac actually uh they will be using uh this forward slash but in Windows you have to use backward slash so that's why to uh I overcome this Issue we we are using path uh actually
path library from path leap see how it will work so let me show you one example so first of all I will open up my python okay now here let me first of all import so from path Le okay import PA okay then let's define a path here so I'll just write p is equal to test let's say slash uh T do PI okay let's say this is My uh folder inside that I'm having one file now what I will do okay it has given me one error let me see the okay so test is
not defined that means I need to give inside double quotation because it's a path now so this should be a string type data H now fine okay now if I give this P to my path uh function now see what will happen it will automatically detect my operating system based on that it will Convert that part see it has given me Windows path it has automatically detected my operating system based on that it will convert this part to the windows path let's say you are using Linux you are using Mac automatically this path would be
converted to that operating system okay so that's why instead of manually giving that path as a backward SL what I can do I can give to the path function path will automatically take care that means we Can execute this project in any kinds of op system okay otherwise uh in some of the operating system you will might get some issue okay that's why we have to always handle these kinds of scenario got it so whenever you are designing your software make sure you are handling the these kinds of scenario and you are making your application
more robust that means in production it won't be failing whenever it is failing in the production this is not a robust application always You need to take care then after that what I'm doing I'm just splitting my folder name as well as the file name with the help of this actually function so inside oper system path. spit so what path. spit will do it will split your folder name and your file name that means inside this file directory it will uh give the folder name inside the file name it will give the file name only
okay so you can also execute and test so let me show you so let's say this is my Path now so this is my folder and this is my file now if I use first of all let me import operating system so if I give WS do path doit okay inside that I'll give the P now see if I hit enter it will give me one couple see so in the first item it should be my folder and the second item it should be my file okay and this is what actually I'm storing inside two
variable okay two variable I'm doing the tole unpacking here okay I have C then First of all I have to create the folder okay I have to create the folder and to create the folder I'm using w. make directories so first of all I'm checking whether this file directory it is empty or not if it is not empty that means definitely I have some file folder name here so I'm Crea the folder you can see then I'm performing the login operation I'm just doing the login in my terminal I'm telling creating a directory file directory
from the for the file of file Name okay this is the log message actually I'm printing in my terminal then after that I'm also creating the file that means folder creation is done then inside the folder I also need to create the file for this I have written this code actually first of all I have to check whether this file path is exist or not if it is not exist that means if it is not created I have to create it otherwise it is already created no I don't need to create it and I also
check The size of the file PA that means the it is zero or not let's see if it is not zero that means inside that file I have written some code so I don't need to delete that file again let's say you want to create again it only be replacing that file instead of that it will skip that file so that's why I also checking the size if size is empty I'm creating this file you can see and I'm doing the long information otherwise I'm telling this file is already exist okay So this is the
simple python code I have written now see the power of this script okay see the magic now left hand side see there is nothing there is no folder structure no file now if I execute this script now see what will happen I'll clear I'll simply execute this template. Pi file see template. Pi uh see automatically it will create the folder structure for me got it so it's just a one time effort guys you have to like write this code only one time and Later on you can use it uh every time let's say you want
to create another project copy this file write to execute it will automatically create a folder structure for you let's say in future I need some more more actually let's say uh folder or let a file let I need another file called test.py I'll just add it here make sure you are giving inside a string okay and you are giving a comma here because it's a list okay now see I'll execute it again it will Automatically create that for me see test. Pi is also created that's how you can add as much as file and folder
you need okay it will automatically create you don't need to manually create it here okay this is the Advan to use this template. P file guys okay I hope it is clear now you can check it out since resource this trial. i1b file is created now inside SRC init created helper created promp Created okay you can see init created helper created promp Createdb also created app created setup dop is created okay everything has created so now let's set up our project as my local package okay so for this I'll be writing the setup.py code and
this is a simple code guys I think previously I also shared so this is the setup.py code first of all give the project name that means uh the package name you want to give let's say gener project because if I show you currently in my terminal uh inside my environment This uh package is not present so I'll just write P list see genv project is not there okay so I have to install as a gener project okay for this we'll be using the setup.py now give the version author name email okay and find package we'll
try to find this underscore init.py and whenever this init.py is present it will consider this folder as my local package okay let me show you now to install this one you need to give one command inside Requirement. file called hypen e space do okay HP in is space this is the command now let me save now let me again install the requirements now see automatically it will install as my local package now see one EG info file would be created here and whenever this EG info file is created that means this uh project has been
set up as my local package let me show you now if I again do the P list operation now see Genera VI should be There see gener VI POS is there now it is my local package okay that means now I can import anything from this SRC itself got it so this is the entire idea now let's commit the code in my GitHub guys so for this I'm going to open up my G bash and here I'm going to write one commment called G ad then G commit Ty a m now let's give one commit
message let's say folder stuct are Added then let's do the push operation so get push origin main okay now it has pushed now if I want to check let's go back our GitHub now let's refresh now see it is pushed in my GitHub and readme is also updated now you can copy all the command and you can execute in your system okay I hope it is clear great now the first thing we'll be doing the notebook experiment okay first of all we'll be implementing The entire project in our jup notebook then we'll be converting as
our modular coding and after implementing jupyter notebook only you just need to copy paste operation okay that's it now first of all let me select the Kel so I already created one environment called medbot okay so let's select it now let's test whether this working or not so I'll give one print statement let's say print okay so this notebook should work now it is telling just try to install some IP 1B cardal package so let me install because I'm using jupyter notebook inside my vs code for this you need some additional package and it will
automatically install for you so it will take some time after that you will see that execution would be completed so let's wait okay guys so as you can see execution is completed and it is giving me okay message okay it's fine now one thing I just wanted to show you let's say if I show you my project working Directory right now let's say PWD so I'm inside this resarch folder okay inside resarch folder but I want to load my data and that my data would be present inside my data folder let's say I'll create a
folder called Data okay inside that I'm going to move the PDF I'm having let me move quick quickly so guys as you can see inside data folder I'm having the PDF okay PDF document now see whenever you are implementing any project make sure you are working in the Project directory okay that means your root project directory so this is my root project directory but currently I'm inside this resarch folder but if I want to go get back okay inside my project working directory so what I have to do I have to execute one more command
so let me import operating system package inside operting system you are having something called change directory okay let me show you so OS do CHD that mean change directory To hire one folder back so for this you can write do slash okay dot do slash means so do do slash means it will get back one folder back okay let me show you now if I execute now if I again show you the PWD project working directory now see I'm inside my root project folder itself okay so that's how actually you can check the folder location
first of all okay otherwise what will happen you will get like path issue sometimes you'll get let's a data Not found and so on okay so that to prevent this issue actually make sure you are working in the project folder folder directory always okay this is the Su I want to give you so first of all let's import some of the library here so here I'm importing uh Lang Chen from Len actually I'm importing P PDF loader and directory loader because I told you I will be using PDF documents and to load the PDF documents
I need this P PDF loader it is present inside a directory Okay that's why I've given directory loader then I also need to perform chunking operation and to perform the sing operation I can use recursive character text splitter okay I think you already familiar with these other thing now let me import all of them now here to load the PDF here I have written a function let me show you so this is the function I have written guys okay extract the data from the PDF itself so the function name is load PDF it will Take
the actually data directory okay data directory and it will only load the PDF documents okay it will only load the PDF document let's say you want to load the docs document txt document at that time you can give the extension here fine so with help of P PDF loader it will extract the information from the PDF itself okay and whatever uh let's information it will extract it will give give me as a documents okay you can see now let me show you the let me execute So it will return me extracted data so here I've
given the path okay so it should be data okay data directory now let me show you so if I execute now see it is extracting all the information from the PDF itself and it is having almost 700 actually pages so it will take some time so let's wait so guys as you can see my extraction is done now if I show you the extracted data see this is the entire Extracted data actually you got okay from all the pages it will show you the data now let me comment it as of now H now what
I have to do I have to uh perform the chunking operation because you remember see the architecture I have extracted the documents now I have to perform the chunking operation okay and to perform the chunking operation I will be using recursive text splitter so with that I have created another function here so split the data into text chunks So it will take the extracted data and it will perform the chunking operation you can see so here is the code so chunk size is 5 500 and chunk over live is 20 I have given and it
will perform the chunking operation and it will give you the chunk RS okay now let me show you so now if I execute the function and if I show you the length of the text chunks now see you got 7,20 chunks here guys 7,20 chunks you got just try to see how much data Actually you have in the PDF okay and this is enough information to create one medical chat B I believe that fine that's why I have taken this big data guys now see this is the inter CH actually I got now what I
have to do I have to use one embedding model to like perform the vector embedding and this embedding I'll be storing to the knowledge base that means inside my Pine con Vector database okay so now let's download one embedding model from the Hugging pH okay so here is the function I have written guys download H hugging face embedding model now for this I need to import this package first of all let me import so this is the package guys hugging face embedding I'm importing from the Lang chain embedding okay now let me execute H now
this function will automatically download one embedding model see this is the embedding model I want to use let me show you this embedding model so this is Present inside hugging phas so the name of the model is all mini LM L6 V2 so this is one actually embedding model and it will give you Vector dimensional 384 Dimension four dimensional Vector actually it will return you and this Dimension you need because whenever you are using pine con Vector database you need to also mention the dimension of the vector like like what is your vector dimension whenever
you are creating the class turn now whenever you are creating The index that time you have to mention this Dimension so that's why always try to remember whenever you are using pine con Vector database try to see what kinds of let's say embedding model you are using and that embedding model uh Returns what kinds of dimensional Vector just try to see the documentation you will al already get it fine now let me download the embedding model so let me execute this function I'll download the embedding Model see it is downloading from the hugging phase Hub
so it will take some time let's wait so guys you can see my has downloaded okay now let's test the model whether it is able to convert my sentence to Vector embedding or not so for this I have given one sentence here hello world and there is a function called embed query so it will automatically give you the vector embedding and I'm also checking the length whether it is giving me 384 Dimensional vector or not now see if I execute the program now see length is 384 and if you want to see the vector see
this is the complete Vector actually you got okay I hope it is clear now let me comment this line as of now got it now what I have to do guys I have to initialize my pine cone uh inside Pine con actually I have to store these are the vector okay the complete Vector actually I'll be generating from my TCH CHS so for this let's open up my Pine Con so if you want to open up the pine con just write Pine con. so this is the website and make sure you created One account guys
just do the sign up operation with your email address you can create our account so for me I already have one account let me login so guys as you can see this is my Pine con uh website okay this is the interface of the pine con and this is one of the amazing Vector database provider guys you will be loving a lot even I Personally like it so here what I have to do guys I have to create a index that means I have to create a cluster in that cluster actually I'll be saving all
of my Vector they have also given the quick uh start like how we can set up the pine cone how we can let's say stored the data see all the code example they have also given okay so we'll be following the same approach to store our embeddings to the Pine con but before that you have to collect one API key so How to collect the API key left hand side you can see one API key option is there just try to click here for me I already created one API key if you don't have just
try to create API key just give the name let's say I'll give test or you can give any name let's create the API key okay so this is the API key now just try to copy the API key guys and try to mention inside this.v file so I'll open the EnV file so let me paste it here and make sure you are giving one Key name here that means this is your Pine con API key okay that's it and make sure you are writing this thing in a string got it h now let me open
up my trials. IP andv now what I will do I'll just create a cluster here so let me go back to the database and don't share this key with anyone guys otherwise they will be also able to access your account okay I'll be removing it after this recording so for you don't need to share with anyone now Inside pine cone either you can manually create the index let's say you can click here create index let's create one index let's say I want to create uh let's say anything you can give test now Dimension let's say
38 4 the same Dimension I will give here the dimension actually I'm getting cosine Matrix just keep it you can also select different different Matrix like what kinds of let's say similarity Matrix it will apply whenever it will do the semantic SCE operation I Want to use cosign score now now just keep it everything as default you don't need to change the provider see AWS is the free one if you let's say sign up for the first time you can create one to two cluster here with the help of AWS cloud provider but if you
want to take this premium subscription that means you want to create a multiple class if you want to take some more space you can see see if you're using free plan you will get 2 GB storage here but if you have More than 2 GB data that time you have to take the premium subscription okay that time actually what you have to do you have to take their plan subscription plan let's see if I take this one uh Google Cloud now see see the price plan see this is the price plan actually they have so
with respect to that you can also purchase their uh let's say machine okay but as of now I'll be using the free free services only I don't need the let's say premium subscription of the Pine gon but if you are creating the actual project if you need more spaces that time you can do it so I'll be selecting the AWS cloud provider and Reon just keep it everything as default only just need to give the name okay name and a dimension that's it 384 okay now let's create the index so this is the manual approach
to create the index guys okay this is the manual approach see this index is getting created it is getting initialized and Once it is initialized see it would be connected okay see it is now connected okay now if you want to connect it you can execute this python code and you can insert your data all the conf code they have already given okay but I don't want to create this index manually I want to create through the python code so let me delete it so if you want to delete just click here and try to
delete the index so you have to give the name okay I'll be deleting because I Will be creating through the python code and how to like create through the python code guys I think they have already given one starter code remember so here you can give the uh index name like which index you want to create and create index Dimension you can give here serverless AWS region okay everything you can pass here so now let's use this approach to create the index so what I'll do I'll go back to my vs code and I already
prepared this code for you so Let me show you so this is the code only you just need to change the API key so let me change the API key the API key I have collected here let me copy and let me change it here or I think I can read it from the environment only so what I can do I can import the environment just a minute so let's import so from EnV import load EnV okay then I will load the EnV now simply I'll get this Pine con API key from my environment and
here it will save inside This variable now this variable actually I'll pass here okay so nobody won't be able to see my API key that time so this is the best approach guys you can also follow now you can give the index name let's say I give medical bot medical bot let's say this is my index name and here you have to give the dimension so make sure which uh actually U embedding model you are using and what is the dimension size 384 that means it's fine 384 keep everything as default Cosine s a US
one okay no need to change anything because we are using free uh cluster here fine now once you will execute this code okay once you will execute this code you will see that uh your U index would be automatically created in the pine let me show you so if I see if I execute discode now see what will happen so guys execution is done now if I get back to my pine cone and if I refresh it here see medical chatboard Should be created here let me show you see medical chatbot is created automatically it
has created and now it is active and running okay now what I can do I can uh convert our uh let's say chunks to the vector embedding and I can store inside the pine con Vector database so for this uh what I can do so for this first of all set this fine cone uh API key as an as your environment variable because uh whenever we'll be storing the Let's say data to the Pine con so Pine cor internally looks for this API key inside the environment variable okay so for this we are using operating
system Environ okay Environ and this is the key name guys I've giving and I'm setting my Pine con API key okay whatever API key I'm loading from myv file so it will set inside my environment variable right now okay so whenever I'll execute my Pine con code I don't need to like again and again pass it okay again and again I Don't need to pass it here this this is the advantage now see to store the vector embedding this is the Cod snippit you have to execute guys so you can see this is the code
snipp it let me comment here now I'm importing langen Pine con import Pine con Vector store okay now inside Pine con vectory store you have one function called from documents so first of all you have to give the documents that means the T chance the Tex CHS I have created remember so this is the text chunks I have created okay so this text chunks you have to give then you have to give the index name so what is the index name I've given guys I think remember this is the index name that means this index
name we it is already created that means inside this index it will store the data got it so make sure you check this other the variable now I need to give the embedding model so I think remember we Downloaded one embedding model OKAY from the hugging face out got it so this three parameter you have to pass here now see if I execute this line if I execute this line it will convert all of my data to the vector embedding and will restore to the uh Pine con vector database and how many chunks I got
guys I think remember I got 7,20 chunks that means 7,20 chunks Vector will be P stored inside my Pine con so it will take some time guys let's wait after Execution after executing this code I'll pause the video and once this Vector will get stored in my Pine con I will come back again see let me show you if I go to my Pine con right now and if you refresh the page here see all of the vector would be stored here even you can also see the count let me show you see as of
now 864 let's say chunks has been added now see th now see you can also see the visualization of the vector it's amazing Right so whenever you used to use local Vector DV it was not possible to visualize our Vector but here it is possible you can see you can also see the score that means um centic score like what is the score between this Vector this vector and this Vector got it this is one amazing visualization guys you will get in the pine cone and see this is the entire Vector representation of this entire
sentence okay entire text okay and even it is Also showing the page page number with in which page it is extracting the data and what is the PDF name as well okay every information you can see here even you can also perform the qud operation that means the way you perform the SQL qu you can also perform the quy operation here manually everything is possible but everything we'll be doing through the python code I'm not going to use the manual approach here now see the vector count guys so 4, 120 Vector has been stored now
let's wait uh once it has saved all my all of my Vector dor I'll come back and I'll show you so guys as you can see my execution is completed now if I get back to my pine cone and if I refresh see all of the vector it has stored here like more than 7,000 Vector it has stored inside my Pine con index okay now what I can do I can use this uh index as my knowledge base that means we'll be connecting my large language Model with this knowledge base and will be asking the
query and it will do the centic search operation here okay this is the idea now let's get back to my code editor now see I already created the index and I already stored my Vector there now I want to load my existing okay existing index if you want to load the IND existing index what you can do you can Use this code cipit okay see again I'm importing Pine con Vector from Lang and pine con and there is another function called from existing index okay from existing index you have to give the index name so
the index name I've given the same index name and you have to pass the embedding model the embedding model you have downloaded now see if you execute this line of code it will load the index that means whatever index you have created and all the Vector actually it will load and will give you the doc Source doc Source object okay this is on object doc Source object now you can also perform the retrieve operation here that me you can perform the similarity Source operation here let me show you one example there's a do as reter
search type similarity search keyword three that means it will give you three relevant answer let me show you so if I initialize my ret object and Now I will ask one question here so this is my medical data I will ask let's say what is acne okay acne so I think you know inside this PDF I'm having acne related dis dis also let me show you so if I press contrl F acne so guys as you can see this is the acne disease what is acne and all so you can see acne and AC related
actually treatment okay medicine they have suggested each and everything So I asking one question here so what is acne it should give me the relevant uh answer okay for the acne from my knowledge base now let me execute and let me show you now see if I show you this Rel uh retried docs it will give you three results actually three results even it is also showing the source okay from where it is it has referred now see different different actually uh acne related response it has given me okay Because I have set SAR keyword
is equal to three it will only give me three response okay but I don't need these kinds of output I need a complete let's say definition okay what is acne got it so for this what I have to do guys I I think you remember I have to integrate my large language model that means here we'll be working on user will give the query this query will go to B knowledge will give the rank results I'll be using the large language model so LM will Process this rank results as well as the qu and it
will give me the correct response okay so for this let's initialize my model so I already told you we'll be using openi model so openi so let's initialize my open model and I think you already know what is temperature and Max token so it is giving me one error okay it is giving me error because I haven't set my open API key so let's set the open API key I'll open up my environment so here is my API Key guys so make sure you create one API key and just try to paste here and similar
wise I also need to set this API ke as in my environment variable okay so let me execute this code so here what I can do I can duplicate this line and here I can write open API key so it will also give me my open API key then here whenever I'm setting as an environment here I can also change open a key open I key okay now let's set this One string expected non time my okay so I think I have to load it every uh again then I have to execute now it should
work now it is working fine now if I get back here now if I initialize it it should work see successfully have initialized my large language model now I have to create the complete chain for this let's import some Library previously I showed you now how to create the rag application there I think You remember we were importing this uh Library create T chain create stop document chain okay create promp chat promp template okay everything I already showed you there so we are importing the same code here now this is going to be my system
prompt so here I want to give this prompt in my language model OKAY large language model now I'm going to create the complete prompt promp template you can see this is the system prompt and whatever input user will give That means the input actually have given it will come here and it will give you the complete output now let me initialize now let's create my chain so create aop chain inside that you have to give the llm as well as the prompt and whatever object you will get in inside create Ral chain you have to
pass it as well as the r that means your vector restore the vector restore you created okay ret object now let's Initialize my chain now let's ask the question so here I will give what is acne okay what is acne sorry acne now see see the response guys see acne is a common skin disease characterized by pulse on the face chest and back it is caused by uh clogged force and become inflamed due to the oil dead skin cells okay and bacteria treatment and so on see that means it has given me the correct response
if you open the PDF now this book right now and If you read it here you'll see that it it has given me the correct response now see if I giving any other question which is not relevant about the PDF what will happen see now see if I give any other let's say question let's say I have given what is stats that means what is statistics it is telling me I don't know because here I have given the prompt now so if you if you're not getting the context that means here this question I'm asking
this is out of context that Means the data I have given to my model OKAY the knowledge B I have given uh to my model so this information is not available here so that's why it is telling I don't know okay so this is the complete custom uh actually system we have created like custom chatbot we have created which is only referring our custom data okay I hope it is clear so guys this is actually entire demo like that's actually be implementing the entire application now you can ask any Kinds of question let me show
you so you can take any other disas let me show you or let's take any other disas okay let's take these dis actually this one I'll copy and I'll come back here then I'll ask here now let's see see uh it is giving you the response growth hormones resulting the abnormal growth okay see that means this is the correct response actually I'm getting Now if you see the image so the same thing okay so you can see um this disease is a abnormal releases particular chemical from the blah blah blah okay so you can read
it so that means we are getting the correct response fine now let's try to convert this project as our end to end uh implementation right now that means we'll be also implementing the UI but before that I have to first of all create the modular coding okay so what I'll do guys I will first of all open up my helper dopy inside H.P whatever let's say uh utility related code I have written let's say loading the data then U creating the chunks then loading the embedding model all the function I'll move inside this ala. Pi
so let me open so inside that I'm going to import all of the library see I just copy pasted the code from my notebook only if you open the notebook the the same code actually I'm composting and why is Giving the error because below you can see I have to select my environment the correct environment let's say medical B now this error will be disappeared now see all the function I'll just copy paste here all the three function I have written now my load PDF then uh this one text split and this one my load
embedding model so here I kept all the function you can see because I want to import from here so these are the function I don't want to create inside My endpoint otherwise my code base would be like messy right that's why I am creating inside this particular file now let me save and inside prompt actually what I will do I'll give the system prompt whatever prompt actually have written so here is the prompt guys I think you remember I'll copy The Prompt and inside prompt. Pi I'm going to mention it here okay so from here
I'm going to open the prompt let's see in future use uh you Want to change the prompt you want to give any other prompt you can open this prom. P you can change here it will reflect in your code okay no need to open your actual code implementation that time F so let me close this at the tab as of now it's not required now yeah so helper is fine now what I will do I will create another file here and I will name it as store underscore Index uh do file okay file why because
let's say you want to update this book let's say you want to add some more information that time again you have to store all the embeddings to the knowledge base that means inside your Pine con yes or no right and to store it I again need to execute one like end point that means store index. Pi whenever you will execute it it will load the new information and it will store to the Vector database again fine so for this what I will do uh I'll prepare this one so see whatever code I have written now
here that means I extracted my U let's say documents I created the chunks downloaded theing model after that I pushed everything to my Vector database okay see all the let's say Vector database code I have written everything I need to write inside dat month so let me import all of the library so you can see I'm importing From SRC helper that means from SRC helper so inside helper I'm having three function load PDF then text split and download Face embeddings okay see I'm importing all of them Hing face embedding okay then I'm also importing Pine
con then Pine con Vector store then ladb and operating system first of all I to set my environment variable that means my credential okay openi sorry Pine con API credential then I will be loading my data extracting the data Creating the chunks then I'll be loading the embedding model see load file that means I'm loading the data so I have to give the data location so this is my data location then Tex spitter will run then download the embeding mod okay then after that I will initialize my Pine con code base the same code guys
if you open the the Jupiter notebook file the same code see Pine con initialization it will create the medical chat board okay Medical Board that means this bot it will create okay then uh it will actually create the index then it will store the embeddings okay so at the very first time you have to execute this line guys okay very first time you have to execute this file then uh whatever actually let's say data you are having it will uh do the vector conversion that means aing conversion then it will store to the vector database
okay now what I'll do I'll just Delete my previous embeddings whatever I created uh this uh cluster I want to delete it because I now I'm doing the modular coding now so I'll be executing everything from my modular coding itself now let me remove it now see it will delete it after sometimes you'll see that it won't be visible here now see make sure guys before launching your application first of all you need to execute this store index. Pi make sure this index should be created Other you won't be able to do the query operation
right so at the very first time you have to execute this store index. Pi okay then you will be launching your application that means app.py and this this one you have to only execute one time not multiple time multiple time when you will be executing whenever you will update the data let's you want to add any other data one more PDF that time you'll be executing this file okay otherwise just keep it as Default that's it now let me save now I think this is deleted now see there is nothing now what I'll do uh
let me show you how it will create everything so I'll open up my terminal let me clear now let's execute this store index. Pi python store index. Pi now let's execute see again it will perform all the operation whatever things we have done in the jupyter notebook so let's wait after uh sometimes you'll see that your uh index would be created and Inside index all the vector would be stored so guys as you can see my uh index is created now if I open it up inside that uh now it will store the embeddings one
by one all the embeddings it will save so guys as you can see my execution is completed and it's it has stored all of my Vector now that means my um this store index is working fine I'm successfully able to store my Vector now let's create the Endo okay that means my um I mean main application so User will uh start this application right now so here I already told you we'll be using flask okay so here I already told you we'll be using flask here and if I'm using flask I need a HTML and
CSS code to create the um one beautiful user interface so I already created one HTML and CSS code for you guys if you don't it's completely fine again I'm telling you there are so many template you will get over the Internet there is a website called Bootstrap previously I also told you I think bootstrap okay bootstrap website so from here you will get different different example even you can also see different different template it is having even if you search like chatbot HTML CSS free template free template you will get different different actually template see
different different template you can Also download the source code you see code is also visible you can also download even you will get some weite see different different chat template okay different different chat template you can use it okay it's up to you now what I will do I'll uh quickly just show you the HTML CSS code I have prepared for this I think you know if you're using flask you need to create one folder here called template okay templates so inside that You have to create a file called index. HTML okay now let me
show you my HTML code I have prepared or it's a chatbot what I can do I can name it as chat. HTML okay chat. HTML now this is the entire HTML code guys I prepared okay save it now I need to create another folder folder CSS called startic so these are the requirement folder folder um flask if you're using flask inside python you need to create this The folder inside startic I'll be creating a file called style do CSS okay now let me also paste the CSS code I copyed from the internet okay so this
is one basic template I created now whenever I will uh show you my application you will see the user interface whatever the user interface I created now let me create my endpoint app. Pi now inside app. Pi I'll be integrating all of the functionality I have prepared so far so first of all let Me all the requirement package all the required package actually one by one so here I'm using flask I have imported flask from the flask I imported all the required functionality you can see then imported uh download hugging face embedding model okay because
I need the embedding object to load my Vector restore I think you know that I need to load this not this actually Pine Con in now so where is the pine Cod index so here is the pine code index so I need to Load this index and to load this index what I need I need the let's say this one embedding object I think remember at the last I showed you loading existing index okay I think you need it okay I think you remember fine so that is why I have to U import it then
I'm importing you can see Pine code is store openi rable keyway that means whatever let's say I was loading here here these are the thing I'm loading everything fine then I also need to load my prompt so Prompt is available inside my prompt. Pi here so I I can UT it so simply I can UT just write prompt SRC do prompt importar that means I want to St everything inside that file okay that's it then I also need something called operating system so import voice that's it now let me initialize my flask to initialize the
flask you have to use this code go to the flask documentation you will get it now now let's set all of my Open and pine con environment okay for this I need to use the clo in environment package that's it then I will be downloading the embedding model then I will uh first of all initialize my embedding object then I will load my existing uh index from my Pine phone so let me show you see the same code from The Notebook I copied see so I'm loading the existing index I'm made B make sure name
is Correct medical B yeah it's correct then it will give me the retriever object then I'm loading the open a model then creating the promp template then after that I'm creating the chain okay now I have to create two route exactly first route would be my default route that means user will get the interface of my application chat. HTML would be rendered got it now second route I'll be doing for the chat operation that means user is giving any Query to the chatbot so that message actually will be accepting and this message I will give
to my llm okay that means you can see I'm doing the inbo operation okay so same inbo operation I'm performing let me show you so here is the invoc operation remember okay invok and whatever response I'm getting I'm printing it as well as I'm showing in in my uh that user interface okay that's it now let me execute my Application so I'm running on my Local Host port number 80 de is equal to r that means if you change anything it will automatically update everything so now let me show you my application so let's open
up my terminal so I'll just execute python app.py sorry it should be python spelling is not cor python okay now it should be running now see just give the allow permission now see your application is Running now let's open up our Google uh Google Chrome search for local host port number 8080 see guys this is the interface we have created and this is one amazing uh interface guys you can see amazing um chat template we have created uh because many people have requested me just try to create one front end part of our application as
well beautiful front end part and that's how we can create a beautiful front end part now here you can ask any kinds of Question let's say I will ask the same question I'll ask what is acting the acne is a common uh skin disease by characteristic by pimples on the face fine then I will ask another question let's say I got fever what should I do see now this is the response if you are experiencing high fever it is important to seek the medical attention Okay so tell me some medicine now see it is also
suggesting some medicine okay related fever now you Can ask any kinds of question from the let's say PDF itself any kinds of disease you can ask let's say this is the disease I want to ask I'll just copy and here I'll just write what is this dision all about see it has uh given me the response okay now you can read this uh I mean PDF and you can see that you can ask the medicine uh you can ask for the treatment anything you can ask so this is our chatbot guys this is our medical
Chatboard we have created and see it is also giving you the one beautiful user interface okay it is also returning now what we'll do we just try to also deploy this application guys but before that we'll be implementing some more end projects then after that I'll uh keep one uh dedicated end to end let's say deployment session there I'll show you how we can deploy these are the project on the cloud platform okay so yes guys I hope you liked it but before that what I Will do guys I will comment these are the code
in my GitHub so you can also comit from the vs code let's comit from here light um updated let's comit and VMV file would be automatically ignored by the git ignore okay because inside git ignore you have already mentioned it do git ignore would be ignored see would be ignored okay so it w be published inside my GitHub now if I open up my GitHub now if I refresh see all the code has been updated now I can write the further uh let's step as well in the read me file okay so I have added
all the step like after that what you have to do uh after that you can uh set your uh uh environment key then run store index then app.py then you just need to local up open up your local list okay now let me comment the changes as well so read me Updated and make sure guys you start this uh repository all of you start this repository you can also for so that you will have the reference okay now if I refresh now see readme is updated and you can see all the step I have also
added that's how you can also prepare your read me file so that so that if other people is referring your let's say GitHub they will be also able to set up this code okay in their system as well fine so yes guys this is the complete Implementation I hope you liked it so that's how we can create any kinds of end to end gener VI B project fine and don't need to worry I'll also show you the deployment even we'll be also discussing about LM Ops uh with the help of LMS also we'll be learning
how we can access any kind of Open Source large language model okay as a hosted open source large language model we can use it so yes guys this is all about from this video if you liked it so please uh Try to subscribe to the Channel Try to like it and try to share it video will be implementing another end to end projects called source code analysis so here we'll be using Lang chain Vector database large language model even I will also show you how we can create a Content application of the entire projects so
uh this is going to be completely end to end implementation guys so I think in our previous video I already showed you one NN implementation We implemented one medical chatbot so here we'll be following the same uh actually process to implement this project as well fine and what is source code analysis I'll tell you I'll show you the entire architecture diagram of this project then I will start the implementation so first of all what I will show you guys uh I will show you the GitHub repository uh creation after uh creating the GitHub repository we'll
be making all the setups then we'll Start the implementation so let's open up our GitHub and let's create one GitHub repository for this so guys as you can see I'm inside my GitHub so here I'm going to create a new repository and I'm going to name it as n to end source code analysis gener API fine so let's keep it as public repository I'll select the rme file and I'll will also take the G ignore wise python then let's take the license I'll Take MIT license now let's create the repository so after that I have
to clone it so I'll copy the link address I will open up my local folder and here I'm going to open up my git bash now let's write the command git clone and press the link that's it see it has cloned the repository okay now I want to go inside this repository I'll just write CD that means change directory to my end to end okay uh source code Analysis now I'm inside this folder now inside this folder let's open up our uh vs code so let me open up my vs code so here is my
V code so guys I think you remember uh the first thing we create a virtual environment okay for a new project so let's create a virtual environment for this you need to follow some of the commands so let me mention in the readme file so you can see these are the step we have to follow so first of all if you're using my repository First of all clone it then try to create an environment with the help of python 3.10 so that environment name is llm m so for me uh what I have done guys
I already created the environment in my system so I'll just activate the environment but for you if you don't have the environment just try to create it so let me show you my environment I'll just write cond activate LM see it is already there for me okay now the next thing what I have to do I have to Set up the requirements of the projects okay so for this I'm I'm going to create a file called requirement. txt inside that I'm going to mention all the requirements actually I need so you can see guys these
are the requirement I need I need open Tik Tok and chroma DB so Vector DB wise I'll be using chroma DB previously I used Pine con you can use any any of the vector DB it's up to you then Lang chain flask so the front end implementation wise I'll Be using flask framework then G python why G python I'll tell you whenever I'll show you the architecture diagram this part would be clear then uh python. ENB just to manage my credential okay and I'll be using open large language model you can also use JY you
can also use open source large language model but I told you if you're using open source large language model uh you should have good configuration machine okay otherwise you won't be able to execute And that part we will be learning in the LM Ops right now let me save everything and let me install all the requirements so for this you can execute this command so let's copy the command and here I'm going to execute see for me I already installed all the package that's why it's telling requirement satisfied but for you it will take some
time now the next thing what I do guys usually I create a template file I think remember project Template file so templates do p okay and previously I shared one template code with you I think remember so I'll just copy paste the same code so if you missed out my previous implementation guys please try to check the ENT and medical chatbot implementation so there I already explained this code okay this code how we we have written and all everything I explain in detail okay see I'm using the Same code same actually template code now what
will happen this code will automatically create the template for me so I already created requirement file I don't need it I'll remove it as of now okay now everything is fine so now if I execute this template. Pi file see uh it will create my folder St so let's see see it has created the fold structure and whatever log actually you have written it is showing in the terminal you can see okay so my folder structure Is created now what I will do uh I think remember first of all we perform the notebook experiment that
means we'll be implementing the entire projects you our jupyter notebook then we'll try to convert as our model coding okay so for this I'm going to open the resource folder inside that I'm have so inside resource folder I'm having one jupyter notebook file called trials. iyb so here first of all let's select my Kel so this is my environment okay now let's test Whether it's working or not so give okay message here yeah so everything is working fine now first of all let me show you the project architecture diagram like what we are going to
perform here so guys as you can see this is the entire uh project architecture so here we'll be basically performing question answer over the code base to understand how it works see what happens actually whenever let's say we Implement any kinds of Project okay let's say previous L I think you remember uh we implemented one project called n2n medical chatbot and this is the code base we have created right this is the entire repository of our project now let's say if anyone new to this field if actually he's not able to understand this project project
like how I have written let's say uh this person will open uh this project let's say he will open this helper. by and now he's not able to understand uh what is This function actually download hugging face embedding or let's say what is this function load uh PDF file and all okay let's say anything anything actually he's not able to understand about the project the project we have implemented he's not able to understand the code we have created right so what actually uh this uh application can do so if you give this uh GitHub repository
link uh to our software uh my software what it will do it will try to first of all Extract all the code from the repository then it will create a knowledge base okay and with the help of large langas model it will try to understand what you have written let's see now if you're asking any kinds of question let's say what is SRC inside SRC let's say you are having helper. Pi now let's say you are asking let's say what is Tex split function what it does so this model will try to read this code
and it will able to give the uh like answer this function Does this thing and that is how actually it is working okay that means it will give you the entire explanation of the code you have created okay so this is called actually source code analysis okay now I think You' got it what is the project actually we are implementing so project name is source code analysis so basically here we'll be performing question answer operation question answer operation over the codee base okay to understand how this code works Okay and this is the entire diagram
you can see guys let's say you are having a source code that means a complete GitHub repository okay that means the complete let's say I mean end to end implementation so first of all what you have to do uh you have to first of all uh let's say extract all of the source code okay then you will be creating um actually chunk okay different different chunks for this actually we'll be using something called Context a splitting okay now I think you can see here uh we'll be using something called context a splitting so here is
the context a splitting and how context a splitting works so let's say this is your entire source code and I think you know inside Python Programming we use functional programming op programming concept right so let's say you have created different different function I think I told you I had created different different function let's say this is one Function let's say this is one example def4 let's say this is the function inside that you are having some code now after doing this context splitting what it will do it will first of all extract all of the
source code then it will create different different CHS let's see inside this function you are having let's say 200 200 line of code okay 200 line of code now let's see your Chang size okay CH size let's say You have given uh or Let's say I can write 20,000 lines of code okay now let's see your Chun size you have defined 500 okay 500 that means what it will happen 500 okay 500 let's say what that means 500 these kinds of actually tokens would be considered as your one chunks now just try to consider how
many chunks it will create it will create different different chunks obviously let's say this is the first chunks this is the second chunks this is the third chunks Let's say it has created three chunks okay three chunks it has created now how your model will try to understand whether this chunks the entire chunks so it is related to this particular function let's say there would be so many function there would be let's say thousands of function but how your model will understand this three chunks I got this chance is related this function de F function
so what context splitting will do it will automatically tag so see It is automatically tagging so this is the chunks of Def F function okay let's you are having another function let's say def test okay this is that this is the function inside that you have subcode first of all it will perform this splitting that mean chunking operation you will get different different chunk okay then it will also perform the tagging operation let's say this is the tag of test function that means your model will try to Automatically understand now now your model will try
to understand okay this chunks actually I'm getting this is related to this particular function okay so that's how it will understand the context it will remember the context okay this is called actually context splitting and this context splitting we only perform whenever we are working with uh let's say code okay code data because previously we work with text Data okay text data is a different thing And code data is a different thing that's why you have to follow this thing okay now let me clear my uh huh screen now see what will happen you have
to create a vector store yes or no you can see after uh actually getting uh all the text okay all the chunks you using embedding model with embedding model you'll be creating vector and here we'll be using something called chroma DB okay chroma DB we'll be using okay then this this would be your knowledge base okay And you'll be connecting your large language model after that whatever uh question actually user will ask okay let's say this is the user whatever question user will ask it will take uh this uh this actually relevant answer as well
as the question it will try to understand based on that actually it will give you the actual response okay so this is the entire application we'll be creating and again you can see here we are implementing some called M rag Application rable augmented generation okay because here we are feeding our custom data custom data wise we are using Code data okay our uh let's say source code we are using here so I hope now it is clear now let's start the implementation guys so first of all let's Implement everything in our jupyter notebook then we'll
try to convert everything as our modular coding now let me clear my screen okay so first of all let's import all of the library I Need so here you can see I have imported all of the library actually I need in this implementation now the first Library I'm importing called uh from git import repo so I think you remember we installed one package called git python okay so what is git python git python is nothing but this is the package like git uh GitHub okay this is the GitHub actually package inside python that means with
the help of this package you can uh clone any kind of repository from The GitHub let's say the way actually cloned I think remember we open up our terminal we just WR one command get clone then we pasted that link right we pasted our repository link I think remember okay and it was was able to clone this repository in my system so now if I want to do with the help of Python Programming how we can do it for this we'll be using this package actually G python okay so the help of that we'll be
easily cloning any kinds Of repository from our GitHub because I told you now user will upload one URL okay that means GitHub URL project URL and from the URL actually it will automatically let's say clone the repository then it will extract the information that means your code base then it will perform the chunking operation embedding model uh stor the embedding in knowledge base then it will connect LM okay that's will be performing the entire operation got it And here I'm also importing another package called language because here we'll be using programming language so I'm going
to use Python Programming so with the help of language actually I'll set I'll try to use Python programming language here then generic loader language sper recursive character text beter okay so these are the actually additional package I'm also importing and I'll tell you whenever I will be using them I'll tell you each and Everything okay why I'm using it and what is the use of it now you can see I'm importing open AI embedding because I'm going to use open embedding model then chroma Vector database I already told you then we'll be creating our chain
okay the complete chain and here we'll be using actually memory conversation summary memory because I think you remember in our Lang chain uh series actually we learned how to create the memory okay like it will remember Your previous context so with the help of this conversation summary memory we can also create a memory buffer okay so this thing actually have imported so far now let me execute now see it should work fine because we have installed everything now the next thing what I will do guys I'll just create a folder here so mkd i r
so this is the command to create a folder I'll name it as test repo okay test repo now see if I show you my project working directory right Now so PWD project working directory I'm inside research folder so inside research folder it will create a folder called okay mkd is not recognized okay so it should be MK okay not Mac now it should work see now in the test folder it has created one folder called test repo inside test repo I'm going to clone the repository and to clone the repository guys you can use this
code snippit so here I'm using using this one Actually I think remember repo so the repo actually I have imported from git so inside that you have one function called clone form now you have to mention the URL like which repository you want to clone let's say I want to clone this medical chatbot repository I've given the entire link so this is the link I have given just copy the link and try to mention here you can give any kinds of URL any kinds of GitHub URL you can mention here that Repository would be clone
then after that you have to define the path in which path you want to clone I want to clone inside this particular path okay this folder now if I execute it will automatically clone this repository inside the test repo folder let me show you execution done now inside resarch test repo see it has cloned all the source code see all the source code you can see it has already grown okay amazing now the next thing we'll be Loading all the codes okay all the let's say codes all the data from this repository itself okay all
the python file will be loading that means we'll be extracting all the source code for this I'm going to use this code snippit so here is the Cod s guys so here I'm using generic loader from file system so I think remember we imported the generic loader from langen okay generic now here I'm giving the repo path that means where my repo is present Then I'm giving the globe that means it will open that means it will extract all the python file okay Pi file then parer you have to mention the language so here it
is the Python language okay that's why you have to give the Python language and Par threshold you just keep it this particular number 500 okay this is the default number you can keep it now it will give me the loader object okay now from the loader object I will load the documents now see it will give me the Entire documents right now if I show you see all the uh content is has extracted okay now if you want to see them now how many documents you got you can take the length length of documents you'll
see that seven documents actually I got now if you want to see any kinds of specific let's say documents you can print it now you can see this is the entire source code you can see it has uh extracted okay from my repository itself now the next thing is we'll be Performing this uh context uh ER splitting okay context ER splitting I already told you we'll be creating the chunks and this particular chunks will tag with the function name for this we'll be using this fun uh this particular Cod cipit let me show you this
is the Cod cipit guys recursive character test spitter you can use and inside that you only need to mention one parameter called language is equal to language. python so it will Automatically perform the context D splitting that means it will tag it will tag the chunks with respect to the function got it and Chun SI and chunks lab I have given now let me execute now I'll apply on top of my entire documents okay documents splitter split documents I'm passing my documents now it will give you the entire chunks so let's me show you so
this is the chunks I got if you want to see the length you can also see the length Okay See this is the length of my CHS got it then next thing what I have to do guys I have to uh actually use the embedding model okay uh to convert all my source code to the embedding representation and I will be storing to my knowledge base that means chrom ad for this I'll be using openi embedding model I think remember so let's set our openi environment uh API key here so this is my openi API
key so I already created and I already set here now let Me sa I'll go back to my notebook now first of all let's load this API key with the help ofb package so here you can see I'm loading it then I will set this API as an environment variable then I'm going to initialize my embedding model okay open embedding model that's it now let's initialize my chroma DB so I think you remember we imported already chroma from langin here so chroma we already imported you can see Vector chroma we imported now inside Chroma DB
uh we are passing our text that means the chunks actually I created and the embedding model as well as the pares directory that means it will create a directory that means database okay so instead of data you can give database okay DB okay DB so it will create a data database like local database inside that it will save all of your uh Bings and chroma DB session I have already taken in the YouTube channel you can check it Out how chroma DB works okay now let me create the vector store now see DB has created
okay inside resource and it is converting all the code to the embedding representation and it is saving inside DB now see it has created the local database okay see okay now what I'll do uh I'll just make it as now to activate this Vector DV you have to execute this line Vector db. parist okay now I'll be initializing my large language mod Model so here you can use any kinds of large language model you can also use gbt 4 I'll be using the default model like gbt 3.5 turbo so let's initialize my llm then I'll
be creating the memory that means uh that sum uh history memory conversation summary memory you can see I'm importing this library inside that I'm passing my large language model then memory key is equal to chat history this that means it will remember my previous context okay then return message is Equal to true so this is going to be my memory object right now then I'll be creating my final chain so for this you'll be using a conversation retrieval chain okay now inside that give the LM retriever Vector database okay and as retriever so search type
I'm giving M MMR keyword it will give me eight relevant answer and here I've given the memory okay now this is going to be my key object right now now let's ask one question so I'll prepare one question Here question is equal to let's say I'll go to the test repo inside that I'm having SRC folder inside help part I'm having one function let's say download face embedding I'll copy this function name and here I will ask what is hugging download hugging face embedding function okay let's see whether my model is able to explain or
not so this is my question so this question I'm going to pass inside my uh QA that means my chain Okay and it will give me and it will give me results and I'm going to only extract the answer okay from the result itself now let's see so guys you can see I got the answer uh the download hugging face embedding function downloads the embedding from the hugging face model uh so this is the model actually okay now see if I open it up uh see it is correctly identified this model actually we are trying
to download and which returns embedding of 304 Dimensions uh these embeddings are used for the pro processing and analyze text Data into context of the application that means correct information okay amazingly it has explained this function now you can give any other question so what I will do I'll call copy this question and let's ask about this function right now let's say I'll ask about load PDF file okay this function right now let's see what it will give Me so I'll copy the same code and here I can mention I think now let's see so
ignore this line okay this line is not required so this is your ACC response the load PDF function loads the PDF file from a specified directory using directory loader class uh which is a filter uh filter of PDF files and then it extract the documents and return the items amazing guys so that's how you can uh perform the keyway operation with Your code base right now if you don't understand any kinds of code if you don't understand any kinds of function just try to I mean give it here and try to ask it so it
will give you the entire explanation of your code base okay this is called actually source code analysis and this is one of the Amazing Project guys Amazing Project got it now this is the actually notebook experiment now let's try to convert this project as our end to end implementation okay so I'll Just do the copy paste operation from my notebook only because I think I showed you in my previous implementation I did the same thing so now what I will do guys I'll open up my SRC folder inside help part so whatever let's say code
I have written okay then let's individually I was loading the uh I was cloning the GitHub I was extracting the data I was performing the uh embedding model operation okay so everything I'll just write as a function inside helper. F okay so now let's do it this is how actually I implemented guys so I'm importing all of the library then first of all I created one function called repo injection it will take the repo URL and it will do the inje operation that means clone operation the same thing we are doing in our notebook only
got it then second function I have written called load repo the same code guys that loader generic loader I'm using here see it will take the repo path and it will Load the let's say I mean code and it will give you the documents after extracting then the third function I written called text splitter that means it will perform the context splitting okay the same thing then it it will will give you the uh embedding model okay it will first of all get the embedding model from the open air it will return you so instead
of actually writing the scripting I've just written as a function that's it now the next thing What I will do I'll uh add everything inside my endpoint which is nothing but my app.py because app. Pi user will execute and uh they will get one user interface and from there actually will they will upload uh let's say they will give the URL they will uh to the keyway operation okay so for this actually I already told you we'll be using flask and for the flask actually you need one time template and static file I think you
remember so let me show you I Already prepared one HTML and CSS code for you so guys as you can see this is the template file and inside that I'm having my index.html and this is the HTML code and again I refer the internet bootstrap website to implement this uh actually HTML and CSS code if you're not aware about it's completely fine you can use my template as it is okay now inside static folder I'm having the uh CSS that mean CSS code okay that mean style uh of our uh web interface and all and
to Execute this style. CSS I am using some JavaScript actually code you can see this is the JavaScript code again I got it from the bootstrap now see how your application will look like I'll tell you before that let me uh write this app.py code so inside app.py I'm going to import all of the library I need so you can see I'm imported all of the library like chroma DB load embeddings okay from the SRC hire because I need the embedding model right then uh repo in I'm also importing from the SRC then my chat
openi okay function then conversation summary okay conversation r chain so everything you can see in the notebook I'm importing all of them here now first of all what I have to do I have to initialize my flask and I will set the open environment key as my environment variable here that's it okay see now first of all I will load my embeddings okay from my Vector database you can see I'm loading the embedding From my Vector database so for this before that see before that what I have to do I have to first of all
generate the vector embedding let's say if user has given the URL so it will generate the embedding right it will uh create the embeddings and it will save inside my Vector database yes or no so for this I'm going to create another file called uh store _ index do file so previously in the medical chatboard implementation I Return the same file I think remember inside that I was doing the um actually embedding storing operation see so you can see repo in load URL so I'm importing everything I'm setting my open API key then load repo
that means see user will give the URL so this URL actually rep injection will load and it will load the let's say documents it will perform the text splitter embedding then this embedding will store inside my uh like chrom ADB chrom ADB will create Like local database inside that it will save everything okay I think remember now here I have commented this two line because I will be doing this thing with the help of user interface that means UI okay not manually actually so that means from the app. pi user will give the URL okay
from the user interface web application user will give the URL this URL I will receive here okay I'll receive here then I will execute this line of code okay so that's why I've Commented this line but if you want if you want to test what you can do you can uncomment it and you can execute this line you will see that it will work fine okay so as of now I've just commented out now let me save it now I'll go back to my app.py now let's create the llm chain so this is the chain
the same code I have copy pasted from my notebook guys I'm creating the llm creating the memory after that I'm creating the qm that's it then I'll be creating two Route then I'm going to Creator route actually from my flask so this is the default route first of all it will render the index.html whenever user will open the application okay then the second route I'll be creating um for the repo injection so whenever user will uh pass the URL pass the URL of the GitHub and they will submit so what will happen it will execute
this store index. Pi you can see I'm executing this command python store Index. piy and if you want to execute any python find that means if you want to execute any command you have to use system voice. system you can see I have imported operating system so voice. system and inside that just give the command it will execute this file that means what will happen whatever URL user will give okay so first of all it will perform the repo injection I think you remember we created the repo injection function repo would be ined in the
repo Folder and um if I go it back now see now it will execute this store index. Pi file that means this file now this code will be executed that means it will load the repo extract the documents create the chunks uh load them model then store all the embeddings to the knowledge base that means inside my chroma Vector database okay now I think you got it now see this is my second route now I'll be creating another last route this is for the chat Operation that means after ingesting the data creating the knowledge base
it will perform the chat operation user will give the masses okay and if message is equal to clear it will remove your let's say enre repository that let's say you want to clone another repository so your previous repository should be deleted okay otherwise what will happen it will take all I I'm in this space okay in my system that's why I'm removing it I'm doing the you can say rm. RF operation That means it will remove it so this is a Linux command that's why I'm executing inside w. system okay then after that whatever input
user is giving I'm getting the results I'm rendering the answer okay that's it now let's open up my application and let me show you now I think this part would be clear so I'll open up my terminal now let's uh execute my app.py so app.py will open it's running now let's go back now just search for local host port number 8080 Now see this is the interface of your application okay now here you have to give the URL so let's say I will give this URL I'll just give it here now see I'll just send
it now see in the back end what will happen it will inest that means it will clone this repository first of all after that it will create the embeddings and it will save this embedding in the vector database now see internally everything is executing if You go here now see repo it is cloning as well as DB would be also created now let's wait and let's try to see so guys as you can see my execution is done so if execution is done you will see this message actually response okay response so now if I
open it up now see my repo has invested and also my DB object is created that means my embeddings has already stored here now I can perform the keyway operation for This I'll be using this template right now and this is the same actually chatboard template I'm using from my medical chatboard okay now here you can give any kinds of question let's say I'll give what is I'll ask the same question again so I'll open the repo SRC helper let's say will ask this one download Hanging face like what is this function and all about
function let's See see this is the answer I got the download uh hugging face embedding function responsible for downloading the embedding model from the hugging face called sentence this model and specifi the 384 dimensional Vector great now I'll ask another question what is this function let's say text is spit see text spit function see text split function purposes to split the extracted uh data into smaller text juns This text chunks are useful for the processing analyzing the text in a smaller Moree manageable section great that means it is giving you the correct response now you
can ask any kinds of question regarding your source code okay so this is the entire implementation guys of our project uh yeah so I think you like it guys and in the next video we'll see the deployment like how we can deploy these kinds of projects okay to the uh like Cloud platform so for this SE we'll be using AWS so in AWS I'll show you the deployment part so before ending the session first of all let me comit the changes in my GitHub so I don't want to push the repo H let's say this
repo I don't want to push so what I can do see let me show you another thing let's say if I give clear message here it will remove the repo folder automatically see see repo folder would be deleted because you have written the code here I Think remember uh inside helper not helper I think inside app. Pi see here if you give the clear it will remove the repo folder okay now if I click here now see then I also don't want to push P this uh DB objects what I can do in the G
ignore I can mention this DB let's say DB would be IGN inside everything just try to ignore now let's try to see ah D is also okay another actually repo I'm having inside resource so let me also Mention resarch also inside resarch I'm having test repo so everything I want to ignore as well as I want to ignore the DB as well now I think okay research spelling is not correct so it should be like that okay research now let me save now see it is ignoring everything okay now let me push the changes so
I'll just write updated Commit and seeing the changes now if I get back and refresh here see all the source code has been updated now let me write the further step you can perform so here I have added the next step like you have to create the environment file inside that just say open a API key and just run app.py and uh you can execute your project okay that's it now let me also update this readme File readme updated now let's refresh here now see everything is mentioned okay all of the command I have shared
with you so before starting with our chain L discussion first of all I just wanted to tell you something uh like why we should use this chain lit and uh as a data scientist actually what are the difficulties actually we Face uh whenever we try to develop any kinds of application right so uh I think uh you have used something Called stream lead to create your uh this kind of web application even uh I have already showed you how we can create end to end like these kinds of Genera VI projects with the help of
stream lit and all uh so uh see whenever we are uh using streamlit it's a a initial interface right initial uh like uh demo of your application uh but nowadays actually this llm cames and people are creating lots of application on top of these kinds of llms and uh Also like there are lots of application there are lots of product in the market right now like chat GPT is there Google B is there right then Microsoft Bing chat is also there so if you have seen these kinds of product this kinds of application so you
will see like this application has like very beautiful user interface right so whenever let's say I'm creating any kinds of conversational agents okay let's say I'm creating any kind of chatbot so we should have one Like very beautiful uh user interface otherwise user might not get interest okay uh with your uh application so that is the main thing actually and gener it provides actually these kinds of beautiful user interface whenever you are trying to create any kinds of conversational agents and all so we can use this chain Le to build uh these kinds of llm
app super fast okay I'll tell you like why it is super fast because you will see uh using very short Line of code will be uh creating these kinds of application very easily right but if you're using streamlit I think you saw we need to write some more code there okay but if you're using chain lead so like more code doesn't required here so only using some line of code actually will be able to complete our entire projects okay so this is like very amazing package I was exploring and I thought let's uh also show
you this one so that actually whenever you are Learning these kinds of large language model generative AI so at the end you need to build some application right so you can use this kinds of chain lit package uh with your application and uh there is another reason actually you should use uh this chain lit you should explore this chain lit let's say you are building some projects for your client so you need to submit that projects to your client so uh you need to sub some good uh uh you can say output from your Side
otherwise your client might not get interest so at that time if you're using this kinds of chain lad interface okay chain lad package to create your apps and all so it would be very much realistic and it would be very much amazing uh to your client whenever they will look into your application so uh that is why actually whenever you are trying to build something you have implemented any kinds of PC and all so at a time just try to use these kinds of Chain lead package to create your apps so that actually you can
submit some good application to your client okay and they will be very much happy with you right with your work so guys uh this is the video agenda so first of all I will uh take you through the chain lad documentation then I will show you like how we can use this chain lad and we'll be building one basic chatboard here so I'm going to implement one food delivery chatboard so basically it will uh asks For the order okay it will suggest you the food then uh it will ask for the location and it will
like take your order and all so this kind of of chatbot actually will be implementing I think you have seen like zatu right so we'll be building some uh something like zatu uh you can build anything but this is the projects actually I prepared for you so that actually you can learn something right uh so I just wanted to like make you this thing as realistic so that uh Whenever you are building your own projects okay will get interest so yes guys this is all about now let's start with our CH discussion so guys as
you can see this is the chain lit documentation and uh chain lit is nothing but uh chain lit is an open Source python package that makes uh incredibly first to build chat GPT like uh application uh with your own business logic and data as you can see if I open chat GPT so here you will see uh it has One beautiful user interface okay that's why user more interested to uh use this kinds of chat GPT kinds of application now let's see if you give any kinds of message so it will take your message even
it will reply like that okay so we'll be building something like that so it would be like chat GPT like interface right then if you see here it has also official tutorial so you can also visit the tutorial they have given so from there actually you can also learn but I Will uh make this video very simple so that actually you can understand each and everything like uh uh you don't need to go through even the documentation I will uh discuss everything whatever things actually you need to explore okay and apart from that anything is
left and anything you want to explore from your side you can visit the documentation and learn okay uh yes guys this is the documentation as you can see and it has also integration with these kinds of Lang Chen then autogen lamb index okay you can either uh use open either use Lang chain either use lamb index anything you can use okay all the things has been integrated with that so in this video actually I'm going to use open AI API that means I'm going to use U GPT 3.5 turbo model to create this application because
I just want to uh show you quickly that's why I will be using this open AI model because there actually I will get one API key and Everything would be done for me right but you can also use open source large language model I I think I already taken some of the session like how we can use Google Pam to Lama then we have Lama 2 then we have Falcon we have Mr 7B anything you can use here you can also integrate Lang chain with that I have already showed you like lots of things with
the help of Lang chain and all so you can explore this thing right so yes guys this is the documentation now here They have already given some starter code like how we can start with this uh chain L and all so I'll show you like how we can launch this chain lead app and all okay but before that what I need to do I'll be creating one GitHub repository so here is my GitHub so let me create one GitHub repository I'll name it as uh llm app with chain lad you can give any kinds of
name so let's keep it as public I'll add rme file then G ignore wise I'll be using Python and License you can take any kinds of license so let's take MIT license then I will be creating my Repository now uh let's clone this uh repository in my local machine so I'll copy this link address and I will open my uh local machine and here uh I'll open my terminal and just write get clone and paste this link and clone it okay now I'll redirect to this folder llm apps with chain lit now I am inside
This uh folder now here I will launch my vs code so you can use any kinds of code editor it's completely fine so I already opened my vs code now here I'll be creating some of the um like folders and file then we'll be doing the environment setup because I need to install some of the libraries like let's say chain lit I need to install then I need to also install open and all okay so let's do it and uh see Like how we can use this kinds of chain lit so here uh I'm going
to create one for folder called SRC then inside SRC I'm going to create one file called underscore uncore init _ dop uh I think you already know why this Constructor is needed because I'm going to consider this SRC as my local package because here I'm going to writing some of the Python file and from that file I'll be importing some of the function and all okay that's why I need To create it I think I showed you in my Bend projects okay what is the use of this then uh inside that actually I'm going to
create uh two file uh one file I will keep my llm model and another file I will keep my uh system related configuration or you can also talk about this prompt okay you can uh also tell it as prompt so prompt dop yeah now uh outside I will create another uh file called app.py app.py then I also need something called Requirement. txt so [Music] requirements.txt then I also need something called setup.py because of this local package installation yeah so this thing is required now let me just write down all the requirements actually I'm going to
use so the first requirement actually I need something called uh chain lit so CH lit so this is the package you need to install then I need something called Open ni uh then I need something called python Dov I think you know why we need this EnV because I'll be keeping my uh like open credential inside this EnV file and using this python. EnV I will read that uh yeah then I also need something called hypen space dot because of this setup.py now let me create that file also EnV EnV so here uh I'm going
to paste my Open my API key so I already created this API key so you can also create your own API key okay don't use my one I will just remove it after this recording uh then uh I need to install these are the requirements so for this I need to create one virtual environment so I'll open my terminal and here I'll just write cond create typen in uh let's name it as uh chain or I can give uh yeah chain lit chain lead demo this is the environment Name and python version wise I'll be
using python 3.9 and H and Y now let's activate my invironment H now I'll installing the requirement so just WR P install henard requirement. txt okay it's giving error because because we haven't added uh like setup.py code here that is why it's throwing error so I think you remember so this code actually we usually add Inside our setup. PI so this code is a responsible uh to find this uh Constructor file and it will set up that folder as my local package okay and you here you can check the uh check the project name version
author name email address and anything right now let me install them so now I'll again execute my requirement. txt so it may take some time let's wait I'll come back when it is done So guys as you can see my installation is done now yeah so everything is done now let me select the environment I created here so it was uh chain L demo yeah that's it now first of all Let's test it whether everything is fine or not so uh I'll close these are the file first of all yeah so I'll visit this documentation
again uh this chain L documentation and if you click on in pure python so they have given one um Like code here so let me just copy so this code actually this is the Eco like if you send anything so it will response the same message okay as Eco so it's just for the testing purpose like uh whether we are able to launch our chain Le tap or not and it is working fine or not okay so first of all let me uh check it everything is fine or not then I'll uh add my logic
I'll add my chatbot logic here and here if you see this code will remain same and here you just need To write your logic okay uh here we'll be writing our custom logic and this code will remain same okay and this is the eobot currently now here uh what I need to do uh I'll open my terminal and if you visit the documentation so there telling first of all you need to write one command called chain lit chain lit init okay first of all you need to initialize the chain lit here so it will create
one folder here called do chain lit and inside that it will Contain all the configuration see it has created this chain lead inside that it has created another toml file this is the configuration file and it has all the configuration okay so here is the configuration so you can change this uh GitHub URL and all okay you can also change the project name here so here is the project name you can also change it so these kinds of configuration you can change so I'll just keep it as default No issue with that then um I'll
again open my terminal again I will run one command called chain lit run app.py okay now if I execute this command so you will see it will launch one uh server for me and this it is running on Local Host port number 8,000 so I'll give allow access and this is the default user interface okay so why this message is coming here welcome to chain lit and all so if you see here it Has created one uh MD file here automatically it has created one. MD file like markdown file and these are the content actually
it has automatically generated okay now let's see if I change anything here so let's say I will change welcome to uh my bot okay if I change anything here now if I go to my app and refresh see it has changed that means if you want to let's say print any kinds of massage okay whenever user will launch your uh Application okay so you can customize here so I already created this uh MD file I already made some customization there because I told you I'm going to implement something called uh uh this one jatu bot
okay so for this actually I created some of the content related jatu and again I generated these are the content with the help of chat GPT okay so let me show you like what are things I have done so here I'll just copy paste see this is the content I prepared hello Welcome to zato bot and here I'm just uh showing some of the item like P we have pizzas then pasta noodles then Asian Cuisines okay so these are the thing actually I have gener generated with the help of chat gbt so here you can
give anything any message whatever you feel good you can give it here and whatever application you are building with respect to that just add the content here now if I go back to my application and refresh it see guys now it has Changed to this content okay now let's see if I give any kind of message here hi so it will give me the same response like hi okay now if I open my chat GPT and if you just compare with the chat GPT okay now see it's looking like chat GPT only right so that's
why they're telling ch is nothing but it's a like open source python package and with the help of that you can create chat gbt like interface okay chat chat gbt like application so I love this chit a lot Because see only using this line of code okay only using this line of code like uh this only using just one function we are able to launch this kinds of user interface now let's see if you're building any kinds of product for your client so if you submit like that so they will be pretty much impressed by
your work okay so that is why actually we should always use some good user interface whenever we are delivering our application right now here you can give Any message like I am buy so it will give you the sa response okay because we have created the eobot how it is working so let me show you so they have created one function and this is The Decorator they're using this chain L decorator so this decorator is handling everything and again they're using asence and AIT because asence and AET is helping us to read this message okay
real time so it is always sensing our uh input and when whenever we are giving something it is Receiving and it is also sending the message to the front end right so this is the like code simple code now here we'll be writing our custom logic so basically we'll be building one uh Jato bot with the help of this um large language model I'll be using uh like open a GPT uh 3.5 turbo and uh here we'll be like chatting with our bot okay with the help of this Chann Le and here if you see
your app name is chatbot currently so you can also change this Name so I told you there is a configuration it has generated now if you go below so here is the option to change the name so by default it is chatbot so we can also give something called U uh chatbot for zato zato okay now if I refresh and again refresh here um I need to I think stop the execution so I'll open my Code now guys see this name has been changed uh chatbot for zato okay now I can uh close the previous
one and again it's the eobot now guys let's uh add our custom logic so so for this first of all I will prepare my llm that means large language model so this is the file so here first of all I'm going to import open AI so I think you know open has been updated uh now it is like a latest version so they have upgraded lots of thing and if you visit that Documentation currently so if I go to this open open.com and if I log in so this is the documentation now let's uh go
to any kinds of task and let's see how we need to import see this is the UT and we need to uh like initialize this client object okay but initially we just uh need to install uh like import open a okay that's how we we can do it but now now actually you need To import like that from open import open a okay so this is the updated version of openi and if you see here we need to upgrade it also okay if you are using previous one so you need to upgrade also okay so
this is the like uh requirement you always need to follow U you always need to upgrade with the documentation otherwise you might get some issue okay so let's import open a so from open import open AI okay so I need to make one client object here client open AI then here I will be initializing one function I'll create one function so I'll name this function as ask uh order okay ask order so uh this function will uh like uh this is going to be like main function so basically this function will uh give me the
response what whatever question actually user will ask so it Will take the masses first of all it will take the masses then uh it will also take the model so model wise model wise I'll be using something called GPT um 3.5 turbo 3.5 uh 3.5 turbo so this is the model I'm going to use you can use any kinds of model you can use dpt4 U anything you can use here I'll be using this model because this model charge would be a little bit less for me so that's why then I will also take uh
this Temperature value uh what is temperature I think you know in llm temperature is like uh uh like how much Randomness you want from your model okay if you're giving it to zero so that means it won't be giving any Randomness it won't be like more creative it will give be give you Auto like always stick output but if your setting is too close to one that means uh you're telling your model to take risk and it will also generat some Random output okay so that's why this temperature parameter is important also now this is
my function now here U they have already given like how to initialize this uh one if you see here so we need to use something called cent. chat completion. create so let me copy this line I'll copy this line then I will uh open my code and here I will add it now inside that I think you saw first of all it will take the model model Model is equal to model then uh it will also take something called masses okay masses so what is masses here so let me show you see message is nothing
but uh it's kinds of prompt actually you are giving uh basically you are giving some U like zero short or few short prompt here like you are telling this is the role okay role means system system means like you are the role okay like you as a uh chatbot this is your RO system and this Is the content so you are a helpful assistant okay basically it is uh I'm telling to my bot you are help assistant and whenever user okay user is giving some content okay let's say this is the content user has given
so you also need to uh reply as a assistant okay as you can see this is this is this should be your answer so that means we are giving some prompt here okay we are just uh giving some zero shot or free shot prompt here to my llm so that my llm can Understand what kinds of job actually I want to perform okay so I think you already learned this prompt engineering and all okay so this is very much important here so let's also Define this one so I already prepared one prompt here so let
me show you so you can also create your custom prompt there is no issue with that so in this prom. Pi I'll paste it out so this is going to be like uh very creative uh the more good prompt you actually will uh give to your llm The more good results actually you will get okay so this is the prompt actually I developed so system instruction as you can see you to bought an automated service to collect other orders from online restaurant okay if you see here so many like content I have written here like
so here I'm just trying to making my B understand like what uh it is going to do exactly and these are the actually item I also like pasted here so if anyone is asking like for the menu so uh It should return something right so these are the actually prompt I have developed from my site okay so this prompt actually I'm going to use now I think you remember you need to create one uh format as you can see roll as system assistant so that's kinds of format actually you need so for this actually I
already prepared so this is going to be the format as you can see massage roll system content and system instruction I need to import from my Prompt so here I'll just write from uh SRC SRC do uh hiab llm no sorry SRC do prompt then I need to import something called system instruction okay so here you here is my system instruction I'm importing this one now here I'm just telling RO system and content okay so here I'm giving the prompt to my llm as you can see now this message actually I'll will give it here
okay Yes now I also need to set the temperature so temperature is equal to temperature okay now uh this will also return something as you can see if you go below so this is the response format it will give you this kinds of format as a response it's a Json format response so from here actually I will um need to collect this content okay if you see here I need to uh collect this content okay content is the response okay actual response and these are some parameter it Will also give you so if you want
to like extract the content so you can copy this either you can also copy this one uh you can also copy this one okay this one also you can copy so let me copy this one and I'll open my part again and here I'll just write return uh response. Choice message. content I only need the content okay so this is my function I have developed so it should be messages instead of masses yeah now it's fine so now uh I will Write my final logic so I'll open my app. Pi so here first of all
I need to import my uh this function ask order so let me import so from SRC llm import uh ask order then I also need to import something called messages okay because here if you see we have also created this messages I also need this one here let me save it h then if I show you now here if you See here whenever I was discussing this thing so you need to prepare this message okay uh we have created for the system okay system that means we have given The Prompt now I also need to
create for the user okay like whatever uh input user will give okay so for this in this uh like a list actually if I show you uh we have a message okay this is the list actually and inside that we have a dictionary so I need to add some more like you can say u u i mean masses As you can see one by one so this is for the system then I also need to add for the user as well okay so I need to just append it so for this um what I need
to do so I'll open my app.py and here um you can appen like that so I'll just write message. appen and here I'm just initializing my user right now R this is user and this is the content what is the content user will give the message okay the message actually will send okay the message actually will send From here so this is going to be the content so it will uh store here okay it will store it here that means in this list because we are doing the append operation right so once it is done
I also need to uh give this message okay I also need to give this message uh to my uh bot so I'll just write ask order inside that it will take the messages okay then it will also give me something called Response it will give some some uh it will give me something called response okay now uh if you see here I also need to uh provide the response here if I show you I also need to give the assistant response as well okay as you can see assistant response as well so again I will
append so I whenever I will receive the response okay I will extract the content so uh as a assistant actually it will uh give me some response so this Response actually I'm appending because if you if I show you here this response should be my content okay as you can see it is returning the content and that content actually I'm storing now once it is done uh now I also need to send the response to my uh this uh chain lad user interface right so to do it actually if you see by default it is
sending the uh like Eco command only the message user is giving the same message actually it is uh sending so instead of that Actually I just need to send my uh response okay this is the response actually I need to send to my uh front end done uh yes so this is the logic only so this is a uh like very easy code we have written now here you can also integrate your open source llm you can use Pam 2 you can use Lama okay you can also use Falcon anything you can use I think
I already showed you how to use them and you can also integrate Lang CH with that okay so if you're using open Source llm so you can also use uh Lang chain okay with the help of Lang chain you can also do it and the same thing you can uh do the uh return here okay as a response now let's test it whether it's working or not so I'll go back to my so I'll go back to my uh chain lit then I will refresh yeah now let's say if I give any message let's say
hi so it is received High because I need to I think uh uh restart it so I'll open My uh open my terminal and I will close the execution if you press uh contr C it will close now now let's restart again so I have restarted now here is my new one now let's see if I give any message hi see it's giving hello uh welcome to zomato how I can assist you so here uh I can give uh my name is buy Uh and I am looking for some I'm looking for something I'm looking
something for dinner I can I'm looking something for dinner okay now it is telling hi buppy I'm here to help you find something delicious for dinner what type of kins you uh you are in mood for we have options like pizza pasta then Asian cins then uh Beverages and Indian cins let me Know you're preferred so I'll uh tell I'll will go for I will go for Indian food okay now see it is giving great choice buppy Indian kins is always uh treat and here is some option for the Indian menu now it is telling
butter chicken we have with none and even it is also giving the cost because we have added the promt here I think you remember then chicken T Tika Masala rice then palak pan Then uh chana masala vegetable Biryani Samosa Lai so I'll give I need this one chicken tikka masala with rice okay so I'll just write um add chicken tikka masala rice in the cart yeah so I have added chicken TI masal the cut uh is there anything else would you like to order or this is all for you uh no it's fine okay all
right here is the summary of your order so you Tak Masala now if You see this is the price now it is telling uh is that correct and would you like to pick up your order or have it delivered so I'll tell have it delivered delivered great we have uh Del uh we'll deliver your order to you could you please provide me your delivery address so I'll give some address here so I'll tell uh I'll tell my address my address is uh this is my address let's give this address okay so thanks for providing your
address buy your order will be Delivered uh to 17th floor uh Brigade signature Tower the total amount for your order $10.99 how would you like to make the payment uh we accept cash on delivery or online payment I'll go for cash on delivery cash on cash on delivery perfect you can uh pay in Cash When the delivery thank for uh the placing your order your chicken Tika Masala with rice will be delivered shortly if you have any other questions Or need further assistant feel free to ask enjoy your mail uh great thank you okay see
guys uh we have implemented one amazing food delivery B uh with the help of this chain lead and now see the interface guys it's like very professional very amazing right very amazing now if you deliver this project to someone else they will be going mad right like whatever things actually you have developed right now now uh what I will do quickly I'll just Uh now uh what I will do I will just quickly commit the changes to my GitHub so that I see you can get the code so I'll just write uh completed comment done
now if I go back to my gethub and refresh yeah so you have committed so yes guys uh this was was the demo of the CH lit I think uh you enjoyed this package a lot and even I'm also enjoying this package a lot okay I'm just trying To use this uh Trend L package to build my llm apps and all and see guys like by using some uh like very less line of code actually we are able to build this kinds of awesome application and it's like very fast right uh now we'll try to
uh see the deployment part like how we can deploy these kinds of projects on the cloud platform so I think you already know we implemented so many NN project right so implemented medical chatboard then source SC analysis they Then we also implemented something called J to chatbot right so different different application we imp implemented so far now in this video I'll will show you how we can deploy these kinds of application on the cloud platform that means here we'll be deploying our application on the AWS Cloud if you want to learn any other Cloud deployment
it is also fine see the process will remain same only the cloud functionality would be different so uh I'll try to create Some more video on the deployment like I'll also show you how we can deploy these kinds of project on the gcp cloud on the let's say the aure Cloud but first of all let's try to see how we can deploy this project on the AWS cloud and see this is not going to be a simple deployment so here we'll be doing continuous integration and continuous delivery or deployment that means cicd so I think
you have heard of this uh particular term called cicd right so We'll be learning how we can deploy uh this project as a cicd I think you already know how to deploy any machine learning project de learning project as cicd but we haven't seen how we can deploy any kinds of genv based project as a cicd right so that's why make sure you are uh watching this video till the end so here I'm going to show you each and every step you have to follow and trust me if you follow this step guys you can
use this step to deploy any Kinds of let's say gen VI project okay and this is going to be production grade deployment okay so here I'm going to use different different kinds of service from the AWS even I I I'll be also using Docker that means first of all we'll be dockerizing the entire source code then we'll be deploying this application on the cloud platform okay so make sure you are watching this video till the end so before starting the deployment guys first of all let me show you what Exactly we're going to perform here
what is cicd exactly then I will also discuss let's say what are the tools and Technology will be using here what are the services we'll be using here then I will start with the deployment and make sure you have the AWS account if you don't have the awx account guys so you won't be able to deploy this project so first of all try to create one account if you're creating for the first time you will get $300 free credit okay and This is enough for you so try to create an account guys then you can
start with the deployment with me so let's open up our Blackboard and try to to see the architecture overview so guys as I already told you here we'll be doing something called cicd deployment right and I already told you what is uh cicd full form it's continuous integration continuous delivery or deployment right so inside continuous integration continuous delivery what happens let's Say you are the developer okay let's say you are the developer so what is your task your task is to develop a project in the development let's say environment so development environment means like your
local system let's say you are using your laptop or computer let's say local computer okay local computer fine then what we are doing I think you remember we are committing This code to the GitHub right we are doing the code management I think you remember that means that means whenever I was adding some new feature I was pushing this code to the GitHub yes or no okay with the help of git client we are pushing the code in the GitHub GitHub server now what happens actually let's say after deployment so this is called actually development
server or I can write uh this is actually development Environment okay development environment so after implementing this project what we have to do we have to deploy this project yes or no let's say um somehow you have deployed this project on the AWS Cloud let's say this is your AWS Cloud fine so let's say you have deployed this project to the a cloud manually manually you you just created a let's say instance there you create you just took a machine there K2 machine and you manually deployed this Project now it will give you some endpoint
okay endpoint so with the help of this endpoint any of the user okay user can access your application now let's say after 4 month or let's say 6 month you want to add some more features in this let's say uh application let's say you are deploying medical chatboard the medic chat but we implemented I think you remember right let's say you want to add some more data You want to add some more knowledge base and you want to add some more features in this application then what you have to do you have to again develop
this let's say features in your code then what you will be doing again you'll be deploying this application to the cloud now just try to see whenever you are deploying the project for the second time let's say this is the first time you have deployed then you are trying to reply for the second time then what you Have to do first of all you have to stop this application in thews okay stop this application then you will be uploading your updated code then this code will reflect to the endpoint then user will able to access
that now let's say in between whenever you stop this AWS server let's say your uh application let's say it took 3 hours it took 3 hours to change the entire source code that means uh change the entire features okay of your application so what will Happen 3 hours user won't be able to access your application so they will come your website and they will see server error okay server error actually they will get so if user is getting these kinds of experience so definitely it would be a negative let's say effect okay on your application
so next time actually they are not going to use your application yes or no let's say if CH GPT is down for the uh let's say 3 hours definitely people will move to the Google b or any other let's say software whatever we are having okay so now see chat chat GPT is also updating their let's say application day by day but did you ever observe this server is down no you are not seeing the servers is down but still they're able to make the changes in their application how because they are following something called
cicd approach continuous integration continuous delivery that means this application is keep on running but in The back end they're pushing their source code they're pushing their let's say new features and this feature is automatically getting updated okay so this is collect cicd that means you are not going to deploy this application manual instead of that what you have to do you have to follow the cicd that means what will happen let's say you have changed something in your code you will push the code to the GitHub okay GitHub will automatically uh let's say Deploy your
code to the AWS Cloud it will automatically push your code to the AWS cloud and your endpoint would be automatically updated okay automatically updated so that if user is using your application okay they won't be filling any kinds of let's say server down issue okay server down issue actually they won't be failing got it so this is what actually uh we have to do that means we'll be creating the entire pipeline entire let's say cicd pipeline so we'll Be just pushing the code in our GTH iub and GitHub will automatically trigger uh this uh action
and my code will updated to the AWS cloud and AWS will update the end point okay now see the automated process we'll be doing now whenever we we'll let's say push our code to the GitHub GitHub will automatically trigger how it will trigger for this you have to use some cicd tool okay cicd tool cicd automation tool so here there are different kinds of cicd tool so the First to you can use something called GitHub action okay GitHub action you can use then you can use something called Jenkins okay genkins then you can use something
called Circle C Circle CI so these actually three famous tool actually we are having in the market right now so people are using more this GitHub action because GitHub action you don't need to set up anything it is already set up everything in the GitHub but if you're using genkins and circle CI you have to set up this server manually okay so here we'll be using GitHub action because it is already inbuilt with the GitHub we don't need to set up anything going forward I will also show you how we can uh let's say use
genin Circle CI these are the services as well fine so yes guys this is the complete uh high level architecture of our deployment now now let's discuss what the services actually Will be using for the deployment see here the first Services actually I'm going to use uh called Docker so I think you already know Docker so Docker is a containerization service so with the help of that you can containerize your code that means let's say you are having a source code you can perform the containerization that means it will create a Docker image for you
okay so this image you can let's say keep anywh let you can keep in the Docker Hub you can keep let's say in the ECR that mean elastic container regist from there you can pull the image and you can execute this image okay so if you using Docker so you won't be getting any kinds of setup issue this is the main advantage here then we'll be using something called ECR here ECR ECR means elastic container resist so this is the service from the AWS okay AWS giving the service so here you can store any kinds
of Docker image it is the same like your Docker Hub okay but it is the uh it is the service from the AWS you can store the private image there but in dockerhub actually the all the image be public this is the difference only now we'll be using another service called ec2 okay ec2 is a virtual machine service from the AWS again it is from the AWS okay so this will give you virtual machine there you can buy let's say CPU based machine GPU based machine okay so all kinds of configuration would be there so
here We're using a local computer so there we'll be getting a cloud computer okay this is the difference only then as a cicd tool I'll be using something GitHub action as I already told you okay we'll be using GitHub action for the cicd cicd management fine so yes Guys these are the services as of now I'm going to use so let me show you the deployment step will be following for this what I have done guys I think remember we Implemented one medical chatbot this is the code and this code would be also available uh
in your resources section even you will also get this code in my GitHub so guys this is my GitHub profile just search entb in the Google you will see my GitHub and here I have created one repository called uh Inn medical chatbot so I'm going to commit this code here so you will get all the source code from this repository itself okay and even I will also add the deployment step In the readme file so that is what actually I have written you can see the entire deployment step I have written here now this is
the cicd deployment will be following so first of all what I have to do we have to uh let's say create a im user uh then we'll be uh creating the2 machine easier okay then we'll be see this is the deployment step as of now just try to follow the step I'm going to show you first of all we'll be building our uh so here first of all We'll be building the docker image of our source code that means Docker image we'll be implementing then we'll be pushing this Docker image to the e that means
elastic container to store my Docker image then we'll be pulling this image from uh from ECR in the ec2 that means we'll be pulling that image in the ec2 machine in our virtual computer then we'll be launching this Docker image in my E2 machine and from there we'll be creating an endpoint and that endpoint User will be able to access okay and these are the policy I have to give whenever I will be creating the IM am user fine so this is the entire step guys I have uh written even I have also given all
the command you need to execute here now let's start the deployment guys but before that you just need to prepare some of the files the first file you have to prepare the docker file so this file will help you to containerize entire your application That means it will perform the dockerization and these are the docker command you have to write so I think remember we are using python 3.10 okay this python version that's why I've taken python 3.10 slim Buster image then working directory it will create a working directory called app inside that it will
copy all the source code then it will install the requirements okay whatever requirements actually we are having here it then it will execute this Python 3 app.py command so that means my endpoint would be executed okay okay my application would be running and make sure uh before actually deployment you just store your index okay in the pine cone so this is the pine con guys it is running and all the vector is already stored here you can see 7,20 uh uh actually record is already stored here so I already executed this file I think remember
store index so it will create all the let's say Vector embedding and It will store in the pine con okay Pine con Vector database so in this project I was using pine con I think remember okay so this should be available here got it so once it is available now what have to do guys then you can create another file called Docker ignore inside Docker ignore you can mention like some of the file which is not required let's say here this file is not required let's say uh I think this file is not required genv
project do. EG info okay so Whenever it will perform the let's say containerization that means uh dockerization so I don't need this uh folder that time what you can do you can mention this folder name in the doer Docker ignore file it is the same thing as your git ignore so it is already mentioned inside inside my git ignore that's why I haven't mention inside my Docker ignore Docker ignot is completely empty as of now I need all the file that's why I haven't keep anything here Now let's say if you want to ignore any
kind of file which is not required that time you can store this other file inside Docker ignore okay that's the thing now the next things you have to create you have to create one folder called do GitHub okay inside that you have to create one folder called workflows inside that you have to create one yl file you can give any name here I have given C.L okay now see this is the complete EML file so this EML file will Help me to do the cic deployment okay that means uh here it will be using something
called GitHub action okay and GitHub action need this yl file so here all the command is already written so first of all you will see that it will perform continuous integration inside continuous integration it will authenticate with my AWS account okay then it will log with my ECR then whatever let's say Docker image actually you will be building you will push that Image to the ECR okay that means elastic container then your continuous deployment will start again it will authenticate with AWS account then it will pull that let's say uh image from your ECR to
ec2 machine then it will execute it here and here all the environment variable I have set you can see hypen means this is the environment variable see AWS access key aw secret key I'll tell you how to generate these are the key then Pine con API key open API key everything I have set here okay so these are the thing we'll be reading from the secret okay GitHub secret I'll tell you how to create the GitHub secret as well so as of now just try to remember this is the ml file you have to use
for the cicd and this EML file you can use as it is no need to change anything in your other other project as well okay that's the thing now let's open up our AWS account so guys this is my AWS account I already logged in with My AWS account so here so first of all what you have to do you have to create a im am user so let's create IM am user so I am so the first thing we'll be creating one user here so let's create one user I'll create a user so let's
say usern name is I'll give uh medical medical B you can give any name it's up to you so let's click on next now you have to give policies okay like what are the policy you want to give in this user I don't want to give my entire Policy that means all the services I I don't want to give the access so I'll be giving some specific access so Access wise I already WR in the readme file let me show you so these are the access I'll be providing my ec2 uh container regist full access
and ec2 full access okay these are the thing we'll be giving so here just search it so this is the ECR okay ECR elastic Container regist full service here I'm giving so let's select and after that Again I can delete it then I'll try to copy the next service this is the service okay this is the E2 machine service so again starts here and select the E2 machine and click on the next now see that I have given this two service okay next thing just create the user so right hand side you can see one
button is there create user just click on create user so guys as you can see my user has created now I'll click on the user then Here you will get one option security credential just click here then you have to create a access key so uh click on create access key select command interface and I understand this above recommendation and click on the next so there is a next button just click on the next now create the access key here now see guys this is your access key and secret access key try to download as
a CSV file so let's download so and just keep it with you okay this thing I need Later on okay and no need to share this key with anyone otherwise they will also access your uh AWS account okay so here I'm showing because after the recording I'll delete everything got it so my IM am creation is done now I'll go back to the home page now the next thing what I have to do guys I have to create a ECR repository to store my Docker image so let's search for ECR so just search for ECR
that means elastic contain register just open It up and make sure you check your region in which region you are working see I'm working uh in Mumbai AP South one if you're in other region just try to note it down okay you have to note it down this this syntax actually AP South one then CA Central one if you let's say in North Virginia you have to give us East one so just try to remember these are the tag okay so I'm inside Mumbai that means AP sou one now I'll be creating a elastic container
regist so Let's create uh so let's click there now you have to give the name let's say I'll give medical chatbot you can give any name it's up to you everything keep it as default now click on next now see my repo is created now what you have to do just copy the URI and uh just try to paste it as of now here because this thing I need later on just for reference I'm saving it here okay H that's it now what I will do I Will go back again now the next thing I
have to do I have to create my ec2 machine then I have to set up uh these are the requirements okay that means I have to set up the docker there then I will be also updating that machine so let's create my ec2 machine right now so search for ec2 here ec2 instance so this is the E2 service Now launch the instance now select the UB to machine and what you have to do give the name name of the machine let's say Medical bot machine you can give anything then um see instance type you have
to take at least 8 GB Ram 8 GB Ram just take 8 GB Ram I'll be taking T2 large okay this machine and then select the key value here if you don't have the key value here you can also create it so let's say I'll give medical so it will create one rsf file so guys you can see after creating and download you will it will give you one Pam file so this Pam file you can use Let let's say if you're using any third party tool to connect with your uh ec2 machine let's say
py or mobile ex stream that time you can use this Pam file but I'm going to launch my let's say machine in my uh second tab with the help of AWS only I don't need this P file okay so now I'll select this two option https and HTTP keep it everything as default configuration storage at least take let's say 30gb and launch the instance right Now so here you can see click on view all instance then see my machine is running so if Stratus is running now click on the machine ID now here you have
to connect this machine so click on the connect button now I'll be selecting the E2 connector and let's connect the machine so it will create a new tab for me so here I will get one terminal block terminal so we we have to use this terminal to set up everything here and this is the production server guys okay That's how your production server looks like see this is the production server let me Zoom now let me clear see here you won't be getting any kinds of UI interf okay that means the way we are using
our computer it is having UI but in the production server you won't be getting any of UI you have to work with the terminal always and this is the Linux terminal I got this is the Linux machine guys so that's why if you know any kinds of Linux command it will help You a lot got it so here first of all I have to update this machine for this you have to execute this are the command one by one so let's copy all the command one by one okay so this is done now I'll copy
the next command and just try to past here now give yes permission and press enter okay so execution is completed now I have to set up the dogger in the E2 machine so this is the command you have To run copy the next one and execute then I will execute this one then the last command so we have to add the docker in the sudo group okay now if it is running on not so what you can do you can execute one command Docker ipen ipen version so if it is showing the version that means
Docker is installed successfully now uh we have to set up our self hosted Runner ec2 as our self hosted Runner that means we have to Connect our GitHub okay you can see this is the GitHub now this is our project GitHub so this GitHub I have to connect with my uh AWS that means whenever user will uh sorry whenever let's say developer will push the code in the GitHub it will automatically get updated to the AWS okay AWS Cloud so for this what I have to do I have to go to my uh project make
sure you are you committed your repository in your GitHub and go to the settings okay go to the settings now Here left hand side you will see one option call Action just click on the action click on the runners now create a new self hosted Runner now select the Linux machine and you have to execute these are the commands so let's copy this command one by one and let me execute here so it will automatically make the connection with your uh GitHub let me show you then the third Command then fourth command now you have
to go to the configuration section and copy this command and execute it here now see GitHub action it will automatically connect now it will uh see it has connected to the GitHub now it is asking for enter the name of the runner group I don't want to give anything press enter now it is asking enter the name of the runner so here you have to give self hypen hosted okay self hen Hosted this is the thing you have to give if you're giving another thing it won't be working okay make sure you're giving self hen
noer now let's press enter now keep it everything in default just press enter again then again press enter now see it is done now the last command we have to execute this one and you can ignore this one it is not required now see it would be connected with my GitHub see connected to the GitHub listening for the jobs okay now if I show you if I come back and now if I go to the Runners let's say I'll go to the runners now see status should be ideal you can see it is connected okay
now the next thing what I have to do I have to set up all the GitHub Secrets okay now let's go back to the secret so secret variable click on the Action Now new repository secret now give the secret uh ID okay so The first ID I have to give AWS access key ID just copy and give the key name here and why you will get the value I think remember we downloaded one CSV file so let me open this CSV file I will open in a notepad++ and this is my a access keyl copy
and here let me paste it then the next one I have to give secret access key ID copy and create another new repository secret and give it here and in the CSV file you will get The next one see it is comma separated so before the comma this is the AWS kid uh access kid and after the command is the secret access K so let me copy the secret ACC and I will past it here okay now the next thing you have to give AWS default region so region I already told you I'm inside uh
Mumbai that means AP South one so I think remember I inside AP South one see AP South one you have to write it so let's Write AP ien South okay ien one AP South one let's try to check AP South one a South one okay fine now let me add it now the next thing I have to add my ECR repo so e rep wise I think you remember we copied one URL so just try to get this name only okay after.com just try to get the name and try to mention here then the next
thing I have to set my um Pine con API key so let's copy and add it here and I think remember in my DOT EnV file I had my Pine API let's copy and this is not available inside my GitHub that's why I'm reading from the GitHub secret okay this is the idea now I also need to add my openi secret okay that means open IP key the last one you can see open IP key let's add it so let's copy the open API and I'm going to paste it now let's add the Secret okay
now see everything is added now let me see anything is left no everything I have added now what I have to do I have to push the changes okay but before that let me get back to my project now I'll push the changes so just try to comit to comit just open up your uh G bash and try to write this command get add space dot then get comment typen in let's say CI CD Edit then get push origin main now if I go back to my GitHub now see if I refresh here so it
will start running my action now just go to the action now see my uh action is running see first of all it is running the uh continuous integration so inside continuous integration first of all it will authenticate with my WS ECR build the docker image push it to the ECR okay everything will happen see automatically It's just a one time setup guys next time you don't need to set up it you just need to write the code and push it to the GitHub automatically it will uh push to the uh let's say server okay now
see so let's wait uh once this execution is completed I will come back so guys as you can see my continuous integration is completed now it is running the continuous deployment now what it will do it will again authenticate with myws cloud and it will pull that image from The e to E2 machine and if you go to the El lasting container right now and if you refresh here you will see the image okay see doer image has been uploaded here now see it is running the uh deployment now see it is pulling the image
so once it is pull then uh we're doing the port mapping uh we'll create the endpoint and uh we can access our application let me show you so guys as you can see my execution is completed that means both uh integration and Deployment is executed if you're not getting added that means congratulation it's done now I will go back to my ec2 machine okay now here in the2 machine uh you will get one public IP address just let me show you so this is the public IP address just try to copy and just try to
open it okay now see if I open it up so it won't be opening because uh see my application is running on port number 8080 I think remember okay so by default port number 8080 is not set so I have to Do the port mapping so what I have to do I'll go back go to the instance uh this is my instance then I'll go to the security okay now here you will see one option called uh Security Group just click here now on top you'll see edit and bound rules okay just try to click
click here now I'll just add the rules custom TCP port number just give 8080 uh just sck 00 0 okay and here you will see save rules okay just click on The save rules that's it okay so rules has been added now I'll go to the instance again I'll open it up now I'll copy the public IP address again and here let me paste now you have to give clone port number 8080 now see if I hit enter my application should be running see guys this is my chatbot is running now if I give any
message let's say uh what is acne see guys it is giving me the Response now see my application is live right now so it is already deployed now user can use my applications now what you can do can buy a domain okay your custom domain and you can publish this application to the audience okay now for the second time let's say you changed anything in the code let's say you added some more function what you have to do only just need to commit the changes it will automatically get updated here fine so no need to
set up anything from Scratch it just a one time setup once setup is completed just try to commit the changes it will automatically running okay now you can ask me if I close this terminal will it work yes definitely you can close it let's say I will close this terminal now if I again go to the chat bot if I refresh see still my bot is running okay so that's how guys we can do the cicd deployment now let me show you how we can terminate all the instance like whatever instance We created if you
don't want to keep it running otherwise it will charge you right so what I can do uh first of all I will stop the easy to machine I've created so select the machine click on the instance State and terminate so terminate will delete it okay so it will terminate and delete the instance then after that I also need to uh delete my EC ECR so let's go to the ECR so first of all delete the image inside that just write the delete Command now try to select and delete okay then uh what I have to
do I have to delete the uh then I will also delete my IM I'm user okay I'm user I created user let's select and try to delete it medical chatbot I have to write here so let me copy let me delete it see everything is deleted now nothing will charge you okay so guys that's how We can perform the deployment and this is the code you can use uh this code as it is okay the code actually have shared so yes guys this is all about uh and I hope you liked it so thanks for
watching guys and what you need to do guys uh the next project I showed you now that uh this one source code analysis you can deploy this project with the same process okay whatever project we have created so far you can deploy with the S uh same process so guys we'll be Starting with another very important concept inside genbi called llm ops so I think you already heard of this name okay called llm ops so the full form is large language model operation so those who are from machine learning and deep learning background I think
you know there is another uh actually concept called mlops so with the help of mlops we can perform machine learning operations that means let's say if you want to build any kinds of end to end Mldl application you can follow some certain step okay some certain pipeline you can follow that means you can follow some certain pipeline to implement that particular application and here mlops usually help help you to implement that particular pipeline right so similar uh things actually we can also apply in the geni so I think we saw inside gen we can use
different different kinds of let's say uh tools and Technology like we can use different different kinds of Framework we can use commercial large language model we can use open source large language model yes or no so commercial wise I already told you we can use open AI platform because openi gives you the model with the help of API access that means you can create a API with the help of that you can access this model in the openi platform so for this you don't need to download the model in your system okay so download you
don't need to do if say if you're Using any low configuration system still you can use that model but whenever it comes to open source large language model and it can of foundation model I think I showed you the entire hugging F platform there we are having so many open source large langage model right let's see if you want to use these kinds of foundation model because apart from openi these are the model are also good now this is also better some of the model you will see it is better than Openi model okay now
if I want to use these are the model efficiently if I want to create a efficient application if even if I want to deploy this particular application on the cloud platform how we can do it because we saw that if I'm using this kinds of model like open source model I have to first of all download the model in our system and whenever I'm downloading you should have good configuration machine yes or no that means you should have good GPU Good memory good CPU then you will be able to execute this model and let's say
you want to fine tune these are the model definitely you need a very good configuration GPU otherwise you won't be a able to fine tune that I think we already saw one fine tuning example of Lama to model there I was fine tuning this model onto our custom data and I was using Google collab so there actually it was taking lots of time uh let's say to train only oneop just try To consider got it so that's why llm Ops comes into pictures so llm Ops will gives you the flexibility to work with uh different
kinds of foundation model OKAY like uh what kinds of foundation model I think we you saw like Lama is there mral is there okay jini is there different different large language models are available so different different foundations model are available uh I already showed you in the hugging P platform okay so if you want To use these kinds of foundation model Foundation large language model with all the flexibility you have to use something called llm ops there are some llm Ops platform are available uh this will give you all kinds of flexibility you can work
with these kinds of foundation model even it will give you all kinds of functionality okay all kinds of functionality it will provide you all you just need to use this platform to implement your LM power Application I'll teach you all the platform whatever platform actually we are having which is like famous in the market I'll discuss all of them one by one okay no need to worry so first of all let's try to see uh why llm Ops is required because we saw that like what is llm OPS exactly now we'll try to see why
llm Ops is required uh why we need it exactly for this I'm going to open up my Blackboard and there I'm going to explain this concept so guys I think so Far you have learned about Lang chain okay Lang chain then you learn learn about llama index okay then you also learn about open source large language model okay open source llm even I also taught you the open AI okay openi and open a provides actually commercial model because all the models are paid here if you want to use them you have to pay for that
right and how it will charge you based on the Token token limit and this will give you the API key okay API key with the help of API key you can access this this model but apart from that there are so many actually Foundation model I think I already discussed inside hugging fish okay hugging fish platform like Lama was there okay then mral was there then Falcon was there okay then then jini Jimma okay and so on there are so many Model so many model if you open the hugging face platform you'll see that there
are so many models are available and these are the foundation model and this model is pretty good guys if you see this model this model is like very powerful model it's very powerful model even I already showed you some of the demo now we created some of the application with the help of Open Source L language model there I think you saw the power of the model yes or no right But I think one issue you saw whenever I was using this open source large language model the issue was this model size is very big
let's say if I'm considering llama 2 just try to consider about Lama 2 so now tell me llama 2 having a how many variant I think you know Lama 2 is having three variant like 7 billion parameter model then it is also having 13 billion parameter variant model it is also having 70 billion uh parameter Model even uh even recently there is another model you will see called llama 3 okay llama 3 and llama 3 is very huge than your llama 2 so you can open up the hugging pH Hub and there you can search
for Lama 3 you will see the model so here I already told you if I want to use this model actually I have to use use GPU based machine GPU based machine GPU based system yes or no and if I'm not using GPU based machine so what will happen I Won't be able execute deser the model but if you see there are so many developer there there would be so many Enthusiast uh they will prefer to use these are the open source large language model but most of them will not have the GPU based machine
okay because it's obvious now because if you see nowadays actually uh whatever laptop whatever let's say uh system we usually buy uh people usually take this um actually GPU let's say 30 60 okay ti so I can Consider RTX RTX okay RT RTX 30 60 TI if you if you do if you have actually less budget then you can buy this uh GPU if you have more budget you can also buy higher configuration GPU but there are very less people they will buy the higher configuration GPU let's say who have the huge amount of earning
they will buy buy it okay but if you but if you have actually less budget because see most majority of the person will have the Less budget whenever they buy any kinds of system so they will have let's say this GPU 30 60 Ty TI okay so this GPU having actually I think around 4 GB 4 to 8 GB I think uh vram okay vram it is having okay now just try to consider let's say I'm loading this Lama to 7 billion parameter okay this model in this GPU so what would be the inference time
as of now let's try to consider let's say it will take 5 minutes okay 5 minutes to give you the Response now just try to consider if your application is taking 5 minutes to execute only one prompt will people will use your application tell me if let's say I'm using chat GPT and if I'm giving any prompt to the chat GPT and chat GPT is taking let's say 5 minutes to give me that response so we I use the chat GPT again no definitely I will not use because nowadays you'll see see that people will
have very less time they they want actually quick response from Any kinds of application so what you have to do the system you have developing this should be very fast okay the inference time would be very fast here otherwise people won't be using your application right then let's see we are loading this 13 billion parameter model OKAY in the RTX 3060 TI GPU let's say now it is taking 10 minutes to execute okay 10 minutes to execute and after that let's see you are taking 70 billion parameter model in the TX 3060 TI and you
will see that sometimes it will give you memory okay memory out of space okay out of space error okay because this uh GPU doesn't have the capacity to load this 7 billion parameter model because I told you it is having four 4GB and 8GB vram okay so that time actually we are not able to use this kinds of foundation model but let's say some somehow we are using this Foundation model let's say we are using Small variant of the model let's say 70 billion or 13 billion but what is happening inference time is very high
here okay it is like very slow here so again people won't be using my application so this is another issue got it now let's say somehow you created one application let's say it is taking 5 minutes to execute now what you have to do uh if you want to publish this application to the audience you have to deploy this project yes or no let's say You want to do the deployment you want to do the deployment M okay deployment now I think previously I showed you one deployment right uh like uh we deployed One n
to Medical chatbot project there I was using actually AWS okay AWS so I was using let's say ec2 service from there ec2 then I was using ECR okay lastic container then I was also using something called Docker okay so these are the services Actually I was using there one by one now see there I was actually deploying the open a based application that means I was using open a model that's why I was able to deploy this project in the cloud platform with the help of these are the simple service only but whenever you are
using open source large language model so that means this model you have downloaded in your local machine and now what you have to do you have to move this model to the production server that Means your Cloud machine and there just try to consider if you are using these kinds of 8GB let's say instance like T to large instance will it work so I think remember we use something called t2. large instance there and it was having 8GB memory only okay 8GB memory with CPU CPU code so it won't be working right if I'm loading
this llama 3 or Lama 2 in this machine it won't be working so for this what I have to do I have to purchase I have to purchase Actually a GPU based machine there so if you see the ec2 instance you will see that there actually will also have GPU based machine so there actually you can also purchase GPU based machine apart from that AWS provides another service called AWS salemaker those were from actually uh MLS background they already know what is aw says maker so salemaker is a complete actually platform so here you can
uh actually Ingest okay train evaluate okay evaluate and deploy any kinds of machine learning or deep learning project okay and it also provides different different virtual instance that means virtual computer okay virtual let's say uh computer it will also give you so there actually you can take any kind of instance and you can let's say host your model it is also possible so salemaker provides actually Lots of service you can search uh over the Google you'll see that it provide different different Services okay now a few month back actually I was uh going through one
actually research article so there I found if I want to let's say deploy any kinds of Open Source lar language model uh as as a let's say my custom created let's say server let's say I want to uh launch my own server and there I want to host my model for this what I can use I can use something Called sales maker okay A W salemaker I can use because salesmaker will give you different different kinds of instance okay so there actually you can host your model and if I want to use says maker and
if I want to let's say deploy this kinds of uh open source large language model because see if I want to use this kinds of Open Source large language model I have to use different different Library like I I told you we have to use hugging face okay we have to use Lang Chain so different different things we have to uh install even if I want to use hugging fist I also need to install accelerate bits and bytes because this model access the GPU right and to access the GPU I also need deserve the package
okay I already explained uh in my hugging F session so if I want to install these are the package when let's say I'm taking any ec2 machine there you will see installation issue so most of the time you will see lots of Installation issue actually will get but what they did actually they created one service called DLC okay DLC that means deep learning container so this service is already available in the AWS okay you can use with the help of salemaker salesmaker you can use this DLC that means deep learning container so inside deep learning
container you can install any kinds of deep learning package let's say you can install hugging face you can install let's say F tensor flow so all The let's say library would be already pre-installed there so you don't need to let's say manually put the effort to install these are the library okay with the GPU configuration everything would be already there okay only just need to take the DLC uh let's say uh instance and you have to connect with the salemaker that is the idea okay so with the help of salemaker with the help of DLC
you can host this model let's say you are having one open source model now You can host this model in the DLC platform got it and to get the let's say Endo from the model what you have to do you have to use something called API Gateway okay API Gateway so this is another service from the AWS with API Gateway what you can do you can cre an API okay API like we saw like open a API now we used to use one API so here also you can create API with the help of API
other people can access your model let's say this is user so user can access your Model with the help of API request so this API will request this uh let's say model that means let's say you are using llama model okay and llama model is present inside DLC that means deep learning container it is running on the salemaker actually instance okay from uh from there actually you can get the response now let's say somehow you deployed this uh model like that okay the way actually I showed you with the help of say makeer DC API
Gateway okay All the services let's say you deployed this model and now let's say it is working fine now there one issue we'll get the issue is cost okay so it will actually charge you huge amount of money let me show you so uh in that research article I got one image actually let me show you the image so I have already taken the screenshot so this is the image guys I think is this image is visible H see what they did actually they have taken Different different llama model you can see they have taken
different different llama model let's say llama 2 7B 7B chat 13B 70b okay and this is the model ID and this is the max tokens like uh what would be the input length Okay of this model I think you know Lama 2 Tes actually 496 token okay and this is the default instance type actually they're suggesting let's say if I want to use Lama 27b model so I have to take M ml. g5. 2x large machine okay and this Machine is available in the Sal platform now let me show you the salesmaker so if you
just search for sales maker okay sales maker pricing just search for salesmaker pricing in Google so you will get one website machine learning service Amazon salesmaker pricing a let just try to open it up now here if you just go below you will see uh salesmaker will give you All kinds of pricee list with respect to all kinds of uh let's say machine type let's say you want to use MLT 3 medium it is having actually 4GB RAM and here you will get two vcpu and this is the price per hour like that actually it
is having actually different different instance actually you can see it is also having large instance so you can see it is also having large instance x large instance okay then 24x large also instance it is also having now see the More good configuration instance you will be taking that means your vcpu is increasing as well as the let's say memory is also increasing you'll see that price is also increasing so this is the par hour price let's say par hour price it will charge you $6 if you're taking M ml m5d 24x large now see
if I show you the chart now so this chart also they have also suggested let's say you are implementing one project with the help Of llama let's say Lama 70 okay Lama to 70 billion parameter and now see this is the configuration you have to take this is the configuration you have to take mlg5 48x large okay configuration machine uh but let's try to consider Lama to 1 billion okay let's say you want to use this this actually configuration model now let's find this configuration MLG fine 12x large so here you can find out that
configuration you can see uh there are different different CI are available this is the M5 C5 you can also find out the G5 so as of now let's try to consider this machine let's say ml uh M m5d 24x large okay let's say I want to use uh this okay this machine I want to use this machine this machine is having U 96 actually CPU and it is also having 384 GB uh memory now what is the Power Hour cost Power Hour cost is you can see uh $6 okay almost $6.5 you can consider $6
as of now now let's calculate the Pricing so what I will do I will open up my calculator uh I'm going to actually calculate the price per day actually per day how much actually it is charge me let's say the price you can see it's $6 okay $6 times I want to run 24 hours this model okay this model I want to run let's say 24 hours in my uh actually says make okay platform because see 24 hours your application should be keep on running it's not like that you are Running for two hours again
you are stopping no people won't be using your application chat ZB is running uh 24 hours okay uh 7 days it is all uh I mean running every every time so you will see there is no downtime so whenever you are creating your application it should be always running in 24 hours okay so let's say I want to consider part day how much actually it will charge me so 24 six six 6 * 24 so it will uh charge me actually $144 only just one day it will charge me $144 now if you're from India
if you're from Bangladesh if you're from any other country just try to convert the currency okay in your let's say money and try to see how much money actually it is taking part day from me got it now let's say I want to run uh this model uh let's say I want to run this model let's say seven days what I do I'll just multiply with seven multiply with seven now see this is the C Day cost I got 1, $8 so it is more than 1 lakh rupees okay more than One lakh rupees actually
it will take if you're running this model uh 7 Days let's say I want to run this model 4 weeks so this is the charge actually it will take now let's say I want to run this model one year so I can multiply with 12 see this is the one year cost actually you are getting $48,000 okay 3384 so now let's see if you're running a uh small startup if you're running is a small company so will you afford that Much amount of money will you be able to pay this this much amount of money
no definitely you are not so at then your cost is increasing if you're using this kinds of Open Source large language model if you're deploying uh on this kinds of let's say platform on the custom let's say hosted server so you will have lots of cost at end so better to use the openi model that time yes or no because openi model will charge you less very less than okay very less than Than the this amount of money got it so that's why so that's why actually llm Ops comes into picture so what big tech
company actually did they came up with actually different different kinds of llm Ops platform so I think you saw um not recently actually this llm Ops platform came actually long back so Google actually came up with one um LM Ops platform called vertex AI okay vertex AI so previously when you used to use mops now also we used to use Something called vertex a for the mlops also because this vertex a provides all kinds of functionality all kinds of let's say pipeline for the machine learning operation as well okay nowadays they also integrated geni okay
in the vertex AI even AWS that means Amazon web service they also came up with another lmos platform called Bedrock okay Bedrock okay Bedrock so Bedrock is also one llm of platform and this is the new service Actually they have launched in the AWS actually let's say website you will see that there there would be one section called Bedrock okay so Bedrock also provides all kinds of llm off Services let's say you want to use any kinds of foundation model you want to use Lama Mistral okay any kinds of foundation model so this model is
already hosted see this model they have already hosted in their server let's say whatever Lama model whatever Mr model whatever model You see these are the model is already hosted in this llm offs platform like that Google has also hosted lots of found model in their server it's not like that they're only hosting their model let's say Google is having their own model they having jini model they having Jimma model yes or no apart from these are the model they also hosted some of the foundation model from any other organization let's say from meta AI
from let's say mistal so they so what They did actually they collaborated with these are the organization even they hosted their model in their platform as well and they created one endpoint they created one API okay they created one API now as an user as a user what I can do I can use this API to interact with this model that means like our openi will send the request okay to we send a request to this kinds of llm of platform with the help of API and here I will mention what kinds of model I
want to Use and this will give me the response that means I don't need to download this are the model in my system okay so this is the main benefit to use this kinds of lmos platform okay and it is having all kinds of flexibility let's say you want to connect the data you can easily connect let's say you want to connect the vector database you can easily connect okay that means all kinds of storage all kinds of flexibility this LM platform will provide you it's not like That manually you have to take from any
third party let's say tool it's not it's not like that complete like let's say pipeline it is already having even let's say you are thinking to fine tune this kinds of foundation model you can also fine tune you can also perform the fine tune operation so fine tuning will happening in their server only not your not your system only you just need to set the configuration of the model then you have to pass your data automatically It will perform the fine tuning in their server yeah it is sure they will charge you but you will
see this price would be very less very less than your this cost actually whatever cost actually I I showed you here now if you're let's say manually creating your Ser if you're using manually different different kinds of services that time price would be high but if you're using this kinds of lmos platform your price would be very low guys trust me it would be very low If you see the pricing now if you see the pricing I'll tell you whenever we'll be exploring these are the LMS platform there I'll also show you the pricing you'll
see the pricing it should be very low cor so that's why actually llm Ops is required we need the LM offs if I want to use this kinds of foundational model it's not like that we only need to rely on the openi model because we we saw that there are so many model which is more powerful than my openi model Even there is another cloud provider I think you know a so a also having one lmos platform called a your openi okay a your openi so they have what they did they collaborated with openi and
they bring the openi platform in the Azure platform only okay on top of that they created some more services and it is running on the AZ right now so if you know about open now open so you'll be also work with the AO okay AO let's say open so openi wise we already saw how we Can use openi with the help of open py how we can access different different kinds of foundation model okay so yes guys this is the complete idea of our llm Ops and why llm Ops is required okay so in the
next video we'll be learning about one amazing actually L platform called verx a from the Google site so we'll see the entire platform what kinds of service it provides even we'll also do lots of practical we'll also do the handson we'll be creating different Different let's say llm power application with help of verx a even I will also show you the AWS Bedrock how we can use the Bedrock okay to implement any kinds of end to end LM powered application even we also learning how we can deploy these at the project as well fine in
this video we'll be starting with our very first llm Ops platform called verx and I think I already told you verx is the platform from the Google so it is available in the Google Cloud So Google has developed this platform so here we'll be doing this complete genv on the Google Cloud we'll be learning how we can use different kinds of foundation model even uh with that how we can uh create different different kinds of LM powered application so guys here you can see this is the complete Revolution at the Google uh inside artificial intelligence
I think you remember in 2017 uh they bring actually one architecture called Transformer so Whenever this Transformer architecture came in the market uh after there actually so many Revolution happened in the field of language model in the field of large language model that means whatever architecture you can see nowadays all the architecture are using this Transformer architecture in the back end even I already uh created one video right I already took one session on the Transformer architecture like what was the main component inside Transformer architecture and why it is so powerful right so then after
that actually in 2018 actually they came up with another model called Bart BT so this is one of the actually language model they brought then they brought actually T5 model then slightly actually in 2020 actually they brought one large language model called Lambda so Lambda was amazing model in the field of large language model then 2021 they uh actually published Alpha fold model then 22 Palm model and in 2023 they introduced something called B model OKAY b a r d b I think we used to use uh Google B right Google B that means uh
it is the Char gbt kinds of interface I think remember nowadays this B has been let's say replaced with the jini so we uh now actually we search for Jin but previously we used to search for B right so this was the first application from the Google actually the published and you can see the model summary like what Are the task this model can perform so as you can see T5 can perform Tex to text generation it's a transfer learning model uh it is having actually 10 billion parameter and it's open source model then Google
Lambda model then you can see the Lambda model uh trained to convers then Alpha fold predict structure for uh for all known proteins then Palm actually this is the industry leading actually large language model they they introduced something called Conversational AI service powered by Lambda okay then in 2024 I think you can see from the Google research we are having gin then uh imag 2 okay this is uh this can actually generate image okay with respect to the text prompt then we're having also something called Jima okay Zima is one of the large langage model
then there is another model called chart this is the model it can generate is to text so that's actually Revolution happened that means uh it took actually Lots of time time to came up in this position and Google introduced actually so many research so many model okay so many technology and nowadays actually we are using these kinds of Technology okay in the field of gen tvi and guys here you can see these are some actually first party Foundation model like Palm for text you can customize language task then uh Palm for actually chat so here
you can perform conversation okay then you can uh do the Imaging okay for the Text to image that means if you give any let's say prompt it will generate the image okay then we also had something called embeddings model that means embedding API let's say you want to generate any kinds of embedding for the text or image you can use this kinds of embedding model okay then chart model spe to text then code generation model was also available so this was the first party Foundation model in the Google research okay then they introduce Something called
Model Garden okay so what is model Garden so here you can access customize and experiment with different Foundation models and its API okay that means what Google did actually they uh introduced one platform called vertex a so vertex a was the platform in the Google and it's been actually long this platform is uh available in the Google uh Google Cloud but only we used to perform mlops operation with the help of vertex a but whenever actually this Genera VI came so they updated this vertex Ai and they introduced something called Model Garden that means inside
verx AI you have one model Garden inside model Garden you are having all kinds of Google model whatever model actually Google has published now all the model you will get all the foundation model you will get in the model Garden apart from that you will also get something called called open source model like whatever open source model I think I Showed you now from the hugging phase from different different let's say organization like llama mistal Falone right different different kinds of Open Source model I think I already introduced in the hugging pH so what they
did actually they also hosted these kinds of Open Source model in the model Garden that means you can use all kinds of Open Source model from the model Garden itself okay from the verx a then it is also having some partner Partners Model so what is Partners model exactly so they collaborated with different different kinds of organization they collaborated with the different different kinds of company let's say cloud is a company then we are having Meta Meta is a company okay that's how we are having different different kinds of company so this company has also
introduced so many large Lang Bas model okay so what they did actually they collaborated with the company Collaborated with the organization and all kinds of actually model they introduce in the model Garden so this is called actually partner models now let me show you the vertx platform right now how it looks like you can see guys this is the previous vxi platform okay that means here we used to only perform the ml operation that means machine learning operation that means you can apart from the experiment train and deploy okay for the data science workbench now
what they Introduce actually on top of this platform they introduce genbi you can see they introduce something called genbi platform that means here you can perform the pr designing fine tuning gen Studio okay Foundation model open source model okay then uh you can also perform actually LM Ops everything you can perform in this platform right now that means instead of replacing uh the previous actually let's say verx AI services on top of that they create Created one additional layer and this is called actually llm of splat form now it is on the vertic L okay
now I think it is clear now as I already told you uh they introduce so many actually Partners model okay I already told you they introduce so many partners model in the model Guiden so these are the partners okay these are the partners actually with Google I think nowadays they also collaborated with so many organization if you open the Google U Cloud vertx We'll see different kinds of actually partner models are available so here you can see AI 21 Labs uh this another organ ation they collaborated with the Google then anthropy C then contextual AI
okay mid Journey then osmo then resistant. a meta okay Runway and so on so these are the actually Partners model actually it will you'll be getting in the model garden now let me show you this uh gcp okay gcp vertex AI so let's open up our account and Let's open up the verx there and try to see whether these are the things are available or not so for this guys what you can do just search for gcp okay gcp console in your Google and make sure you have one gcp account okay if you don't have
the gcp account guys so you won't be able to use the vertx so make sure you if you don't have the account just try to create one account initially it will give you $300 credit okay so let me uh login with my console So guys as you can see this is my gcp console so here what you have to do in the search button just search for vertex AI okay verx AI so you'll see this is the platform this is the lmop platform you'll get see this this is the complete vertx AI now left hand
side you will see different different kinds of services so here I already told you it is having model Garden so let me click on the model Garden so here you will see all kinds of model so guys as you can see Whatever model I showed you all kinds of model are available apart from that uh they have also actually brought new new model here you can see uh so Foundation model okay then Partners model see all kinds of models are available okay as you can see Meta model is available that means llama 2 llama 3
okay then Palm model uh you can also show all the model here so see here I can click on the show all model see all kinds of model are available see mral Al also is there then Jamba model then you can see cloudy okay apart from that you will see different kinds of uh actually Foundation model you will get okay and these are the actually partner okay partner company apart from that you can also filter out let's say you need language model so here you are having 63 language model you can filter out all the
language model let's you need visual model you are having 11 Vision model you can also filter out all the vision model okay now Let's say you want to use this model just click here and you will get all the guideline how to use this model andal okay they have already given the source code everything they have given okay how to uh use the python uh SDK for uh let's say using this model and all all the let's say guideline they have already given okay I'll tell you how we can use these are the model one
by one okay now let's say you want to also play this Model in the model Garden okay it is also possible let me show you so let's say this is my model Garden let's I want to uh like play with this model jini 1.5 Pro uh so I'll just click here now you can see open in a Vertex AI studio so here you can open this model in the AI studio so they are also having one AI Studio let me show you even you can also open with the help of collab notebook and there also
you can also play with this model now let's click on The I agree and continue now here you can pass any kinds of prompt let's say I give hello and I will send this prompt and this is the response actually I'm getting okay so that's how actually you can play with different different kinds of model before using it like how this model is working on all what is the performance of the model everything you can perform here okay then you can also search by the model name let's say you Need Jimma model Lama model anything
you can search here you will also get these are the model got it and Guys these are the model is already hosted in the Google Cloud so you don't need to manually download these are the model in your system you can use this at the model with the help of uh API request as I already told you that's how so that is why we call it as a llm of platform okay so they gives the API canel functionality with the help of API Request we can use these are the model in our uh let's say
project implementation and don't you worry I'll tell you how we can access different different kinds of model we'll be also implementing some kinds of application now apart from that vertic also provide so many services you can see it provides the pipeline services this is the pipeline for the mlops let's say you want to perform any end to end pipeline data ination data validation Model training model evaluation you can use the pipeline Services apart from that it also provides the notebook services that means let's say you want to uh let's say use one large language model
uh you want to find in one large language model you can use their collab Enterprise that means you you can U take the machine okay good configuration machine good configuration collab notebook in their Google cloud service okay then you can also take the Workbench as well then you can see different different studio is there you can perform different different kinds of let's say u i mean task task demo let's say you can perform chat demo Vision demo translation speech okay you can play with these are the functionality as of now I'm not showing then you
can also download different different kinds of extension for the geni from the extension Market here you will also get something called feature restore data Set okay labeling task then training experiment okay so some of the services you will see these are the MLF services not the Genera Services AP from that you will also get something called vectors RCR this is another super important service from the verx a let's say you don't want to use any other Vector database what you can do you can use uh this verx a vector SE this is another Vector database
okay and this is hosted in the Google Cloud that means in the Verx AI so going forward I will also show you how we can use this uh Vector search Okay Vector search functionality from the verx a itself so yes Guys these are the services actually verx a provides so unless an Al one not exploring all the functionality okay you won't be getting so try to open this account guys and try to explore all kinds of functionality okay now there I told you you can perform different different kinds of operation okay Different different kinds of
this operation it will give you all kinds of functionality okay and I showed you the model guarden and all uh so then I already told you we can also perform something called fine tuning so it's not like that you can only use these are the Foundation model for the inference no you can also perform the fine tuning operation that means this vertx I will give you the functionality you can also improve the model performance with the Help of fine tuning okay on top of your custom data you can also perform something called human feedback to
increase your model usefulness like I think previously you saw uh in the chat GPT uh it used to give one feedback box right let's say how use your experience so that you can uh give your experience rating so that kinds of rating also you can pass if you're using this kinds of verx AI then here you can also generate customize for your business then here You can also improve Google's predictive AI with your own custom data okay everything is possible here now you can ask me what kinds of task actually I can perform here you
can perform all kinds of task so you can perform language image video okay documents speech tabular so here all kinds of task you can perform with the help of vertx AI okay and what verx actually provides uh as I already told you it will give you stateof the Art Foundation model even You will also get end to end governance that means you can do the pro prompting tuning okay you can also perform mlops operation on the verx a even here you will get the data privacy so whenever let's say you are uploading your custom data
to the verx a it's not like that your data would be leaked uh it would be secure okay so this kinds of actually ensure you will get if you're using verx platform that means you will have the complete data security so no need to Worry about your data if you're having any custom data all the data would be secured here so now I think you got the difference between mlops and LM Ops so in the verx I we have the AML Ops um like Services already on top of that they created the LMS okay so
here we don't need to throw out your existing mlfs investment okay so here what you need to do you just need to understand the unique MLS need for the Gen that means whatever additional let's say Functionality they have given just try to understand and try to use uh let's say those are the service in the genbi field okay that means uh you can see in the verx AI you will see multitask model fronting okay then model Garden customization managing new artifacts evaluation monitoring okay connecting with the Enterprise data let's say you want to connect with
different different kinds of data sources it is also possible in the vertx a so now as a high Level you can see the complete mlops on the verx a this is the so this was actually our traditional verx a that means here we can perform experiment training and deploy uh and predict okay for the machine learning operation and now in the vertic a you can also perform something called uh prom predict okay then you can also use something called Foundation model that means all kinds of Z VI based functionality you will get here then uh
I already told you you can Also connect with uh different different Enterprise data let's say you want to connect any external sources any API any website you can also connect okay with the help of vertx a so guys all kinds of flexibility you will get here if you're using verx LL platform so no need to worry about anything so yes guys this is the complete uh introduction of our verx a the complete generative AI on the GCB cloud in the next video we'll try to see how we can build different kinds of Transformative llm powered
application okay with the help of this verx a platform so far we have discussed about llm Ops and this uh generative vertex AI now in this video we'll be uh trying to use this vertic as a practical that means let's say if I want to use any kinds of foundation model uh with the help of python code so how we can do it okay that means we'll be doing the entire handson in this video so try to open up your Google uh cloud guys Everyone and just try to open up your verx I I already
show you how we can open the open up the vertex a right now after opening the vertx a the first thing you have to do you have to enable all the recommended apis so there you will see one option enable all the recommended apis so make sure you have already enabled all the recommended apis so after en in uh in the notification bar you will see uh See all the API has been enabled so these are the API you Should enable otherwise sometimes actually will get actually different different add so that's why make sure you
enabled all the API now here you can either open up your Google collab uh to perform all the let's say operation I'll be doing here either you can also open up this collab Enterprise or workbench okay so here I'm using this Cloud platform that means LM Ops platform so in this video I'll show you how we can use their services actually that if I Want to use collab Enterprise how we can use collab Enterprise you can also launch this Marben so what is the difference between Marben and collab Enterprise let me show you so in
workbench actually you can select the machine configuration so here you can create a new workbench just try to click on create now here you can select the machine configuration like uh what should be the machine name what is the reason then you can also select the Instance type okay that's how actually you can uh change the configuration okay all the configurations see you can change here okay everything you can change here that means if you want to create a custom machine as per your requirement you can change here and here you can see the pricing
summary that means per hour it will charge you $0.18 you can see price is very less it's not like very huge amount of money they will be charging it's like very less money so You can also create your custom configuration but here I'm not going to create the custom configuration what I'll do I'll use the collab Enterprise okay this one so it will give me Google collab kinds of interface and there I can um actually do my coding so after clicking on collab Enterprise so here you will get one option create a notebook Okay so
just create a notebook you can also select the region but I will keep the default region e Central one okay that's it now you can give the name I can rename this notebook let's say I'll rename it as um vertex AI demo okay you can give any name I've given this name now if you want to increase the size of the notebook if you want to increase the size of the Cale let's say if I write something so the text is like very small here so I can also so increase the size so for this
what I can do I can click on the Settings uh just go to the editor and here you'll see this font size just try to make it as uh I can take 28 okay 28 I think it is fine now the next thing what you have to do guys you have to connect this notebook so let's click on the connect button see guys once you will be connecting the notebook it will give you this kinds of popup screen that means you have to select account okay like which account you want to use so make sure
whatever Actually account you are using for the Google Cloud console that means is for the vertic a that account you have to select here so this is my Google Cloud account I'll just try to select now it is also asking for password so let me give my password and I'll click on next so if you're doing for the first time so it will give you this kinds of authentication uh it's just a one time authentication okay that's it so guys as You can see my runtime is connected now if I execute the cell if you
want to execute the cell just press shift and enter the same thing the way actually you execute any collab notebook you can execute as a same now see I'm getting the okay output now you can also download this notebook so there is a download button you can also download The Notebook so first of all uh let's do the coding after that you can also download uh this notebook even I will Also share this notebook with you in the resources section fine all right now see here we'll be using actually verx okay we'll be using verx
services that means we'll be using model Garden so from the model Garden itself what we'll do we'll try to access different different kinds of foundation model got it and if I want to use model guardan that means the complete vertx I I have to install one Library so you have install vertex AI SDK for python because we'll be using Python code right and if you're using python code you have to install this Library so here I want to install the upgraded version of the package and this is the package name Google Cloud AI platform got
it if you want to use Vex a you have to install this python package now let me install okay now after installing it will tell you restart the runtime just click here so it will automatically restart the run time after that you just need to import Vertex AI okay just to test whether everything is fine or not now see if I import it should work see it's working fine there is no error fine now see those who are running this code from the collab notebook you just need to do uh two configuration here so let
me show you so the first configuration you have to do um this one that means you have to authenticate with the Google Cloud first of all so here let me write the code so First of all you have to authenticate with the Google Cloud okay from the um collab notebook and for this you have to execute this line of code so here everything I'm running from the vertex only that me I'm using collab Enterprise from the verx a I don't need to perform the authentication but if you're running from the Google collab that time you
have to do the authentication guys okay so let me show you one example let's say if I want to Load any model uh from the collab Enterprise that means if I'm using Tex a services I don't need to authenticate it I don't need to authenticate it I can easily load it let me show you so for this let's import some of the library first so here I'm importing some of the library you can see gen generation config generative model okay I'll tell you why I'm importing them and what is the use of it first of
all let me import all of Them now let's say I want to load one model let's say I want to load this gini 1 uh Z pro model so you can see there is a model zmin 1 Point uh Z Pro just click here uh if I click here you will see the model ID you just need to copy the model ID from here now if you just go below they have already given the source code here you can see this is the model ID just try to copy copy this ID and try to paste
it here okay that's it now see if I execute this code it will Automatically load the model see there is no error but if you executing from the collab notebook you will see you will get one error that means your notebook is not authenticated with the Google Cloud okay so that's why you have to execute this lineup code it will uh what it will do it will authenticate first of all it will ask for the email address that means which email address you are using for the Google Cloud account that email address you have to
Select first of all got it then the next uh actually configuration code you have to run this one let me show you this one if you're using collab Enterprise guys you don't need to execute but if you're using Google collab you have to execute now see this is the code you have to execute so here you have to pass the project ID as well as your location okay that means region now how you will get the project ID so here you can see on top you have the Project ID just try to click here now
see I'm inside my first project so here is the project ID just try to copy the project ID and try to mention it here okay in the uh in this box actually you just need to paste it here then you have to give the region name and what is your region name you can see I'm inside us Central one so try to mention us Central one here okay then if you execute it will automatically initialize with the vertex a that means it will authenticate With the vert.x a with your configuration now it will get to
know okay now you are you are the correct user so once it is authenticated then you will be able to use all the services from the vertx a that is all the foundation model all the services you can use got it so this two line actually you have to execute if you're using Google collab see collab only and if you're using collab Enterprise no need to execute guys okay you can run as it Is I hope it is clear now okay so now I think you saw how we can load the model to load the
model you have to use one function called generative model you can see I already imported this generative model from the vertex a preview. generative a models now P generative model inside that you just need to give the model ID now if you want to use any other model simply just try to open let's say you want to use 1.5 pro model so just click here okay so you can see Jini 1.5 pro model now just try to go below and here is the model ID so try to copy the model ID so this is the
model ID just try to copy and try to paste it here so that's how if you want to use any model from the model Garden just try to copy the model ID and try to past it inside this function it will automatically load that model for you so now let's uh test this model so here I'm going to pass one prompt so here I have already prepared one prompt let me show You this is the prompt guys I have prepared okay see so here I'm using the model object and I'm calling one function called generate
undor content inside that I'm passing the prompt why is Sky uh so here is the prompt why is the sky blue and stream is equal to True why I have given stream as equal to true because if I show you if I let's say open the j j or chat chat GPT if you can open anything let's say if I give this same prompt here now so if I let's say Pass the same prompt I is the sky blue any kind of prompt if you give it will give you stream kinds of output see it's
not uh it's not actually onot output it will give you stream output so I think here you saw it will give you stream output that means after one word it is coming another word another word okay that's how it is giving stream output not in one shot so that's how we also want to get the output from my response okay that's why I've given this Parameter stream is equal to true that means it will give you stream output got it now see if I execute so here you will get one response object see this is
the object okay so it would be a object see this generator object now here I can apply any kinds of iterator that means looping and I can get the response now let me show you so here you can apply looping so here I'm applying looping for response in responses and I just wanted to print the text okay whatever text Actually I'm having from the response I want to print it now so if I execute the for Loop right now you will see the response see this is the response actually I got and this response is
uh actually short response that's why you didn't saw the streaming output but if it is generating bigger output you will see the streaming output would be visible got it now you can try with different prompt let's try with another prompt so I'll give another prompt here Let's say I'll give U uh what are the latest developments in the auto uh automotive industry okay now if I execute so it should give me the response okay see this is the staming response I'm getting guys I think you saw okay that this is called streaming response see the
complete response I got great now see here I pass the prompt directly okay in the generative uh generate content function but if you want to pass as a parameter you can also Do it for this Ive created another example so this is the example guys just a minute huh just see that so here I created a variable called prompt and inside that I've written The Prompt let's say this is my complete prompt create a numbered list of the 10 items each item in the list should be a trend in the take industry in Trend should
be a less than five wordss okay try uh now you can give any kind of prom now here what I'm doing I'm calling the model Object generate content inside that I'm giving the prompt okay whatever prompt actually I have defined stream is equal to two then again I'm performing the looping operation now see if I execute it should give me the output okay now you can also customize the prompt let's say here I can give uh translate in Hindi see I'm also getting the Hindi translation as well got it so that's how you can also
customize your prompt it's Up to you now I think you remember whenever I was uh like going through the model Garden so I can also open this model in a vertic a studio I think I showed you so let's open this model in the verx a studio see if you open the verx a studio right hand side you can see you can also set some of the model parameter like temperature output token okay so these are the thing you can also change so how we can change with of python code because as a developer we
Have to use with the help of python code not from the UI interface got it from the UI interface also you can use the model but as a developer we have to use from the python SDK so here we can also set the model configuration so let me show you so for this you have to use this function actually so I think you remember the generation config okay so let me show you so I already prepared see in that generation config I'm passing all the parameter okay let me The temperature top P top K candidate
okay Max output and so on and I think you already know this the parameter I already explained previously what is temperature what is top b top K got it here temperature means here you are assigning the model creativity like how much creativity you want from the model if it is close to one that means it will take risk it will be more creative if it is close to zero it won't be taking any kinds of risk okay that's the idea now This generation config you have to pass as a parameter of generation config here here
you can see generation config is equal to generation config and the same prompt I have given stream is equal to two again I'm loing through the response now if I show you I'll see that your uh output would be more creative okay so previously I think you remember we we got actually uh very short output now see this output is more creative right now because we given the uh Configuration model parameter right now okay I hope it is clear now let me show you another thing you can do this is called actually test chat prompt
now what is test chat prompt let me show you one example let's say here I ask one question why is the sky blue okay now this promt actually have given and my J will try to remember this prompt now if I ask give me the answer In bullet point okay now see here I haven't mentioned I want the this answer I just given give me the answer in bullet point it will automatically remember my previous let's say context okay that means I want to ask this particular response now see it has given me in the
bullet point okay so this is called actually memory that means it is trying to remember the chat history so that thing you can also perform here so let Me show you so for this you have to use one more line here this the line guys model. start chat so this start chat will automatically remember all the conversation you are doing with your large language model now this is the prompt actually I've have given my name is buy you are my personal assistant my favorite movies are the lord of rings and Hobbit suggest another movie I
might like so this is the prompt I'm giving and this is the stream is equal to two I'm giving and now see if you're using chat that time you have to use chat. send message okay chat. send message because here you have initialized the model you remember now see if I execute the program so guys as you can see I got the output okay now what I will do now what I will do based on this answer I will ask another question so because it has remembered my previous context H so here now asking are
my favorite movies based on uh a book series this is the This is what actually I'm asking now see if I ask this question see it will refer my previous actually context yeah you can see both uh the lord of rings and The Hobbit trilogies are the based on the book series okay by JRR to tolken okay now I can show you the story so if you want to see the story just write chat. history so it will show you the entire history see so first of all you ask this prompt your model has given
this answer Then again you ask this question your model has given this answer got it I hope it is clear fine so that's how actually we can use any kind of language model now let's try to see uh any multimodel that means we'll be also using some visual model so I think you know U here actually we are also having some Vision model as well so this is my model garden now just click on the vision model you'll see there are different different kinds of vision Model are available so from here I'm going to use
One Vision model so Jin 1 Point Pro Vision model so you can search this model here now if you want to load this model you can load like that this is the model ID so let me show you this model I'll copy the name and in the model Garden so here you can sech this model you will see J 1.0 Pro Vision model okay now see this is the visual model so with this model actually you can also work with the images and videos Okay now this is the model ID you have to give now
I have given the model ID now let's load the model fine now here I'll be working with images and videos and to work with images videos I need some helper function this that means this helper function will help me to load the image visualize the image okay so here I have created some helper function let me show you so these are the helper function I created you can see different different helper function I created uh display image so it will display the image if you want to get any image as a bytes from from the
URL you can get uh with help of this function load image from the URL you can use this function get URL from the GCS you can use this function print multimodel prompt that means if you want to print any multimodel prompt you can use this function okay so that's how we can create some utility function and this function you will get over the internet If you just simply search how to display image give me a function you will get these kinds of cod okay so I have initialized these are the utility related function as of
now okay so here the first thing I'm going to show you uh how to generate actually caption from any image is okay so for this uh let's see one example this is the example guys so here I'm downloading my image you can see this is the image uh URL so so guys you can see this is the image okay this image actually will be downloading so here let me show you and how we are downloading the image we're using U GS util command okay so if you execute this command it will download the image from
the URL then I'm loading the image then I'm giving a prompt describe this image okay I'm passing the image as well as the prompt then I'm using the multi model that means the model we have loaded I think remember This uh gini 1 Point U zero Pro Vision model okay you can also use any other Vision model so simply go to the model garden and try to choose any other Vision model you can use as it is okay now let's get back now whatever actually let's say output I'm getting I'm just rendering okay that's it
now let me show you so guys as you can see this is my results I got so here I given a prompt and this is the image now it has given Me the caption so the image shown a gray cat with black strips working uh in the snow the cat has yellow eyes and long tail you can see yellow eyes and long tail cat is working towards the camera okay that means this is a great caption my model has given me now let me show you another example so I prepared another example H so guys
you can see I'm using another image now if I show you this example see this is my uh prompt I have Given describ the image that means describ the scene now see this is the response I got now this is the response to ponon boats are anded in the charless river in Boston in the background are two Breeze and Boston skyline so guys as you can see this is a correct caption my model has given me okay now if you're use load image function that means the function we created in the utility function uh to
load this image URL you can use this Load image URL function as well and with the help of that we can also do the same thing okay there are different different technique you can load the image you can also use open CB to load the image okay it's completely fine now see I'm getting the same output now here we'll be learning another thing here we'll be combining multiple uh multiple image and text promp for the F shot prompting that means here I'll be passing the F shot to My model and my model will try to
give me the response so let me show you one example I prepared so this is the example guys so here I'm using three image so let me open all the image so this is stored in the Google Cloud Storage okay you can see this is another image so you can see this is one of the image this is another image and this is another image so this three image actually I'm using here and with the help of load image from URL I'm loading Both three images then first of all I'm giving the prompt for two
images you can see this two images I'm giving the prompt this one and this one this one I'm not giving any kinds of prompt okay you can see I've given city is equal to London because you can see this is a London city and the landmark is a big b okay it's a big ban so you can search on Google big ban I think yeah see this is the big ban okay so that's why I'm giving some prompting that means I'm Giving the label okay now promp two the second image I'm giving this is the
Paris city city is equal to Paris and landmark you can see it's iffel Tower okay you can see it's I to and the third image I'm not giving any kinds of prompt now let's try to see my model is able to generate the actual uh let say promt for that particular image or not so here you can see I'm passing my image one prompt one image two prompt two and I'm only giving the image three here not the Prompt now I'm using the multimodel and generate content now let's try to see whether it is able
to uh identify that image or not see uh this two actually I have given it's fine now see third one city roome Landmark uh kosm okay so let me copy and let me try to verify uh see correct it's the correct one okay so beautifully my model is able to generate the response so this is called actually few shot prompting now also let's try to see some video example So what I will do I'll download one video so this is the URL of the video I'll download this video let me download and let me show
you here so I think you remember Google has launched one phone actually called pixel okay you can see this the pixel this is the pixel advice advertisement you can see this is the pixel advertisement okay so this video actually have loed from the URL now I will give this video to my multimodel and I will give some prompt let me show You so I already prepared one prompt this is the prompt answer the following questions using the video only what is the profession of the main person what are the main features of the phone highlighted
which uh CT was this recorded in provide the answer in the Jon format now see if I give this view uh video and if I give pass it to my multimodel it should give me the response see guys I got the response now you can See the person name is sha shim shimada profession is a photographer brand is Google phone brand is Google model is pixel 8 and uh and you can see the features video boost night sight City Tokyo country Japan amazing okay amazing actually response I got now you can use any kinds of
video you can do the caption generation got it so that's how guys you can export different different kinds of mod model in the uh model Garden it's not like only Google model You can also let's say use meta model as well let's say you want to use Lama model just simply click on the llama and try to open this notebook so they have already given the all the code example and everything okay only just you need to execute and you have to see how we can access the Lama model and all got it so just
try to explore guys uh this is uh up to you because because all the model I can't show you so I have just given the demo like how we can access Different different kinds of foundation model now you'll be exploring whatever model actually you need later on got it okay and one thing guys after implementing all the code what you can do you can download uh this notebook and after that you can delete this instance okay no need to keep on running just try to delete the instance other way you can you can also stop
the instance but I will try to delete it okay I'll just try to do the confirmation so after some Time actually you'll see that this notebook would be deleted okay now see this deleted so yes uh this is the actually Hands-On part of this vertex AI so in the next video we'll try to see how we can uh do the set up everything on our let's say local machine that means uh how we can connect this vertx with the help of local machine that means uh we'll be coding everything in our vs code from the
vs code itself we'll try to access the uh like verx a Model that means Foundation model okay so we'll try to access the model Garden from the vs code so for this whatever setup actually required I'll tell you each and everything see how we can uh use this vertex AI from our local machine that means we'll be uh doing the setup everything in our our local so previously I showed you how we can use collab Enterprise that means everything I was performing in the gcloud but uh in this video I'll try to use my vs
code And to access this vertx a what are the configuration we have to perform we'll be learning each and everything because see going forward whenever you will be implementing projects Right End to End projects so you have to implement through the vs code right and if you want to implement through the vs code you have to know how to configure this verx AI in your local machine okay so that's why it's super important to know how we can set up everything in our Local machine itself so for this guys uh you need one tool actually
so the tool name is gcp uh CLI okay gcp CLI install just search gcp CLI install on the Google so after searching this gcp CLI install if you just go below you'll see this link actually install the gcloud CLI so just try to open it up mhm now here you will get the installer see download the installer so I'll click on the link so it will Automatically start downloading this CLI for me see it has downloaded okay and if you're using any other operating system like say Linux then Debian Red Hat sento Mac voice okay
all the command is already written here you can follow the command and you can install this gcloud CLI in your system because with the help of this gcloud CLI will be doing the authentication with our account okay that is the thing now I have already downloaded it now what you have to do Just double click and try to install so the way actually you install any kinds of normal software you have to follow the same step okay there is nothing new so for me I already installed see after installing just search for Google so you'll
see this Google uh Cloud SDK shell just try to open it up so we will see this kinds of window okay if you are getting this kinds of window that means you have installed successfully okay now let me close it Now what I will do guys I will open up my local folder and here I'm going to open up my um terminal you can open up your any kinds of terminal whether your anagon terminal GB terminal anything now let's also open up my vs code here all right so here the first thing what you have
to do guys you have to create one virtual environment after that you have to install all the required packet okay then after that we'll be writing our actually code okay With the help of vertic C so to create the environment you can execute this command so cond create typen in let's say uh vertex AI verx a demo python is equal to let's say I'll take 3.10 ien y now let's create the environment okay it's done now let's clear uh before clearing I have to activate the environment so this is the command let's try to copy
and try to execute it here H Now let's create a file called requirement. txt so I'll just create this file requirements. dxt inside that I will mention all the requirements I need so these are the requirements guys I need I think you remember so I think you remember to use the vertx a we have to install this package Google Cloud AI platform I think in the collab Enterprise also did the same thing then stream lead I want to use because I want To create a simple user interface uh there actually you can give a prompt
and you can get the response and python. because I'm going to manage my gcp credential okay that's why now here let me create this file as well so do EnV okay inside EnV you need actually your project ID okay then you need region I'll tell you how to collect this project and region but before that let's do the configuration okay without gcloud so First of all let's me install all the requirements so I'll just write P install ien our requirement. dxt all right so installation is completed now let's do the configuration uh with our gcloud
so for this uh you have to execute this command called gcloud in it okay just try to execute this command guys so if I execute this command you will see that see if you're Doing it for the first time you will get this kinds of window you must log in to continue would you like to log in I'll give yes so guys now here you have to select the account the account actually you are using for your gcloud account so this is my account I select now you have to provide the password so this is
my password and I try to do the next operation okay then click on the Continue and allow okay now you can see uh I'm done with the authentication now if I come back uh see now actually I'm getting my project ID you can see I'm getting my project ID got it so let me show you if I go to my dashboard now if I click here this is my project ID uh studos uh loader 4361 I think you can see that uh Studio loader 43 61 and2 okay so this is my project IDE yeah now
if you already have One existing project you can select it either you can create a new project so for me I already have the projects I'll just select the one I'll select the one that means my uh existing project now it is asking for do you want to configure a default compute region and zone I'll give yes now it is telling which region you actually want to take so what I will do I'll take this region actually so Central us so let's say I will take this One US Central 1 C so I'll give seven
here okay now I think it is done now let me clear my terminal okay so everything is set now let's uh create one file here called main. by inside that I'm going to write all of the code okay that means we will be trying to access one that means here we'll be trying to access one um model from the model Garden but before that you have to set this EnV so you have to give the project ID so here is the project ID guys I think I showed you How to get the project ID this
is the project ID just try to copy and here try to pens then you have to give the region so I think region I already showed you so here you can check the region click on the collab Enterprise left hand side you can see us ipen Central one just try to select it us ipen Central one okay now just try to verify T Central H uh one okay huh so it's fine now let me save Now let's create the main. pi so first of all let's import all the necessary Library yeah so these are the
library I need guys I think remember I was also importing them in my collab Enterprise so verx AI then stream L I need for the user interface then I'm also importing some of the necessary uh let's say function from the vertic preview generative models okay so I need generative generation config generative model and EnV I need because I want to Load this EnV file now let's select my current environment so this is my environment okay now this error will be disappear now the first thing I'll uh load my project ID as as well as the
region from my environment okay for this this codit you have to run got it then I'll be initializing uh then I'll be authenticating with my verx a so this is the authentication code guys verx a. init you have to give the project ID as Well as the project location that's it now let's load the model so here I'm going to load one model from the model Garden called gin 1.0 Pro so let me show you if I go to the model Garden so this is the model guys I'm trying to load just click here and
you will see the model ID so just try to copy the model ID and you can use it so this is the model ID guys here just try to copy and mention it here okay that's it then I have created a simple function Let me show you so this is the function I created inside that what I'm doing I'm just first all creating my uh streamly title first of all I'm U first of all I'm setting my page configuration like verx a demo so it will give you one name then I'm uh setting the header
that means that means what will happen it will give you one title so here you can give any name so here I have given let's J mini on vertx a or you can give vertex AI uh vertx A local demo because everything we're doing in the local machine okay then uh user will give one question so that's why I have taken a uh text input box and this question actually I'm passing to my model you can see okay if question if there is a question it will execute this uh model otherwise it won't be executing
okay after that actually I'm going through the response and I'm printing in my console okay that means in my stream Le console that's it now if I want to execute I have to call the function as well so let's call the function that's it now let's try to execute and try to see whether it's working or not so I'll write stream l run app.py sorry main.py needs now let's give the allow permission so this is the interface guys now here you can ask anything let's say I'll give a hello let's see you can also see
the Terminal to get the log okay you can see it is giving me one error it is telling uh Google authentication default credential your default credential are not found to set the default configuration you can visit this link so let's open this link um and I think it will suggest me some command there ah so let's copy this command and now I'll execute from my terminal let I'll stop it now let's execute I'll select my account the Account actually I'm using for my gcloud so I'll give my password next so if you're doing for the
first time so you might get this kinds of issue so once this setup is done now uh you can use it as it is okay now I think it is done now let's execute the application again now let's keep hello okay guys now see I got the output there is no issue that means what you have to do you have to set the credential as a default uh default Actually okay credential so that's why we uh we actually executed this command gcloud o application default login got it now see I'm getting the output now here
you can pass any kinds of prompt so here I can give let's say uh tell me about python okay see I'm getting the entire response okay got it so you can now ask any kinds of question that means from our local machine itself now we are able To access the model Garden that means by verx a Services all right okay great so yes guys this is all about from this video now we'll be using this technique going forward whenever we'll be implementing any kinds of project okay so in the next video we'll try to see
how we can uh Implement a rag based application okay with the help of actually uh verx a platform and we'll be using some vertica Services there all right so as of now we saw how we can use Model Garden uh that means how we can access different different kinds of model on top of that how we can perform the inference but let's say if you have some custom data and with the help of that custom data you want to create a knowledge base and you want to connect your large language model there okay and you
want to perform the quy operation on top of it so how we can perform it and this is called actually rag I I think I already explained okay in my previous Video so if you're not sure about the r guys first of all check those video then you will be able to understand okay what I'm trying to implement here so that means uh let's say here I'll be using my custom data and with that custom data what I will do I create a knowledge base okay and I'll just try to connect my large language model
there then I'll be performing the quy operation on top of my data the data actually I have uploaded this is the thing we be be Implementing and here technology wise actually will be using the verx entire verx a platform that means here everything will be using uh Google cloud services okay we are not going to use any third party services that means uh to store our data we'll be using Google Cloud bucket and to store our embedding we'll be using uh Vector restore okay Vector restore service from the vertx AI this is one of the
vector database they have created okay this is called Actually Vector restore so we'll be using all the Google cloud service only not any other service okay in this video so for this guys you can see I prepared one notebook in the collab Enterprise so first of all try to connect the notebook after that you can check with the help of okay command and if it is giving the okay message that means notebook is running then the first thing what you have to do we have to install these are the library as I told you we'll
be using Uh some custom data I'm going to use PDF document that's why I'm installing this Pi PDF 2 okay this library then Google Cloud AI platform because if I want to use the verx a I need to install this package then I already told you we'll be using custom data that means custom document and to store this custom document I'll be using uh Google Cloud bucket I think you Google Cloud also provides bucket service let me show you so if you just Open Google cloud and search for bucket so this is the bucket so
here you can create different different bucket and inside a bucket you can store the data see I already created one bucket called start content 2024 inside that I'm having one documents I'll tell you how to uh upload this document here as of now just try to consider uh we'll be using this bucket services to store my content okay and no need to create manually it will Automatically create I've already written the code for it so these are the package you have to install for me I already installed everything so I'm not going to install again
now see you have to import these are the library right now so here I'm importing stories from google. cloud because I told you we'll be using bucket and to use to bucket I need this uh actually function then I'm also importing text embedding model from the verx AI language model because here We have to perform the embedding generation and to generate the embedding and need some embedding model and here I'm going to use Google embedding model OKAY from the vertx a from the model Garden that's why I have imported this text embedding model function then
I also imported AI platform Pi PDF and these are the library I also imported here and here I'm using UI ID okay why I'm using UI ID let me tell you see there is a reason actually I'm using UI ID see if you're using vertex AI this rag would be slightly different okay slightly different that means let's say whatever documents you will be publishing let's say I will be publishing one statistics PDF I'm having let me show you the statistics PDF so guys as you can see this is the statistics PDF actually I'm having so
this particular data actually I'll be using as my custom data now what will happen first of all I have to extract The documents from the PDF yes or no so after extracting what I will get I will get uh different different documents I think you know I'll get different different documents that means doc one okay then doc two doc two and so on now what will happen this document I have to convert to embeddings as well that means it would be converted to embeddings so let's say you will get some Vector here let's say 0.29
comma 0.55 0.88 and so on let's say this is the vector representation of this doc one and Doc two and so on let's say I'll I'll just create some dummy Vector here now here what I want to do with the help of this ey ID I want to assign some unique ID let's say the document one it has extracted I will assign one unique ID let's say this is the unique ID let's say 1 2 1 0 let's say this is the unique ID okay it has assigned to the doc one Then the doc two
it will assign another unique ID let's say 1 2 1 1 okay now what will happen this unique ID also would be assigned in the vector because this Vector is representing this documentation yes or no okay that's why this ID would be there 1 2 1 0 then this ID would be also there because this Vector is representing this document so 1 2 1 1 now what will happen let's say whenever you will perform the similarity s operation that time what happens Actually it will return the relevant answer yes or no I think you know
whenever you implement any kinds of rag whenever you perform similarity operation it will give you some relevant answer okay relevant relevant answer you will get okay and from where you will get the relevant answer from the documents only you will get the relevant answer the documents you are having okay now let's say it will give you this relevant answer let's say Doc one Whenever it will give you doc one relevant answer that means it will return the vector because your model will only return the numbers not the text okay and this particular number will map
where in this document that means this document is representing this Vector that means whenever your model is generating this Vector that means it will map this documents that means the text okay how with the help of this ey ID okay with the help of this ey ID so It will look for where this U ID is present that documents actually it will show you as a relevant answer okay now I think it is clear why I'm using this UI ID to give uh to assign actually unique ID uh like uh in front of my documents
and in front of my Vector embeddings okay now I think it is clear now let me clear my boat now the next thing what you have to do guys you have to initialize some of the variable now see if you're doing this project from the Google cab first of all you have to authenticate with your gcloud and how to authenticate I already shared you two code I think you remember two configuration code I shared with you in my uh like verx demo session so you can refer the notebook and you can first of all authenticate
with your Google Cloud okay said until you are not doing the authentication it w be working I'm running from a collab Enterprise that's why I don't need to do the Authentication so if you're running from the collab Google collabs so you have to activate this line actually project so you have to give the project ID here how to collect the project ID this is the project ID got it so here I'm running from the collab Enterprise so I don't need this variable I'll just try to comment it out so here what I need I need
the location and this is my location uh then uh PDF path so why do you have to upload the PDF so in the Files actually you have to upload the PDF let me show you so here you can right click and upload and try to select the PDF whatever PDF you are having just try to open it here it will automatically upload here see for me I've have already uploaded and this two file would be generated as of now U see I already executed this project okay that's why all the artifacts you can see okay
but for you it would be empty initially so this two file would be Created how it will be created I'll tell you as of now just try to consider we have uploaded the statistics of PDF file got it then you have to give the bucket name so you can give any kinds of bucket name and make sure this bucket name should be unique so this bucket would be created automatically in the bucket section see it is already created for me again this would be created automatically you don't need to create it manually then here you
have to give Embed file path what is embed file path because see I already told you uh first of all we have to extract the documents from my PDF then what we have to do we have to generate the embedding as well with respect to the documents and this documents and this embedding I I'm going to save in a Json file okay that's why I'm just creating some Json file you can see these two Json file I'm creating okay starts embedding Json and start sentence. Json inside sentence it will Store all the documents that me
the English text and inside embedding it will all the uh your embedding okay embedding of the sentence that means whatever let's say documents you are having that means whatever documents you have okay with respect to that it will store the embeddings as well okay with the unique ID so that is the thing then you have to give the index name index name means let's say here we'll be using Vector search now and inside Vector Search we we just create an index the way actually used to create the fine code index I think remember we create
index here so this is the index name it will automatically create that index now here I have written a function this function will extract the sentence from the PDF so it will take the PDF path and it will give you the sentences that means the all the documents it will give you then I have written some of the helper function guys you can see Generate text embedding that means this function will generate the text embedding it will take the sentence and it will generate a text embedding with the help of text embedding model you can
see I'm using text embedding Geo model uh so this is the model G text embeding Geo 001 so this model is available in the model Garden uh in vertx AI got it then I have written another function generate and save embedding that means that means this function will actually Add the unique ID uh in front to of your sentence that means documents and in front to of your uh embeddings that means Vector embeddings okay that means it will generate this both file okay and it will it will save as a it will save as a
juston file you can see here I'm doing it I'm first of all uh actually creating my embedding okay and I'm adding the unique ID I'm also creating the sentence I'm adding the unique ID then I'm saving in a Json file okay this Is the thing actually we are doing that means this function is responsible for creating this two file and it will store uh your sent sentences and it will store your documents as well as the embedding with the unique ID okay that's it then there is another function I created called upload file that means
after uh let's say generating this file I have to upload this start sentence dojon in the bucket and how I will push to the bucket we'll be using this function upload file You can see if you go to the bucket section so this file is already present okay so if you want to push any kinds of file you can use this function that means we created some helper function this helper function will help me to do certain operation this is the thing now here another function I have written called create Vector index so this function
will create the vector index in the vector s okay in the vector s Vector database and each of the dimension would Be 784 I think you remember previously we used sentence Transformer uh actually embedding model and we we used to create I think 384 Vector representation I think you remember right but if you're using this uh text Geo model from the model Guiden your vector Dimension would 780 sorry 768 this is the dimension and you can see with the help of AI platform actually Library we are creating the Index so this function will automatically create
the index for you got it then after that I'm calling these are the function you can see first of all I'm calling generate and save the vector I'm giving my PDF path sentence file path as well as the embedding file path then I'm also uploading this file to my bucket that means if I execute this line what will happen it will create this two file first of all let me open it so this is my sentence that Means this is my document and you can see first of all it will show you the ID then
it will show you the sentence okay that means this is the document it has extracted from the PDF with respect to that it has also saved the embeddings as well let me show you so this is the embedding and here you also you'll see the unique ID that means the same ID and now this is the embedding representation okay I hope now it is clear now so once it is executed then upload file will Execute and it will upload this sentence dojon to my bucket now if I show you see bucket would be created and
inside bucket actually you will see this start uh sentence dojon would be present okay because later on whenever I perform the let's say similarity s operation I need that file I need to download that file with that I'll will do the source operation okay similarity Source operation that is the thing now the next thing what we'll be doing we'll be Creating the vector index that means we'll be creating the index and inside index actually will be storing all of my Vector okay so here you can see I'm giving the bucket name that means from the
bucket name it will load this file okay it will load this file that means the sentence file so sentence file it will load then it will also take the index name and with the help of actually that embedding model it will generate the embeddings of that documents the Documents actually we are having and it will store to the vctor database that means inside my Vector SCE now if I execute this line okay if I execute this line it will take some time it will that means it will process all the data it will restore to
the vector embedding after that if you open it up let me show you so if you open it up so you'll see that it will create an index the name actually you have given start index okay now you can see both Places you are getting this green TI icon that means everything is ex uted successfully if you're getting any red icon it's not completed okay that means there are some issue now if I click on the deployed index you'll see one ID here so we need this ID later on so guys you can see this
is the ID the last number you can see this is the ID so we need this ID later on okay see this is the ID this ID actually we need later on so now let's Get back to my colar now again just import some of the necessary library then again said these are the actually variable like your location sentence file path and index name that means the same index name name okay same index name you have to give start index okay that's it then just load the model large language model here so here I'm using
JD you can also load any other model then just try to load the index that me index endpoint you have to load and to load The index endpoint you just need to give this ID and how you'll get the ID guys I already told you so this is the ID just try to copy this ID and just try to paste it here now if you execute it will load the index ID got it that means this is going to be your knowledge base right now and this is going to be your model now again I
have written some of the helper function generate embedding that means whatever user will give the prompt now let's say I have given one Prompt this prompt should be also converted to the embedding okay then this function will help me to do that then I written another function generate context and load file that means it will load that documents file and it will generate the context okay that means if we are performing similarity s operation uh it will give you some rank results and how it will give the rank results with the help of this two
function it will give the rank results now here I've Loaded the file then uh you can see then this is my query let's say what is corelation first of all I'm generating the query embedding with the help of that embedding model and if you want to see the query embedding see this is the vector representation guys this is the entire query embedding now let me comment it now this query embedding I will pass to my Vector SS okay now you can see find neighbor it will give me 10 rank results okay 10 rank results Actually
it will give me then this response actually I'm giving to my model okay this is the code cipit my model will try to analyze this response even the query I have given based on that it will give me one response now if I execute see if I execute this last kind of code see here you will get the response code delation is a statical measure that indicates uh see blah blah I'm getting the output and here you you can also give the prompt based on the Context uh delated in the brackets answer the query okay
that's how you can give any kinds of promp and that's actually we can build a retable augmented generation kinds of application okay with the help of vertic platform and all the steps guys I have mentioned here so if you're implementing any kinds of rag based actually application you can follow the same step okay even you can also perform in your local machine that means I told you how We can connect our local machine that means how we can connect our vs code so with help of vs code also you can create this kinds of R
rag application and you can create a user interface let's say you want to create a flask interface you want to create a stream late interface you can also create so this task actually I want to assign you just try to create one user interface so you have to use the same logic only you just need to integrate one user interface that is The idea okay I hope it is clear now see once it is done now if you want to terminate all the instance what you can do so first of all uh just try to
delete this uh so first of all just try to delete the bucket to delete the bucket just try to select the bucket and there is a delete button you will see just try to click on delete and delete confirmation you have to give so it will delete the bucket then you also need to delete the index so to Delete the index just go to the index Vector SE so first of all you have to do the r deploy then you can delete it right now I think now there is a delete button just try to
click on delete so it would be deleted okay so that's how you can remove all the instance you will be creating okay so I think guys uh this is helpful I showed you the entire uh uh like process how we can Implement rack See I think you already saw how we can set up everything in our local machine that means if I want to connect with my verx so what are the setup actually I can perform so setup wise I already covered in my previous video so you can check it out so here actually I've
already prepared everything because see most of the code are similar whatever things we implemented previously so make sure first of all you configured everything your account uh then your Let's say uh configuration everything you just prepared after that in the EMV file just try to uh change this project ID and region so let me copy my project ID so just try to click here so this is my project Ed gu just try to copy and let's mention it here okay and reason I'm inside us Central one only I don't need to change it now the
next thing you have to create an environment how to create the environment I already told you okay I'm Using the same environment Veri demo now you have to install the requirements so one more requirement I have added called flask so let me install the flask because in this project I'm going to use flask okay flask framework to create the front end application so for this just write this command P install ien add requirement. txd now it should install the requirements so guys many people were asking me how to uh install actually uh GPU Nvidia GPU
uh in our local system for this what I did actually I created one video in my YouTube channel let me show you so if you go to my YouTube channel guys this is my YouTube channel you can also subscribe me here now if I go to the video section so here I already created one video uh here you can see the video set up Nvidia GPU okay so here I showed you all these steps you can follow to set up your Nvidia GPU for the Deep learning Task got it so let's say if you're having
good GPU and if want to let's say execute open source our language model uh in your local machine instead of using collab you can refer this video that time and make sure guys you subscribe uh to this channel because here also I create lots of content related genv mlops then end to project implementation Okay computer vision everything now let's get back so I think installation is completed yeah now see What I have done I created one template file and inside that this is my HTML code because you already know if I want to use flask
I need a HTML okay HTML code to create my front end part and again I used the bootstrap website to prepare this front end I'll tell you whenever I'll execute how it will look like now this is my endpoint app.py and here I imported all the necessary library and these code are common I think you know we are doing the Authentication with my verx a now this is the route I created this route is my default route it will launch my index.html page and this is my prediction route that means whenever user will give any
query okay user input I'm getting the user input and this input actually I'm passing to my model and whatever response actually I'm getting I'm showing in my plus cver that's it now let me um execute the application so what I will do I'll run This application in the port number 8080 save it now I'll open up my terminal and let's execute python app.py okay so it is running now let's get back Local Host now see this is how your user interface will look like now he can give a any commment now here you can give
any query so I'll ask what is python now send the request you can also see the log in the terminal see I got The response so this is the complete response I got guys got it this is the complete response I got now if you want to ask another query just try to refresh again you can pass that query okay now that's how actually what you can do the previous rag uh application we implemented with the help of verx you can create your custom rag that means you can up use your custom documents like uh
we did one project like I think remember medical chatbot so that's how You can use your custom documents and you can create this kinds of application and this is the front end you can use as it is got it all the HTML code I will share with you so yes guys that's how we can also create llm power application with the help of vertex a so as I already told you in the model Garden you are having so many Foundation model and if you want to F tune these are the model so what are the
step you have to follow in this video I'll tell you each And everything so I think remember previously whenever we used to do the find tuning operation first of all we had to download that model in our system then I Was preparing the data then I was fine tuning these are the model but here if you're using these kinds of llm Ops platform you don't need to download the other the model in your system so here what you have to do you have to use that model endpoint that means you will be accessing this model
through the API Right that means uh first of all you have to authenticate with the verx I think I already told you how to authenticate with the verx a after that just give the model ID which model you want to use it will automatically hit that model so here the only thing you have to do you have to prepare your data after that all the thing would be happening in their server only okay now let me tell you the STP actually you can follow to perform the fine tuning Operation so for this what I will
do guys I will use the collab notebook so I'll open up collab notebook you can also use collab Enterprise you can also do it from your local machine it's completely fine but as of now we have explored uh like V code we have also explored like collab Enterprise now let's try to see how we can also use the collab okay like our Google collab so here let me change my account so let's say I'm going to use this account okay This account actually I'm going to use and make sure the account actually you are using
for the gcloud the same account you have to use in the Google Club as well okay otherwise uh you might get some authentication issue now what I will do guys I'll just create a new notebook here so let me create a new notebook now simply just connect this notebook and in between let me show you the fine tuning step you have to follow so see you can follow this uh blog Actually so this is from actually Google so Google developer so they have already given all the step actually you have to follow to perform the
fine tuning operation so I already prepared all the steps you have to follow to perform the fine tuning only I will show you one thing from this blog actually the data format okay so here let me show you the preparation and loading the data see if you're fine tuning on the vertx AI you have to prepare your data in the uh Json Json L format okay see this is the Json L format data you can see so here they have already given one example let me open up so you can see this is the U
train juston L format data and it is present inside this GitHub so what you can do from this GitHub actually see this is the data you can download this data okay in your system you can download this data so I've already downloaded let me show you so this is the data I'll open it with the notepad++ Now if you open the data guys you will see it's a Json okay Json structure that means you will have one key here called input text and this is the input text and this is the BBC data guys okay
BBC News data so in the input text actually they are having the entire news and they're having another actually key here let me show you or I can perform the contr F operation now now here I will just write output underscore Text now if I find it now see guys this is the output text okay so inside output text actually what they are having they are having the summary okay summary of that news so this is the data actually that's how actually you have to also prepare your data if you're having any custom data what
you can do you can prepare your data with respect to that that means what I'm trying to say that means in the Json uh file actually the first key will have the Input okay in input _ text okay inside input text what you will have you will have the entire story okay entire story and you will have another actually uh key called output text sorry output output text okay so inside output text you will have the summary okay summary of the entire story so this is going to be your data format okay that's why actually
you can take as many as record you can got it I hope now it is Clear the format uh so whatever dat you are having try to convert in a jonl format and how to convert in a JL format there are so many converter you will see inside python simply you can execute those converter it will automatically convert your data in a jol format got it that is the idea now you can ask me sir where I will get these kinds of data I think I already told you now so you can use hugging phase
Hub actually so there actually will have like so many kinds of Data set so you can use any kinds of data from the hugging ph up you can download it and you can convert to the jonl format and if you're using your custom data that time you have to convert everything in a Jin format okay this is the idea okay you can see the image also they have already given the data format so first of all you have to take the input text this is the entire BBC News you can see and this is the
output text just means this is the uh Summary of that uh story okay this should be your data format now apart from that they have already given all the step you can follow so I've already prepared one notebook let me show you what other the thing you have to do now let me also add this link in the notebook so that you will have the reference later on so this is the link I have added now here the first thing you have to install some of the library so here I'm going to uh install Google
Cloud AI platform because I want to access the verx a then data sets then Google Cloud pipeline components okay these are the library I have to install one by one let me install so after installing these are the library you have to restart the Kel and if you want to restart the Kel you can use this code snippit actually uh see you can also do it from the UI just click on the runtime restart the runtime So there is a restart runtime option okay restart the session otherwise you can also execute this line of code
it will also restart the your run time okay now what you have to do you have to authenticate with your gcloud account for this you can execute this line of code so here I'm using actually uh Google Authenticator with the help of Google Authenticator actually I will authenticate with my account got it so now let me execute so it will look for Your account so it will give you one popup screen let's give that permission select your account let's say I want to use this account because this is my Google account now I'll try to
select it now you have to provide the password so guys as you can see my uh authentication is completed there is no error that means you are successfully authenticate with your uh this one your gcloud account fine all right now the Next thing you have to select the project so to select the project you have to execute this line of code so let me select the project so I think you know how to select the project click here copy the project ID and just try to mention it here and with this actually we'll be initializing
our vertic so let me initialize my verx then let's say uh set some actually variable that means my region and the Project ID again I'll will give the same project ID now I have to set one configuration here for this you have to execute this line of quot okay you have to execute this command gcloud config set project this is the project ID okay the project ID actually I'm having here now let me set fine now let's import some necessary Library so guys you can see I'm importing some Necessary library now you can ask me
from where actually I got this code if you go to that uh uh blog actually so they actually they are suggesting install and import see they're also importing these are the library okay so just try to read this blog actually will see all the steps actually they have written I'm following the same thing only now let me import all of the library okay so I have imported all the library now the next thing uh I will Load my data so let's upload my data either you can upload in the gcloud bucket so from the bucket
also you can download the data if if you're having let's say use data that time you can keep in the bucket I think I told you so it is already having one bucket Service as well so you can also open up the bucket inside that you can store any kinds of data and you can also load it here okay so if I show you the bucket so this is the bucket guys so here my data Size is not used so what I will do I'll directly upload here so I'll just upload this is the data
now with the help of pandas actually I'm going to load my data so now we can call Cy the path and you can paste it here now let's execute okay see this is the data I have loaded now you can see this is the input text and this is the output testt now if you want to see the shape of the data You can also see this is the shape of the data and now we can start the training and it's like very easy see only you just need to execute this line of code your
training would be started in their server only okay not in my Google collab actually uh it will use their instance got it now see here is my fine tune model name so this model name actually I'm giving BBC fine model you can give any name so after fineing what would be your model name okay that is What you have to give here then you can give the Preen model that means which model you want to find you here I want to find text Bon 002 model and if you go to the model Garden you will
see this particular model and if you want to use any other model you can also give the model ID here got it that's the thing now you have to pass the data that P by DF the DF actually I got and you have to set the number of epoch training Epoch you want to let's say train so here I Mentioned 100 Epoch okay I want to train as of now and you have to give that tuning job location okay so I have given EUR W 4 you can keep the default size if you go to
the like blog you will see that they are also using this particular location now see if I execute the program it will give me one URL and if I click on the URL I I can see the pipeline that isans completely uh that means I can see the complete pipeline my training process is running and it will Take guys uh like one to two hours to complete the training but I can't wait one to two hours what I will do I'll stop the execution so guys I stop the execution because it will take time see
I'm telling you after actually executing this line of code what you have to do so you can see one URL just try to open up the URL so this is the URL guys and try to make make sure you have selected your account okay that gcp account then you can see this pipeline okay this uh Pipeline would be running and this is the jobs so here you will see all the let's say logs so in the pipeline section you will see all the logs that means whether your training is running or not okay and how
how how much time actually it will take to complete so everything all the log actually you can see here got it so once this execution is completed you will see it will automatically stop after that your model would be ready okay and then you can use This model and if you want to delete this pipeline so what you have to do guys so there is a delete button just try to click on the delete and try to delete the pipeline okay it will automatically delete the instance okay and guys one thing after completing the training
if you open up your bucket you will see two bucket would be automatically created okay and inside this bucket actually it will save all the model checkpoints okay let's say Whatever training it is doing all the artifacts would be saved inside this bucket itself okay you can see but once you complete the execution see it is it will already save the tune model okay inside that that you can see the text B model got it so that's how actually it will save all the artifacts in the bucket and from this bucket itself we have to
deploy our model as an end point okay and this model would be available in your model Garden so if you click on The model Garden so there you will see one option so here is the option my uh view my endpoint and model if you click here you will see that that model would be present here okay I hope it is clear now let me show you the next step you have to perform so previously I already did this actually uh find un name so let me show you what you have to do after that
see after that actually what you can do you can use your train model that means the model actually you have Trained and you can do the inference operation so here I'm giving this uh actually uh prompt actually summarize this text to generate uh generate a title so this is the entire actually prompt I'm giving and this is the output actually I'm getting got it now you have to deploy your model as an end point for this you have to execute this line of code okay so this will uh like deploy your model as an endo
okay to the gcp server that means this model would be Available in your model Garden right now so if I go to the model Garden so here you will see one option view my endpoint and model just click here in this section actually will see your model would be published okay from here actually it will load the model but what I did actually I deleted the model because again I'm using Cloud platform I don't want to keep anything okay otherwise it will charge me so that's why I already deleted okay then after That I was
loading that model and I was again performing the prediction you can see this is the results actually I'm getting from my model then I was also loading the base model that means the actual model and I given the same actually prompt and I was just checking the actually output okay whether this output and this output uh actually good or not that means which model is giving the better output and here you can see both model uh giving the good response Okay it's not a bad although I only trained 100 Depo but you can see this
response is pretty good got it so that's how actually you can perform the fineing operation after that you can deploy your model as an end point to the uh gcloud okay now I I think it is clear now you can pick up any other model and you can perform the same F tring operation all right so this resources would be shared guys in your resources section from there you can uh download this notebook And you can perform the fine tuning operation so guys uh as of now we have seen the uh vertx a demo the
complete let's say verx a demo we have seen we have implemented different different kinds of application with the help of vertx a so now it's time to explore another H name ofs platform called AWS badrock so this uh servic is available in the AWS Cloud so make sure you have the AWS account so if you don't have the account just try to create one account Initially it will give you $300 free credit but you have to add your card there got it so after login with your console so you will get these kinds of window
okay in the AWS so this is my console so now here what you have to do you have to search for bedro so let me search so see Amazon bedro and this is the service actually the easiest way to build anale gen application with the foundation model so let me open up the B drop m Now I'll just click on get started see if you're opening for the first time this Bedrock what you have to do you have to request for the model access so there is a button you will see request model access just
try to click here and see for me I already requested the model okay for me I already requested the model and I got the access you can see some of the model actually they didn't given me the access for this what I have to do I have to again uh let's say Resubmit the access request then I think I might get but for you what you will do you will select all the model and try to send the request okay send the request will see that in 5 to 10 minutes actually they will provide the
access got it and make sure you check your region actually in which region actually you are sending the request okay because that in that region actually your model would be available so I'm currently inside AP South one that means Asia Pacific Mumbai okay so now if I go to any other region let me show you so if I let's say go to North Virginia so there you will able to see I don't have any access of any model now see it will show you these kinds of window enable all model okay now if I click
here now just select all the model and try to click on the next okay now you have to submit the request okay you have to submit the request so let's submit the Request sometimes actually they will ask for some information so you also need to uh let's submit some information let's say your name email address okay then you can send the request after sending the request in 5 to 10 minutes you will see that they will give you the model access so guys you can see after sending the request actually I got the access you
can see granted access and some of the model you can see it is still in progress so I think it will take some Time to give me the access okay so that's actually you can apply for the access so once you got the access that means congratulations now you can start with the Bedrock services that means now you got all the let's say access of the foundation model now let me get back to my that region actually mbai region H now see this is your bedrock interface if I click on the overview now you are
having different different Foundation model here and what is Foundation model I already told you these are large language model and different different companies also has collaborated with the AWS even they have also published their model like you can see Amazon is having their own model apart from that cloud has like updated their model then coh Lama mistal okay so you can see all the provider okay all the provider they have collaborated with the AWS and they hosted their model on the AWS that means AWS bed drop okay now What you can do you can access
the the model through API got it so you don't need to download the the model in your system that is the m Advantage here now you can also play with the playground let's say if I click on the chat so here is the chat guys so you can select the model let's I want to select this model um let's say I'll select this llama 3 8 billion instruct model now if I send any request let's say Hello let say this is my prompt now see it is giving me the answer so that's how actually it
is having different different kinds of playground text playground chat playground image playground because it is also having multimodel as well so you can see here different different model let's say I you want to use this uh uh Amazon model so just try to click here see all kinds of Amazon model you'll be able to see it is having Titan uh text G1 light Titan Text G1 Express okay Titan image generator then uh Titan multimodel embeddings now if you want to use let's say this model what you can do you need to just copy the model
ID so this is the model ID and with the help of model ID you can use it similar wise for The Meta Also let's say you want to access llama okay llama model you can see so for this you have to use this uh you have to use this model ID okay even from here also you can directly open in the playground And you can also play with these other the model even if you go to the example section they have given different different example even you can also copy these are the code and you
can execute in your system now let's click on the Bas model you can see different different kinds of base model it is also having even if you want to train your custom model it is also possible just go to the custom model you can also perform the fine tuning operation so from the UI Itself you can do the fine tuning operation even let's say after fining you can also keep your model here okay in the custom model section so that you can load this model as an API okay every functionality actually this Bedrock will give
you so you can use them so yes this is the overview of our Amazon Bedrock that means it's a platform it's LM of platform here we are having different different kinds of foundation model even all the functionality it is also Provides with the help of these are the functionality you can create any kinds of gni based application so in the next video what we'll do guys we'll just try to uh uh do the handson on this uh AWS B drop that means we'll be accessing these are the foundation model through the python okay python code
this video we'll be learning uh how we can let's say access this kinds of foundation model uh without python code that means we'll be set uping everything in our local Machine I will be writing everything from our V code okay so what I will do guys uh I'll open up my local folder and here I've already created one empty folder let me open up my vspot okay so I'm also going to open up my terminal here and make sure you create one environment so for me I already created one environment so let me activate so
cond activate llm app I think remember this uh environment we created previously so I'm using the same environment here and you can see the python version I'm using it's 3.10 so you can use 3.9 3.8 it's up to you okay here then first thing you have to set up the requirements so let me create a file called requirements. dxt inside that let me mention all the requirements actually I need so I need the langin then I also need stream Le because I'm going to create a small Let's say user interface here and I need two
more Library boto three and python. python. I think you know why we need it because we here we're going to create one EnV file okay EnV file so inside that you have to mention the um I mean your secret credential that means we'll be generating access key AWS access key and AWS secret access key with the help of that we'll be authenticating with our AWS account got it so we'll be uh Keeping these are the key inside this EnV file and to load it I need this python. EnV and what is boto 3C with the
help of boto 3 python package will be making the connection with our AWS okay so this Library will give you the functionality you can connect with your a this now let me set up everything in my environment so I'll open it up and just write P install requirement. dxt okay for me it is already installed For you it it will take some time now the next thing what I have to do guys uh I have to create a im am user so let's go to my AWS I'll go to the homepage and here you will
see or you can also search I am okay identity access management because I have to create an user and there actually I will only give the permission related my Bedrock I click on the user now just create a user here I'll give let's say Bedrock test attach policy Now search for bedro I want to give the bedro Amazon bedro full access now click on the next and create the user then after that I will click on Bedrock test and go to the security credential and here you'll get one option uh create access key just try
to click here uh select common line interface I understand and click on the Next now create the access key now so guys this is your access key and this is your secret access key now you can also download as a CSV file let me download now I have to configure it so for this you have to install one tool actually so just search for AWS CLI okay AWS CLI this is the first website you can open so for all the operating system they have mentioned okay how we can install it for Linux Mac macway okay
see Linux You have to execute some of the command these are the command you have to execute for Mac wise also you have to execute some of the command and for Windows you are having one MSI file so you can directly download this MSI file and you can you can install Okay as a software that means the way you usually install any software now next next install you have to do the same thing after doing it let me cancel it because I already have after doing it just try To open up your terminal okay and
execute this command AWS configure AWS configure now see if I execute it should give you these kinds of window if you're getting these kinds of window that means your installation is fine now what I have to do guys I have to set this at the access key now just try to copy the access key open the terminal and it is asking for the access key just try to paste just right click it will paste and press enter Then you Have to give the secret access key just try to copy this is my secret access key
and I'll paste it here and don't share guys this key with anyone otherwise they will be also accessing your account I will delete it after the recording and now it is asking for the region now you can see which region you are in I'm inside mumai that means AP South one so I have to give the reason so you just check which reason you are working on based on that you can decide so AP South one Now default output format just keep it empty press enter that's it see my credential has been set so this
was the first method and the second method what you can do you can uh I think you remember we created this EnV file so here inside EnV file also you can mention so let me show you so inside that you can mention AWS access key ID secret key ID and region name so let me replace all of the secret key again is My access key ID and this is my secret access and sou one it's fine now let me save now here I'm going to create a file I'll name it as main do PI inside
that I'm going to do the coding so here first of all let's import some librar so these are the library I need let me select my environment uh llm app I'm using now here you can see guys I'm importing Bedrock from Lang Chen Lang chen. LM Bedrock I'm inputting Bedrock then llm Chain then prompt template boto 3 and stream lit okay then I need another package uh this uh one actually because I want to load this at the credential and how to load guys I think remembered so you have to use this code snippit so
load ENB and you also need operating system with help of operating system we'll be loading these are the key okay from the environment variable now here it will restore everything now if I want to initialize Bedrock client so I have To use boto 3 so this is the syntax if you go to the documentation you will see that they're using this synx Bedrock client you have to make a Bedrock client now boto three client you have to mention the service name I want to use Bedrock okay Bedrock Services that's why I've given Bedrock run time
you have to give the region AWS access key ID and secret key ID so with the help of that it will authenticate with your account okay I hope it is clear now The next thing you have to give the model ID like which model you want to access I'll give the model ID is equal to let's say I want to use one Foundation model let's say I want to use this mistal model I'll click on mistal now you can see mistal having different different variants 7B 8X 7B then Mr Large now let's I want to
use srct so what I will do I'll copy this model ID so that's how just try to open any kinds of model and Copy the model ID and here you just need to mention it that's it okay so it will automatically load this model now let's create our llm rapper so inside Bedrock you have to give the model ID okay and here you have to give the Bedrock client okay client is equal to bed drop fine and this is the temperature parameter that means the creativity parameter I think you already know now here I'm going
to create a function so this function will take the Input from the user like in which language they want to let's say get the output and what is the input prompt okay so this function will take this input you can see language as well as the user text then it is create then inside that I'm creating a prom template I think you know what is prom template right so this is the input variable language and user text and I'm creating a prom template youf chatboard and you are in this language and this user text now
based on That you have to give give me the response now here I'm creating a llm chain and I'm passing my llm as well as the prompt and finally I'm passing my language as well as the user input to my uh Bedrock chain that means the chain I actually have created okay because inside that I'm having my large language model as well as the prom and whatever response I'm getting I'm just returning that's it now let me create a stream Le title stream L let's say user interface Bedrock chatbot demo now user will also able
to give the language so let's say here I'll take a language input from the user whether uh he want to get English response Spanish response or Hindi response you can so here you can also pass any other language it will also work because I think you know these kinds of large language model can also support multi language then this is my final logic so if language is there so they will give the input that means what Kinds of question they want to ask then once they have let's say given the prompt this particular prompt I'm
checking first of all this uh they have given the prompt or not if they have given I'm passing it to my my chat bot you can see this function so it will take the language as well as the user input language and user input and whatever response actually I'm getting I'm writing in my stream lit application now let me execute and let me show You I'll save it now simply I'll just open up my terminal and this is the command stream lft run main.py so this is the interface guys now you can select the language
that's I want to select the English now here I can ask any query let's say what is python now you just need to press shift uh sorry you just need to press control and enter so guys as you can see uh if You give any kinds of prompt now here is the response so here I've given what is Python and here is the response I got now you can ask any of quy let's say I will ask uh uh give me a python code to add two numbers see here I got the python code to
add the two numbers okay so that's how you can pass any kinds of prompt and you can uh get the response it's amazing Guys and and here we're using Mistral model OKAY Mistral model we are using okay and and it is present inside Bedrock platform that's why this inference time is very low you will see that I'm getting very fast response but whenever I was using mral model I think remember in hugging pH okay hugging pH Hub so there I had to download this model in my system and the execution time was like very slow
there yes or no right so that's why we have to use this Kinds of lmos platform if I'm implementing any kinds of project efficient project okay in the industry so yes guys now you can uh explore different different kinds of model different different kinds of like Foundation model you can also export Co model meta model anything you can export only just need to keep the model ID and everything you can access got it so yes guys I hope uh it is clear now how we can use this Bedrock platform and if you Want to get
this kinds of content more so please try to support our channel so please try to subscribe to the Channel Try to share these kinds of content with your friends and family so that they can also get to know okay this kinds of session is going on so please try to support the channel guys try to like the video try to subscribe to the channel you can subscribe to the code Commander Channel even you can also subscribe to my channel D with buy okay both Channel Actually I'm operating you will see that in both Channel I'm
creating lots of content related data science machine learning genbi mlops okay n project implementation okay everything I'm creating the content if you don't know what is AWS Bedrock AWS bedro is a platform from Amazon web service we can use this particular AWS Bedrock to create scalable genbi application with the help of large language model so if you want to use this particular platform Uh Amazon Bedrock so you should have one AWS account so so for me I already have the account so if so if you don't have any account so please try to create one
AWS account then you will be able to access this particular platform now if you can't see this particular service here so what you can do you can search here called Bedrock okay if you just search for Bedrock so you will get this particular service called Amazon Bedrock now let's try to click here so it will Launch this particular platform for you see guys so this is the Amazon Bedrock platform and this is the easiest way to build an scale genbi application with the help of foundation model so it is having lots of foundation model I'll
tell you what are the models actually is having so it has one beautiful documentation you can go through this particular documentation like what it can perform and all so you can perform Tex generation you can also create a Chat BS you can also perform the searce operation Tech summarization image generation and personalization okay so this many task actually you can perform here and these are the actually uh Foundation model so it is having let's say lots of model it is having Jurassic 2 model cloudy model command Stone defusion and Amazon Titan and you have another
model called Lama 2 okay Lama 2 is a model from met AI so this is one of the like very popular and Powerful model Okay you can use so in this particular project actually I'm going to use this particular Lama 2 model okay and I will also use any other model to show you like how we can access this particular Bedrock model OKAY from the Bedrock platform now what you have to do you have to click on this particular uh geted button so if you just click on the get started button okay so initially it
will tell you just try to request for the model OKAY request to access all the Model because if you're using for the first time you have to send a request like toess access all the model okay so I already made the request and I already got the permission and I I can access all the model OKAY from this particular platform now guys as you can see so in the overview section so these are the models are available now let's say if you want to use this particular model jasic to model just click on this particular
model now if you see Jurassic Model is having different different variant okay so it is having Jurassic to ultra and it is having another model called Jurassic to Mid okay you can either use this mid model either you can use this Ultra model okay so similar wise if you if you're using let's say Amazon okay so it is having different different model so it is having let's say Amazon uh this Titan embedding G1 text model so this is the embedding model then it is having Titan text G1 Model light model then Titan text G1 Express
model okay that's how it is having different different model even it is also having Titan image generation model okay that's actually let's say if I click on meta okay that means Lama 2 so it is having different different variant of the model Lama 2 so it is having Lama to chat 1 billion model it is having Lama to chat 70 billion model even it is also having to 30 million model and 70 million model okay so these Are the preent model and these are the chat model let's see if you want to perform question answering
and chat operation you have to use these are the model and let's say if you want to only generate let's say a text and if you want to let's say do the sentence completion at that time you can go with this particular model okay so that's how actually it is having different different model OKAY based on the base model actually you can see here so again I will go to the overview now you can see I already showed you the foundation model now it can perform different different task I already showed you let's say you
want to do any example so we just go to the example section now here you can select what kinds of task actually want to perform so here you can see different different task even it is also having the playground let's say I will open this particular playground okay so playground I want to perform This chat operation so now here what I have to do first of all I have to select the model I want to use so let's say I will be using this particular model called Jurassic to mid model so I'll select this particular
model and apply okay after that I will get this kinds of interface now here you can give any kinds of promp let's say if I give hello so here let's say if I send hello okay to this particular uh model so it will give me the Response see it is also responding like uh hello how are you how how I can help you today that's why actually you can also change the temperature parameter topic parameter okay so these are the parameter you can uh change so what is the temperature parameter temperature parameter is a creativity
actually parameter so let's say if this temperature parameter is like let's say close to zero that means you are telling your model uh to be stick okay it don't Be taking any risk and if you are setting this parameter to close to one that means you are uh like telling your model just take any risk okay okay and try to give a uh creative output so that's actually you can change these other parameters so by default you can so by default just keep this default parameter only now you can give any kinds of prompt here
okay it will response with respect to that only now guys this is for the playground okay but Whenever I'll be creating the application I I'm not going to use this particular model like that so I have to uh use this particular model okay with the help of this Python programming language so with the help of python client actually will be accessing this this particular model but how to access this particular model because this model is already available in the AWS okay so how to authenticate with the this particular account so for this first of All
you have to create one IM user okay so first of all let let me create one IM user and let me uh like add this particular secret key and access key then I'll tell you how we can access this particular uh Foundation model OKAY from the AWS bedro uh so guys as I already told you to authenticate with this particular AWS and if you want to access this particular Bedrock services and all so you have to first of all create one IM user and you have to Collect that particular access key and secret access key
okay then you have to configure that so for this you have to search for this particular service called I am okay that means identity access management now here I'll go to the first option I am now left hand side you can see there is a user option okay let's click on the user now here I'll be creating one user create user just give the name so here is let's say I'm doing a Bedrock project So I'll just give it as Bedrock user okay then I'll just click on next then I will be adding the policies
directly so here you need to select this particular option now here you have to search for the Bedrock service okay because I want to only give the Bedrock access okay not the enre service access okay from my WS account now here if you see Amazon Bedrock full access okay so I'll select the first option then click on next then once it is done I will just create the User now my user has created now I'll go to the user I'll go to the security credential and here you will see something called create access key okay
let's click on the create access key now I select on the common line interface then I understand the above recommendation then click on the next then once it is done I will create the access key okay now see guys this is my access key and this is the secret access key and don't share this particular Credential with anyone else otherwise they will also able to access your account okay I'm sharing uh because I'm going to delete this particular user after the recording okay that's why I'm showing now what I have to do I have to
download this thing as a CSV file so click on this particular button okay it will download as a CSV file okay see I've already downloaded now if you want to authenticate okay if you want to authenticate with your actually code so What you have to do first of all you have to install one tool okay called AWS CLI Okay aw CLI download just search on Google aw C download so this is the website actually you will get and you have to install this particular tool now you have different different operating system let's say if you're
using Windows so you will get one MSI file okay as you can see this is a MSI file so try to download this particular MSI file and try to install it the way actually you Do install any regular software Okay then if you're using any macway or let's say Linux operating system you can follow these are the command to install this particular aw CLI so here I'm using Windows machine so I already downloaded this particular MSI file okay let me show you so this is the MSI file so inside windows so this is the MSI
file if I click here so it will start download see guys it will start download okay now we cancel it okay after Downloading you have just need to double click and you to install as a regular software okay then once it is done then I will open up my terminal so first of all let me open up my local folder okay so this is my local folder so here I'm going to open up my terminal okay but before that let me create one folder here so inside that I'm going to do my project so here
I'm going to name this particular folder as Uh um Genera application okay generative AI uh gen project okay bed Dr let's say this is the name now inside that I will I'll open up my terminal okay so let's open up my terminal yeah so now here uh what you have to do first of all you have to test this particular aw whether it has downloaded successfully or not so for This you just need to execute this particular command called AWS okay configure so if you execute this particular command so it will give you this kinds
of interface see it will give you this kinds of interface and if it is not giving this kinds of interface if it is throwing any kinds of error that means you are not able to install successfully okay you have to install it successfully so for me I already installed successfully that's why I'm Getting this particular option now here you have to provide that particular credential okay that FW secret key ID you have downloaded now let's go back to the CSV file I downloaded so this is the CSV file let me open it up so guys
I already open up this particular file with the help of my notepad++ now as you can see this is the access key ID now you have to copy before the comma because this is the comma separated this is a CS v file so I'll copy this Particular credential and I'm going to open up my terminal and I'm going to paste it here okay so just right click and paste it okay so it hasn't com copy yet I think so I'll copy again open up the terminal and let me remove now I'm going to paste okay
now see if you right click it will paste now if you hit enter it will automatically save okay now you have to also set for the a secret access key okay so this is The secret access key so copy before the comma and again open up the terminal and past it here okay then it is telling select the region okay so so make sure you are checking the region currently which region actually you are in so for this just go to the home okay AWS home so here you actually will see that particular region okay
see now currently I'm inside North Virginia that means Us East one okay but if you're in any other region you have to select this Particular name let's say you are inside Asia Pacific that means Mumbai you have to write AP South one that time but I'm inside Us East one okay that means not Virginia so I'm going to write Us East one here so I'm be opening the terminal and here you just need to write in the same way us past one okay now I'll press enter now you have to keep it as default press
enter now see all the credential has been set now I can easily authenticate with my Bedrock okay Bedrock platform with the help of this particular terminal okay now what I have to do so let me open up my folder the project folder I have created now inside that I'm going to open up my V code okay so let's open the vas code so here let me write down all the steps so that that you can refer later on also in the readme file I'll add everything so here let me close it now first of all
I'll create a file called readme okay readme do MD so here first Of all Let me give the project name so this is uh N2 okay n2n generative AI a project okay using AWS bedro bedro okay so this is the title of my project now inside that I'm going to write the step okay you have to perform to set up this particular project so first of all you have to create one virtual environment so I already written So let me just uh give it it here so these are the step I have to perform okay
if I want to execute this particular project so first of all I have to create one okay I have to create one virtual environment so let's give the name so this is let's say Bedrock project okay so Bedrock Bedrock okay Bedrock project Bedrock Pro Bedrock project now you have to activate this particular Bedrock uh Environment okay you have created and make sure you using python 3.8 you can also use more than python 3.8 is completely fine then you have to install the requirements because I'm going to write some requirements okay for this particular project you
also need to install them then I already told you you have to uh download this particular aw CLI okay and this is the link I have already given then you have to set up it okay after set uping you have to Configure that and add all the credential okay so these are the step actually you have to perform then uh what I will do I'll quickly uh create some of the folders and file here so first of all I'm going to create a folder called research okay research inside research I'm going to create one file
called trials okay I can write B Bedrock trials okay Bedro trials do file then outside I'll be creating another file called uh main. F okay then I'm going to create a requirements file requirements okay. dxt yeah so as of now I need these are the folders and file if you want to add uh like G ignore and license file you Can add it if you want to comit this particular code in the GitHub but I'm not going to comit okay so that's why I only need deserve the files and folder now first of all let's
add all the requirements actually I need for this particular project so first of all let me show you what are the tools and Technology I'm going to use so for this I'm going to open up my Blackboard so here uh I'm going to use some of the tools okay so the first thing I'm going To use Lang chain okay I think you already know what is Lang chain Lang Lang chain is a generative VI framework okay so we can use this particular Lang chain to create generative VI application okay so it it is having different
different functionality uh so it will help you to create that particular application okay so then second I need something called P PDF okay Pi PDF why I need P Pi PDF because I'm going to um upload my PDF Data okay and from there actually I'm going to uh do the retriever operation okay that means qu operation I'll tell you like what kind of of application we're building okay so we'll be building something called rag application so there actually we'll be creating one knowledge base okay with the help of custom data actually I'm giving so that's
why my data source would be PDF data you can also use dogs data or txt data it's it's completely fine but I'm Going to use PDF data here so I'm going to like also show you the architecture okay architecture of this particular project like how everything will work but before that let me write down the tools and Technology I'm going to use in this particular project then I'm going to use something called stream lead okay stream stream lead streem L I'm going to use to create the user app Okay because I'm going to create a
user app so so that actually User will get some interface to use your application so for this I'm going to use is stream lit you can also use flask okay first API or any other let's say web framework it's completely fine but streamlit would be easy for me to like you can Implement because there I'm not going to write any HTML and CSS code okay then after that fourth I'm going to use Vector DV so Vector DV wise I'm going to use f okay so F so f is a vector DB so this is for
Facebook Okay Facebook team has this particular Vector DB called f and this is the local Vector DB okay you can also integrate Pine con then we chroma DB okay it's up to you but I'm going to use f so in future maybe we'll be creating some other projects so there actually we can use any other Vector DB okay so these are the tools and Technology I'm going to use for this particular projects and definitely the final tool and platform I'm going to use something called bedro Okay bedro from the a okay so Bedrock I'm going
to use and llm wise I'm going to use something called Lama 2 as I already told you I'm going to use Lama 2 okay from M so this is the LM I'm going to use so yes so this is the tools and Technology I'm going to use now let me quickly uh I mean install these are the tools and technology so what I will do I'll open up my code and here let me write down everything one by one so first of all I need something called Len Okay Len then I need something called P
PDF P PDF then I need something called stream lit okay then I need something called fire CPU P CPU okay so let me save it now first of all I need to create an environment so I'll copy this particular command and I'm going to open up my terminal and let me create this particular environment first of all so guys as you can see my Environment is created now I have to activate so this is the command G activate so now let's activate also now see I'm inside this Bedrock project okay that means this particular environment
now I have to install the requirements so for this I'm going to execute this particular command see now it is installing everything so it will take some time okay so let's wait I'll come back when once it is Done so guys as you can see installation is completed now what I have to do yeah so now I can uh start the coding part so yeah so make sure uh yeah so now what I will do I'll go inside this particular resarch folder and first of all I will test this particular Bedrock okay like how we
can connect this particular bedrock and how we can access the model and all because I already showed you it is having some Foundation model and if you want to access it so for this First of all you need to do some authentication so that authentication we have already completed so let's go inside this particular Amazon Bedrock yeah so now see these are the models are available okay and I think you already know this Lama 2 then Mistral so these are the model is also open source model okay so this model is also available in the
hugging face okay hugging face is also having these are the model let's say I want to access Meta Lama model okay so this is the model actually I can see so all the model actually you have 7B then 70b 13B all the model actually you are having but if you want to load this particular model okay you have to download this particular model from the hugging P itself okay for this you need GPU machine okay you need a GPU machine and you you you should have a good memory there okay otherwise the you can't actually
load this particular model even If you have already used this particular model okay so you'll see like inference time would be also High because it's a huge model again okay uh because everything you are running on your local machine okay and then you are executing the are the model but whenever I'm talking about this particular Amazon Bedrock so this model actually is already hosted here okay uh as an API so you can connect this model through API okay API request so first of all you Just need to make a client okay with the help of
this particular client you can send a request that means you can send a prompt and you can get a response okay from this particular model and this execution time would be very fast okay so for this if you want to let's say create a production grade application and if you want let's say Fuster inference okay at that time you can go with this particular Amazon Bedrock service okay but let's say you have uh Good instance okay you have a good instance you have lots of money you can buy good instance good GPU configuration machine and
all and if you are not worrying about the fer inference at the time you can go with these are the model okay you can like manually download this particular model you can use it but here we'll be using the client okay to access this particular model and every request actually you'll be sending okay will have some of the cost okay now if you Want to check the cost also so you can check it out let's say if I want to use llama model so Lama is having there uh like cost okay cost per request so
here you can see the view pricing option so part token actually it will charge you okay so see so if you if you're using this this model Jurassic to mid model so let's see the Llama model I think llama model is also there yeah meta Lama so this is the price uh per thousand tokens okay see Per thousand tokens this is the price actually it will charge it's like very less okay lesser than your open I think okay so uh there is no like more charts so you don't need to worry about so if you
want to create a produ grade applications you can go with this particular Amazon Bedrock service okay you can visit that particular pricing pce and you can try to understand now what I have to do guys uh first of all let's try to access one particular model OKAY from this particular Amazon bedro so let's say I want to access this particular model called this Jurassic model OKAY Jurassic model I want to access that's I want to access this particular model Jurassic mid model so if I want to access this particular model I have to hit the
API request okay so here I have to send the API request but I'm going to send the API request throughout the python okay so guys now let's see how we can uh use this Particular Amazon Bedrock that means the different different actually Foundation model okay so first of all we'll be doing this particular experiment and if we are able to access these are the model okay so next actually I'm going to show you how we can create this particular uh Inn rag application with the help of this particular Amazon bed so I'll go to my
code editor and first of all here I'm going to import this Bedrock from the langen because it is Already available in the langin okay Len framework so we have already installed this particular langin so make sure you have selected the correct environment so so let me refresh so this is the environment uh I'm using yeah so Lang chain do uh you have to import llms then inside that you have something called this particular service called Bedrock now I'm going to import this particular Bedrock class okay so bed drop then I also need something called llm
chain Because uh here whenever you will be creating that particular QA object okay so you need this particular LM chain so let's create this particular LM chain so from L chain dot chains so if you already know about Lang chain I think you already know what is the things actually I'm importing because these are the requirement actually I'm expecting you are already familiar with this particular Lang chain framework okay so chain I'm I'm going to Import something called LM chain then I'm going to import something called prom template so here I'm going to write prom
Lang chin uh do prompts okay I'm going to import something called prompt template then I'm going to import another Library called boto 3 okay so if you don't know what is boto 3 boto 3 is a library python Library actually we can use to connect with our AWS account okay But here we haven't installed this particular package so let me install also so here I can write boto 3 now let me open up my terminal now again I'm going to install this particular requirements so installation completed now let's create the client quickly so with the
help of this particular um client actually I'll be accessing the bedro so we have imported B of three then I also need to import uh stream Lead Okay because I'm going to create one Basics uh UI for my user okay so that's why it's needed so let's import it so I import stream lit as STD yeah okay it should be uh import okay not form yeah now I think everything is fine now first of all I'll be uh defining the Bedrock client okay so let's define the Bedrock client Bedrock Client so for this I'm going
to use this particular boto 3 so boto 3 okay dot client and uh here you have to mention the service actually want to use so service servicecore name so I want to use this particular Bedrock service okay so you have to write Bedrock underscore runtime okay bedro bedore runtime then you also need to define the region like which region actually you want to use in the AWS you are having I already told You how to check the region so we are inside this particular Us East one okay so let me also write down this particular
region so I'm inside this Us ipen East ipen one okay so this is the reason so yeah guys this is my client so I'll store this particular client in in a variable so let's name it as Bedrock okay Bedrock client then I need to write the model ID okay which model actually want to access from the Bedrock okay so you have to Define the model ID now how you will get this particular model ID so again go back to this particular bedro and I already told you it is having different different Foundation model let's say
I want to use this particular Jurassic 2 model okay I'll click here now it is having different like variant Jurassic and Jurassic mid let's say I want to use this particular Jurassic mid I'll click here now if you just go below this is the model ID okay now let's copy this Particular model ID I'll copy and I'll open up my code and here I'm going to paste it okay so this is the model ID now here what I have to do I have to create a llm rapper so let's create LM rapper llm is equal
to uh bedro and inside that first of all you have to define the model ID OKAY model ID so model ID means the model ID actually we have defined okay this particular model I want to access then after That I have to also Define the client okay so client is equal to this Bedrock client we have created okay with the help of this bedro client it will authenticate and how it will get this particular credential I think you remember I think you remember we already set uh those credential okay in the terminal uh by running
that particular AWS configure command Okay so it will take the access key and secret access key from there and it will take by this Particular Library called boto 3 okay boto 3 automatically will take like take this particular credential okay from the environment then uh what I have to do I have to define the model uh keyword okay so let's define the model keyword also so model arguments so inside that I'm going to mention one arguments called temperature parameter okay because this is the parameter we usually twak a lot actually whenever we are using llm
and all okay So here is the temperature parameter but it should be defined as a string right then here you have to provide the value of this particular temperature parameter so let's take 0.9 that me I'm taking close to one I'm telling my model just take any risk and try to give my and try to give a creative output okay whenever you are generating any kinds of let's say output and all so you can set this particular parameter based on your requirements so I said 0.9 now what I will do I'll create a uh I'll
create a function here called my chatbot mycore chatbot okay my chatbot so inside that actually I'm going to write the final logic okay so it will take the language uh I also want to give the language okay language let's a choice let's say you want to generate this particular response in English language or let's say uh Spanish language or Let's say Hindi language okay you can give any kinds of language so you can give this particular parameter as language then it will also take the user okay user text user text like whatever text user is
giving it will also take that particular input then what I have to do I have to create a prom template here so prom template I have already imported this particular prom template now inside prom template first of all I need to specify the input Variable okay so input variables okay so here input variables I'm having the language okay language and this user text okay the these two actually I have this input input variable okay here now what I have to specify I think I should also close this particular quotation yeah then I also need to
specify my actual template okay so the template is equal to so here uh you can give any kinds of let's say prompt so I'll give this kinds Of prompt actually so you add a chatboard uh you are in this particular language and this is the user text okay so this is the prompt I have given to my llm okay that's how you can Define this particular prompt template now let's instore this particular promp template in a variable I'll just name it as prompt okay prompt now what I will do I'll just uh create a LM
chain so LM chain is equal to uh I'll Define this particular llm Object okay llm is equal to the llm rapper we have created llm then I also need to give this particular prompt so prompt is equal to okay prompt is equal to prompt we have created okay then I store this particular things in a variable I'll just name it as Bedrock okay bedro _ chain now uh user will uh ask the query and they will get the response so for this I'm going to use utilize this Particular Bedrock chain I have created so Bedrock
chain so it will take first of all the language okay input as a language because first of all user will give one language so it will take that particular language so language should be language okay then it will also take the user input user text so user text uh should be user text I think everything is fine let's test okay it should be Text okay not a test so that's why this error is coming user text user text and now I think everything is fine now the response actually I'll be getting so it will uh
it will return me the response okay so I'll return this particular response so return this response yeah now uh this function is ready now I can uh inference on top of the model actually Uh I'm I'm using here now what I will do I'll create a Basics stream L uh uh you can say UI so for this first of all I'll give one title s. title s. title so here you can give any name of the application so here I can write U Bedrock okay Bedrock demo or Bedrock test okay Bedrock uh Bedrock test I
can just write uh then uh first of all I'll be taking one uh you can say select box here because user will uh you can say choose the language So for this you can uh use this particular code snippit so see here I'm creating one uh like slate box and user will select the language so here you can add as many as language you can so I have only added three language English Spanish and Hindi okay you can also add Bengali then you can also add let's say uh I mean U like Russian okay anything
you can add here it's up to you now what I will do uh I'll just write one logic if user is given any kinds of language So what you have to do first of all you have to take the input from the user so I'll take this input and I will store in this particular variable user text okay user text is equal to so here I'm going to create a uh select uh text area Okay because uh I'm going to just create a box in inside that particular box user will give the input okay so
for this you can use std. sidebar s. sidebar okay then you can take something Called text idea okay text idea yeah then inside that you can Define the label that means uh the question actually you want to render here so I'll give a message what is your okay what is your question okay and you can also Define like how many let's say uh character user will able to put okay so here you can just Define Max character is equal to I will just specify 100 okay so 100 character user will put here then once It
is done so I also need to check this user text that means if user has given any kinds of input text that means the prompt okay to the llm so what I have to do first of all I have to uh give it to my model so I'll call this particular function my chatbot inside that first of all I'll give my language okay the language user has selected then I will also provide something called user text okay whatever question actually user is asking then it will give me the response Okay so it will give me
the response yeah then after that I will write this particular response in the Stream lit okay stream L application say St dot write and here I'm going to write this particular response so I'm going to only extract the text okay from the response so it should be text yeah so I think it should work now let's test whether it is working or not so I'm going to open up my terminal so let me Cate this Particular terminal and I'm going to execute this particular file this Bedrock trials. P so for this you have to execute
this particular command okay so let me also add this particular command so if you want to execute the file so you can write stream lit okay stream lit run uh this is inside resarch okay slash uh the name of this file is bedra Trials Okay do p is the name now let's copy the command I'll open up my terminal and let me execute okay so it is giving one error it is telling research okay so research spelling is not correct let me check so research okay now I think it's fine now let me copy and
execute okay now my application is running now if I go to my browser now if I open up my browser okay so it will open up one one new tab so this is the Tab it has opened but again I can see there is a error so what is the error so unknown service error so unknown service Bedrock undor runtime okay so let me check this particular C snippit so here uh okay so it should be Bedrock hypen runtime okay not a underscore runtime now I think it should work I go back and rerun this
particular application okay now see this is my interface now you can select any kinds of language so let's select the English language now ask any Kind of qu so I'll ask what is okay what is uh let's say um what is C++ okay what is C++ so now let's see whether it is able to give me answer or not now if you want to execute you need to press control and enter so here it is giving me the response C++ is a general programming Lang language uh that means uh yeah so it's a correct response
now you can any ask any kind of query so you can ask like what is python okay and control Enter see it is also giving the response now see guys although I'm using this particular model okay this large language model and I'm also using this open source model OKAY zuras is also open source model but the response time is like very uh first here because this model is already hosted okay over the wraw and we we can access this particular model with the help of this particular client okay the client actually we have created okay
this Particular client we have created okay so that's is why actually uh whenever you want to create any kind of production gr application and you want to let's say get a quick response at that time you can go with these other the services but whenever let's say you are loading these other the model on your machine at that time your inference time would be very very higher okay like it will take lots of time to execute that particular code so that's why Actually you can use this particular bedrock and this like very scalable okay like
it is like very efficient solution whenever you are trying to create any kind of genbi application now see guys uh I already showed you how we can access different different model now let's say you want to also access Meta Meta model okay so what you can do you can come to the meta and you can take this particular model ID now let's say you want to access this particular model This seven 7 70 billion parameter model so you need to give this particular ID okay so here you just only need to change so let me
show you if I open up the code so here you only need to change okay inside the model ID and everything will remain same but here only we're doing the inferencing okay on top of the model but now what I'm going to do I'm I'm going to create one NN rag application okay so NN rag uh like you can say project we'll be implementing so What is rag so let me give you one uh idea so first of all I'm going to create a architecture okay of this particular project I'm going to develop so here
first of all what I'm going to do as I already told you I'm going to use PDF documents okay let's say I'm having some PDF documents okay PDF docs okay so it can be multiple PDF also it can be one PDF also it's up to you now what I have to do first of all I have to extract the text okay I have to Extract okay extract data okay from this particular PDF after extracting what I have to do I have to create a chance I have to create a chance why I have to create
a chance because all the model is having their input length okay if I show you uh let's say I'm using this uh metal Lama 2 okay going forward I'm I'll be using this metal Lama 2 so metal Lama 2 uh is is having actually uh input size okay this token input size Let me show you so if I go to the official metal Lama 2 website so let's go to the website so here you can see guys so it is having different different models 7 billion 13 billion and 70 billion and if you see here
the context length is 496 token okay now what will happen so whenever you are using any kinds of PDF documents so if you extract all the data that means all the documents okay from the PDF you will see sometimes it would Be more than 4,096 token okay it would be more more than 496 token at the time if you give this particular input to the model okay to the model your model will throw input error so to prevent this kinds of issue what I have to do I have to create a chunks okay chunks means
I have to create a different different chunks okay let's say this is your entire text okay let's say this is your entire text okay this is your inter text so What you will do you'll just take a chance okay chunks of the text so this is one chunks this is another chunks okay this is another chunks okay that's how you will be giving to the model okay as in inut so different different chunks actually you are giving instead of taking all the text together you are giving different different chunks so that time you have to
also Define the Chun size okay and there is another parameter we can also refine called Chunks overlap okay so this is the idea of this chunin now once this chunks is created now what I have to do I have to download one embedding model okay I have to download one embedding model so in this case I'm going to use this Titan embedding model I already showed you okay so embedding model Titan embedding model with the help of this particular embedding model I will convert this particular data to Vector Embedding okay so Vector embedding Vector embedding
that means numerical representation Vector embedding then what I have to do I have to store this particular Vector embedding in a knowledge base that means in a vector DV so here I'm going to use something called FAS okay f is a vector DB it's a vector DB Vector DB okay so this is nothing but this is my knowledge base knowledge base okay now what will Happen user will ask some query so let's say this is the user so user will ask some query so let's say this is the query user has asked so what I
have to do I have to pass this particular query to my knowledge base okay so it will do something called semantic sear okay semantic sear semantic index S I think you already know what is semantic sear inside Vector DB then it will also return some of the rank results okay it Will return some rank results rank results okay so it will return this particular rank results B on the K parameter let's say if K is equal to three so it will return only three relevant answer okay with respect to the query you have asked now
what I have to do I have to connect the LM so I'll be collecting the llm so in this case I'm using something called Lama 2 okay Lama 2 so I'll fit this Particular rank treasur to the llm okay then I'll also give the query to the llm so LM will try to understand the query it will also try to understand the results that means the answer then it will process this particular query answer and it will give you the actual response okay actual response so response okay so let's say you have asked something related
python so first of all it will check in the knowledge base whether I'm having answer related Python or not if it it is having so it will take most relevant three output okay from this particular knowledge base then it will process okay based on the query then it will give you the correct response that means what is python exactly okay this this is the idea and this is called okay this is called rag application okay rag pipeline this is called rag pipeline okay this called rag pipeline that means retrieval augmented Generation so instead of fining
the model what I'm doing I'm just adding some additional knowledge okay additional information in okay information okay additional information to the model okay so this is my additional information so I'm just processing this information I'm creating the embeddings okay and I'm storing in a knowledge base and I'm connecting okay I'm connecting the llm to the knowledge Base and whatever user is asking this kinds of query okay I'm just uh like getting the response and I'm showing to the user okay so this is called rag Pipeline and what is fine tuning fine tuning means whenever you
will be like say training that particular model that means you will be changing the we parameter that that is called actually fine tuning and fine tuning is like very hard task and it's like very costly task okay so that's why you don't need to Perform this fine tuning every time okay first of all you will be uh applying this particular rag concept and if rag is working fine then you'll be going to going with the r rag concept and if rag concept is failing somewhere that time actually you have to perform the fine tuning so
feature actually be also learning how we can perform the fine tuning okay on top of the custom data that thing we'll be also learning so yes guys this is the entire architecture of Our like you can say application now I need to code this particular example so for this I'm going to again open up my code editor now here I'm going to open this particular m.p file so guys I think you already got the idea how things are working and how we can make the connection with this particular bedro and all so let me just
quickly write all the codes so first of all I need to import some of the library here so let's import all the library so these are the Libraries are required okay I'm importing boto 3 because of that particular you can say Bedrock I want to access with the Bedrock streamly I need because I'll be creating the web application then I'm also importing something called Bedrock embedding okay because I already told you I'm going to use one embedding model OKAY from the Bedrock so Bedrock having also embedding model so if I show you let's say Amazon
is also having one model called Amazon Titan okay so let me show you yeah Titan embedding G1 text okay this is the model so this embedding model actually I'm going to use okay that's why I'm importing this particular embedding from the uh you can say Lang chain okay and this is the Bedrock eming then I also need this llm that means Bedrock then P PDF directory loader because I'm going to keep some PDF okay in the directory so let me create a directory here I'm going to name it as Data okay and I can name
it as PDF data okay PDF data inside that I'm going to keep my PDF so with the help of this particular package I'm going to load this particular PDF then character recursive character text splitter so I already told you something called chunins so if you want to perform the chunins you have to use this particular method called character text splitter okay recursive character text splitter then I also need something called Vector DV so Vector DV wise I'm going to use F and it is already available inside langen and prom template and retal keyway because I'm
going to perform then I'm importing this particular retal keyway okay this particular method because as I already told you I'm going to create this rag application and on top of the knowledge base actually I'll be performing the query operation okay so that's why actually this is need it now the first thing what you have to do You have to Define one prom template okay so this is the default prom template I just prepared okay so here I just written so you human will ask some query okay based on that you have to return the response
okay by taking the context okay context means the knowledge base so this is the prom template I have created now what you have to do you have to collect the you have to connect the client okay so I think you know how to collect the client So this is the code for the client so boto three client and you have to provide the service name and the region name then what you have to do you have to get the embedding model so to get the embedding model what you can do you can use this particular
Bedrock embedding okay Bedrock embedding to load the embedding model so here is the code okay so that's how you can load this particular embedding model as you can see Bedrock embedding here you have To give the model ID so Amazon Titan embed text V1 so this is the model actually I'm using so Amazon Titan embed text V1 okay so this is the model ID I'm using let me show you yeah so this is the model ID after that you have to also assign the client okay the client actually you have created then first of all
what I have to do I have to load this particular PDF documents okay so I don't have any PDFs so let me upload some of the PDFs here so I'm going to Open up the folder and here I'm going to keep this particular PDF okay so this is one research paper actually as you can see so let me show you this is one research paper and uh the name of the research paper is development of multiple combined regation method for rain rainfall measurement okay so this is the research paper actually I have have given here
so you can give any kinds of resarch paper any kinds of PDF here any kinds of PDF documents it's up To you okay so I already have this in my system that's why I'm giving this particular PDF now let's uh see how we can extract these are the PDF text now what I will do I have already created one function so let me show you so inside main. Pi here I can mention this particular function called get documents okay so first of all I will load this particular PDF okay as you can see so here
I need to give the path of the PDF copy the name Yeah so this is the folder inside that actually whatever PDF actually you are having it will load everything then after that I'm performing this recursive T splitter that means I'm performing this chuning operation okay so as you can see so this is the chunin actually I'm doing I'm after extracting the documents I'm performing this chuning operation that means it will create the chunks different different chunks and this is the Chun size that means it will Take thousand character okay in one chunks let me
show you so let's say whenever it will create this particular chance it will consider thousands character Okay thousands tokens in a one chance okay so that's you have to give this particular chunk size and chunk overlap means how much overlap actually you want from the previous okay from the previous um like you can say chunks you have created okay so this is called chunks overlap now once this is done Then you have to uh apply this particular split document that means it will perform the chuning then after that I'm just returning the documents then I
also need to load the vector okay so to load the vector you can execute this particular function let me show you you can load this particular function called get Vector store now it will take the documents the documents actually you have extracted now here I'm using this F okay F Vector DV as you can See FES and I'm initializing the files I'm giving this particular documentation the documentation I have extracted and also giving the vector embedding okay the vector embedding I have initialized okay vrog embedding then what it will do it will automatically take those
other the documents and it will convert to Vector representation okay and it will save inside my local system with the help of this particular name okay F index now once it is done I also need The llm op okay so I have created another function called get llm so here I'm initializing this particular uh you can say llm now if you see I'm using metal Lama to okay so now if I show you my Bedrock now if I go to the overview and I'm using this Lama 2 and I'm using this particular model say 7
billion model okay now this is the model ID okay just copy and here you can paste it okay here you can mention now client and here you can give the model arguments okay Now here I have given Max gen length okay that means how much actually like let's say tokens he want to get as a output okay from the llm so here I've given 512 okay that is 512 token it will give me as a output and after that I'm just returning the LM then what I have to do I have to create the prom
template so I think you remember we already defined the uh you can say prompt here so prompt template now here I'm just defining the template okay inside the Template I'm just defining these other the thing now input variable should be context and question because here I mentioned okay question and context now once it is done uh I have to uh create the response function because here it will take the input and it will give you the response okay so this is the simple respon function so it will take the llm that means large language model
and Vector destroy that means your F and query now First of all it will create this particular Ral object okay inside that it will take the llm and chain type is stop because I'm going to create a simple chain that's why because in Lang chain there are so many chain type you can explore over the documentation now R object means your knowledge base okay here I'm using the f as you can see store F and as reter and I'm performing similarity s okay and this is the uh this rank result parameter that means K Is
equal to three it will return only three okay relevant answer then I'm just returning okay this particular Source document as well then you also need to give the promt template you have created here then you need to ask the query okay the query actually user will give and whatever answer actually will get I'm just returning this particular answer okay then here I just need to create a simple user application so first of all I'm going to create a new main method Let me show you so this is the main method so here first of all
I've given one title let a rag demo and here I've given the name of the project into rag application okay then here I'm just giving one input okay input text okay text input from the user user will ask any of query from the PDF itself now first of all what I have to do okay let's say if you want to ask any question on top of the knowledge based first of all you have to build a Knowledge base that means you have to execute this particular function called get Vector store okay but before that you
have to execute this particular function called get documents that means it will extract the documents after that it will store as a embedding okay so I have to execute this two function so for this I'm going to create one button here okay so let me show you so here I'm going to create one button as you can see so here I'm Crea one slide but and Here I've given update or create a vector store now if someone is clicking on this particular button store Vector first of all it will run this particular document that means
this particular function and it will extract the documentation and whatever documentation actually I'm getting okay sorry whatever documentation actually I'm getting from here I'll be passing inside this particular function called get Vector restore okay as you can see Get Vector restore this documentation I'm passing then once it is done I'm just sending the success message then uh if my uh knowledge base is created that means Vector store is created now I'll be performing the question answer of on top of it okay so again I'm going to create another function I think it should be inside
this particular a yeah so again I I have created another function called uh send okay send button so whatever user will Give the query so whatever user will give the quy here okay first of all what I have to do I have to load my uh knowledge base okay as you can see I'm loading the knowledge base f. load actually I'm doing and this is the index name then I also need to give the embedding object then you also need to give this particular parameter called allow dangerous der serialization is equal to two Okay if
you're using this F so this is the uh parameter you have to Pass because I saw from the documentation they are also using this particular parameter then I I need to load my llm okay that means I already created one function to load the llm so as you can see get llm object okay then after that uh here is the final function I'm calling so get llm response and it will take this llm Vector store and query so llm Vector store and query the uh query actually user is asking and I'm just writing the response
okay whatever Response actually I'm getting I'm writing on top of the stream L okay now let's see whether it's working or not so to execute this particular file you need to run this command so let me also mention this command so it should be stream L run main. Pi Okay because I'm going to run this particular main main file now let me copy the command and I'm going to open up my terminal so this is the terminal now I'm Going to stop the previous execution let me clear and let me execute this particular file now
again this is running so if I go back and I will close the previous one and this is the new one it has loaded okay I'm not getting anything because let me check my code main. Pi okay so this main actually I'm not calling okay so I need to also Define this particular main function in this particular if statement okay if Underscore name underscore mean then I'm going to uh access these are the code okay now I think it should work I'll go back and run this particular code so guys now as you can see
I got my interface now first of all I need to store the vector okay because I need to extract the documents I need to store the knowledge base so let's click on this particular store Vector so once it is done you will see the success message so guys as you can see the Execution is done now if I show you my code left hand side you can see it has created this particular index okay that means here actually it has stored all of my Vector but it is in a pckl format okay because you can't
visualize these are the vector again it's a local Vector DB okay if you want to see that particular Vector you can use pine con okay Pine we there actually you can visualize these are the vector like how it is look like and all now what I will Do I'll ask some of the query so let's ask query with respect to the PDF actually I'm having so let me open the PDF so let's say I'll be asking related uh this particular uh rainall measurements okay so rainall measurements I'll copy this particular term and here I can
ask so tell me about rful measurements now if I just uh I think I need to click on the send button okay now I think it should work now guys let me ask another qu so I'll ask tell me about uh tell be about methology tell me okay about methology of the paper now guys here is the response I got uh so guys here is the response I got the paper compares the performance of the various regression models for the rainfall prediction and the matology involves the comparing of the performance of difference regession model statical
metrics such as R2 me and MC and rmsc okay if you read this particular paper you'll see the same thing actually uh they're also explaining okay about this particular paper and all that means it is working fine okay now not only this particular paper you can uh place any kinds of actually PDF here in this particular folder okay any kinds of PDF any kinds of PDF books and you can again uh click on this particular button so let me show you so this is my application you can Again click on this particular button that means
store Vector it will again store those new Vector that means new uh text and it will create a new knowledge base and on top of that you can perform the question answer okay so this is like awesome application actually you can create with the help of this particular a uh Bedrock okay not only this model you can also use mral AI okay you can also use this particular uh stability AI okay you can try these are The model at least okay after this particular uh you can say uh video you can try with different different
model and you can create different different application on top of it so yeah guys this is all about from this particular video I hope you got the entire understanding how we can create our different different application how we can create a you can rag application with the help of this particular AWS Bedrock so here you can explore this Particular playground as well as the model actually different different Foundation model you can export and you can create different different application okay so yes guys uh this is all about from this particular video and uh this is
like awesome actually platform they have developed uh if you see the inference execution time actually it's like very fast although I'm using like bigger bigger model let's say I used metal model this particular 7 70 billion parameter model and my execution time was like very fast there okay but if you're loading this particular model on your system you will see uh you will get lots of difficulty there okay so yes guys this is all about from this particular video I hope you like this particular video so thank you so much guys for watching this video
and I'll see you next time
