hello all my name is krishak and welcome to my YouTube channel so guys if you like this specific setup please do hit a like because I'm going to probably upload more number of videos in a similar way now coming to the topic that we are going to discuss today in this video we going to talk about various text summarization techniques I know you may have heard about one or couple of text summarization techniques but here I'm going to show you around four to five different text summarization techniques which you can specifically use along with Lang chain for solving various business use cases now the other thing is that many people assume that we can only do this with paid apis you can also do the same task with open source apis also right or open source llm models also but in this video I'm going to focus with respect to open AI llm models the reason is very simple because I will be able to get more accurate results right but anyhow if you're interested in understanding about open source llm models already I've created multiple videos in my Lang chin playlist you can go ahead and check it out so let's go ahead and let's understand what all things we are going to do there are four to five different text summarization technique which I'm going to complete in this session and in my next video I will be using all these techniques to create an endtoend generative AI project which will probably solve text summarization techniques in different different use cases now in companies there are huge number of PDFs documents and text summarization can play a very important role in solving various use use cases so let's go ahead and let's see step by step how we are going to do it and all the code everything if you're following my langin playlist you will be able to understand in a similar way in a very easy way in short right because I've covered all the specific topics in my langen playlist also so the first thing in the code one is that I have used the open API key and this open API key I'm going to use for different different open AI llm models and with those models I'm going to perform different techniques of text summarization and obviously Lang chain as a framework has different functionalities to do the things with respect to text summarization so first thing first over here you'll be able to see the first line of code is taking an opening API key and setting up an environment over here with the name open aore API key because this key will be used in my llm models when I'm initializing whether it will be a chat model or whether it'll be a simple llm model itself so the first thing over here you can see the first topic the first type of text summarization I will be starting with something called as best basic prompt summarization now here in basic prompt summarization the main aim is to just give a simple prompt and probably do the entire text summarization of any any kind of data that you have so over here we are importing from lin. chat models import chat open aai then from Lang chain. schema import AI message human message and system message now you know the importance of AI message human message and system message right human message is any kind of message that probably a human being is giving AI message is what the response from that llm models are and system message is the initial message let's say that I want to make the chat bot to work like a poet okay so let's say I'll say hey you need to act like a poet and whatever text I probably give you have to probably create a poem out of it okay so that will be the default message that we want the system to act as so the first thing first I'm going to import all these things and here you can see that it has executed successfully now coming to the next part here I have taken one amazing speech from our prime minister SRI Narendra Modi G okay so this specific speech I have downloaded or I have copied it from Wikipedia and pasted it over here now our aim is that in a simpler Way by just taking the specific speech how I'm going to summarize it okay so this is the speech over here people across the country involved in government po political and social activities so and so all the entire information is there I'll be providing you all this information all this code in the GitHub mat which will be provided in the description of this particular video so here is the speech and let me just go ahead and execute this so this is my entire speech now with respect to this particular speech as I said I'm going to import system message human message or AI message right we have imported it so we will go ahead and set up the default values with respect to all these things so first of all I'm creating a list of chat message and this chat message in the system message I'm saying the content and I'm I'm telling the AI that you are an expert assistant with expertise in summarizing speeches okay so let me go ahead and make this as speeches so here you can probably see the entire code you are an expert assistant with expertise in summarizing speeches and then the second message is basically set up with human message please provide a short and concise summary of the following speech and here we have given the text which will be in this particular variable so this is my input variable that you can see over here the speech part right so once I probably consider all these things this will basically set up my human message system and the AI message system right so by default the system message system not AI message as AI message is the response that we specifically get from the llm models so the system message and the human message is basically set up now in my third line of code what we have done is that we have initialized our chat open API right right so chat open AI now this chat open AI will just act like an llm model like we are providing we we are asking any question and the LM model will provide us some response and here specifically we are using chat GPT 3.
5 turbo and if you probably go into the open AI page there are different models that will be specifically used for this particular purpose so once I execute this so my llm model is initialized okay and I've given the two default prompts which is basically assigned with system message and human message very much simple I hope you're able to understand it if you're able to understand it please do hit like and now this is just the first type similarly we are going to see four to five types of text summarization technique which is super important again I'm telling you it is super super important because some of my students are working in companies who are working in llm models and they are also doing this kind of task right using again Lang chain so here is all my initial setup now we will go ahead and see if I need to understand that how many tokens are there in my speech okay so what I will do I'll use that llm model I'll say getor numor tokens so this will actually help us to understand how many tokens are there or how many tokens it is going to consider from that particular speech okay so if I go ahead and execute this you'll be able to see that we are having 866 token okay so 866 token is there and understand one thing is that in various models there will be some token restriction size so if I probably take an example of GPT 3. 5 turbo it has somewhere around 4096 tokens I guess okay if it is not true you can go ahead and validate it just let me know if it is true or not but I think I've heard that particular number and I've seen it long back okay but if we talk about GPT 4 it has different token size it has the capacity of different different token size and similarly like now there are lot of models that are probably coming up right where this token size will vary now let's go ahead and see the entire summary so I will be using something like llm so what we will do I will go ahead and probably create a code and here I will write llm and let me just go ahead and write my chatore messages okay so once I execute this this will basically give the entire summary of the chat that I have actually given of all the speech that I have actually given Now understand one thing the output has something with AI message this basically means this is the output of the llm model so here it says that the speaker discusses their participation in the vixit bhat sankalp yatra so all the information is basically given over here now if you really want to just focus on the content go ahead and write dot content and here you will be getting the output okay and again it is is hitting the API and it is getting the data okay and it is probably giving you the result over here so in this speech the speaker discusses the importance of ensuring that government plans and schemes so and so information and you can also print this and with respect to the print you'll be getting this so this is one kind of summarization technique that I used and this system will specifically this way will be used in chat models right where we are saving a conversational chat bot right this is all information like system message human message AI message where we are setting it up it will be super beneficial and super important when we create conversational chat bot and in my langin playlist I've already done that and I've created a detailed video now coming to the next type okay the next type is that we are going to use prompt template for text summarization so first was directly by using this AI message human message system and uh I think one more was something called a system message right so all those things we have specifically used now let's see with respect to to prompt template text summarization now here the idea is very simple we will create our own custom prompt and then we will give that particular prompt to our llm model now there have multiple ways to do this and first of all I'll will discuss a basic way and then when we have different types of prompt templates also we will see that how we can actually do the same task so whenever we use a prompt template in Lang chain we have something called as llm chain because prompt template can have multiple prompts so for executing that we require chains of llms right so that it executes one by one and gives you the final output so for this I'm going to import from Lang chain. chain import llm chain from Lang chain import prompt template with the help of this prompt template I'm going to create my own custom prompts okay so that is what I will be looking at now once I execute this over here you'll be able to see that I've executed this now let's go ahead and create our generic template so over here I have written write a summary of the following speech right and here I'm saying that the speech is equal to speech so this will basically be my input variable that we will be sending the information over here and then I'm also saying translate the precise summary to this specific language okay translate the precise summary to Hindi so this language information also I will be giving okay as an input now this will basically be my generic template information now coming to the second line of code here you can probably see we are creating a prompt template and we are saying the input variable will be speech and language and the template that we are going to use will be this generic template and this is how we actually create our entire promt template with all the information all these things has been taught from basics in my Lang chain playlist so you can go ahead and have a look so once I probably execute this over here this prompt template is ready now if I probably want to see my prompt okay so I will go ahead and write prompt do format and here I'm going to probably give my input variable speech with my speech that is stored in this specific variable speech itself the entire speech is stored in the speech variable and here I'm going to probably use the language which will be nothing but Hindi okay I'm saying that okay convert this entire text into Hindi so probably if I execute this then this exactly becomes my entire prompt so here is the information that you see write a summary of the following speech speech is equal to people across the entire speech and later on this prompt translate the precise summary to Hindi it will become Hindi okay so all this specific information is there now what we are going to do I'm going to save this entire prompt in my variable complete prompt okay so this becomes my complete prompt now in order to see that how many number of tokens are there for this complete prompt it is the same text so for this also we are also going to get somewhere around 850s only the reason it is 885 right now because we have added some more information over here translate the precise summary to this specific language so additional information has been put in the prompt itself so here is what if I go ahead and execute llm dogor numor token with complete prompt you will be able to see some error let's see why this error is coming okay error was not there so here I'm actually getting 885 tokens right so the number of total number of tokens has increased from 850s to 885 I think before it was somewhere around let's see 866 now it is somewhere around 885 now how to run this entire entire chain from the promt template itself as you know we have imported llm chain so here you have my llm chain I'm giving my first parameter with respect to llm which is my model and my prompt is nothing but the prompt that I have actually designed so I'll be using this llm chain and when I have to run this entire prompt template I will write do run and my first input parameter is speech again I'm giving my entire speech which I have stored in the variable speech over here and the language here I've said it as Hindi you can also give it as French you can give it any other languages that you want now once you probably execute this you will be able to see now I will be able to get the answer of the entire summary and the summary will be in the form of Hindi okay so it will probably take some amount of time because uh 855 tokens right sorry 880 tokens again it is going to hit the API it is going to take the results okay so till then I can just show you the summary over here so here you can can see or something like that okay so all this information is basically there uh I hope you like the specific two techniques which you are able to understand this is still getting executed it is going to take somewhere around more than 1 minute when I tried it for the first time uh yes some amount of time it is definitely going to take now you can see that we have discussed about two important things one is with the generic template and one is with a prom template and we have seen both the text summarization technique now let's discuss about the third technique that we are going to see which is called as stuff documentation chain now what does this specifically mean stuff documentation chain okay and similarly if you go ahead and see there are more techniques that we are going to see like which is which is also called as map reduce okay map reduce and we'll see one more technique which is called as refine so one by one we will be seeing this till then this is running I think this should get over in another 5 to 10 seconds let's see uh yeah because the speech is quite big you are first of all getting the entire translated summary in English and then you are converting that into Hindi so obviously it is going to take some amount of time yeah and this is with the paid API itself if you probably go with GPT 4 I think then this it may not take that much time and it may get executed okay so once this is done let's go ahead now see it has taken 2 minutes 2.
9 seconds it's okay but uh now you have GPD 4 so this is again going to take less amount of time for that okay but I really wanted to show you with 3. 5 turbo then you can use 4. 5 you can use uh 4.
0 4. Turbo Turbo version is also coming com recently which GPT has launched or chat GPT has basically open has launched itself right now let's discuss about stuff documentation chain now in stuff documentation chain what we are specifically going to do okay stuff documentation chain says that let's say if I have a specific document if I have content in a document it will take the entire content and it'll push to the LM model it'll get the response okay now you may be thinking what is the difference with respect to promt template and all so here the aim is very simple if you have any document take the entire content give that entire content to the llm and get the uh summarized version of it it is similar like prompt template where I'm probably taking the entire content but this is one way okay so I will show you the differences between stuff documentation and map reduce also there's very important differences and when you should not use stuff documentation I'll talk about that also and even the normal problem template also so first of all here what I am doing let's say you have a file and that file is not that big as I said that there is always a token restriction right with respect to various GPT models okay if I talk about GPT 3. 5 turbo it has somewhere around 4096 tokens capacity that that means that at once it you can give 4096 tokens to the llm model and get the response okay but what happens if you have a huge document and you are not able to convert that into so many number of tokens if it is more than 4096 token then this method will not be useful so first of all I will show you how this method will basically work so here you have from Pi Pi PDF 2 import PDF reader I have a PDF from Drapj Abdul Kalam and this specific speech PDF is stored over here I will also be giving you this in the GitHub file so here is the entire PDF file you can probably see I will read this PDF okay so let's go ahead and read this PDF and again with the help of Pi PDF 2 it is becoming very easy it becomes very easy to read any kind of PDF that you want so once you read this you get an object of PDF reader and then what we do we enumerate through every Pages inside the PDF reader we extract the text and we store it in a variable called as text okay so if I execute this over here and if I probably see the text here you can probably see the all the test that is present over here see over here APG Abdul Kalam okay let me just open my vs code again so here you can probably see apj Abdul cam departing speech I'm delighted to address you all every all the information with respect to that text is available now now what we are going to do I am going to give this entire text to my llm model and understand the speech that is probably there it is a two page PDF only two two page PDF okay not more than two page so when two page becomes the text is really less when compared to where you have PDF with 30 40 pages so here what we are going to do first of all if I want to give this in the llm basically to my llm model if I'm using this stuff chain documentation or stuff documentation chain what I will do I will first of all convert this into documents and for this I have imported some libraries okay I will show you the libraries over here so let me do one thing so there is a library which is called as documents okay and here you can see from Lang chain.
document. store so here let me go ahead and copy this and this document we need to convert because the llm model when we using this technique we'll be expecting all the text data to come in the form of document so here I'm going to probably import this particular document okay and then what we going to do in we'll convert that entire text in the form of a document okay so here we are writing document page content is equal to text so that basically becomes all the text is now pushed in one document okay that was our aim our main aim by using this stuff documentation chain is that I have my entire content I'll push inside a document and I'll give it to my llm model and get the response of whatever task I'm probably doing like text summarization now this this is my entire document over here and here you can probably see the page content now I'll go ahead and initialize my chat open API okay and then let's go ahead and use some important information that we have over here so first of all I'll initialize prompt template I'll load the library I'll use chat open API as already so no need to again reload it here is one important function which is called as load summarized chain now with respect to all the techniques of t uh of text sum iation that we'll be seeing we will be using this specific library that is load summarized chain now load summarized chain here you can probably see there is a parameter which is called as chain type now for stuff it will become stuff over here for map reduce we have to just write go ahead and write map reduce over here right over here you can probably see there is something called as stuff only the parameter will change and automatically this entire text that you'll be probably seeing will work in the accordingly based on the type of parameter we select over there if I write stuff so what it is going to do the document that I have created will take the document give it to the llm and it'll give get the response for map reduce what will happen I'll tell you just in some time so that is the reason we are specifically going to use load summarized chain so let me just go ahead and import all these libraries now let's go ahead and write a template now so for template I have written write a concise and S summary of the following speech speech is equal to text I've given the text over here we have created promt template input variable is equal to text template is equal to template so once I execute this now see I'll use this s load summarized chain the first parameter is llm the second parameter is chain type is equal to stuff stuff basically means now this function understands that you you are putting the entire content in a document so it'll be waiting for that document to probably come and then it'll go ahead and call the llm model and then finally you'll get the response then you you're putting prompt is equal to prompt over here and then one more verbos parameter is there just to show you that if you want additional information that how the call is basically happening you can make it as true okay otherwise just keep it as false okay because I don't want to see much much information in my tab or in my output tab itself and finally I will just use this chain which is my llm chain do run and I'll just pass my documents which I have actually created earlier we have created this document right so this document the same document I will pass it over here and once I execute it you'll be able to see that this is also going to take time because now I have this huge content right I'm putting inside a document and sending it to my chain so the output summary you can probably see over here in this speech ABJ Abdul Kalam reflects on his 5 years as president of India and highlights all the important information and all now now the best thing was that I had some kind of documents or text or PDF file which had limited data not huge data one to two pages of data right and I was able to probably document that entire text and send it to the llm model and get the response quickly okay but in companies as I said earlier you will be having data which will be a huge one a PDF of 500 MB of 1 GB of 2GB so in companies you will be having files and documents of huge size so in that case when there is a restriction of to token you cannot use just simply stuff document chain technique for that you have to use map reduce technique and that is where we are going to discuss about summarizing large documents using map reduce now what is the concept behind map reduce the concept behind map red is very simple you have a huge PDF you divide those PDF into chunks okay you take every chunk give it to the llm model and for that chunk you get a summary so for every chunk you have summaries so every chunk will be given to the LM model and you get a summary the next chunk again is given to the LM model you get a summary right and finally the llm model will take all the summaries com combine them and generate a final summary so this is the technique whenever you have large documents you can use the same technique by using the map reduce technique map reduce is very simple you have a huge document you divide those document into small chunks for every chunk you generate a summary you take all the summaries and generate a final summary so that is what we are going to probably see how it is basically done so here we are going to import promp templ chat open API load summarized chain again and we also going to use one thing how we can divide this documents into chunks so for that we are going to use recursive character text splitter so again we'll be using this I will execute this now for that same apj speech you can see we are reading it over here we are extracting all the text by using the same code okay we are initializing a chat open AI model we are seeing that how many number of tokens are there 41 we will take this entire text and we will generate into chunks so we are now splitting the text so here you can see recursive character text splitter with chunk size 10,000 if I make chunk size small I will be getting multiple chunks now if I get multiple chunks I have to push every chunk give it to my llm model and get the response back right in this technique so that is the reason I have used a huge number 10,000 and by this when I create the document chunks you'll be able to see my chunks will be only two that is basically created over here okay so this is my entire chunk uh you'll be able to see if I go and show you what is the length of the chunk also you'll be able to understand so once I execute it you can see the length of the chunk is two so that basically means when I use 10,000 of my chunk size it is basically creating two documents just to show you an as example example so this two chunks will be sent to my llm model one by one and after the summarization of both the chunks will be combined and given a final summary so here we are again going to use load summarized chain but this time the chain type will be map reduce right so here you can probably see map reduce okay chunk is already made and I'm giving that same chunks over here see chain do run chunks so if you probably go ahead and see it it will be taking this h document already we have divided that into chunks now chunk by chunk it is basically given to the llm model you get the response and finally it'll be combining all the chunks and now here is my final output you can see former president of India apj Abdul Kalam gave a departing speech expressing gratitude for support okay and that is the entire summary that you can probably see again a very good technique and this should be definitely used when you have huge files and that file size is very huge you have lot of content inside that okay and if you want to implement a text summary you can do that one more thing that I want to show you as you know we convert the document into chunks right and for every Chunk we give that to the llm model now while giving also we can develop our own custom prompt to do anything that we want right for the chunks and for the final summary also we can write our own custom prompt right and that is what is basically shown over here so first of all I've written chunk underscore prompts I've said that please summarize the below speech speech is equal to text summary right something like this then then I have created The Prompt template and given the input as text the final combined prompt so this this prompt is basically happening will apply to all the chunks that is getting created this final combined prompt will be applying to all the summarized information that we have got from the chunk right right here I write written provide a final summary of the entire speech with this important points add a generic motivational title okay start the precise summary with an introduction and provide the summary in a number point for the speech and speech is equal to text I've given right and then I've have converted this into a prompt where text is my input so for the chunk there will be a different prompt and for the final output there will be a different prompt and again I've used load summarized here I've used map reduce in chain type for the first prompt for the chunk prompt we have something called as map prompt okay map reduce something see the meaning of map reduce there are two components over here for map this will be my Chun prompt okay and for Combined prompt there will be a final combined prompt template okay so here you can probably see finally we write summary do chain. run summore chain.
