Build POWERFUL Streaming Apps Like ChatGPT with ZERO Code (Full Guide)

now flutter flow recently released a bunch of new features but really the biggest one of them all is the ability to do streaming API calls and this is an amazing feature that allows you to have that kind of Chad GPT Behavior with all kinds of apis and so in today's video I'm going to be explaining to you why this feature is so important I'm going to be showing to how you can leverage this feature and then we're going to take a look at my proof of concept apps that you can learn how to implement this

feature yourself first of all why is this feature so important well if you've used Chad GPT before or other similar products you're probably familiar with that streaming answer functionality where you ask a question and instead of dumping an entire answer after like you know 10 20 or more seconds it starts responding immediately and answering your question word by word as it starts to figure out what you're looking for and that is called a streaming behavior and so in my opinion and in probably your opinion as well this is a much better user experience than the

alternative where you the user is going to be getting an answer after the system figures it out that's that might take a long time while the user is waiting and wondering if the system is going to deliver an answer for them and so this way the user knows that the system has an answer so they can simply wait until the streaming behavior is completely finished now the problem here is before we had this functionality in flut of flow you could not implement this feature so even though your apis things like chat GPT or other llms

or other apis may have supported this streaming functionality there was really no other way there was really no way to implement this in flutter flow and so you had to to create your own custom code you had to create a custom action which is something that I did in one of the previous videos to demonstrate to you guys how you can implement this functionality when you do not have native API support but right now you have that native API support which means that you can choose between implementing a standard non-streaming API Behavior or a streaming

API Behavior now before I show you a work proof of concept that further demonstrates how you can do this with flut flow let me show you a diagram that best explains how this feature works so right here you're looking at a standard diagram that showcases the standard which is the non-streaming API Behavior so we have the user here we have the UI we have the logic and we have the back end and so all these three pieces is your app right right so the user is interacting with the UI there's the logic uh behind the

scenes and then you're saving or reading the data from a back end and so in a typical API call which is this module right here you're going to be connecting to a remote server and you're going to be sending some kind of a request and this request could be as simple as getting some data or it could be a little bit more complex where you're saving some data and it's important to understand that all this time while the server is doing its work this system here right your API call is waiting right it's synchronous Okay

so until this finishes nothing else can happen you cannot go to the next step until this is done because the system knows that at some point in time it's going to respond with a full response whether you know the request was successful or you know if if the data that it's looking for is ready and so the system is not going to do anything until it gets that full response now on the other hand if you take a look at a situation where we can make an API call and we have the streaming support it's

going to behave slightly different so this this first row here is exactly the same but where the big change is is the how the API call is going to behave okay so we have our API call we send a request which works exactly the same way as before but the difference here is in the response okay so the system is going to call a call back which which means it's going to be letting us know if it has some data and it's going to be doing it in an asynchronous way and so in this case

the API system is going to make a request and then it can just do something else it's not going to be waiting for a response in a synchronous way where it's blocking the system and blocking the user from attempting to do anything else and that is because the way that this is going to work is that as the data is going to be coming in this system is going to get notified in a call back style okay so what it means essentially is that it sends a request and then it forgets about it but at

some point ins maybe 5 seconds later or maybe 50 seconds later or an hour later it's going to be awakened with some data right via a call back and so this call back can be one of three of these conditions right it can either be data that's coming in it can be an error or it can be a finishing message which means that there's no more data coming in we can close this off and do something else and so while this is processing your app can do just about anything that it wants to do because

it knows that it will always be getting this data and it will be able to do something something once it gets this data and so that is how this streaming API call system works in a nutshell okay and so like I said in the beginning of the video this is something that was not supported by flut flow right this is something that you had to do using custom code that is why in some of the previous apps I built custom actions that did just that but right now we have a way to configure our API

calls to support streaming all right so now that I've shown you an architecture diagram just to explain to you guys how it works behind the scenes let me show you a working proof of concept app that I buil that shows you the standard API response Behavior as well as the streaming API response behavior and then we're going to jump back into the app and I'll show you exactly what you need to do and remember guys if you're a member of our amazing patreon Community you'll be able to view and or clone this app right away

way which means you don't have to worry about any of this you can simply add this app to your own Library so that you can build on top of it so that you can learn how it works and you're going to be getting a ton more value as opposed to just viewing this app on your screen right now and so let me go ahead and run this app so that you can see how it behaves all right so here's our app and this app has two pages so the first is the standard Behavior and the

second is the streaming Behavior now I'm actually using claw 3.5 Sonet llm for this demonstration but you can use pretty much any llm they all work pretty much the same including Chad GPT or any other llm they all work very very similarly if not exactly the same and so if we come here and let's say I asked the llm hey how are you and we had send we see the message but we have to wait until the whole response is done so there's no streaming here if I say something like list 10 cities in the

US we need to wait right there is a delay as is thinking okay and then it comes up with the entire answer here on the other hand if we come to this second page and this is the streaming Behavior we still see this conversation here but the responses from now on are going to be in a streaming format Okay so for instance let's say we ask it list 10 biggest cities in the world okay we hit send and now it's thinking and it's coming up with the answers immediately okay so the user doesn't need to

sit around and wait for the whole answer in one shot right they can be sitting there and seeing this system come up with an answer word by word token by token so let's say ask it something uh something else L 25 capitals around the world okay so this means it needs to come up with a lot of data and streaming is going to work really nice so you see it's coming up and this is the response that it came up with whereas if I come in here and I say list 30 random cities um it's

going to it's going to think for a minute right it's going to think for some time right it's still thinking thinking thinking okay and now it has this entire list right and so in my opinion this is something that I talked about before streaming is a lot better it's a lot more fun it's also a lot more userfriendly and so this is definitely the recommended method going forward all right so let's jump back into FL oflow and I can show you exactly how this new feature works okay the first thing that you need to do

when you are setting up your API call you need to do two things first you need to send something like I have here so with this API we need to say stream equals to True right we need to tell the system to stream the response back to us okay otherwise if we don't do that it's not going to know right so it's going to it's going to default to the standard Behavior so that is the first thing that we need to do the second thing that we need to do if you head over to advanced

settings you want to make sure that you enable this right process streaming response needs to be turned on right so I have it turned off in the first API call right I have it turned off and then the body I'm not saying uh streaming is true right I'm not saying that right because that's the standard Behavior whereas here I have it turned on and I have this set here okay now once you do that you're going to be getting different responses so if you come to response and test here and this is my standard response

and I do test API call you're going to see this but if you click on raw body you're going to be seeing a bunch of responses now this here is a scrollable list okay it may not seem this way because you might not see a scroll bar right away but you can scroll down and what you're looking at here is the tokens that the system is sending one after another and so when you send a streaming request you're going to be getting back these tokens okay so you need to parse the data that's in these

tokens and display it okay so if we take a look at the first token and you scroll down all the way to the right you're going to see that there's no data here right it's just a token but if you go to the second one you see that text is empty here there's nothing here if you come over here you see text says it says hello if you come over here it says how and then if you scroll down we see how can right so it's going to say something like how can I help you

because I said something like you scroll further it says I assist you today right and so we're getting PE piece by piece answer here right because the request I sent was hello world and so first it says how then it says can and if we go to the next chunk here it says uh feel and then if you go next one feel free uh free to ask me and then if you keep going it says any right so these are all chunks we're getting piece by piece answers and so what we need to do now

is we need to take these chunks here parse them and display them in a way that the user will understand right in that kind of streaming Behavior here okay and so the first thing that I decided to do here is I decided to create a custom chunk type okay and so it says here parse each each chunk as a data type right so if you go over here I have this custom response type all of this and then you know this thing is is a data which is Delta and then you have Delta here so

the easy way to create it is by simply going back to your API call here coming in here doing this test API call actually for this one you want to do that you want to do it for this one and this is my tip to you is that what you can do is you can simply copy this data here copy it go back here and you can click here and you can paste it here and then you can call this you can say well this is my response type okay and then create I already did

that and that is why you're seeing this here and that essentially takes that Json response and breaks it apart okay and this is great because now we can parse the responses into this data structure and that's going to be very very helpful to us later okay next let's go back to the app so I have a very kind of standard chat up and I've done many of these chat UPS before so if you've seen my previous videos you know exactly the structure here okay so um before I show you the chat up if we head

over to App State you can see that I have a list of messages which is a message type and this message type is has just three Fields the role which is going to be either user which is me or you or the assistant which is the llm AI then we have the message and we have the time stamp here okay and this allows us to store the messages with you know knowing who is the sender and when the message was sent okay and so if we go to our app State I have these messages a

list of messages here next we have here the main part of the screen and that here is a list to of different rows that have a stack as the parent and that allows us to display roles in kind of this alternate style so either it's going to be you know my message or somebody else's message we're never going to have two messages at the same time right because we're going to have one row you know with with my message another row with somebody else's message right and so that's going to so we're never going to

have it this way it's going to be either you know maybe a message here a message here or message here message here message here and then another message you're never going to have two messages on the same row because we also have conditional visibility so this here this row here is only going to be displayed if this is a message sent by the AI whereas this row is only going to be displayed if this is a message sent by the user now if we select the button here if you open up this right here you're

going to be seeing three actions here so the first thing we're doing is we're adding the message that the user sent to our list of messages right I say add to the list and I give it a role of user okay the second thing we're doing is we're creating another message that the AI might have sent all right and this message is just going to say thinking right so I'm creating another message and I'm saying thinking and that way once the user sends a message we immediately get get a response that says think right and

then when we get the real message from the AI we remove that thinking message and we display the actual AI message now the key Point here is that in this API call right we select this API streaming and because this thing support streaming we have three callbacks that we can now utilize which was not the case before right so with a standard API you don't have any callbacks right cuz you execute it and you just display result right so if I go back to my regular call here and I open it up this is what

it looks like this is a regular API call I call it if it's succeeded this is going to happen if it's not succeeded we're going to display some kind of an error message right this is a synchronous Behavior right this just goes from here to here whereas in a streaming Behavior right if you come over here we don't have anything here right that is because it's all happening here right so the first call back is uh the on message once we get a message the second call back is on error if you get an error

do something and the third and last callback is UN closed right when we are done when it finishes sending stuff maybe we want to you know take some action based on that right here I'm only using this first call back on message and so if we click here we now have another action flow that is executed or when we get a message right and and and when I say a message it's a chunk when we get that individual chunk well remember when you saw those multiple chunks in a sample API request well every time we

get a chunk this whole thing gets executed now this first half here right what we're doing here is we are keeping track of how many chunks we have so far because if it's a first chunk which means that you know this is the first chunk that we are getting right we don't have any data we didn't have any data before that we need to remove this message right so what I'm doing is I am removing that thinking and I'm replacing it by an empty string here okay next this part is the key here because what

I'm doing here is I'm checking whether the response chunk has the text field because if you remember in that sample API call we had a lot of chunks but they were you know some of them were different right and that will happen they're they're not going to be all the same right they may be slightly different they may be radically different and so it's very very important for us to first check whether this chunk has actual text because it might have something else okay or it may not even have text okay so we need to

make sure that we have this and so here what I'm doing is I'm getting this chunk right this action parameter right so if I go over here I have this action parameter which is the UN message input and remember we are parsing it as that message type and so we're parsing that Json uh breaking it up into different fields according to what we have uh as that as that data type there right and so what I'm doing here is I'm checking whether it has this text feel there right because if it does then we have

data coming in we have actual text Data if it doesn't we don't really care about it right it might have something else it's not very important and so if it does have it we are going to be updating that message right we're not going to be creating M new messages right because otherwise we're going to have multiple messages as chunks we need to update that specific message and which message well the last message okay so remember we for the first message over here very first message we create a message here and so regardless of what

happens there's going to be at least one message right typically you're going to have a lot more than one message but at the very least you're going to have at least one message right so if you jump back over here um what we are doing here is if we have the text we're going to be updating that message by appending this new message to it right so what I'm doing is I'm doing a text combination and I'm getting that original text and I'm appending a streaming response which is that actual text that we're getting back

from the API call on the other hand if we do not have a any text coming in I just have a dummy action that doesn't really do anything right so we have a text combination it just returns uh the message so basically it sets the message M to itself and last but not least we're going to clear that bottom input so the user does not need to clear it when sending a new query and then as a result we execute the API call but when we do that the system is not going to be blocking

right so over here if we come over here if we execute an API call and we say something like you know list 20 random cities here when we click Send the system is going to block because it's waiting for the response and when I say Block it's not going to be allowing the user to do anything else right so you see how it's waiting here right it's blocking because it's waiting for the response and now it has a response whereas here if I say you know give me top 25 cities it just sends it right

away right it doesn't it doesn't block it gets a response it sends it but it's not going to be blocking while it's getting the response because it does it using callbacks right it does it asynchronously and so as a result we don't need to worry about what we need to do after the API call all of that is taking care for us uh via callbacks right so it's a it's a different behavior and because here we're displaying a list of messages and here we are updating updating the messages the individual message the last message that

needs to be displayed we are getting that beautiful streaming behavior and you have to make sure that when you're updating this app state right when you're pending the messages you need to have this rebuild current page because if you said no rebuild you're not going to be seeing that streaming behavior and so all of this is going to be for nothing because you're not going to be rebuilding that widget you're not going to be rebuilding that page when you are updating the App State so this is very very important and thankfully by default this is

going to be set but just in case so that you know you have to make sure that you're rebuilding this current page and as a result we have this beautiful streaming Behavior functionality without needing to resort to creating custom actions which is something that I had to do in previous videoos so this is a great great feature and even though here I'm showing it to you with claw 3.5 Sonet uh this is going to work with all kinds of API calls even non lln there's lots and lots of cool API calls that give you the

streaming functionality and now you can't leverage it in your FL oflow apps now earlier in the video I mentioned that you're going to be getting access to this app when you join our amazing patreon community and really it's a great great benefit because this app is done for you right you don't have to worry about anything all you need to do is enter your API key right here you can simply use uh open AI system they use the almost the same API format if not the same exactly the same uh all you need to do

is replace your API key replace the URL and you can get this app to work with Chad GPT as well and so this is Cutting Edge functionality right it was just recently released and you can start building apps to take advantage of it without needing to do any custom code right that's the beauty of no code right it's to give you the the the features the functionality the resources so that you can go out and build the app that you want to build without getting bugged down with code and so as a member of our

amazing patreon Community you'll be able to view and or clone this app plus you're going to get access to all my apps including lots of complex apps lots of different Niche apps lots of different apps that showcase all kinds of Cutting Edge features plus you're going to be getting access to our extra content such as q&as live streams behind the scenes content as well as our patreon supported master class Series where I do deep Dives of different topics apps and tools it's all on there in fact just couple of days ago I did a huge

huge complex app that was more than five hours long and that entire app is on there and so if you're looking to build a complex app I have lots and lots of complex apps I believe I have over 170 or maybe even more apps at this point because every single app that I did as part of a tutorial on this channel is on there right so you'll be able to get access to all my apps extra content and you're going to be supporting this Channel and supporting my work so if you're getting value from these

tutorials then you're going to get even more value inside of our amazing patreon community and so if all of this sounds good to you check out the first link in the description below the video and become a member