For example, a chatbot or a process automation tool using generative AI needs to go through some of these phases. This is the phase where a lot of companies today are in, and more on the POC form or maybe a prototyping form, right? There are some intrinsic sort of changes to some of these systems, the way it operates and also the way the field is evolving.
What we are seeing now is a set of guiding principles around how generative AI technologies are deployed. Hello everyone. Welcome to yet another episode of AI Connect.
Today, we are going to be discussing. An important point, a pivotal topic, as we have seen in last few years, the evolution and development of AI has been nothing short of spectacular. In fact, according to a recent report by PwC.
The overall contribution of AI in the global economy is going to be in the tune of 15 trillion by 2030. That's a massive number. And one of the big portion of that contribution will actually come from generative AI technology.
In last four to six quarters, we have seen again a huge amount of development within the generative AI space. And if we see 23 and 24 is was basically spent on building and testing different models and different technologies. 2025 is probably going to see large scale of deployment for generative AI applications.
Now this is the pivotal moment. This is where we will see that so much of promises that are tied to this particular technology. We will be seeing the result, the performance of this technology.
So keeping that in mind, I'm going to invite Shiva to our AI podcast episode today. And I'm going to ask Shiva how generative AI is going to see the life cycle from 2025 onward. And where is it now, Shiva?
Yeah, thanks. Thanks, Jaydeep. Thanks for having me here.
So indeed, as you said we are seeing a lot more adoption of generative AI technologies from last year and it is continued to it is expected to grow further. What is happening right now is what we are seeing is essentially there are applications which are being developed using generative AI technologies, which are getting more and more into the mainstream. So if 2023 and 24 was more around experimentation, trying out.
various use cases. As we go along, a lot of these applications will move into production. What is becoming more and more important is some of the traditional aspects of software engineering, like reliability of systems, accuracy.
A performance cost optimization, all of that frameworks or all of those measurements need to come back into generative AI systems or applications using generative AI technologies as well. However there are some intrinsic sort of changes to some of these systems the way it operates and also the way the field is evolving. So all of that needs to come together.
And what we are seeing now is a set of guiding principles around how generative AI technologies are deployed. Build first and then deploy and then as well as how they are operationalized, right? So these guiding principles will evolve as we go along, but then I think right now we are seeing some of those initial frameworks which are ready to be adopted and so that regenerative applications can move into production scale.
Got it. Got it. Can we then look at or compare that as more of generative AI ops?
Correct. Correct. Yeah.
So very similar to DevOps or MLOps that we know we are also seeing the evolution of what is being called as gen AI ops. So that the right through the like proof of concept build and the operational life cycle, you're continuously evaluating metrics around the performance of these applications, which are using underlying generative AI technologies. Got it.
So let's take this broadly into three pieces that you mentioned, right? And maybe start with the first one where you said experimentation. Correct.
Can you give us a big, a broad outline without going into too much of deep? What it is going to be likely for that phase for generative AI? Perhaps we are already in that phase.
Yeah. Yeah. So typically, any application that needs to be heard be it for example, a chatbot or a process automation tool using generative AI needs to go through some of these phases.
First one, I would call it as experimentation right where you're trying to understand, which is the base underlying large language model or generative AI model that needs to that would fit. The business use that you are talking about, right? Obviously, that is very important because we need to look at characteristics of models, like whether they are multimodal, whether they are specialized in, let's say, a particular area like mathematics or medical.
Choosing that technology or underlying model is more slightly, I would call it as an experimentation phase, right? Because you need to evaluate the accuracy. You need to look at cost.
For example, you might not want you might want all of that data, all of that application to reside in your environment, then you might want to choose an open source model which is deployed in your environment rather than an API based model like cloud or GPD, right? Those are aspects that we need to consider. The measurement or evaluation metrics at that stage are.
around accuracy, then performance, then cost, and then also specialization. What is the purpose of the model that you're going to use, right? So you evaluate, and then we typically finalize the model base model.
And also the important factor is Usually, there are like three techniques for making the utilizing the model. You do it via prompt engineering. You do it via drag or sometimes if these two approaches are not sufficient for your application, then you go for a fine tuning of the model.
So that again is. This is a metric that we need to evaluate basis and choose the right method, right? This is the accuracy of responses through these methods.
So that I would call is an experimentation phase. So to round it off. This is the phase where lot of companies today are in and more on the POC or maybe a prototyping form, right?
Just to see that what really works in terms of their use cases or, correct. Correct. Yeah.
A lot of the companies are there, but then in order for you to make use of that, you need to go through the other stages. Build the application, build and scale. Where once you do a reference implementation and the experimentation phase, we need to scale that, right?
So that's where a metrics around How do you manage your prompts? So how do you manage your various kinds of content? That is coming in.
How do you manage your How do you evaluate the accuracy of a model? For example, there are like various scores around which you can fluency you can measure fluency, accuracy, and all of those things. And then all of these, we'll probably need to have a matrix to measure against.
Exactly. Exactly. So they, you define metrics and then then.
Then models output will change based on user behavior data coming in. So how do you continuously measure the performance of the application with respect to some of the parameters that you have talked about? So all of those will need to be defined at the end.
So we have done that, we have experimented, we have defined, we have figured that these are the matrices that we should be measuring. What's next? The next is, you build the model and then you operationalize that model, right?
So that's when the operationalization happens. Yeah, the implementation happens and then the model is in production. That point in time, there are aspects around observability how the model output is changing with respect to new data coming in, user behavior.
Yeah. All of that new models are also coming. Yeah, absolutely.
This field, as we have been seeing is evolving rapidly. There are like versions of models. Recently I saw a framework which was built circulated by where you can actually experiment.
Models in the same code base and so you might want to use try and experiment with some other model, newer model, which has come up with a way which has come recently, right? So those kinds of things need to be constantly evaluated and measured against the application's desired performance, right? So that's when the operational part of the measurement comes in.
Okay. So this is where the operationalization is happening. And then all the applications are put in production environment.
The outputs are measured, checked. fine tuned what next after that? Yeah.
Typically, what is becoming more popular and then we have also used some of these techniques is you create a evaluation framework, which actually sits along. With the pipeline in production, so that you are able to see the outputs fit, accuracy, performance in real time, right? And also cost and all of those things in real time.
So there are like dashboards there are matrices that get published on an ongoing basis, which the ops team can look at. And then. Obviously, if there are some, they're not meeting your criteria, then you want to go back and make the change.
So that's the sort of operation evaluation piece that needs to get integrated with the model pipeline so that you are continuously evaluating your output. Got it. So this is a continuum of course, right?
Absolutely. Yeah. From a precision standpoint how are we Addressing this for our customers.
Can you shed some lights on that? What? How you plan to look at and address these continuum to ensure that the operationalistic stabilize the performance is consistent and there is no disruption In the production environment, correct.
So we are also seeing a lot of these frameworks coming up from some of the large vendors, like Google, Microsoft, etc. And this is our experience. Now that we have productionalized a few of these applications.
What we have built is a set of matrices and evaluation frameworks that run along with the that sort of run along with the model when in production, right? And then we are able to evaluate the model performance on an ongoing basis. So that's an evaluation framework that we have come up with as well.
Ah, okay. That looks is a natural progression. Yeah.
To be able to support going forward the life cycle of generative AI applications and technologies at it. Correct. Great.
So any other points you think is important to mention here in the context of life cycle of generative AI? Only thing I would say is, this. Field is rapidly evolving and changing.
So what needs to happen is we need to be agile in our approach from a measurement and evaluation perspective and then integrate. These metricses, these frameworks right through the lifecycle. If you we have, we also recently published a blog around the framework to select a modern at the experimentation stage or the model.
We can put that link in our description for our, for the video. And we are soon coming up with the evaluation framework that we are building right now and which we have a sort of basis, some of the experiences that we have that will be useful in the build stage as well as in the operational stage. So these are some of the tools and technologies.
That we are coming up with. And there are also, aspects of it, which is available from some of the larger vendors, but then obviously, you need to customize. So all these things are essential.
So if you look at it from a software engineering point of view. For example, we whenever we make a change, we regress and then we a lot of those tests are automated, right? Regression test.
Similarly, even in the general generative AI context and ops context, we are coming up with a lot of these automated features which can measure the performance of the application on an ongoing basis. Got it. So this is an important conversation and I think this is going to be an ongoing dialogue.
At least from our end and you will see that in the future, we will do more focused on those three different phases as we spoke today and deep dive into each of these. Bringing more insights, more learning and share with you here in this podcast. One of the important thing probably missed out and probably that is going to be part of the phase one or maybe even phase two as well.
But phase one is critical is the other aspects of generative AI, right? One important quality or, Concerns that is associated with generative AI is trust, right? Yeah.
Whether it is security, whether it is bias, whether it is hallucination, all of that will I guess is going to be part of the phase one. Exactly. Yeah.
Phase two as well, and that's why it's continuing to sort. Correct. Each model will probably bring in a better version of those aspects and I think we'll continue to report that as we discuss that.
Correct? Correct. Yeah.
So When I talked about performance, and that includes some of these aspects as well, not just like application performance, but then performance with respect to trust, reliability, and safety as well. So all these parameters need to go into the evaluation matrices and are being adopted as we speak in a large sort of scale environment. Great.
So I guess with this, we'll wrap up this episode here in the future we are going to cover each of these aspects more in depth for now, Shiva, thank you very much for your time. Thank you. And we'll meet again on this talk.
Sure. Thank you. Thank you everyone.
