[ MUSIC ] >> Welcome, all. This is Raj. We are thrilled to have you here today.
In our previous sessions, you heard a lot about chat use cases and advanced features like fine-tuning, including vision and the teacher-student distillation process. Today, we are excited to share two unique customer stories that are beyond chat use cases and have leveraged these advanced features. One, Mars Petcare Diagnostics, and number two, Methanex in anomaly detection for methanol production.
But before we dive into these fascinating stories, let's take a quick look at how Azure AI is driving innovation. Did you know that Microsoft itself is a major user of Azure AI? From Microsoft 365 Copilot, GitHub Copilot, and even DAX Copilot.
So we are integrating Azure AI into various tools to enhance productivity and support different functions. This level of integration is a testament to Azure AI's scalability, flexibility, and enterprise readiness, making it a backbone for Microsoft's own offerings. At this Ignite, we announced Azure AI Foundry.
This is our holistic platform that brings together foundation models, open-source models, task models, and industry models, all accessible within Azure AI model catalog. It's not just about having access to models. It's all about providing you with the SDKs and tools for discovering, customizing, and optimizing these models to meet your unique business needs.
I'm going to spend a minute on the Azure AI catalog. As the two use cases we're going to show, use the models from the catalog. Within the Azure AI model catalog, we empower users to find the perfect model for every use case.
This catalog is designed to prevent vendor lock-in by offering curated selection of models, including large-language models like GPT-4, multi-model models, regional and domain-specific models, and both open and proprietary options. The goal here is to make it easy for enterprises to explore, compare, and swap models quickly, ensuring the highest level of flexibility for a range of applications. Now that we have seen what Azure AI has to offer, let's shift our focus to the customer success stories, starting with Mars.
So before I welcome our lead to the stage, did you know that Mars has an amazing Petcare segment? How many of you were aware of this? Raise your hand.
Ah, that's good. Okay, so it's my pleasure to introduce Michael Fitzke, Global Senior Director of Next-Gen Technologies at Mars, to the stage. Welcome, Michael.
MICHAEL FITZKE: Thank you. Thank you, Raj. [ APPLAUSE ] RAJ MADHAVAN: So Michael, before you start, I would like to ask a few questions.
Mars is famous for products like M&M and Snickers. Whoever thinks about Mars, M&M and Snickers comes to our mind. Today, we're going to talk about animal health.
So can you give us an introduction about Mars in the petcare segment? MICHAEL FITZKE: Yeah, just maybe very quickly, Mars, as a family-owned business, has an over 90-year heritage and experience in taking care of pets. So long experience, long heritage in that space.
RAJ MADHAVAN: So Michael, what's your driving motivator for AI? And what are the main technical hurdles you have to overcome? MICHAEL FITZKE: So the driving motivator for us is creating a better world for pets.
The fact that AI is a very good tool to do that is awesome. And if I think about the hurdles, I am super grateful to lead a team that works on different hurdles every day when it comes to veterinary AI. One specific one comes with the structure of the data that we are engaging with is in.
So natural images have their specific structure. And a lot of the research that you find in the literature is coming from that world. But we often have different structures.
Multiple images form one radiology study, for example. There's different text and metadata connected to that. So we're working around techniques to leverage that.
We have two research papers that I can think about. One is a study formal one. One is a Coraca one.
If you're interested, definitely check those out. RAJ MADHAVAN: Thank you. So one last question before we get started.
So for the benefit of audience, can you talk about how the AI implementation manifests your five core principles at Mars? MICHAEL FITZKE: Yeah. So maybe one step back on that.
I know that a lot of -- I've seen this in my career, that often AI teams basically are unicorn teams, very separate from especially businesses that have a long heritage already. And with us, that's absolutely different. We are deeply ingrained.
We are deeply part of the Mars culture, being associates often with long decades of experience within Mars. And that makes it easy for us to connect to other associates, to the culture, and implement the principles when it comes to AI. One principle that I can think of specifically that makes it very important for us, of course, quality.
And we are really focusing in and making sure that our models are of the highest quality. RAJ MADHAVAN: Thank you. So take it away.
Interested in knowing all the details of what you did. MICHAEL FITZKE: Yeah, of course. So quick question.
Who here is a pet owner? Oh, wow. So that's great.
So yeah, you will have an easy time to emphasize with someone who has a pet, who goes to the veterinary clinic, for example, the worries that you might have, the thinking what might be coming out, and also the worry that you have to wait to get some results. And when you do that, a lot of different tests are often conducted. And sometimes also imaging, like x-rays, for example.
And that might result into an image like this one. Now, here you have the heart in the middle, you have the spine, you have the ribs. And, unfortunately, in this case, you also have four abnormalities, four masses in this image.
And I want to give you a little bit of time to figure out where those abnormalities might be. So that was exactly ten times as much time as RIA has to do the same analysis. And that's what it produces.
So who of you had all four of them? Three? That's great.
Awesome. Now, you might say, "Okay, but I'm not a veterinarian. Why would I need to analyze that image?
" And you don't have to. But only 5% of veterinarians have a special training in imaging, in interpreting images. So building Copilots, if I may say so, Raj, building Copilots that can help those amazing veterinarians is crucial for us.
And we do this with implementing RapidRead. Rapid read X-ray is a product that I will talk most about. But we have other very, very exciting products under RapidRead as well.
One thing I'm extremely excited about, for example, is RapidRead Dental, a product that will enable veterinarians to optimize clinical workflows in a different way as well. Now, a question might come up of why or what were the factors about us being able to build this. And in AI, like, every time, one of the biggest ones is data.
Now, on the left side here, you have an image size of an amazing data set that was structured by colleagues at Microsoft. It's called Rad-DINO-MAIRA-2 data set. And if you work in this, in AI for health, you recognize it's an amazing data set because it's really large compared to what we had even five years ago, even three years ago.
1. 4 million images, as big, as large as the famous ImageNet data set. Now, the data that we work with and we worked with in this use case is tenfold as large.
So you can see the potential of what we can build with this data. Now, our data, like I said, it's not only coming with images. It comes with a lot of other data as well.
With text, metadata and especially on the text side, you have to have a way to analyze that data to then make it usable in image analysis and training of deep neural networks. And we did this over the last couple of years with various techniques. We used rule-based techniques.
We used standard machine learning techniques. We used models like BERT, early language models and also LLMs that we had to deploy ourselves in the beginning and run ourselves. Now, that completely changed when Microsoft brought the model catalog, because all of these steps take a lot of time for us, right?
So we have to do a lot to, I mean, try to run a large language model, especially two years ago when we didn't have the tooling that we have today on-premise. It was quite intensive work. And when the model catalog came out, that changed for us.
We can concentrate on different things, different things that are important for us. What are those? In our opinion, what we see is one, that you really want to be connected to the latest research on how to extract the best information out of LLMs.
We know from even the early works on prompting that how you ask a specific question makes a big difference. And that changes all the time because the providers of LLMs are continuing to work on it, integrating new technology, new techniques. And so you have to be very, very close to the latest research to get the most out of your LLM in this very dynamic field.
So that's what we could concentrate on way more when the catalog came out. The other one is strategic model selection. And I remember that you wanted to ask me a question on that.
RAJ MADHAVAN: Yeah, so what is the process that you went through before you decided on Mistral large model? MICHAEL FITZKE: So one of the biggest ones for us is, of course, testing, testing, testing, testing. Because of the catalog, we could test, a lot of different, a lot of wide variety of different models.
And Mistral was performing at a very high level, that Mistral large that was. And it was very fast as well. So that's also important when we're talking about this type of large data sets that we are working with.
And it also had a license model that worked with our use case. This also makes a big difference from LLM to LLM. So that's on the model selection part.
Last part that I also have to stress is that we implement a robust process to embrace the limitations. And there are limitations with current LLMs. So having a strong governance process, thinking about where and how to put humans in the loop, and doing all of that is a very crucial step or a very crucial part of our process working with LLMs.
I have mentioned the great MedProm paper. So when we're thinking about research in this space, also here in partnership with Microsoft Research in this space, let us introduce a lot of these techniques early on in working on our data as well and improving the results, getting out of that. And the improvement, the scaling, in the end of the day, results like always in what we see, at least over the last couple of years, specifically in AI, results into better models.
Better models that mean better models at the point of care in the end of the day. Excuse me. But the best model is -- I mean, it's great.
Maybe you want to be on a leaderboard, and you want to be up there. That's all awesome, right? And I know that that's often what we're talking about when we are thinking about AI.
We're talking about models, we're talking about accuracy, we're talking about capabilities of the model. But the best model can only be leveraged fully if the back end of the model inference system has the right architecture and is fast enough. And I really want to say that I think back end engineering, back end engineers are basically the unsung heroes of the AI revolution that we see right now.
Because you need to figure out how to make these models run fast, reliable, and also how to do, for example, model investigation early or fast. When it comes to running them fast and running and scaling up, we are using the Azure Kubernetes system a lot. And for us, this is a great way to scale up capacity of our models.
And equally important for us is that we understand how the models act in the wild, that we validate at test time, and that we implement technologies like drift detection to understand when our model is not working anymore. And to do that, to having a good database and having a good database structure is absolutely crucial. And we leverage Cosmos DB because it allows us, because it's cloud-native, to do this, again, quickly and then concentrate on everything else, all of these complexities that come with it.
Now, when we think about the impact of a system like that, that's always what I personally think about. In RapidRead, we have a system, an early alert system, that sends an alert and sends a case to an imaging specialist. If we find a finding that is, A, life-threatening, and, B, urgent.
And since we launched RapidRead in May of this year, we did this 2,500 times. So in 2,500 cases, there was a life-threatening and urgent finding identified by the AI. The imaging specialist was informed.
The case comes right on top of their desks. They can review it, and they can contact the veterinarian on the ground at the point of care. Can call them and say, "Hey, you might need to act swiftly.
You might need to act now, because that can actually make a difference in a pet's life. " So the AI, and this is basically the output that we are talking about, the output that the AI creates, takes into account not only the images, but also the clinical context of the case. So thinking about what does the veterinarian observe at the point of care.
And then it creates all of those different AI findings. And then it creates the assessment that you see down there. So all of this is created by our AI, and then sent back to the veterinarian.
So I want to end with a video on RapidRead as well. But I really hope that we can go into conversation after the session. Definitely happy to answer any questions.
And thank you so much. JERRY MARTIN: At Mars, we believe that pets make the world a better place. Our science and diagnostic product development is dedicated to ensuring we can provide the very best for the pets we love.
Dr MARK PARKINSON: There's a shortage of veterinary trained radiologists in the industry. So to save more lives, veterinarians need access to the right tools for faster, high quality results with more meaningful insights. We built Rapid Read to combat that global shortage.
Azure AI allows us to leverage a wide range of pre-built models, such as Mistral. And that helps us restructure data and enhance our accuracy. The Azure ML platform and Azure Kubernetes significantly speeds up large scale model training, including vision language models.
This has allowed us to run incredibly large X-ray models. We believe the largest in the veterinary industry. JERRY MARTIN: Clinics are now getting results in minutes, not hours or days.
The significance of that can't be understated. Dr MARK PARKINSON: When a pet is sick or injured, every second counts. As a pet owner myself, it's about minimizing worry and for them to get the best possible care when they need it.
JERRY MARTIN: Over the last 12 months, Rapid Read has potentially saved hundreds of pets in life-threatening situations. That is a great metric by which we can measure our success. RAJ MADHAVAN: So Michael, thanks for sharing the insights and knowledge with all of us at Ignite.
So at Microsoft, we are truly grateful for our partnership, thriving partnership, and above all, your trust in your AI journey. Thank you very much. [ APPLAUSE ] So for the audience, if you have any questions on this case study, you can scan in this QR code to get more details.
So with that, I'm going to welcome Julio to the stage. JULIO FIGUEROA: Thanks, Raj. [ APPLAUSE ] RAJ MADHAVAN: So Julio is an IT manager at Methanex.
He's leading several AI initiatives and helping manufacturing sites operate more safely and reliably. Take it away, Julio. JULIO FIGUEROA: Thank you, Raj.
Hello, everybody. Thanks for coming. Well, I'm here because I would like to take the opportunity to share our experience implementing solutions for some business problems that we have at Methanex using foundational models.
But first of all, let me briefly explain to you the context of Methanex. Methanex is the largest methanol producer of the world. We have manufacturing operations in six countries in different continents.
You might think you don't use methanol. Well, of course, because you don't go to the shop and order a gallon of methanol. I know that.
But I can tell you guys that you use it on a daily basis. In different forms. For example, you might use it as a paint, as a plastic, and even as an energy form.
But in order to produce this chemical, we have to do it in large industrial facilities. Have you ever been in an industrial facility or in a manufacturing site? Some of you?
Oh, I can see several of you. So for those that haven't gone to a manufacturing site, you might know that those places are dangerous. Once you go in, you can feel there are many hazards.
So that makes us conscious of that, and we spend a large amount of our time thinking how to make our operations safer. So it's natural that with all of this huge AI boom, we start thinking, how can AI help us to make our operations safer? This is how we started this journey.
And the first initiative that we implemented, it was using LLMs for health and safety data. Let me give you some context. After several years of operations, actually, after decades of operations, we have developed significant organizational knowledge.
And this organizational knowledge is stored in formal databases. We have incident management system. We have different procedures, forms, and so on.
But knowledge is not always structured. Knowledge is transferred through socialization process also. And that implies that large part of organizational knowledge is stored in unstructured databases.
You probably will feel empathy with this, because a lot of information is saved in forms like PDF files, drawings, and even pictures. So how can we efficiently retrieve this knowledge? And this is where LLMs can support this initiative.
We implemented a RAG architecture that reads these unstructured databases, enabling our users to retrieve this data using natural language. So as you can see in this example, people are able to retrieve information, but not only finding files, but also it can act as an assistant for relevant business processes, like job hazard analysis. In this case, as you can see, the assistant is recommending or highlighting some lessons learned from previous incidents in order to make safer a really dangerous work that is going to be completed.
But building trust of these systems is key. And what is the biggest concern that people are prone to after seeing something like this? People fear of hallucinations.
Azure AI provides different mechanisms that complemented with some prompt engineering techniques can help you to control or at least significantly mitigate hallucinations. In this picture, you can see an example that can sound really dumb, where I am asking the system, "Can you tell me how to make ice cream? " But notice the answer.
It is telling you, "I don't know. " Even people is not able to say I don't know sometimes. So after implementing this, let me share with you just two quick learnings that might be useful if you are planning to start this journey.
After years, organizations develop their own culture, and vocabulary is something critical of the culture, you know, the vocabulary shapes our culture, which means that you create some words that only apply for your context. Naturally, LLMs are not aware of these words. That's why, in our first iteration of our RAG system, we base it purely on semantic search.
And even though results were precise, the precision were good, the recall was low because the LLM was not aware of our own vocabulary. In order to solve that, a hybrid search approach significantly improved the recall. So I would highly recommend you, if you start this journey, to take this consideration in order to get better results.
As a second key learning from this experience, it was the user adoption. The first iteration was built using a totally different website, something that people need to be trained. But after integrating our results, as you saw in the picture, within Teams, the user adoption boost up, which makes sense.
So I would like to highlight that Microsoft Teams, the developer portal, provides a comprehensive analytics tool that helps you to understand how much the system is used and you can understand return users and so on. But as Raj said at the beginning, there is AI beyond LLMs. There is AI beyond LLMs.
So it's not just chatbots. So I would like to highlight our second experience using pre-trained models in our organization. Let me add a little bit more of context before presenting the solution.
Manufacturing plans requires to be reliable. They require to be predictable. And why?
Because reliable plans are safer plans. And in order to make it reliable, you need all the components working in harmony. Let's make an analogy.
Your car. In order to be reliable, you need the car, you need the engine working in harmony with your pumps, your compressors, the transmission. Everything needs to be tightly working in a nice harmony.
However, while you are driving, you are always looking at small things and a weird vibration, or a weird sound, and you are always paying attention to the anomalies. Because anomalies can be an indicator of a problem. And our problem affects reliability.
And when we are talking about plants, we might be creating a hazard. This is why we deployed Nixtla TimeGen, which is a production-ready model for time series. This model allows you to deliver accurate forecasts for your time series data.
And what I really want to highlight here is it requires minimal coding and you don't need a dedicated machine learning team. Let me explain to you the workflow of this solution. In a typical manufacturing setup, you will find a workflow like this.
You have your plant full of sensors, IoT sensors. You measure practically everything. You try to measure things like pressure, temperature, vibration, speeds of the rotators, and so on.
All of this data, that is gold, is stored in historians. Historians, they are databases for time series data. And then end users consume this through dashboards or they get alerts when some values cross some thresholds and so on.
What we did is we connected our historian with Azure AI services to the operational data and from our previous learning, we inject the results, which are the forecast, within the historian, making the users able to analyze this data in the tool that they already know. How this would look like? This is the result.
Okay, I know that probably they are not all familiar with this kind of chart, so let me do a zoom of this and then let's discuss a little more of this. What you are seeing here, the red line represents the actual value of the asset that we are analyzing. In this case, it's a pressure.
The green line represents our forecast. And the orange and purple lines are presenting the tolerance range of the normal behavior. So if the red line goes out of the normal behavior, we are in the presence of an anomaly.
An anomaly is not a faulty. It's like when your car started doing a weird noise, you know. So I want to highlight that these kind of systems, this is predictive maintenance, is not replacing at all any of the formal maintenance strategies.
So this is something complementary to everything that we already do. Now, let's go to the zoom out again. Understanding now the chart, when you look at this, you can even visually see how accurate is the prediction.
You see how close the red line is to the green one? So visually, you can quickly see that this is very accurate. But in numbers, we have measured in this particular chart an error of 1.
6, which is excellent for this kind of prediction. But, okay, if someone in the room is familiar with machine learning techniques, it's highly probable that you might be thinking, "Okay, Julio, but this is not new. We have been doing time series forecasting for years.
" It's true. This outcome is not new. This is not the value that I'm presenting here.
The value that I'm presenting here is we didn't train any model. In fact, the only line of code that you need to do a forecast is this. That's it.
With this line of code, you are able to produce forecasts. No training required. And you know what's the real impact for organizations?
We can focus on the business problem and not in the technical problem. Furthermore, predictions are important. But it's important also to understand the forces behind this prediction.
Let me explain to you. This is another chart that you can get from TimeGen where we are explaining what are the different variables that I'm going to show you. In this case, variable one is the one with the greatest impact or weight in the forecast that I presented before.
And as you can see, variable 16 is zero, which means there is no impact. That can be explained because variable 16 can be, for example, the backup system for variable one. But let's imagine this weight, this balance of weight, is hardly changed.
Even though you find that outcome keeps within the normal ranges, if the forces behind your forecast are different, it might be a signal of an anomaly that operators wouldn't be able to see. So as you can see, the line chart that is below is showing variable two that at a given moment of time, the weight changed. So even though the outcome, again, even though the outcome was within the normal ranges, we can start seeing that there is a noise in our car.
There is a weird vibration that might need an inspection, a preventive inspection. In conclusion, our journey started with a RAG pattern system that allows our users to restore all this organizational knowledge that I don't want to say lost, but it was really difficult to capture from our unstructured databases. Our key second learning here is pre-trained models like TimeGen are showing similar precision numbers than models built on fundamental principles.
Which means if you want to create a model based on fundamental principles, you need to have a lot of knowledge about the process. However, with TimeGen, you don't need that. So you are getting the same quality of results, but without a specialized team.
And finally, Azure AI is suitable for organizations like us because it makes all our solutions scalable across all the manufacturing sites regardless of their geographical location. Thank you. [ APPLAUSE ] RAJ MADHAVAN: Thank you, Julio, for sharing all the benefits of Azure AI model catalog and also the key learnings.
Really appreciate it. JULIO FIGUEROA: Thank you. RAJ MADHAVAN: So thank you, everyone.
We appreciate your time with us. For more info, you can contact Michael or Julio, or you can stop by expert meetup to get your questions answered. Your feedback is very important to us.
Visit this URL, and you can submit your evaluations. And these are all the breakout sessions. Either you would have participated, if not, we strongly recommend you listen to the recording.
They are tied to what we talked today. So with that, I'm going to invite Michael and Julio back to the stage. And we have a few minutes for a Q&A.
If someone has questions, please come forward and ask questions, please. [ APPLAUSE ] [ INAUDIBLE ] JULIO FIGUEROA: Yeah. Every week, basically.
So because we have these pipelines, now built, we're going through new models and understanding very, very regularly. So definitely on a weekly basis. [ INAUDIBLE ] RAJ MADHAVAN: Let me repeat the question.
The question from the audience is, human in the loop. So how do you decide when you need to have human in the loop? So I'll let you, Julio, start.
And then Michael, you can also provide your perspective. JULIO FIGUEROA: Well, it definitely depends on the context. Manufacturing sites are highly conservative because the risk can be a fatality.
So everything, and actually this is one of our biggest blocks in order to deploy these kind of solutions. Because we don't want people to blindly rely on these solutions. So I would say that in manufacturing environments, these solutions are only an assistant.
Always they require to be validated by a human. But it depends on the context. MICHAEL FITZKE: Same for us, actually.
So for us, people that use this technology are trained veterinarians, highly trained specialists. So we always have a human in the loop in this use case. RAJ MADHAVAN: Thank you.
That's a good question. Anyone else? So you can always meet us in the expert meetup if you have questions after you get to digest all the information, that great information that Michael and Julio shared.
So thank you all for attending. Have a nice evening.
