one of the powerful features of agent force is that you as developer can extend its capabilities by bringing your own large language models you do this with model builder so with model builder you can connect to various popular models like GPT Claude or Gemini hosted on AWS gcp or Azor and then Ed them right within agent force but what about other models hosted maybe somewhere on different platform the new llm open connector lets you bring in any model you want hi I'm moit in this video we'll learn how to bring in any model from your
preferred provider into Salesforce using the allnew llm open connector let's dive right into [Music] it so what what is llm open connector llm open connector is our API specification that we publish on GitHub that your model providers apis can follow so that they can be connected to this model builder now let's take a look into the spec so you can see that you know this currently we support chat completions so chat completions apis are pretty standard across different models and model providers and this lets developers create conversational AI by sending messages to an AI and
then getting responses and thereby enabling natural dialogue and you can see that it has a certain request and the response structure and you can you can browse using this website here now as long as you have an endpoint that adheres to this spec Salesforce lets you bring in your model in the model builder now there are a lot of providers who follow this spec because this is very standard but let's say if a provider does not follow you can build an endpoint that confirms to this specification and then that endpoint you can have code to
make calls to your provider API so it's sort of like a middleware so let me show you how to build one of these end points so this is hugging phase where developers build large language models and open source and this has chat completion support as well and there are a lot of models that support this chat completion so let's say I want to bring one of these models for example mixol into Salesforce model builder so you can see that hugging phase lets you deploy this model as inference API using the serverless inference for prototyping uh
and if you look at the the apis for this it's a little different than the llm open connector specification so if we were to compare this hugging phas specification to our llm open connector specification you will see couple of differences the first one is if you see the authentication hugging pH supports basic authentication while llm open connector follows the keyas authentication there's nothing wrong in both these approaches it's just different so that's the first difference the second difference that you see here is the response and the request structure hugging phase has a little different structure
compared to this standard structure that we have in the llm open connector so what you could do as a developer is build a middleware application using the language of your choice and and the Frameworks of your choice which confirms to the open llm specification and then that has an end point that can make a call out to your provider's API in this case it's the hugging phase API so you can see that this is a simple nodejs application that I've built and what I'm doing here is setting up this endpoint and here you can see
what I'm doing in this uh endpoint point is simply taking the the request and converting into a format that hugging phase requires and make an API call to hugging phase and then transform back into the response format that again confirms with the specification specified in the llm open connector of model builder so once we have all these I deploy this application on Heroku and once I have this endpoint ready now I can bring this model directly in to my model builder so let's see how so here we are within the model builder and if you
click on this add Foundation model you will see we have all these Foundation models but now we'll let you also connect to your own llm and bring in your own provider so you can click on this and then say next and here I can enter the name of my model in this case it's mixol uh and the URL which is my hoko application and then provide the key and then uh optionally you can provide the model name here and you can set your token limits and simply click on connect so behind the scenes it's testing
the connection with the sample prompt and you can see that it's successful and now I can create a configuration for this model so all I need is say new and you can see I can adjust the capabilities and uh you know at this point we only provide chat completion capability but you can adjust the temperature settings of the model to see how much creative you want this model to be and make sure that you actually test this configuration with maybe a simple prompt like for example here I can say write a welcome email for guest
registration something like this and you you can start generating to see if the model is responding correctly next you save the model by providing the name so here I am within Salesforce prompt Builder that lets you build prompt against large language model and then use this either in embedded AI applications that you build on Salesforce or building agents in agent force by exposing these prompts as actions now here you can see that I have tested this prompt with a standard models that we provide now you can easily switch to custom model and you can see
that same mixt model that we generated through the model builder now available here for us to test this so we simply go ahead and want to test this and you can see that we get the response from the mixl model which is our custom model in this case now and you can also see that it still applies all of the security protection from the Einstein trust layer so even with a custom model that you just brought into the model builder all of our llm intractions from Salesforce is still happening to that Einstein trust layer giving
that security benefits even with the custom model now here I an agent Builder that lets you build agent and you can see that I have an action generate personaliz schedule this action is using the same prompt template we just saw and now I can ask for a personalized schedule for the contact Sophia Rodriguez now this action that's happening is using the mix draw model which is the custom model that we just brought in and you can see that uh using this model it's going to bring up the the response and the mixture model can also
do a bit of planning and the reasoning as uh you can see from the response finally you can also use these within the models API in Apex or rest API to get responses from your custom model like here you see I'm using the the custom mixture model that we just saw to generate an email requesting for uh much uh needed PDO to summarize you have seen how to build an endpoint using the llm open connector to bring any model from the inference platform of your choice into Salesforce using the model builder now this allows you
to take advantage of all of the security that comes with the Einstein trust layer and all of that audit and the feedback capabilities that we have um and also all of the Native platform features that comes with our agent force tools like for example U agent force actions you know models API promp Builder and all of these features uh to get started we have an open-source repo that provides recipes and cookbook and all of the code that I used for this app and others examples here is now available as recipe here and this is a
blog post that walks through all of the configuration steps that are required in the model builder uh to bring in your own custom model now this resource slide has all of the link Consolidated so I hope uh this is useful to you and thank you for watching don't forget to subscribe to our Channel and like this video if you have found it helpful thanks again [Music]
