Building and training ML models with Vertex AI

hi i'm priyanka vergaria and this is ai simplified where we learn to make our data useful we're helping an imaginary company that is in the process of building their machine learning workflow and to accelerate ai innovation they are looking at vertex ai which provides tools for every step of the machine learning workflow for managing data sets to different ways of training the model evaluating deploying and making predictions in the previous episode we started by looking at the first part of any ml project the data sets vertex ai supports managed data sets for image tabular text

and video data in this episode let's move further in the ml workflow and talk about building and training the models the managed data sets we created earlier feed directly into the model building and training although you can bring external data sets while building the model manage data sets bring some added benefits as i laid out in the previous episode we have two options to train the model we can use automl which allows us to create high quality custom models with minimal effort and machine learning expertise and the second option is to write custom model training

code where you can run custom training applications in the cloud using pre-built containers of your own now when should you write your own model code and then should you choose automl it depends so let's see if your use case fits a supported automl offering then starting with automl is a great choice and these use cases include cases where you have images or video classification or object detection text classification entity extraction from the text sentiment analysis of text tabular regression and classification but if it does not fit any of those categories or your model takes a

mixed input type such as you have images and then the metadata is in the form of text then it makes sense to use custom model the other technical factor could be that you have specific requirements for the underlying model if you need control over your model's architecture framework or the exported model assets for example maybe your model needs to be built in tensorflow or pi torch then you use custom model otherwise for the basic use cases just use auto ml the next factor could be how experienced is your team with ml and ai if you

have a team with limited machine learning expertise in building custom models then exploring auto ml makes sense before looking into the custom model code because it is the lowest barrier to entering another deciding factor could be your team size now if you have a small data science team and less machine learning expertise in that team it perhaps makes more sense to work with automl because custom model code would require more time and maintenance and use automl if you want to develop a quick initial model to be used as a baseline or a proof of concept

which could eventually end up being your production model but if you're looking to improve on your existing baseline or heuristics then you might want to write your own custom model to test out the possibilities hopefully these factors have clarified which path to take when it comes to building the models now let's see how to build and train the auto ml model using vertex ai with that on to the google cloud console we learned how to create data sets in the previous episode but for the sake of completeness let's create a data set in this example

we are using credit card fraud detection data set available on bigquery public datasets to create a credit card fraud detection model i have included the link to this dataset below if you would like to follow along once the data is uploaded we can analyze our data in the rows and columns right from the data set details page we can select train new model define our objective which is classification here since we are labeling examples as either fraudulent or non-fraudulent this is where you decide whether you want to use automl or custom training in this video

we are focusing on automl so we will select that stay tuned for the next episode where i will cover custom model next we give our model a name under the target column we select class this is an integer indicating whether or not a particular transaction was fraudulent zero for non-fraud and one for fraud in the advanced options you can specify a specific train test split if you don't want vertex ai to handle that automatically for you in this case we're using the random assignment but you can also specify this manually when you upload a csv

of your data or you can have the data split chronologically if it has a time column in the choose training options section we will go to the advanced option and since this data is heavily imbalanced less than one percent of the data here contains fraudulent transactions we will choose the auc prc option which will maximize precision recall for the less common class in this case we care more about ensuring our model can classify fraudulent transactions correctly now the last step is compute and pricing here we enter one as the number of compute hours for our

budget and leave early stopping enabled training your automl model for one compute hour is typically a good start for understanding whether there is a relationship between the features and labels you're selected from there you can modify your features and train for more time to improve model performance then we're ready to start training from here we can leave the ui and we'll get an email when our training job completes alright so our model is ready let's explore the model evaluation metrics we head to our model tab and evaluate the model we just trained there are many

evaluation metrics here we will focus on two the confusion matrix and feature importance a confusion matrix tells us the percentage of examples from each class in our test set that our model predicted correctly in the case of an imbalanced data set like the one we're dealing with here this is a better measure of our model's performance than overall accuracy remember that less than one percent of the examples in our data set were fraudulent transactions so if our model accuracy is 99 there is a good chance that it's just randomly guessing the non-fraudulent class 99 of

the time that's why looking at our model's accuracy for each class is a better metric here the confusion matrix shows that our initial model is able to classify 85 percent of the fraudulent examples in our test set correctly this is pretty good especially considering our significant data set imbalance next we could try training our model for more compute hours or gathering more training data to see if we can improve that from this 85 percent now this shows us the features that provided the biggest signal to our model when making predictions the feature importance is one

type of explainable ai a field that includes various methods for getting more insight into how an ml model is making predictions the feature importance chart is calculated as an aggregate by looking at all of our models predictions on our test set it shows us the most important features across a batch of examples this chart would be more exciting if most of the features in our data set were not obscured we might learn for example that the type of transaction transfer deposit etc can be the biggest indicators of fraud in a real world scenario these feature

importance values could be used to help us improve our model and to have more confidence in its prediction we might decide to remove the least important features next time we train a model or to combine two of our more significant features into a feature cross to see if this improves our model's performance we are looking at feature importance across a batch here but we can also get feature importance for individual predictions in vertex ai and we'll see how to do that once we've deployed our model now that we have trained model the next step is

to create an endpoint in vertex ai a model resource in vertex ai can have multiple endpoints associated with it and you can split traffic between endpoints on our model page there is deploy and test tab there we click deploy to endpoint give your endpoint a name like fraud version one or two and then leave the traffic splitting settings as is and then select a machine type for your model deployment we used an n1 high cpu2 here but you can choose whichever machine type you would like including one with gpus then select done and click continue

leave the selected location settings as is and then click deploy your endpoint will take a few minutes to deploy when it is completed you will see a green check mark next to it just like my version one here now we are ready to get predictions on our deployed model there are a few options for getting model predictions the ui and the api let's check out both one by one on your model page where your endpoint is shown scroll down to the test your models section here vertex ai has chosen random values for each of our

models features that we can use to get a test prediction you're welcome to change these values if you would like and then scroll down to the bottom of the page and select predict in the prediction results section you should see your model's predicted percentage for each class a confidence score of 0.99 for class 0 for example means that our model thinks this example is 99 non-fraudulent the ui is a great way to make sure the deployed endpoint is working as expected but chances are that you will want to get prediction dynamically via a rest api

call to show you how to get model predictions here let's set up a notebook for that we navigate to notebooks option and select tensorflow without gpus once created open the notebook instance and then we open python3 notebook from the launcher in the notebook we run the command in a cell to install the vertex ai sdk and below that we add and run a new cell and the example from our test set next we need to create the endpoint object and then change the endpoint id and project number to your specific endpoint you can get the

endpoint from the url and the project id from the project dashboard we use that to make a prediction request to our endpoint and we see our prediction around 0.99 for the zero class which means the model thinks there is a 99 chance this transaction is non-fraudulent with that we've learned about the two options to train our models in vertex ai auto ml and custom training and when to use which one then we went into the console and trained an automl model from scratch in the next episode of ai simplified we will look at building and

training a custom model in the meantime let's continue our discussion in the comments below and don't forget to like and subscribe to catch that next episode