welcome back in this video we will introduce the concept of machine learning operations and why it is useful let's start from a situation that every data scientist and machine learning engineer is familiar with which is the academic or kaggle experience this is a workflow for machine learning that is typical of academic classes tutorials that you can find on the net academic research and data science competitions like kaggle competitions in this workflow we start from a data set that is given to us we perform some data exploration then we dive into the modeling and iterate over
the modeling and possibility exploration based on a metric because here the target is to improve as much as possible a target metric say accuracy or precision or recall so we can use very complicated and elaborated modeling strategies that might also be very computationally expensive as long as they improve our final metric and so in this kind of workflow we spend essentially zero time on data creation because the data set is given to us we spend maybe 30 percent of our time on date exploration and cleaning and most of the time on the modeling coming up
with smart and smarter ways to improve our target metric we also have a minimum time spent on inference for example when we have to compute the predictions on the validation or the test set now of course these numbers are just guidelines but i believe that this is a pretty realistic picture of the time allocation in such a workflow using machine learning in production instead looks very different here we have many more stages we start from a problem definition typically coming from a mix between the customer needs what is technically possible and our intuition about how
to achieve the goal and the first question we need to ask ourselves is whether we actually need much learning at all there are problems out there for which a prescriptive solution is actually possible and such a solution is orders of mod into cheaper and simpler than deploying a machine learning model therefore if one solution such solution exists most of the time it makes sense to adopt it and avoid machine learning altogether but of course there are many problems where we do need machine learning and then we start a project by first collecting our data and
possibly labeling them if we are within the framework of supervised learning and we do not have labels then we move into the data exploration phase where we understand the limits of our data set and we uncover problems and queries of quirks of the dataset we then move to data cleaning and pre-processing then to model development and we obtain a trained machine learning system then we need to make it available in a production system where it can finally provide answers to our customers we also need to monitor such a system to avoid performance drifts and other
problems so the data collection and the production part of the process are completely new with respect to the workflow that we've seen before and are not present in an academic setting most of the time so let's stop one second and explore a few of the consequences of having the problem of production so a model that is not in production in this kind of situation is at the moment at least almost completely useless and certainly is not going to make any money which is needed to sustain a company so we absolutely need to think from the
very beginning of a project if and how we will be able to deploy our model in production second as a consequence having a model that is seventy percent accurate that can be deployed in production is infinitely better than having a model that is maybe ninety percent accurate but cannot be put in production because maybe it is too complicated or it requires data that are not available in production so when choosing your modeling strategy and in general how to structure your project you have to consider how complicated your model will be to be put into production
for example considering if the data sources that you are using are available at the required speed and cost in production then because the focus of this whole framework is on the production part you have to make sure that your model is dependable which means you have to monitor it for performance drift and correct the drift when detected for example retraining your model on new data and then lastly you work in a team and your projects should be understandable by all the people in the team and also the people that will come after you in your
role so your process must be repeasible and transparent
