Artificial Intelligence, Standards, and Certifications for Automotive Applications

All right. So, thank you again for joining me today. Um, my name is Molly O'Brien and today we are going to be talking about artificial intelligence standards and certifications for automotive applications.

So, just a little background about me. I have a PhD in computer science from John's Hopkins. Uh where I did research in computer vision, deep learning, and surgical robotics.

Now I'm a senior safety engineer at Exoda and I'm one of our subject matter experts in safe machine learning research efforts and projects. I work also on hardware and software functional safety assessments, FMEAs and FMEDAS. and I do work in the automotive process and semiconductor industries.

And so today we're talking about achieving an overall certified or validated vehicle. Um, and there's a lot of different components that are going to go into that. Um, here I'm just showing for illustration some example um, so safety elements out of context SEOC's as well as an example item.

Um, so if you think about safety elements out of context, you could have a perception AI model. You could have firmware for the vehicle. Um, maybe some internal processor that's hardware only that's being used as well as there would be multiple different items.

Here we're showing an example autonomous driving system where you're seeing AI, software, and hardware all working together for that task. And so then the question is um if you're developing these different pieces that go into the vehicle, which standards are going to apply to each of these different elements? So there are a number of different relevant functional safety standards for AI and automotive and and these I would say are the the key four standards.

Um so first off we've got ISO 26262. This is our automotive functional safety standard originally published in 2011 with the most recent second edition out in 2018. ISO 26262 is the standard that uh you know existing hardware and software is being assessed and certified to for automotive.

It defines a functional safety life cycle as well as requirements for hardware and software um and supporting processes for this functional safety life cycle like software tool evaluation and qualification which we're going to talk a bit more about in a couple slides. Now, recently in December of 2024, ISOPASS 8800 was released. And so, this is an automotive ML and AI functional safety standard.

Um, it defines a specific safety life cycle for machine learning and AI. um it offers additional requirements that aren't in ISO 26262 specifically for ML and AI like data related requirements and you know how you would want to do analysis for a machine learning model. Um it also gives guidance on what pieces of ISO 26262 or ISO 21448 would be applicable to different types of machine learning models or or elements.

Now speaking of ISO 21448, this is the soda standard uh or the safety of the intended function standard. So this standard is going to cover hazardous behavior when the intended functionality is not achieved. Um, so if you think about a machine learning model where the weights that you've learned for a specific example don't make the correct or the ideal prediction, that's an example of something not performing its intended function.

Um, and specifically the soda standard is for systems that have sensing, processing, and acting components. Um so if you think back to that autonomous driving example um that is a case where you would be sensing taking in in uh images and information LAR radar of this outside world using a control algorithm and then outputting some next action or direction uh for the vehicle. Then there's also ISIL IEC TR5469.

Um so this is a technical report for ML and AI in safety. Now it's a technical report. So that means it does not have specific requirements but it gives information um and sort of a framework for thinking about ML and AI in safety critical tasks.

It has a classification for the risk of different ML and AI models based on what the models are used for and the complexity of the model. It outlines different risk factors for ML and AI and proposes risk mitigation, validation and verification techniques. Now, uh talking a little bit more detail about ISOPASS 8800.

So this is the new standard for ML and AI in functional safety for automotive applications. In the standard itself, they give some guidance on how isopass 8800 relates to other standards. Um so this table right here, table 1-1, gives that guidance of how ISOPass 8800 relates to these standards.

Um, there's also just to call out a couple of other sections that are useful if you're looking at how ISOPass 8800 relates to these other standards. In clause six, they give guidance on what work products they expect are going to be relevant for ML or AI um, development projects in table 6-1. And they give guidance on what parts of the SODF standard they think are going to be relevant in table 6-2.

Um, and there's also an ISO 26262 gap analysis for machine learning that they provide in NXC. Now, if we take that information and think about how these different standards apply to the elements in this, you know, integrated vehicle that we're talking about. Here's sort of an illustration of what typically is going to apply.

Um so ISO 26262 this is generally still going to be applicable to to just about everything. Um hardware and software is still going to be assessed and evaluated in the same way that it's been over the last almost 15 years with ISO 26262. Now, for AI elements that are either part of an SEO or part of an ele item, um we expect you're going to be looking at ISOPAS 8800 with a tailored version of ISO 26262.

Um so, you know, naturally not all of the same techniques for software, let's say, you might not be using those same techniques, but change management is still going to apply, quality management is still going to apply. Um, and so anyway, we expect that people are going to be using a combination of ISOPASS 8800 and 26262. And then for the soda standard, we typically see that applied more at the item level.

Now, the soda standard, it does make a reference to safety elements out of context. Um, but generally if you think to that sensing, processing, acting framework, it's going to be applicable more for things that have a function closer to the vehicle level. Um, now to talk about some examples of safety elements out of context that might have um sort of still be applicable, I would say think of something like a lanekeeping safety element out of context where you're still performing some function that's close to the vehicle level.

Um, so some of the concepts from from ISO 21448 may be applicable. In contrast, if you think of a general purpose um processor or a general purpose um vision system that outputs predictions, but it's not clear, you know, what the final user will be doing with those predictions. Those would be cases where we would say that we think that the soda standard would would likely not be so applicable.

Now with that in mind, let's talk through an example ML design life cycle. Um, and we're going to use this as sort of a framework then when we talk to the different stakeholders who are likely going to be involved in some of these machine learning development projects. Um, so first here we're talking about there's some item or some higher level that is choosing to use machine learning.

then that t those requirements that functionality is going to be allocated to the machine learning subsystem. Now very likely there's going to be pre-processing and post-processing with the actual trained ML model. Um just sort of for for background typically if you've got some machine learning model you are often not feeding directly let's say sensor inputs or data right into the model.

There's probably going to be some filtering, some pre-processing before that data is handed off to the model and then likely there would be some post-processing afterwards to let's say look at the model predictions over the last several frames and then make a a decision uh at the vehicle level, you know. So those types of um temporal filtering or uh looking for consistency of predictions across frames um that's going to fall into this post-processing block. Now to train the ML model you need to select the architecture, the number of layers, the size of the different layers, what calculations are done between the layers in the machine learning model, the optimization algorithm that you are using to actually train the weights and and arrive at your final model and then the data.

So super important curating the data that is going to be used both to train the model and then also test and evaluate the performance. These pieces all feed into the model training itself. And then first here I'm showing a model verification.

Um this is to illustrate the verification done just with the model itself. So if we think of uh a perception let's say an object detection machine learning model um this would be the per frame accuracy or the per frame um you know precision and recall for whatever task you're performing. So that is going to happen at some point.

Here we're showing that first. Um then you're going to want to evaluate the machine learning model in the context of its subsystem. So with with post-processing maybe temporal filtering um with these pieces overall how that whole ML subsystem works together and then finally that ML model and the subsystem are going to be integrated into some larger item um could be could be the item level and then eventually it will also be the vehicle level.

So we're expecting that people are going to also be doing verification on all of these pieces together. Now, we talked through a typical ML design life cycle. Um, but in practicality, those pieces or those steps of that design life cycle may not all be done by the same group or team.

Um, and so here are some typical roles that we see in ML system development. First would be the integrator. So this would be the group who's developing the item or the vehicle.

Um then the ML model developer. So these would this would be the team actually training the model and very likely testing the model on its own before it's put into the system. And and depending on the project, this might be the SEO level and then also the tool chain supplier.

Um, and so these tool chains can be used during training, could be used during data collection. Uh, we'll talk about this in some more detail. Um, but there's often a tool chain supplier who is not necessarily the person developing the model or integrating the model into a system.

Now, if we take that ML design life cycle that we talked about, here are kind of two examples of how these tasks might be allocated to these different stakeholders. So on the left, we're seeing an a development where the integrator is really doing everything. you know, they're creating the item, they're defining the ML subsystem, they're training the model, they've collected the data, they're using their own training platform, and then they're going to be verifying things every step of the way.

Um, on the other extreme is a more heterogeneous development where there's an item integrator. they define you know what functionality they need from the ML subsystem but then they reach out to a model developer uh who is actually training and testing the model in isolation and in this example this ML model developer is using a third party tool chain um and so for this type of heterogeneous um development the question really is for each of these different groups which certifications are possible ible and helpful, meaningful, you know, for each each person and then also for for the team that's going to be using the products down the line. Um, so this is really one of the key questions that we're going to talk about today.

First off, we're going to talk about the tool chain suppliers. Now, I've mentioned this tool chain term several times. Uh, and so here I just want to show an example of what tools are often used in an ML development project.

Um, and so here is a is a great illustration showing that you might have data recording tools, conversion tools, database tools, tools for labeling the data, which we know takes a lot of time and effort. um version control tools, the DNN framework. So, PyTorch and TensorFlow are two very common examples, but this framework is really how you're running the training, you're running the optimization, you know, you're adjusting the weights.

Um, you could also have experiment tracking tools. Once the model is trained, you may have target conversion tools to go from, let's say, the Python code to something that's going to run efficiently on your hardware. Um, field monitoring tools.

So, here you can see that there are are likely many tools that are going to end up being used in a machine learning project. And we can group these into some different categories. So data processing tools, model creation tools, target tools and field operation tools.

And so when someone is using this ML tool chain in a functional safety project, um the question comes, okay, how do we know we can trust that this tool uh is going to work correctly? So software tool evaluation, this is something that's been covered for many years in ISO 26262 in part 8 clause 11. Um, and more recently in ISOPAS 8800, software tools are covered in clause 15.

And so a question that we get from customers is, you know, if they're using a tool for machine learning, um, which standards do they need to comply with to help their customers to be able to achieve certification? So here what we have is an illustration of the different requirements in ISOPASS 8800 clause 15. and and I've added in where 8800 refers to these other standards we've talked about um just to give some clarity on how people can can meet these requirements and really the the key point I would say is this requirement 15.

3. 2 two, this is talking about confidence in software tools. Um, and they say in a note that you can really follow the tool qualification and tool evaluation requirements called out in part 8 clause 11 of ISO 26262.

Um, so in that sense, it's it's fairly straightforward for our tool chain customers that you can follow this one set of requirements. you're going to meet the requirements for ISO 26262 and for any customers who are interested in ISOPASS 8800 those requirements will also meet what's defined in ISOPASS 8800. Now I'll mention you know if if you're interested in this tool chain evaluation qualification and certification um these are services that Exoda offers.

So what we find is for these types of complex tools, it's really the tool chain developers themselves that are going to have the knowledge to do a a meaningful tool evaluation or qualification. So what we offer is templates and support for for these teams so that they are able to uh quickly and effectively re reach a a good solution. Um, often what this might look like is is we might have a kickoff workshop where we work with the team um to go through their tool and explain what they need to do, explain templates if they're interested in purchasing templates from Exoda.

Um, and then we have interactive support meetings. Um and so then once that tool evaluation is done which will define the tool impact and the tool detection exit and naturally will review that provide any feedback um based on the TI and the TD that determines the tool confidence level which which is then used to determine whether additional qualification is required. Um and likewise we provide iterative reviews and support for the tool qualification.

And then we also offer tool chain certification uh for customers who are interested in in going that one more step. Uh I would say if you're interested to see examples, please check out our safety automation equipment list on the website. We've got examples of our tool certifications.

Um and so then that provides certificate and an assessment report detailing the use cases that were looked at and how the specific tool meets the requirements of this standard. Um now if you're interested in tool chain certification um you know this is something that is going to demonstrate to future customers that this tool will work as intended you know based on the use cases defined. Um and so if we think back to that neural network training tool chain um having a qualified certified tool chain will say okay when you're training in this in this framework and you run back propagation you know you can have confidence that that gradient's going to be calculated correctly those weights are going to be updated correctly.

Um, one thing that is not included typically in tool chain evaluation will be comments on the model functionality itself. Um, naturally, if you think about that tool chain uh for ML development example, the tool chain developer, you know, they don't know for everybody who's using the tool what model they're trying to train, what data they're going to use, what functionality they require. Um, so that really is typically going to require the model developer to be involved to be able to comment on the functionality of the model.

Um, and with the tool chain, it really just gives confidence that yes, this is a tool that can be used um, with this functional safety project, you know, without concern. Now, speaking of the ML developer, that brings us to our next stakeholder um and the ML system development. And so, here we're going to talk a bit about what certifications are going to be helpful, meaningful for the ML developer.

Now, as we talked about um for developing the ML model, isopass 8800 is going to be in scope. Um, and so here what I've got is just a quick summary of the different clauses in ISOPAS 8800 and on the right an indication of what we think is going to be applicable for the ML model developer. Um so if we think about AI safety management, assurance arguments, derivation of AI safety requirements, selection of technologies and architectural measures, then moving on to other sections, data related considerations, verification and validation of the AI system, safety analysis of the AI systems and measures during operation.

Really the the punch line is this is all going to be applicable for the ML model developer. Um in this example we've already talked about the confidence in use of the AI developments and software tools being covered by the tool chain developer. So if that um evaluation and qualification has already been done then that is something that the ML model developer does not need to do themselves.

Um and so really what we're seeing is that for the model developer all the concepts talked about in ISOPASS 8800 are going to be applicable. Um then I would also say depending on what is in the scope of this of what the ML model developer is is creating. You know if it has traditional software and traditional hardware those would be cases where we would expect ISO 26262 to also be looked at in combination with these AI topics covered with ISOPASS 8800.

And then now let's move on to the integrator and kind of see what we think is going to be applicable for them as well. Now here I'm going to I'm going to call out two different options for the integrator. The first being an integrator who's taking this ML model that we're going to imagine has already been uh certified.

first without modification and then the case where the integrator takes this certified model but also does modifications for example fine-tuning the model and then the question is in a typical case which of these clauses are still going to be applicable for the integrator and what you'll see is once the integrator has modified the model we expect that pretty much everything is still going to be in scope um and if the integrator has not made modifications. They're going to be some topics that we think can be covered by what's already been done by the ML model developer. Um, and so then here if we kind of keep looking through the different sections, you know, for example, the data related considerations, if that's been done for the model and the model hasn't been modified, you know, there's no more training, we think that that's likely could be covered uh by this by the model developers actions.

Um but then once you again start changing those weights uh making modifications we think that's something that is going to be important for the integrator to look at as well. Um naturally if you're taking this model and integrating it into a system you still need to look and make sure that the requirements at the system level at the item level are covered. Um so this is something that we expect is always going to be in scope for the integrator.

Um, and then just jumping along like for the safety analysis, if that's been done for the model and the model hasn't been changed, um, we think that the ML model developer analysis can can be used as long as there's still kind of a rigorous verification at the integrated item level. Um, but again, if you're changing the model weights, this is the safety analysis is something we think you're going to need to look at again. Um and so so as we're going through this, you know, we're seeing that there's a lot of these pieces that we've got marked in scope for both the model developer and integrators.

Uh um and also say we're we're not sure how likely it is that there will be many integrators who won't need to make modifications. So we think it's probably going to be most likely that there are going to be some fine-tuning or some modifications by the integrator uh after the ML model developer. So you might be wondering hm is is it really going to be that beneficial for the ML developer to be doing some of these certification related tasks if we think the integrator is going to have to do them as well.

Um, and so I'd say really we do think this is going to be application dependent on let's say how closely the ML developer is working with a particular integrator. Um, how complex the task is, how much effort it would take to kind of for the integrator to do some supplemental verification. Um, we we think we're probably going to end up seeing both approaches.

Um, on the one side, if the ML developer does do some of the activities related to certification, it helps partition and spread the cost of the effort for both the ML developer and the integrator. Um, but especially if you're planning and you know that the integrator is going to be making a number of changes, there is a risk that that could limit how much essentially credit you can claim for the ML model certification activities. Um, so we we expect this is going to to differ case by case and you know if you find yourself in a position where you're curious what could be a good approach for you, please I would encourage you to reach out.

We we're very happy to to to meet with you and and understand your application and try and work and figure out what's going to make the most sense uh for you. Um I would also I also this is also something that I like to tell um customers who are thinking about this is also think about maybe apart from certification what information you can provide to your customers that can help them or facilitate their own integration certifications you know if if that's something that they're interested in. Um, and the example I like to give is if you think about FPGA manufacturers, which I know seems totally not applicable.

Um, but for FPGA manufacturers, which we certify a lot, we work with them a lot. um for that programmable logic fabric. They don't know exactly what their customers are going to configure in that logic, but they provide information.

They might provide tools. They provide support to make it easy for their customers to do certification to do their F FM femda calculations. Um, and so I would just encourage you to think about what information as an ML developer you could provide that even if the model certification still needs to be done by the integrator, um, can make that task easier.

So again, you know, it it's still like we know it's a new topic. Um, so it's interesting to see how everything is is progressing over time. Now I would be remiss if I didn't mention that you know these are all services that Exoda provides.

We provide certification services for ISO26262 ISOPATH 8800 the SODF standard. Um we can also provide integration services if you're looking to do some analysis of integrating pieces together. Um but we do also offer services that often are useful before someone is ready for certification.

And so these we call the advisory services and the analysis services. So here are just some examples of what these advisory and analysis services can offer. Um we can do AI performance and data analysis.

So this is a statistical analysis of a trained model to understand the performance throughout the operating domain. Um typically people are going to choose to use machine learning for complex tasks like you know autonomous driving tasks or perception tasks. And so there can be a lot of variation in the highdimensional input space.

And so understanding how the performance changes throughout this space um can often be very beneficial and it can help people understand what additional data might be most helpful if you want to tr continue training your your model to achieve better performance as well as what test data would be helpful to clarify the performance of your current model. Um so that's one one service where kind of whether you're interested in certification or not um finds off of often helpful for people. Um we can also work with you to identify acceptance criteria.

So this is typically the level or levels of performance that are meaningful for your task um to so that if the model can reach this performance you know you kind of have reached a meaningful uh level of functionality. Um so again this is something that can be helpful and meaningful outside of the context of certification. um if you think you might be interested in certification in the future, we can also work with you to understand how this criteria is going to relate to future SIL and ASEL targets.

And one more I want to mention is um sometimes when we talk with customers, one of the challenges they're facing is how to communicate the performance of their current model. Um because like like we even talked about earlier when we were talking about verifying the model versus verifying the ML subsystem for anything there's going to be typically a per frame performance metric but there's also if you make some aggregations over time that performance might be different depending on where a potential customer wants to use the model the performance will be different. Um, so this is really an activity to help create a clear, concise report so that it's easy to communicate potentially to customers, maybe to different groups within your own team of how well a model really does work.

Um, so I just grabbed kind of three services we offer to put on this slide, but if you're if you find that you're interested in something that maybe isn't listed here, I would also encourage you to reach out. you know, we love to talk with um potential customers and and understand what you're looking for and and see if it's something we could offer. Um yes.

Now, I'm also happy to announce for people who are interested in machine learning that we are offering a new course ML 301 ML and AI in safety critical tasks. Um so this is a course I'm going to be teaching this course in September, September 9th to 12th. It will be online for uh half four half days.

Um and in the course we're going to have theoretical sessions, coding sessions, and group discussions. Um so I if you're interested, I I hope you consider signing up and and saving the date. And so with that, that's that's all I have for today.

Um thank you so much for your attention and I'll now be happy to take some questions. Okay. Okay.

Great. So, I'm I'm seeing a question here on what if an integrator applies modification and then think that the model developer certification can be skipped. Um, so that's a great question.

What I would say is the the task of analyzing the model's performance can't be skipped but it as long as the information from the ML model developer is available that is something that the integrator could could sort of supervise or or or be the one who's going through that. Um and so we would we would need to have information on let's say testing of the model, the outputs, um some information on kind of how the model was designed. So I think it's definitely possible for the integrators to sort of be the one driving.

Um I would expect that we just might need some input provided from the ML model developer. Um, but I'd say it's not so different from if sometimes in existing certifications a tool is used and the our customer might need to reach out to the tool supplier to get some information, but generally it's not a problem for the certification to be able to run. Um, so great question.

Thank you very much for that. Um, let's see. The next question is asking about if this is applicable to IEC61508 systems as well.

Okay, so that's another great question. Um right now the the third edition of 61508 it's in process it's not released yet um so there aren't explicit guidelines from the 61508 ecosystem on machine learning but I would say the fundamentals of for an ML subsystem you know we we need to have just just like everything we need to have processes in place where we write down what we do we verify that we've done something correctly um you're going to need to have some guidance on or or do some analysis of what data has been used, the performance of the ML model. Um so I I would say those key concepts I expect are going to be very similar to what's decided in the 61508 um ecosystem.

Now, what that would mean if you're interested in certification is we we just wouldn't necessarily be able to list the specific ML standard in the 61508 ecosystem yet because there there isn't one yet. Um, but I I think in terms of of fundamentals that these concepts are going to be similar in both industries. Okay.

All right. Well, thank you so much everybody for attending and for your questions. Um, if you think of any questions after the fact, please don't hesitate to reach out to Exoda.

We'll be sure to get back to you um with answers. And uh I hope you have a great rest of your day. Oh, I'm sorry.

I'm seeing one more. One more question being sent in uh asking about what are the services for data qualification and related assessment certification. Um yeah, so that's that's a great question.

Um when we're looking at the data qualification, um we are going to want to look at the data that you've used for training and for testing. make sure that it's covered the entire operating domain that you're interested in for certification. Um, and and really kind of relate the model performance to the different operating domain conditions.

Um, and this I'll say this is something I'm very passionate about. If you ever want to talk about this again, please reach out. Um we've have a couple webinars talking in more detail about the specific statistical analysis techniques that we use.

Um but so so basically with the machine learning model with the weights themselves we don't understand how it's making predictions. So when we do the data analysis and the statistical analysis, that's really where we can get confidence that we know where the model was trained to to behave in that we've seen evidence that throughout that domain it's going to perform well and that we have the statistical evidence to back up how well the model will perform. Um and so then those are are really key pieces that have to be inputs to the certification because unlike you know a software where you can do a software review and and read traditional software where you can read what the behavior is going to be that really is is the evidence you have that the functionality of the model is going to be correct.

Um yes. Okay. And then I'm seeing another question in um does the model certification last forever or will the model need to be reertified again after some time?

Okay, that's another great question. Um so for you know any existing hardware and software uh when Exod does a certification it's valid for 3 years and then after those three years we will do a renewal. And so what we do is we look have your processes changed.

We look at field failure data to see whether the observed failure rate for the model being used is below what we predicted in our FMEDA analysis. Um, and so that same type of renewal schedule would apply for machine learning. Uh, I'll say since it's so new for we may even have some check-ins earlier than three years, you know, that's something that we would of course talk about on a per project basis.

Um but just like for the field failure analysis that we do currently for hardware um we would want to look at field failure reportings for the machine learning model uh look at if the performance is changing over time you know data set drift is one concept that relates to that that over time the the types of the let's say the statistics of the data you're seeing could change and so you know those are things that we'd want to be monitoring periodically to make sure that the analysis we did as part of the certification um still holds up for the current data. Okay. All right.

So, I'll say again, thank you know, thank you so much everybody. I I appreciate the the questions coming in. Um and if you have any further questions or you know you'd want to talk some more about what possible analysis services or certification services, please reach out to Exoda.

We'll make sure that that that you and I get connected and uh we'd love to talk with you further. Thank you.