Scaling Vehicle Simulations with MATLAB Parallel Computing
Adithya Vignesh Jayaraman, Rivian Automotive
Vehicle simulations play a critical part in enabling engineers to design a product to meet attribute targets at a much faster pace and lower cost than real-world testing. While simulations can benefit several engineers, their use is often restricted to vehicle modeling experts that can navigate the complexities of configuring simulations on commercial software. The accessibility to vehicle simulations is further challenged by the need for computing power beyond the capability of their work laptops. The Vehicle Simulation Interface (VSI) is an in-house software platform based on MATLAB® designed to enable any engineer, irrespective of their simulation expertise, to configure and run vehicle simulations using a simplified GUI front-end with curated vehicle and load case inputs tailored to Rivian products. VSI leverages MATLAB object-oriented programming to create simulation objects that encapsulate simulation input data and can each call a different commercial software in the backend depending on the needs of the load case. Each simulation object is then distributed to a local or remote worker using Parallel Computing Toolbox™ and MATLAB Parallel Server™. This workflow allows users to run multi-software simulation jobs in parallel and scale computing resources with additional remote clusters. In this presentation, gain insight into how this platform powers quick decision making by allowing engineers to run simulations with ease and generate key metrics within minutes. You’ll also hear details on how various MathWorks products such as App Designer, Parallel Computing Toolbox, and MATLAB Parallel Server were instrumental in creating this platform.
Published: 22 May 2023
So hi, everyone. It's great to be here. I'm Adithya. I'm from Rivian. And I'm just going to extend on theme from the previous speakers.
And in this talk, we look inward. We look at what this means for the engineering user. So my talk is called Scaling Vehicle Simulations, and I'm going to talk about a toolchain that we built in-house to make full vehicle simulations accessible to all engineers within our company. So we'll get into it.
Before we get started, for those who don't know or may not know who we are, we're an electric vehicle manufacturer. We were founded in 2009, and we've got over 10,000 employees spread across the USA and Europe. And we have three products that are currently out there in the market, the Rivian R1T, which is an electric pickup truck, the RR1s, our electric SUV, and then the Electric Delivery Van that we provide for Amazon.
So we'll start with some basics on how simulations are useful for the engineers in the automotive industry. And then we'll go into a bit of detail on who the key stakeholders for these simulations are, what a simulation entails, and how we build this tool that kind of democratizes simulations for all users within the company. And then we'll also talk about specific aspects of the back end, which uses various MATLAB products to really make this a scalable platform to run large scale simulations.
Let's get into some basics. So physical testing was a simulation. So we're trying to build these amazing electric adventure vehicles, and we want these vehicles to do everything in the world. And you can imagine that there's a whole lot of things that we need to test before we get a vehicle out there. So for an electric vehicle, range is a key aspect, acceleration versus a 0 to 60 time, how's the handling, and so on.
And an obvious way to test this is by building a vehicle, getting it out there on the road, and testing it in real time. And as you can imagine, this can become really expensive and the coverage that you can really achieve with physical testing is only so much. And it really depends on the amount of time that you spend on it.
An alternative to run these tests is using a commercial simulation software where you have physics-based models, and you have a commercial software that can simulate these various testing scenarios. And this is a cheaper way to run these tests. And you're also able to run these tests at a much faster pace than real time. So to give you some context on the vehicle simulations at Rivian, our users, using this tool as you'll see in the upcoming slides, have run over 9,000 full vehicle simulations in the last six months. And that accounts to about 2 million simulation miles.
So with this understanding, let's get into who really cares about these simulations within the company. And we'll see these people being referenced later on in the upcoming slides. So to start with, we have the leaders. And the leaders include everyone from the chief engineers, to the VPs, to the product heads. So these are the people that are responsible for making key product-level decisions within the company.
And then we've got the attribute engineers. And this is an umbrella term we use internally for various engineers, including range and efficiency engineers, performance engineers, and so on. And each of these engineers owns a specific attribute or a characteristic of the vehicle.
And then we have the modeling engineer. These are engineers that use Simulink or various software to develop component-level models that make up the full vehicle model. And then we have the simulation engineer who assembles the various component-level models, builds a full vehicle model, and runs the simulation for the rest of the business.
So with this understanding, now that we know who the stakeholders are, we'll have a look at how these various stakeholders interact with each other as they work with simulations. So we've got the leaders as we saw earlier. These leaders provide targets to our engineers. So, for example, a leader could say I want the launch times or the 0 to 60 times for my new program to be under 5 seconds.
And then we've got the attribute engineer or the performance engineer, in this case, who says, OK, this means that I need to simulate these scenarios. So for the 0 to 60 time, it could be a launch on a flat road, or it could be a launch on a grid, and so on. And the simulation engineer gets input from the modeling engineers as well where they provide the various component-level models to them.
So the simulation engineer is then tasked with building the full vehicle model using all of these inputs. And the simulation engineer also goes ahead and runs the simulation on commercial software and feeds the results back to the engineer who then post-processes it, analyzes it, and passes it back to the leaders for decision making. So this is a very simplified representation of the overall simulation workflow.
Now with this understanding, we'll look at some key challenges involved in this process. So to start with, let's start with the simulation scenario. So you'll notice that as the scope expands, as we start bringing more programs into our company, there are a lot of things that we want to simulate. And there's only so much that the simulation engineers can handle. So the number of simulation requests that we get within the company can soon turn into a bottleneck that can really slow down the development process.
And then we also have the complexity associated with being able to run these simulations through commercial simulation software. And this is exactly where the simulation engineer comes in and says, OK, I'm the expert of the software. I'm going to run it for the rest of the business. But they are limited by the amount of time and resources that they do have.
And then we also have other challenges associated with the standardization of the input data. So where's the gold source of the data. Or how is this data being post-processed and reported to the rest of the business? So these are key questions that you will hear as you work with vehicle simulations within an automotive company.
So with this understanding, we'll talk about why it's important to make simulations available for various engineers within the company and how we are able to achieve it internally using our in-house toolchain. So we bring back our stakeholders. We've got the leaders as well as the attribute engineers on the top. And then we've got the simulation engineer and the modeling engineer. And you'll notice a new stakeholder there, and that's the toolset engineers.
So that's the team that I'm a part of. We are MATLAB tool developers within the company. So we work in collaboration with the simulation engineers as well as the modeling engineers to build a simple, easy to use tool. And that's built using MATLAB.
And what we provide to our end users is an abstraction layer where the users can now directly interact with our tool instead of directly interacting with the commercial simulation software. And this tool is tailored specifically for the use cases that we work with at Rivian. So in the upcoming slides, we'll take a deeper look at the tool and also go into a bit of details on the back end.
So this brings us to the Vehicle Simulation Interface. And this is what our tool is called, and it's abbreviated as the VSI. So before we look into the tool, we'll have a look at what a simulation actually means or what does a simulation entail?
A simulation in its simplest form is a combination of a vehicle which in itself is a combination of various components and a load case, which is all the external conditions that this vehicle is being put through. And within our tool as you'll see in the upcoming slides as well, we make vehicle selections available to the users through dropdowns. And this GUI here is a toolchain of its own. It has an object-oriented back end. And it's capable of automatically generating vehicle models using various component-level data.
And on the other hand, we have the load case selections. And in our tool, we make it possible as a predefined list of load cases that we make available for our end users. And these load cases have all the external conditions that goes into the simulation from the ambient conditions to what the driver model is, what the speed profile is, and so on.
And our users have the capability to change any of the parameters that go into the load case or the vehicle as needed. And they're also able to import their own vehicles or load cases. So it's really a modular platform designed to really empower our users to be able to run any simulation that they want to.
And just a quick note on how long it could take to run some of these simulations as we look at benchmarking later on as well, so a typical short simulation as we call it can take about three to four minutes to complete. And at about 5x real-time speed, this is about 15 to 20 actual minutes on the road. And a typical long simulation in a cycle that takes six to seven hours on the road could take about 60 minutes to run on our simulations.
So with this understanding, here's your first look at this tool. This is called the Vehicle Simulation Interface. And you'll notice that parts of this GUI were shown in the previous slide. So this GUI is built using the MATLAB app designer framework. And it's deployed to our end users as a MATLAB toolbox or as a standalone compiled app for users that do not have MATLAB installed.
On the top left, as you'll notice, we have the vehicle configurator that we looked at in the previous slide. And on the bottom left, we have the test run or the load case selection. And a key feature of our tool is that different load cases can have different software requirements in the back end.
So this means that I could run two simulations back to back that end up calling different simulation software in the back end. And as you can see, the tool is designed in such a way that the end user is agnostic to the simulation software. So this really helps us with abstracting out the complexity for the end user.
And with a couple of quick clicks in our tool, you're able to configure simulations and are able to add them to the staging area. And we've got this table on the right side, which basically every row in this table is a unique vehicle and load case combination. And you'll notice gray rows and black rows. Black rows are sections, and gray rows are actual simulations.
And you'll also see here that each row has a status color associated with it, so red indicating that a simulation is yet to be run. Green indicates that simulation has already been done. It could be done by some other users. And you're able to reuse those results using this tool. And amber indicates that the simulation is done, but some of your input data may be out of date. So you have to re-run the simulation to get the most up-to-date results.
So with this understanding, let's get into a bit of detail on what happens when you hit the Run button after you configured your simulations in the GUI. And this is where it gets a bit technical and it gets a bit geeky from a MATLAB point of view. So in our simulation staging area, you're able to create a study that you want to simulate and are able to choose simulations that you want to run in a particular session. In this case, I have highlighted five simulations out of 12. And you'll notice that the active simulation shows a purple status, which means that this is the simulation that's currently ongoing.
So when the user hits the Run button, the first step that happens is we auto generate the vehicle model using our vehicle database tool. And we generate the model in a format that's compatible with our simulation software. And in our workflow, we use a combination of Simulink and an external commercial software. And we use the base workspace for all our model variables.
So once we hand off the simulation to the commercial software and the simulation is done, the log data from the simulation is sent to the MATLAB workspace. And we have scripts that process this data and converts it into a table, which is then saved in the parquet format. And the parquet is an open source file format, which really allows non-MATLAB users to make use of the simulation data as well.
And, finally, after we generate the data, we save it in a shared directory so other users can make use of it as well. And then we have various in-house MATLAB data processing toolboxes which this data is processed through. So, for example, going back to the launch example, we have tool boxes that are dedicated to processing the launch times and extracting the 0 to 60 times out of simulation or the 60 to 100 times and so on.
And we've also got options in our tool to report the simulation status against the targets that are set in our requirement management system. And, finally, we've got various workflows which allow our users to post-process the results and generate outputs in a standard format that can be shared with the rest of the business.
So with this understanding, there's a whole lot of things happening here. And we'll try to understand a little bit about how all of this is handled in the back end using MATLAB's object-oriented programming framework. So going back to what defines a simulation, we have the vehicle inputs, which is a combination of various component-level inputs. And then we have the load case inputs as well. And these are selectable using our VSI tool.
So once we have the inputs, in order to run the simulation, we need to run a bunch of processes using these inputs. We need to create the model as we saw earlier. We need to hand off the simulation, run the simulation, post-process it, and then finally archive the results so that other users can make use of it as well.
So, conceptually, you can think of these as properties and methods in a MATLAB class definition where the inputs turn into properties. And the execution processes, they are treated as functions or methods of a MATLAB class definition. And for the purpose of this presentation, we'll call it the VSISim superclass.
So this superclass forms the base framework for all the simulations that are being run using our tool. These objects work as containers for each one of the simulations that goes through our tool. And what we're able to do with this framework is now we are able to extend the superclass, and we're able to build various subclasses out of it. And we're able to customize this subclass for software-specific requirements.
You'll see here that software 1 inherits all the properties and methods from the superclass, while also adding a couple of properties that could be software specific. And with this concept, you're able to build any number of subclasses that are specific to certain software. And you're really only limited by does this software have a MATLAB API that you can use or can it be called using a system command? As long as you can do that, it can be fit into the framework of this tool.
So with this understanding, we look at a different aspect of the tool, which is really aimed at optimization. How can we enable simulation reuse so our users across the company don't end up running the same simulations for different parameters? So in this use case, we look at any user A, B, and C.
So user A goes ahead, opens a tool, and creates a study. And user A see that out of the 12 simulations that they wanted to run, five of them have already been simulated indicated by the green status. And some of them have not been run, and some of the results are out of date.
So user A says, OK, I just need to run the seven out of the 12 simulations. I don't need to run the other five since I can reuse the results and goes ahead and runs them. Now user B comes in and says, I have a different study to run. And keep in mind that user A and user B need not know each other. They could be in totally independent teams. And user B comes in and says, OK, here's the simulations I need for my work today. And I see that, OK, four out of six have already been done, so I just run the two.
And very often, we encounter situations like user C where they come in and say, OK, here's a simulation I want to run, and we already find that it's been run by some other user. So this really shows you how you're able to save a lot of time in running simulations that could be run by other users as well. And our tool enables us to share these simulation results between different users.
So with this understanding, now we can see that this tool has several different components. And it's really aimed at standardizing the vehicle simulation process throughout the company. Now how can we do this at scale? How can we enable our users to run a large number of simulations at once? What's the way to maximize the throughput? And that's what we'll be seeing in the upcoming slides. And this is where the various MATLAB Parallel Computing products come in as well.
So we'll talk about two kinds of parallel simulations. We'll talk about parallel local and parallel remote. So parallel local allows you to run parallel simulations on your local machine. And it uses the MATLAB Parallel Computing Toolbox. And it ends up calling the parapool and parfeval comments on the back end. And then we've also got the parallel remote approach, which now allows you to offload your simulations to a cluster, which has the MATLAB Parallel Server and the MATLAB Job Scheduler products installed in there.
So we'll have a look at what this workflow could look like at a high level. So the user goes in and creates a study as we've seen earlier. And to enable parallel simulations, they go ahead and click the Parallel checkbox, which brings up a dialog box asking them to choose between do I want to run it locally or do I want to run it on a cluster?
So in this case, we look at the local simulations. And when the user selects local, the GUI is going to prompt them to enter the number of parallel codes that they want to allocate for the simulation. And in this case, as you can imagine, the user is limited by the amount of codes that they have on their work machines.
So once the user creates or selects the number of codes they want to allocate, we go ahead and create a parallel pool using the parapool command. And then we go ahead with creating a simulation object for each row in our simulation table. So once we have the simulation objects, these objects really encapsulate all the information required to run the simulation. And now we're all ready to call the execution methods on the simulations.
So once a simulation objects are created, what we do is we call a rapid function, which ends up calling various methods of our simulation objects. And this rapid method really enables us to go through the sequences of running a simulation from creating the model, to running the simulation, to post-processing, and saving the results. And when we use the parfeval framework in MATLAB, you get the handle to what's known as a parallel future object. And this allows you to monitor the status of your simulation in real time.
And you're really able to see what the [INAUDIBLE] looks like. You're able to promote and demote jobs. And you're also able to stop a simulation If you find it running for much longer than you would anticipate.
So once these simulations are complete on these individual workers, our parallel future object is going to tell us that this simulation is done. And this is indicated by the state changing from running to finished. And once we know that a particular simulation is done, we are now able to extract the output from the simulation, which is, again, presented as a simulation object.
So this is very similar to the simulation object which is passed in as an input. And the output simulation object has a bunch of additional properties populated in addition to the input. So we have pointers to where the results are saved, what the key metrics are, where the plots are saved, and so on.
And then the rest of the drill remains the same where we post-process this data through our in-house toolboxes. And we save our results in a shared directory. And we save our results along with all the properties of our simulation objects, which means that we have all the metadata that was used to run the simulation co-located along with our simulation results.
And that brings us to the end of this parallel local simulation workflow. So this really shows you how we are able to take a single workflow, and we're able to run it now simultaneously, which allows us to scale this up. But we are still limited by the amount of physical codes that we have on our machine.
So how can we extend this concept and run parallel simulations on a cluster? And you'll notice that this slide looks largely similar to the previous slide. So we'll just highlight the differences between the two workflows on this slide. And we'll highlight them in yellow over here.
So once a user selects the cluster option, we're going to call the parcluster command part of MATLAB Parallel Computing Framework. And we're going to point it to our cluster profile. So we use AWS EC2 instances to run our simulations. And we point our parcluster command to this particular cluster profile.
Once we've connected to the cluster, we are able to create the simulation objects. And we're able to create a simulation job from the simulations configured by the simulation. So think of a job as a collection of simulations. And think of a task as individual simulations in this job.
So a user can submit any number of simulations just limited by the amount of computing resources we have. And they're able to submit this as a single job, which has a bunch of subtasks where each task is a unique simulation. And we're able to do this using the createJob and the createTask methods.
So you'll notice that the other key difference here is that obviously we don't run this on our local machine. We are able to offload the simulation onto our cluster. And this is where the MATLAB Job Scheduler comes in. The AWS EC2 instances that we use for our workflow have the MATLAB Job Scheduler and MATLAB Parallel Server installed on them, which means that all the processes associated with queuing the job, with submitting the job, with extracting the results, all of this is taken care of in MATLAB. And this is where the MATLAB in parallel computing framework has been really instrumental for us in creating this workflow.
And it shows us how we are now able to extend the same concept of being able to run parallel simulations and now run it at scale. If you reach a point where the computing resources are not enough and you want to run more number of simulations in parallel, you're able to do that just by extending your computing resources. And you don't really have to make any changes in your tool in order to scale it and make it run larger number of simulations.
So with this understanding, let's look at some benchmarking numbers. So we looked at various ways in which these simulations could be run. You could run them sequentially. You could run them parallelly on a local machine or parallelly on a cluster. And in this case, we're looking at the time it takes to run these simulations for different amounts of load.
So a common theme that we see here is for shorter simulations or for the short simulations as we saw earlier, it really doesn't make a difference obviously whether you run it sequentially or you run it on a parallel cluster. And in a lot of cases for short simulations, offloading a simulations to a cluster can be slower than running them sequentially since there's an overhead associated with creating jobs and tasks and retrieving the results and so on.
And we noticed that as the number of simulations increase and as the length of each of these simulations increases, we can see that the cluster option really starts to shine. And so depending on the use case that you're working with using our tool, you're able to choose between each of these options that's most optimal for your use case.
And here's also a quick slide on the amount of time it takes to actually run a simulation versus the amount of time it takes to configure a simulation, deploy it to a cluster, and so on. And we're seeing the same story here.
On the top left, we have a single short simulation that's being run using the sequential method without any of the parallel products involved. And you'll see that this is the fastest method for a single short simulation. And as we really go diagonally further down to the bottom right, you'll notice that for longer simulations and for more number of simulations, the cluster approach starts making more sense since the overhead time associated with creating jobs and tasks is now minimal when compared to the amount of time it takes to actually run these simulations.
So that brings us almost to the end. So here's a list of all the MATLAB or MathWorks products, I should say, that are used in our workflow. So closing point, so what have we been able to do with this toolchain? We've really been able to achieve the simulations for every one goal where we're able to really empower our engineers to run simulations by themselves and not have to rely on the simulation experts in order to run simulations. We're able to eliminate the simulation bottleneck that simulation engineers could face.
And a key aspect is the simulation reuse, which has really helped in eliminating a lot of redundancies across the company in terms of the simulations that we run. And, finally, as we saw with the parallel simulations, we are able to maximize the throughput. And we're able to really scale and now support a large number of users and run a large number of simulations simultaneously.
So in terms of what next for [INAUDIBLE] platform, we are actively working with MathWorks and seeing how we can collaborate with them to improve our workflows and our tools. So key features that we want to implement are we want to have the capability to create a DOE directly from the GUI. You are able to do it today to a certain extent. You're able to create sensitivity studies. You're able to just vary parameters without having to do it manually. But we want to have a dedicated framework aimed at running experiments at scale.
And we're also exploring how we can deploy our tool as a web app. Currently, we have it as a toolbox as well as a standalone desktop app. Now can we deploy it on a browser which can really make it more accessible to our end users? And we're also looking at ways in which we can enhance our computing resources from a cluster point of view.
So with this understanding, I also want to just drop a quick note on the MATLAB ecosystem at Rivian. So we've got the Minimart, which is an in-house MATLAB store, which houses all the MATLAB-based tools that we build. And there's a lot of them as you can see. And we're really heavily invested in the MATLAB ecosystem. And we found these toolboxes to have a great adoption within the company.
And, finally, I'd like to thank my teammates obviously. There's not a single person effort. There's a whole lot of work going on in the back end.
I'd like to thank my manager Oli. I've got Avi Chekuri who's a simulation engineer sitting right here. So thanks to him. He's the brains behind all the simulation work in this tool. And Liam Barry who was really instrumental in ideating a lot of concepts in this project. That brings me to the end. Thank you so much for your attention.
[APPLAUSE]