On-Device Training of Machine Learning Models with MATLAB and Simulink

Hello. My name is Brad Duncan from MathWorks. I'm here to talk about an interesting new topic in artificial intelligence, or AI, in devices that we may all soon be using or wearing every day. The title of this webinar is "On-Device Training of Machine Learning Models with MATLAB and Simulink."

To let more about myself, I'm a product manager in the area of machine learning using the MATLAB programming language. Machine learning is a method used in AI in which models are trained using data.

Hello. My name is Jack Ferrari. I'll be joining Brad today during today's webinar. I'm a product manager at MathWorks, working on AI for embedded devices. I've been at MathWorks for four years and work closely with our customers and developers to ensure our tools for AI deployment stay aligned with the evolving demands of edge AI.

One of those demands we've been hearing from our users is on-device training. And we're here today and excited to give you a look at how this can be implemented using MATLAB and Simulink.

Let's start with a representative example of on-device learning. This example uses machine learning to diagnose faults on an air compressor. This type of air compressor may be found in a factory setting. What we want to do is design a system that can alert an operator that some kind of fault has occurred with this air compressor. And in order to do so, we need to be able to learn new faults that may occur that were not planned ahead of time or recorded ahead of time, but may be occurring right there in the factory. And the operator wants to be able to detect those types of faults in the future.

Well, why is this? Well, every machine is unique. It may experience unique faults. It may sound different if we're using a microphone to detect the fault. And also, once it's installed in the factory, there's the factory environment to think about. So what we're looking at here is a machine learning solution to detect faults and be able to adapt to its environment.

So this is part of machine learning. And what I'd like to do first is review the basics of machine learning in MATLAB. So this is the typical machine learning workflow that you would use in MATLAB. It has these basic four steps.

The first step is usually data preparation. In this case, data is coming in from a microphone. And we'd like to use that data for something. But you can't use it as raw data. You need to do some processing to prepare it for AI modeling.

The next step in the workflow is AI modeling. There are multiple AI models available. And machine learning is used to learn the behavior of-- learn the behavior from data, where basically the data that came from the first step can be used to predict some kind of state or some kind of response. In this case, we'd like to predict a different fault state of the air compressor.

Once I have the data prepared and the model built, then I will basically simulate this kind of system that I'm trying to design. I'll place the model into a system and run the system and test that it's working. That step is called simulation and test. After that step, it can be deployed to the device. And now that device will be able to make predictions using the machine learning model.

In the first step, you determine what type of data and processing will produce the best inputs to classify the fault sounds accurately. The available data is labeled with faults identified for different segments of time. Since our air compressor example uses audio recordings, the Signal Labeler app shown here works well. MATLAB has similar labeling apps for other domains, like images, videos, lidar, point clouds, and more. And you can use that to create the data that labels the signal that you're trying to characterize.

In the next step, we also have an app in MATLAB that can be used to train different machine learning models. The Classification Learner app is shown here and allows you to test many different models and find the best model that has good accuracy and to tune that model for your application. So this app shown here gives quick access to train and test many available models for classification problems. We also have an app for regression modeling.

So you've preprocessed some data, prepared it for training, and trained a model. You're happy with how your model performs on the data, but now it's time to integrate that within the context of a larger system and ensure it's still behaving appropriately. And so Simulink, one of our platforms, lends itself well to this task. But it's worth noting as well that this can be accomplished in MATLAB.

On the Simulink side, we can easily integrate machine learning models into our larger systems that we're modeling with a variety of native blocks that we've provided support for and have updated over recent releases. These blocks include blocks for traditional machine learning models as part of the Statistics and Machine Learning Toolbox, deep neural networks from the Deep Learning Toolbox, as well as some application specific blocks for different vertical engineering domains, like Audio Toolbox, System Identification Toolbox, and Computer Vision Toolbox.

In Simulink, the system can be tested in realistic scenarios to ensure that the machine learning model you've trained will work well in different cases where it may be accepting different inputs. In a moment, I'll demonstrate how Simulink blocks are used to model the air compressor fault detection system that we've been talking about as part of this webinar.

Once we've simulated our model and are happy with its behavior within different scenarios, now it's time to think about how we can deploy it to our target device or edge device. We have tools for deploying machine learning models to other domains as well, such as cloud systems. But in the context of this webinar with on-device training, we'll talk about our tools for edge deployment.

And so we can generate low level code, such as C, C++, CUDA, or VHDL for FPGAs, that translates our designs from MATLAB and Simulink, including AI models, like machine learning models, into low level code that can run on these edge devices with automatic code generation.

There are various products we have for this, such as MATLAB Coder, Simulink Coder, GPU Coder, and Deep Learning HDL Toolbox. Today we support AI model deployment to CPUs, microcontrollers, Nvidia GPUs, and FPGAs.

Today's webinar is about on-device learning, which follows a very similar workflow to what was just shown, but is expanded to allow the AI model on the device to continue learning after the deployment step. Recall that previously all of the modeling was happening in the second step, but what happens when you want that AI model to continue to learn after the deployment step? Well, that's called on-device learning. Why is on-device learning special? Why do we need the AI model on the device to be one that can keep learning after the device is deployed?

Sure. So there's a few reasons why on-device learning is special. The first is that edge devices typically have a limited amount of resources, especially memory that can be used for tasks in supporting different applications on the device. Training or refitting machine learning models is typically resource intensive and memory intensive. And so on-device learning is specifically designed to be as hardware efficient as possible and minimize memory usage so that it can be realistically run on these small, resource-constrained devices.

Another reason that on-device learning is special is that it enables real-time inference of new inputs to a machine learning model. With traditional retraining or machine learning ops methods, typically, the machine learning model would be retrained in a cloud environment before it's redeployed to an edge device. With on-device learning, we can ensure real-time inference continues without a separate step with retraining and redownloading the model.

Finally-- or sorry-- thirdly, on-device learning is special because we can adapt our model to respond to and make inferences for more than just the number of classes or types of inputs that we trained it on initially. So we can grow to learn about new types of information.

And finally, on-device learning is special because it enables remote operation. There are several interesting use cases for edge AI that are now enabled with the ability for on-device learning for typical-- or for devices that are not-- do not have the luxury of a constant internet connection or deployed to remote locations. So with on-device learning, they can continue to be updated without a connection to the cloud.

Excellent. So the AI modeling workflow that we're talking about is very similar to what I showed earlier. If you can go ahead and click, we can see that it's very similar, but it adds another step where the device will keep learning. For on-device learning, we choose AI models that can continue to learn on the device. These are called incremental learning models. And they're a special type of machine learning models that train iteratively on new incoming data, instead of training all at once on all of the data. After the deployment, then the model can adapt and learn simply by providing it with new data.

So new AI blocks in Simulink enable incremental learning models. As Jack will show, we're using Simulink to model our workflow. Simulink provides blocks for incremental learning, for fitting the model, for updating the metrics that define the accuracy of the model, and for prediction. Simulink blocks handle all the steps for incremental learning and allow the model to predict using real-time data and to update the model using labeled data.

There are four basic steps to incremental learning. One is updating the metrics in terms of determining how accurate the model is. Another step is that it can detect changes and decide whether or not to fit, depending on the use case. The fit step, where the model is actually updated, and the predict step, where the newly updated model is now used to create predictions.

After the model is deployed on the device, the model continues to learn with new data. So in the first block that we're showing here, the model will change with new data as it comes in. In the predict block, the model will continue to predict with real-time data as it gets updates from the fit block. And also, up in the metrics block, the metrics will be calculated so that you can monitor the status of your machine learning model. And depending on the type of operation you're trying to do, you can use these metrics and evaluate whether to update the model.

So Brad just took us through an initial look at this kind of workflow that can be modeled using Simulink. Next, we're going to show how on-device learning works using our use case with the air compressor demo. And we'll hop over to Simulink, where we can step you through our model and show you how incremental learning is implemented.

As a reminder, our example is a use case of diagnosing faults on an air compressor. We'll use audio files or audio data collected in the field for different types of states that the air compressor can be in, one being healthy, the other is being different kinds of fault states. Each machine is unique. And different fault states could occur that maybe the initial designer of the machine learning model didn't account for. So our incremental model will adapt to new sounds and be able to classify them as faults instead of incorrectly classifying them as some other class that was initially defined.

So let's hop to Simulink, and we can take a look at this demo. So now we're in Simulink. As you can see, our Simulink model here is made up of four primary subsystems, the first being where we take in audio inputs. As mentioned, we'll know some of these classes up front that the initial machine learning model will use is trained to identify. These are healthy, LOV, LIV, and piston.

Additionally, we'll have the option of feeding through a simulation with different-- with three other classes that we never train the machine learning model on-- riderbelt, flywheel, and bearing. And we'll show you that we can, with incremental learning, begin identifying these three unknown classes as a new type of fault. This subsystem merely feeds in the audio files as inputs.

Next, we need to use some preprocessing. As we mentioned earlier, typically, we preprocess input signals so that we can extract features of interest that will allow us to identify each input or differentiate each input from one another. These are fed as inputs to the machine learning models. In this case, we're using a series of MATLAB function blocks. Oops-- excuse me. We're using a series of MATLAB function blocks to handle this preprocessing stage.

Once we've extracted the features of interest, we can feed those as inputs to a machine learning model. And so we have a classification stage next. In this case, we're using two machine learning models. The first is trained to classify one of the four known classes-- the first being the healthy state, and the latter three being failure states, LIV, LOV, and piston faults.

On the bottom part of this diagram, you see something called Learn Unknown Fault. That's where we've packaged an incremental learning model to learn the difference between a newly trained fault and some other sound.

So taking a step further into this subsystem, we can see a triggered subsystem. This subsystem, as its name implies, will only execute when it's triggered. In this case, we're using a switch back at the top level of the model to simply switch on this subsystem and begin the process of incremental learning.

And we only do that when the operator is detecting that there's a new fault that they don't already know. So basically, this is a manual trigger that we're going to use in this use case.

So we can take one step further and look at this incremental fitting system. And this might look similar to the slide that Brad showed previously. Do you want to speak to this?

Yeah, sure. So the audio input data from the new sound is coming in from the left. And it makes its way into both a fit block and an update metrics block. The fit block is able to update the machine learning model. And update metrics produces data that can be monitored to look at the fitness of that model.

In this case, we're not feeding metrics back into our decision making process about when to fit. We're simply going to monitor it. But depending on your use case, you could decide to use that as some condition that must perhaps when, for example, the error metric exceeds a certain threshold.

Maybe that meets a condition that triggers a retraining or refitting of a machine learning model. In this case, though, we're just going to use that simple switch on and off to turn on training. But it's good to know that this is available to you if you need to use it for your use case.

And, of course, the new models that are fit from the fit block are passed out to the predict block. And as mentioned, this block is allowing us to identify whether this is some new sound we haven't heard, or if it's part of the initial set of classes that we trained the first model on.

So heading all the way back up to the top of the model, I think we're now ready to start the demo. And we can see how incremental learning works in practice here. So I'll start this simulation, and we'll get started.

So on the left-hand side, you'll see this knob. I can turn this around and essentially choose different input classes or sounds of the air compressor to propagate through the model. And on the right-hand side, we'll see the predicted class as determined by that pair of machine learning models.

Healthy, LOV, LIV, and piston are the four classes that the model has already been trained on. And we can see that passing those inputs through the model, we can see that it's correctly identifying each sound-- each sound's class. So we can correctly identify the LOV fault, LIV fault, and piston faults.

But as you recall, right about flywheel and bearing, we didn't train the model on those sounds. What happens if, for some reason, the air compressor starts-- slips into one of those fault states and the machine learning model no longer recognizes the sound? Well, let's play one and see. Whoops.

So if we play the flywheel sound, for example, we'll see that the predicted class is piston fault. It is a fault, but it's not the correct fault. We can't blame the machine learning model. We didn't train it on this sound, so it doesn't what it is. It's falling back to maybe the next closest sound to flywheel, which happens to be piston.

But what if we want to identify that as a new type of fault? Well, that's where incremental learning comes in here. So again, you could have a different scheme for kicking this off. But in this case, we're going to use a simple switch. We'll switch on our magic incremental learning switch, which is going to trigger that subsystem. And we'll see in a moment here the predicted class will update itself to now be new fault.

And we're going to use new fault to represent some new, unknown fault that the operator has detected. In practice, the operator can then inform somebody about this new fault. And the machine learning models could be updated. But for now, it's going to have this one kind of placeholder that's going to register as new fault when the trained fault is detected.

That's right. So we'll let it train here for about a moment longer. And we can turn off the training. And we can show that the original model maintained its memory about those initial four classes. If we head back and play a healthy sound, we can still classify a healthy state. LOV, it can recall. LIV, piston, and back at flywheel, we can classify that as a new type of fault.

And just to show one more example here, perhaps we hear a second new kind of fault. Perhaps we didn't hear flywheel. Maybe we heard bearing instead. We can retrain this model, and identify this one as a new fault as well.

So we'll just flick our switch on-- initiate the incremental learning. We'll see the predicted class update to new fault there. And we'll let this begin to learn. And we can disable incremental learning, and once again identify that the initial sounds can be recalled as well-- healthy, LOV, et cetera.

So that, I think, essentially concludes our demo. This is running on my PC on my laptop in Simulink. And as the title of this webinar is on-device training or learning, I also want to show a brief bit of code generation to show how this design can now be deployed onto an edge device and run out in-- the field as well.

So if I just stop the simulation and I go to the Apps tab and open Embedded Coder, and this will allow me to generate C code from this Simulink model. So I can hit Generate Code here. Allow this to run. It may take a second.

So code generation is complete. And we can see the generated code here opens up on the right-hand side with various source files, header files necessary to run this application on the embedded device. In this case, I selected an ARM Cortex compatible hardware device. And so the generated code is optimized for that target.

And if I go to one of the subsystems here, let's take, for example, the incremental learning, or the fitting subsystem here that's triggered. I can see the corresponding generated code show up on the right-hand side.

Very cool.

So automatic code generation allows us to extend our incremental learning system, which we've just tested in Simulink, over to operating in the field on an embedded device.

Well, thanks, Jack, for that great demo. This showed incremental learning operation in Simulink blocks and demonstrated the different concepts of incremental learning that we talked about earlier. Next, let me go through some takeaways from our webinar.

So the goal of this was to bring AI models to the edge that can learn and adapt. Some of the key takeaways are that on-device learning enables new capabilities for AI on devices. Machine learning models that can adapt to unique environments where they are deployed-- they can learn new, unseen data, enabling new possibilities.

You can employ the same machine learning workflow that we've used in traditional machine learning. And it's easy to add and simulate incremental learning models into that workflow. One key thing to think about with this type of design is that labeled data must be provided periodically or on demand in order to update the training of a model.

Now, we'd like to take your questions. If you haven't already, please enter your questions in the chat window. And we will try to address them in the next few minutes.