Embedded AI Integration with MATLAB and Simulink

Hello, everyone. And welcome to today's webinar on incorporating AI networks into existing models using MATLAB and Simulink. My name is Reed. And I'm a partner manager at MathWorks, the leading developer of mathematical computing software. Since my master's in robotics and AI, I've been fascinated with incorporating AI into complex controls problems. Today, I hope I can help you all start incorporating AI into your existing Simulink models. We have a packed agenda today focusing on how you can integrate AI into your system models effectively, and deploy your AI and MCUs.

The agenda for this webinar is an introduction, third-party network import, system level verification and validation of neural networks, cross-platform code generation, and closing statements. By the end of this webinar, you'll have a clear understanding of importing third-party networks into Simulink, performing verification and validation of imported neural networks, cross platform C+ code generation from neural networks for deployment.

Embedded AI is AI brought to the absolute edge of the network. These are models deployed to memory and power-constrained hardware like MCUs. The benefits of this method of deployment is that network latency and connectivity are no longer potential issues. This is great for any safety-critical system. These are usually tiny neural networks with orders of magnitude less parameters than deployed networks in non-resource scarce environments. Use cases for these networks may be virtual sensors, predictive maintenance, fault detection or classification. And even controllers in some instances.

This is the type of network that we will be employing today. Over the past couple of years, AI has really been highlighted as a way to provide solutions to complex issues. While most of the news is primarily around large models, smaller models are also being explored and adopted. No matter how one chooses to design and build their neural network, we want to facilitate the inclusion of these networks into existing MATLAB and Simulink models.

By providing the tools to design and build in native MATLAB neural network, or using one of our third party framework importers, you can choose the right network for your solution. Once you have a network, you can have access to all of our simulation and testing technologies that you can use to make sure the network behaves in your existing system as intended. With code generation via Embedded Coder, you can then generate C+ code to deploy on various MCUs. Today, we'll explore importing a PyTorch network, testing this in a larger Simulink model, and deploying this to multiple MCUs.

Taking AI from a concept and bringing it into production is a significant step in the maturation of an AI-enabled product. There are unexpected challenges to the real world that your AI may not be able to adapt to without proper steps taken. These steps can be taken at multiple points in the development, but usually fall within two levels, design or system. At the design level, we have many tools to make labor-intensive steps like data labeling much more efficient, such as a signal label or app.

There is also the Deep Learning Verification Library, which helps verify properties of your deep neural network, such as robustness to adversarial examples. And estimate how sensitive your network predictions are to input perturbations. In this webinar, we are focusing more on the system-level capabilities for integrating our neural network. Capabilities such as requirements tracking with linked test harnesses to ensure your network performs as desired within your existing system. And the ability to translate your neural network directly to plain C+ code for deployment on your embedded device of choice.

For those who may be newer to model-based design, this is a methodology that allows you to design, simulate, and verify systems before deploying them. As engineers look to add AI capabilities to their products, model-based design offers a scalable way to integrate, test, and deploy these features. A significant part of this is the verification and validation of your AI within the context of a larger system. In large systems, requirement management and establishing traceability between artifacts can both be difficult.

MathWorks already offers several tools that can help with this via Requirements Toolbox and Simulink Test. By using these tools, you can effectively integrate your AI with an existing complex system that builds confidence in your deployed AI. The demo videos you will see today come from a virtual sensor position observer for a permanent magnet synchronous motor. An FOC algorithm requires real-time rotor position feedback to implement speed control, as well as to perform mathematical transformation on the reference stator voltages and feedback currents.

Traditionally, such algorithms relied on physical sensors to detect mechanical rotor position. However, with increased accuracy and cost effectiveness, sensorless position estimation solutions can act as a better alternative to physical sensors. This has benefits, such as increasing cost effectiveness while providing comparable accuracy to physical sensors. The virtual sensor can also be used as an indicator for predictive maintenance.

Now, let's move on to our first major topic, importing third-party neural networks into MATLAB and Simulink. This section is crucial for those of you looking to integrate neural networks developed in different frameworks into your Simulink models. Interoperability is vital in AI workflows, especially when you're integrating AI into larger systems. Different teams might use different tools and frameworks. And being able to use these effectively is a key to efficient development.

With the ability to import networks from PyTorch, TensorFlow, and ONNX, you can integrate models developed by the AI community or your own internal teams. Frameworks no longer bind you to a particular language. This interoperability allows you to leverage the strengths of each framework while maintaining a cohesive workflow. Once a neural network is imported into MATLAB, it has the benefits of being a native MATLAB network.

If your goal is to take a network and retrain it with your own data, this becomes easy with Experiment Manager. If you were looking for explainability of predictions and verifying model robustness, you can now use capabilities from the Deep Learning Toolbox to do exactly this on your imported network. If your network was not already designed for deployment to a resource-constrained device like an MCU, our compression methods can be applied to your network.

For those seeking to replace a traditional control method with a neural network, you can easily implement this using a Predict Block in Simulink that points to your MATLAB network. Or now export your network to MATLAB into a Simulink as layer blocks in R 2024b. Finally, you can employ our code generation tools like Embedded Coder for direct translation of your neural network into C+ code. While this webinar does not cover how to design a deep neural network to fit on embedded hardware, there are many resources provided by MathWorks to help those interested in this.

MathWorks authors several tools to help those designing their network for resource-constrained devices. There are two methods of compression, structural and datatype conversion. Structural compression or compression methods that modify the structure of the network, such as pruning, or our proprietary method, projection. Datatype compression changes the weights within the network to lower precision datatypes, and can be done with quantization and conversion to the bfloat16 datatype.

I just finished speaking about compression, but a good practice is to use a combination of compression methods to achieve the resource and prediction accuracy requirements of a neural network. Starting in 2024b, it is now possible to simulate and deploy quantized network in a Simulink model. Previously, this was a capability reserved for MATLAB, meaning that projection was the recommended way to compress neural networks for use in simulation codegen. However, now you can combine this with quantization to achieve the best compression and prediction accuracy possible in your Simulink models.

The function exportNetworkToSimulink takes a network object and returns a Simulink model that contains the blocks that correspond to the layers in the input network. Let's zoom in on how you would import a PyTorch network into Simulink. We recommend using the importNetworkFromPyTorch function to import PyTorch models. Once you've imported your AI network, the next step is to ensure that it is error free. MATLAB provides the analyzeNetwork function, which will visualize the model. This step is helpful in validating that the network has been imported properly.

Sometimes you might encounter unsupported layers during the import process, but MathWorks offers several solutions or workaround solutions for these cases. You can create equivalent layers that are supported, or write custom layers in MATLAB to handle specific functionalities. Finally, for importing your network into Simulink, rather than using a predict block which points to the MATLAB deep learning network object, you could use the new function, exportNetworkToSimulink, which creates a new Simulink model with the aforementioned layer blocks. This network would then look something like this in Simulink.

In summary, you can fully import, inspect, and export a neural network in only three lines of code. Now that we have covered the import process, let's move on to our next topic. AI verification and validation, or VNV in Simulink. Verification and validation are essential steps in the AI model lifecycle. These steps ensure that your AI models perform as expected and meet regulatory standards. This is particularly important in industries where failures can have severe consequences.

Verification and validation of AI models is crucial in safety-critical industries to ensure the accuracy, reliability, and trustworthiness of AI enabled systems. The progress made in developing standards and regulatory frameworks for AI in these industries is a significant step towards ensuring the safe and effective use of AI in various applications. It is important to note that the traditional VNV workflows, such as the V-cycle, may not be sufficient in ensuring the accuracy and reliability of AI models.

In response to this adaptations of these workflows emerge to better suit AI applications, such as the W-shaped process shown here. It's important to recognize that the W-shaped process can coexist with the V-cycle, which is frequently used for development assurance of non AI components. Additionally, although it may look like a linear workflow, it is iterative in nature. For us in this webinar, we are focusing on the last part of the process, system requirements verification.

Requirements Toolbox lets you author, link, and validate requirements within MATLAB Simulink. You can create requirements using rich text with custom attributes, or import them using requirement management tools. You can link requirements to MATLAB code, System Composer, Simulink models, and tests. The Toolbox analyzes the traceability to identify gaps in implementation or testing. The design, highlighting, and traceability matrix summarizes where links exist across multiple artifacts and guide you to address any gaps.

When requirements change, linked artifacts are highlighted. And you can determine the upstream and downstream artifacts affected using a traceability diagram. Generated code from Simulink designs include code comments that document where requirements are implemented to assist with reviews. When used with Simulink, you can create links to blocks with a simple drag and drop.

Simulink Test provides tools for authoring, managing, and executing systemic simulation-based tests of models, generated code, and simulated or physical hardware. It includes simulation, baseline, and equivalence test templates that let you perform functional unit regression and back-to-back testing using software-in-the-loop, processor-in-the-loop, and real-time hardware-in-the-loop models. Utilizing Simulink Test, we can develop a suite of automated tests designed to rigorously verify the functionality and performance of the AI model across various scenarios.

This approach enables us to systematically validate all aspects of our work from the accuracy of the model's predictions to its integration within the larger system. In a large system, requirements can easily spiral and become difficult to manage. Therefore, a project needs a scalable solution to its requirement management. To centralize this part of the product development process, we created the Requirements Editor. This provides teams a way to create requirements, link each link each requirement to a test, and view each requirement verification status all in one screen.

This linkage is crucial for closing the loop in the development process. By running all implemented tests, we can verify that our requirements have been adequately implemented. By linking each requirement to specific functions and tests, one can establish clear traceability that enhances the transparency and accountability of development efforts. Furthermore, the ability to systematically verify each requirement through direct testing from the Requirements Editor underscores our comprehensive approach to requirements verification. This marks the culmination of the W-shaped process, affirming that our AI models meet the stringent criteria set forth for safety critical applications.

One process I want to highlight is out-of-distribution detection. OOD data detection is a process of identifying inputs to a deep neural network that might yield unreliable predictions. OOD data refers to data that is different from the data used to train the model. For example, data collected in a different way at a different time, under different conditions. Or for a different task than the data on which the model was originally trained.

By assigning confidence scores to the prediction of a network, you can classify data as in-distribution, ID, or out-of-distribution, OOD. You can then choose how to treat OOD data. For example, you can choose to reject the prediction of a neural network if it detects OOD data. If this is interesting to you, I highly recommend checking out and downloading the Deep Learning Toolbox Verification Library and some of our examples at mathworks.com.

Using our demo, we can look at what Requirements Editor looks like in a Simulink model. Here we have our PMSM FOC neural network position estimator demo. If we go into Current Control, we can see the subsystem where the neural network observer lives. To view the requirements, we can go to the App section and click Requirements Manager. By clicking the label, we can see the source of the requirement originates in the Simulink model. You can also see at the top ribbon where you generate your traceability matrix and diagram and the Requirements Editor.

Clicking the Requirements Editor opens these in the Requirements Editor. Here, we can see all the requirements for the model as they relate to the integrated neural network. If we click Inspector, we can see the description of the requirement. By clicking Test Harness, we can see our tests in the results. Since our requirement was speed error of close to 1% and position phase lag with a lag tolerance of 1% when driven by a neural network, we can see how it performed to our baseline. First, we'll check out position, and then speed. From this test, we can see that our neural network was able to perform similarly to the existing solution.

Welcome to the final section of our webinar. In this segment, we'll explore how to take your validated AI models and deploy them effectively to various MCUs. Deployment is a critical phase in the AI model life cycle. It ensures that your AI models can operate effectively in real-world environments, whether it's on embedded system, edge devices, or in the cloud. Proper development, proper deployment can significantly impact the performance, reliability, and scalability of your AI solutions.

The primary problem with manual translation is you must maintain two separate implementations that are not automatically in sync with each other. If requirements change during development, which almost certainly happens, it's quite inefficient to go back and modify the C+ or C++ code. And then test it all again to ensure its functionality is the same as the original. Coding errors are also usually introduced during the manual translation process, leading to longer testing times and no guarantee that all bugs introduced are caught.

Finally, the entire process is time consuming and costly, as it's expensive to not only maintain separate algorithms in two different languages, but also separate test benches in C+ and C++ that might not get implemented. With automatic translation, you spend much less time writing and debugging low level C+ and C++ code. And more time developing and tuning your algorithms. Once you're happy with your code, you can start generating MEX files automatically.

With this approach, you maintain a reference MATLAB or Simulink Project and get to C+ or C++ code much faster. Having a single reference makes it much easier to make changes as the requirements change. You can also get to test it more systematically and thoroughly to account for more real-world scenarios. Hardware integration is essential for deploying AI models to specific devices. Having the ability to deploy your neural network to various hardware is valuable since it provides flexibility and choice during the design process.

When given a neural network, being able to test its performance on different hardware is necessary due to resource requirements of the neural network and larger system. The ability to do this quickly through a variety of hardware creates a much more scalable process. To enable this, Simulink supports a wide range of hardware platforms from companies, including ARM, Texas Instruments, NVIDIA, and STMicroelectronics. Hardware support packages for Embedded Coder facilitate seamless integration, ensuring your models can run on the target hardware.

Performance optimization is critical for ensuring that your AI models run effectively on the target hardware. Translating a neural network into C+ can help optimize performance, especially in terms of execution speed and memory usage. This is particularly important when AI is used in real-time applications when running on resource-constrained devices. Embedded Coder generates efficient optimized C+ or C++ code for your MCUs. This code can be deployed to a variety of hardware platforms, ensuring that your AI model can run effectively in real-time applications.

You have the choice of generating plain, generic C+ or C++ code, or. C+ or C++ code that contains optimization libraries for specific targets. We have worked to create competitive, out-of-the-box performance with the plain C+ code we generate for neural networks, with large improvements made in the last few releases. Hardware support packages offer significant benefits for developers. Seamless integration between MATLAB and Simulink and hardware is crucial as it streamlines development.

By enabling direct algorithm implementation without extensive coding, engineers can focus on refining performance rather than dealing with integration issues. This reduces development time and errors, accelerating the transition from concept to deployment, and enhancing productivity. Rapid prototyping allows engineers to validate designs swiftly in real-world conditions. Deploying models on hardware for real-time testing ensures reliability and performance. Processor-in-the-loop testing helps identify issues early, reduce time to market, and increase system robustness.

An extensive range of supported devices offer engineers flexibility in choosing the right hardware for specific needs. MathWorks support allows optimization of performance by leveraging unique hardware features. This enables engineers to tailor solutions effectively, maximizing hardware potential, and meeting technical and business objectives. Here are some examples of hardware support packages available in Simulink.

For Texas Instruments, we have the C2000 block set. STMicroelectronics has Embedded Coder Support package for STM32s. For NXP, there is a model-based design toolbox authored by NXP. And for Infineon, there is Embedded Coder Support package for the AUREX TC4x with support for their GPU accelerator. And now we also have support for Qualcomm's HEXAGON with their DSP and NPU.

So let's see a practical demonstration of a deployed Embedded AI model with Simulink. Here, I have a train position observer deployed to a TI Delfino F28379D launch pad attached to a motor. In this video, we keep speed constant while changing the torque. And keep constant torque while changing the speed. This was deployed using the Texas Instruments C2000 block set.

The same neural network with no changes to the network itself, was also deployed to an STM32. This was made possible by the Embedded Coder Support package for STM32s. As you can see, translating your neural network directly to C+ for cross-platform code generation allows you to test your AI on a variety of hardware without deep knowledge of the hardware or embedded device, thanks to our hardware support packages.

To recap this webinar, you learned about the ability to import a neural network, test in a larger system, and deploy it to MCUs. The demo used in this webinar comes from the motor control block set, it is is now shipping in R2024b. The motor control block set for MathWorks is a tool designed to help engineers develop and simulate motor control algorithms. It provides reference examples and blocks for field-oriented control, sensorless algorithms, and motor parameter estimation. The block set supports code generation for deployment on various microcontrollers, facilitating the implementation of real-time motor control systems.

I want to thank you for taking the time to join me today for this webinar. I wish you all the best on your Embedded AI journey. And please check out our other online Embedded AI resources. Thank you.