Real-Time Testing – Deploying a Reinforcement Learning Agent for Field-Oriented Control - MATLAB
Video Player is loading.
Current Time 0:00
Duration 4:51
Loaded: 0.80%
Stream Type LIVE
Remaining Time 4:51
 
1x
  • Chapters
  • descriptions off, selected
  • captions off, selected
      Video length is 4:51

      Real-Time Testing – Deploying a Reinforcement Learning Agent for Field-Oriented Control

      Deploy a trained reinforcement learning policy to a Speedgoat system for real-time testing. Use the capabilities for implementing deep learning inference in Simulink® and plain C code generation for deep learning networks to deploy a trained reinforcement learning agent.

      In this demo, a pretrained reinforcement learning agent for field-oriented control of a permanent magnet synchronous motor (PMSM) is used to showcase this workflow. Refer to "reinforcement learning for field-oriented control of a permanent magnet synchronous motor" to learn how to set up and train an agent for this application.

      Published: 6 Dec 2021

      In this demo, we show the workflow of deploying a trained reinforcement learning agent to a Speedgoat target computer for real time testing using the new capabilities for implementing deep learning inference in Simulink and plain C code generation for deep learning inference. The workflow is as follows.

      We replaced the reinforcement learning agent block with a MATLAB function block that performs inference on the trained policy using the new deep learning capabilities that allow inference in Simulink. We then leverage the plain C/C++ code generation feature for deep learning networks, introduced in R21A, to build and deploy the complete Simulink real-time application to a Speedgoat hardware.

      Let us consider field-oriented control of a permanent magnet synchronous motor as an example application to showcase this workflow. In this example, the current controllers of the standard field-oriented control architecture, that usually contain 2 PI controllers, are replaced with a reinforcement learning agent.

      Let's look at the Simulink model that implements this field-oriented control architecture. This model contains two control loops, an outer loop that is implemented with a PI controller to regulate the speed, and an inner loop that has been replaced with a reinforcement learning agent to regulate the d-axis and q-axis currents. The plant and the inverter subsystem contain the inverter and the PMSM, modeled using motor control blocks. The reinforcement learning agent used in this model is pre-trained to regulate the d-axis and q-axis current. To learn how to set up and train a reinforcement learning agent for this application, refer to the video link posted in the description.

      Let's go ahead and run the Simulink model. We can see that the tracking performance of the controllers are good and are able to track the desired speed and the dq-axes current. Let's save this result for later comparison.

      Although we can load a pre-trained agent and run inference in Simulink using the RL agent block, this block does not support code generation as of R21A. Now in order to perform deep learning inference in Simulink and have the option to generate code, you can either use the product block or MATLAB function block and replace the current RL training setup. For this demonstration, we shall use the MATLAB function block to call the policy function that performs inference on the pre-trained network.

      The policy function used in the MATLAB function block is generated using the generatePolicyFunction API on the trained agent object. With the updated setup, let's go ahead and run the model. We can see that the tracking performance is identical to the results from RL training setup.

      Starting R21A with the plain C/C++ code generation feature for deep learning networks, we can generate C code for our MATLAB function block, that we replaced reinforcement learning agent with, without linking to third party libraries, such as MKL-DNN. The generated code can then be compiled and deployed to a Speedgoat system.

      In this demo, we will generate code for the complete application, including the plant model, and apply it to a Speedgoat system to test the trained reinforcement learning policy in real time. To configure this, you can open the Simulink real-time app that automatically sets system target file to slrealtime.tlc. For building and deploying code a Speedgoat machine.

      Let's go ahead and build a Simulink real-time application. We can see that the build process is successful, and an application has been created in the current directory. Now let's go ahead and run the application on the Speedgoat machine.

      As the created application loads and starts on the target machine, we can see the log signals in Simulation Data Inspector. We can see that the speed tracking and current tracking performance of the controllers on the Speedgoat machine are good. And on viewing this performance with the previously saved output, we see that performance of the controllers on the Speedgoat machine is comparable to the desktop simulation.

      That was a quick look at the reinforcement learning deployment to a Speedgoat machine. This concludes the video.

      View more related videos