Elements problem due to the deep learning toolbox 'Predict'

I use the deep learning library predict block for RL agent in simulink, while the error indicates Invalid setting for input port dimensions due to the total number of input and output elements are not the same. While it worked well with RL block and matlab function (realized by 'generatePolicyFunction') and I just replace the block with Predict block. It seems the output of Predict is the whole elements sets of actions rather than single action, which is the actual output of RL agent. How do I solve it? Thank you very much.

[Yihao Wan]

Answer 1

[Emmanouil Tzorakoleftherakis]

Hello,

I see the problem. Typically, RL policies have post-processing part, which may vary from agent to agent, and the Predict block does not account for that, meaning you would have to implement this manually.

However, there is a quick solution to that if you use generatePolicyFunction and a MATLAB Fcn block instead of the Predict block (you should still use R2021a to get the plain C code generation benefit).

Here are the steps you need to follow:

1) Execute

generatePolicyFunction(agent);

This will create a) a .mat file that contains the extracted neural network (so you don't need to extract it yourself), and b) a matlab function called evaluatePolicy that performs inference on the policy (including any post-processing required).

2) In the Simulink model, set 'Simulation Target Language' to C++ in 'Model Settings'. Note that this setting is only used for simulation. If you were to right click the MATLAB Fcn block to generate code, the default option in the 'Model Settings'->Code Generation is 'C'.

3) Copy and paste the contents of 'evaluatePolicy in a MATLAB Fcn block (or call 'evaluatePolicy' directly from within the block) and run the simulation. If all the layers you have in the policy support plain C code generation, this should work.

I just ran this on my side and works. Hope this helps