How to extract neural network of reinforcement learning agent?

Is there a way to get the neural network of trained reinforcement learning agent?

Answer 1

[Emmanouil Tzorakoleftherakis]

For critics:

criticNet = getModel(getCritic(trained_agent))

For actors

actorNet = getModel(getActor(trained_agent))

Note that you can use this regardless of whether the agent is trained or not.

Hope this helps

[Yihao Wan]

Thank you for the reply. Is it possible to use gensim for this if I just use the fullyconnected layer in the deep neural network? (I know that gensim is only for shallow neural network)

[Emmanouil Tzorakoleftherakis]

gensim is not compatible with the new API used in layers like fullyconnected.

Are you trying to extract the net and put it in Simulink for simulation? Would the RL Agent block not work? Or do you want to generate code?

You may want to take a look at the new DL blocks that shipped in R2020b that let you bring deep learning models into Simulink for simulation and code generation. You could also use a MATLAB Function block instead. Here are some examples:

https://www.mathworks.com/help/gpucoder/ug/deep-learning-in-simulink-by-using-gpu-coder.html

https://www.mathworks.com/help/gpucoder/ug/deep-learning-in-simulink-using-deep-neural-networks-library.html

[Yihao Wan]

Thank you for the reply. I actually realized through these ways. What I want to do is generating the code for RL for the DS1202 RTI, and try to avoid C++ code generation (the way in attached link can only used for C++ code generation of deep learning network). So I want to build the simulink block of extracted neural network similar to that of gensim, which is actually much easier for the code generation for the DS1202 RTI. Do you have any suggestions for this? Thank you very much.

[Emmanouil Tzorakoleftherakis]

What I mentioned above should work then. Use the DL blocks or ML Function block to bring the trained network into Simulink, and the plain C codegen feature I mentioned in my other answer to deploy. Plain C code generation shipped in R2021a.

[Yihao Wan]

Sure, I will try. Thank you for the reply.