Import the NN weights of a trained agent (policy,actor,critic) from Julia

I trained an agent using the PPO algorithm in Julia (using the ReinorcementLearning.jl package) which uses the clipped PPO methodology . The weights of the NNs used for the approximation of the Actor - Critic can be extracted in Julia into a BSON file . I was wondering if there is any compatibility between Julia and Matlab so as to import the weights of the trained NN of the agent to matlab . The reason I want to do this is because in the lab we have an electronically controlled engine that runs through simulink and we wanted to test its behavior with the weights extracted from Julia (already tested with Matlab).

Answer 1

[Aastha]

Hi @Kyriakos Economou,

I understand that you have trained an agent using the PPO algorithm in Julia and would like to transfer the trained neural network weights to MATLAB for testing with a Simulink-controlled engine. While MATLAB does not support direct import of models from Julia, here is a workaround you can follow:

You can manually reproduce the same neural network architecture in MATLAB and then use the "setwb" function to assign the weights and biases. The "setwb" function in MATLAB accepts a neural network object and a single vector containing all weights and biases, allowing you to set them manually.

For more information on the "setwb" function you can refer to the following MathWorks documentation:

https://www.mathworks.com/help/releases/R2021a/deeplearning/ref/setwb.html

Hope this helps!