Reinforcement learning with action updated once every few (say 100) time steps

Hello,

I am trying to learn a controller in Simulink environment. I am tryng to use reinforcement learning where the action determined by the agent is updated once every few time steps, i.e., an action once determined by the agent is used for by Simulink to run the simulation for a few time steps before it is updated again. Please provide me with suggestions on this. Thank you.

Answer 1

[Shivani]

Hello @Suyash Agrawal

The workflow you wish to implement is discussed in detail in the folowing MATLAB documentation link: https://www.mathworks.com/help/reinforcement-learning/ug/reinforcement-learning-environments.html#:~:text=The%20agent%20and%20the%20environment%20interact%20at%20each%20of%20a%20sequence%20of%20discrete%20time%20steps%3A

The following MATLAB answer thread discusses a similar problem statement and may provide relevant information on how to address this issue:

https://www.mathworks.com/matlabcentral/answers/555799-reinforcement-learning-sample-time

Additionally, there are a number of tutorial videos that may provide more details on this implementation:

• https://www.mathworks.com/videos/reinforcement-learning-for-field-oriented-control-of-a-permanent-magnet-synchronous-motor-1587727861081.html
• https://www.mathworks.com/videos/getting-started-with-reinforcement-learning-1643977291095.html
• https://www.mathworks.com/videos/reinforcement-learning-part-2-understanding-the-environment-and-rewards-1551976590603.html
• https://www.mathworks.com/videos/practical-reinforcement-learning-for-controls-design-test-and-deployment-1657031639527.html

[Suyash Agrawal]

Thank you for the response. I understand the workflow and have read the descriptions you have provided. However, my requirement is slightly different. Specifically, after the agent generates an action, I want to simulate my simulink model for 100 time steps at a step time of 0.1ms, and then generate a new action from the agent. Therefore, the step time of simulation is 0.1 ms, whereas the agent need to output action every 10 ms. I hope this clarifies the query further.