I believe the RL environment template creator has an error in the reset function but I'm not sure

2 views (last 30 days)
when using rlCreateEnvTemplate("MyEnvironment") to create a custom template I came across this line in the reset function:
% Line 123
InitialObservation = [T0;Td0;X0;Xd0];
Initial states seem to be reversed, I believe they should be [X0;Xd0;T0;Td0]?
the same seems to apply to the example loaded with openExample('rl/MATLABCartPoleDQNExample'), although I cannot see the reset function, the example gives the same results as the template and when I tried validating these two against the environments created with openExample('rl/CreateMATLABEnvironmentUsingCustomFunctionsExample') they yield different initial states.
but I'm still kinda new to this and I'm afraid to be missing something here. Could you please clarify?

Accepted Answer

Emmanouil Tzorakoleftherakis
You are correct the order is wrong. That being said, the order of states depends on your dynamics and how you set up the state vector in the step function.
Hope that helps
  1 Comment
Cecilia S.
Cecilia S. on 22 Jun 2021
Excellent, thank you for clarifying. So, I believe in this case the starting angle would be constant throughout the episodes and the resetting randomization would be only of the X variable

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!