Reinforcement Learning does not show that training occurs?

6 views (last 30 days)
Hi, I have a reinforcement learning in a continuous state/action space. I trained it for 2000 episodes, each episode contains maximum of 10 steps, and stops episode training when reaches a positive reward more than 10 or when reaches the maximum number of steps. Here is the training procedure of this off-policy reinforcement learning. This reinforcement learning visually shows that the training happens, when tested on some samples. But I cannot understand why it doesn't show the original training trend of RL (start from low reward to high rewards). I checked some of the answers provided in MathWork like changing OU noise, deep neural netwrok setting of actor and critics, and changing the reward function, but it just fluctuates as follow. I appreciaate if someone could help me in this case.
  3 Comments
Emmanouil Tzorakoleftherakis
It's also not clear what the question is. How did you get the plot above? The x axis does not show all training episodes
shadi abpeikar
shadi abpeikar on 18 Mar 2021
Let me give you some more information. My RL is going to train some swarm behaviors, so in each epiosed it recives a positive reward and stops that episode, when the behaviour is flocking, and gets a penalty when it is a random behaviour. In the second condition the training of the episode iterates for maximum of 100 steps, until reaches flocking or maximum steps. I just checked the new generated states of RL in some random behaviours, and they changed to flocking, as I expected, But I cant see the increeasing trend of rewards, in the RL plot (the same happens with 2000 episodes).

Sign in to comment.

Answers (1)

Emmanouil Tzorakoleftherakis
Thanks for the info. I think this is a scaling issue with the plot. The Episode Manager has this option where you can uncheck "Q0" (orange line) which prevents you from seeing the training trends more closely
  2 Comments
shadi abpeikar
shadi abpeikar on 18 Mar 2021
Edited: shadi abpeikar on 18 Mar 2021
Thanks for your response, Emmanouil,
But I unchecked Q0 as well, and again there are a lot of fluctuations, and not a smooth increasing in the reward values.
Emmanouil Tzorakoleftherakis
Edited: Emmanouil Tzorakoleftherakis on 19 Mar 2021
Well, that means that your agent is not learning anything in which case you have to go back and see what you can change to improve training. I would recommend starting from the reward signal

Sign in to comment.

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!