MATLAB Answers

Agent is suddently doing random actions and training diverge

1 view (last 30 days)
I am training an DQN agent to replace a controller. Everytime the agent is about to converge, it starts to make random move and diverge. Although greedy epsilon was set to (0.3, 0.5, 0.6 etc.). Any idea what the reason might be?
Thanks in advance!

Accepted Answer

Emmanouil Tzorakoleftherakis
Edited: Emmanouil Tzorakoleftherakis on 22 Mar 2021
This is normal behavior - one common misconception is that once the reward starts going up, it will remain up. This is not true as the agent may start exploring a completely different part of the state space and that can lead to sudden dips in the reward as you can see.
Once you observe good behavior for a few episodes in a row/good average behavior over a number of episodes, that's a good indication that you can stop training. So I would stop training after episode 50/60 in your case and see if the result works, or I would let it train for more time and see if it recovers/surpasses the previous max (this is also common)

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!