Clear Filters
Clear Filters

Multiple goals in Grid World

16 views (last 30 days)
GCats
GCats on 15 Feb 2022
Answered: Anshuman on 22 Jan 2024
Hello everyone,
Is it possible to train the agent to reach different goals in the same episode? I'm working with the standard GridWorld environment given here: https://nl.mathworks.com/help/reinforcement-learning/ref/creategridworld.html#d123e1207 and I would like to have the agent reach for example state [2,1] and then [3,5] and finally [3,3] in this order. Is it possible to do so?
If i try to modify the terminal state option as
GW.TerminalStates = ['[2,1]'; '[3,5]'; '[3,3]'];
The agent only reaches the closest one to the initial state and not all of them. Any hints?

Answers (1)

Anshuman
Anshuman on 22 Jan 2024
The "createGridWorld" function is typically used to create a simple grid environment where an agent can navigate to a single goal. By default, the environment is set up to terminate the episode once the agent reaches the specified terminal state.
To train an agent to reach multiple goals in sequence within the same episode, modifying the "TerminalStates" property alone won't suffice because, as you've observed, the episode will end once the agent reaches the first terminal state. To do so, you need to implement a custom reward function and modify the environment's step logic to handle sequential goals.
  • Initially, don't set any terminal states, as you want the episode to continue after reaching each goal.
  • Define a reward function that provides positive reinforcement when the agent reaches one of the goals and then updates the next goal. Basically you need to create a custom reward function.
  • Enhance the state representation to include the current goal or the sequence of goals that the agent needs to reach.
  • Customize the step function of the environment to check if the agent has reached the current goal. When a goal is reached, update the state to reflect the next goal in the sequence(but do not terminate the episode).
  • The episode should only terminate when all goals have been reached or if a maximum number of steps is exceeded.

Products


Release

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!