Is it possible to carry out multi agent reinforcement learning in MATLAB ?

I wish to train multiple agents(2 agents to 10 agents) in a grid map environment with same reward function.

Is it possible ? If so, suggest me some articles or books to start with !!!

I am comfortable with single agent training.

Thanks in advance !!!

Answer 1

[Harsha Priya Daggubati]

I understand you are looking for a way to simulate a multi-agent Reinforcement Learning environment. Unfortunately, the Reinforcement Learning Toolbox currently does not support multi-agent scenario. You would need to write your custom environment and training algorithms for such scenario.

Due to large amounts of request on this feature, the development team is actively working on the multi-agent feature now and it will be available in a future release. If you decide to write your own environment and training algorithm, the following documentation would be a good place to start:

https://www.mathworks.com/help/reinforcement-learning/matlab-environments.html

https://www.mathworks.com/help/reinforcement-learning/training-and-validation.html

[Magnify]

I had the same idea to be implemented as him, so I hope you can check out the link to help those out who have the same requirement as me

Answer 2

[Emmanouil Tzorakoleftherakis]

As of R2020b release, Reinforcement Learning Toolbox lets you train multiple agents simultaneously in Simulink. Please see following examples for reference:

1. Train Multiple Agents for Path Following Control
2. Train Multiple Agents for Area Coverage
3. Train Multiple Agents to Perform Collaborative Task

Hope that helps

[Shabnam Noor]

Hi, I am currently trying to train multiple agents. I trained 2 agents quite smoothly but when I increased the number of agents to 3, I started getting error messages (please see attached image). All three agents have the same observation and their own individual actions and rewards (which was the case when there were two agents). I am not sure what changed when I went from two to three agents! I followed the "Train Multiple Agents for Area Coverage" example to set things up: the only differences being that I didn't using the selectors and I didn't multiplex the actions.

I would be very grateful if someone could let me know what I'm doing wrong!

Thank you :)

[Ari Biswas]

Hello Shabnam,

If you are not using the selectors please ensure that the observation and action dimensions are correct for all agents. This means the observation and action signal dimensions for the agent block must match those defined in the I/O specifications when you created the agents.

Feel free to share your model and necessary files so that we can take a look. If you dont want to share them in this forum you can open a Technical Support Case here.

[Shabnam Noor]

Hello, thanks for your reply. I managed to solve the issue but it looks like it wasn't because of incorrect observation and action dimensions.

I have been having strange issues with my multi-agent RL models. In the case of 2 agents, when I provided slightly different observations to the agents, only one of them worked the way they were supposed to while the other produced strange results (both agents produced correct results when I provided the same observation). If I increased the number of agents to two or more, I got the error messages that I mentioned above. I rechecked all connections, observation and action dimensions etc. but I couldn't see a problem anywhere. Interestingly, in both cases, when I created a new Simulink model with new blocks (there was no change in operation), things started working correctly! I had encountered a similar problem with a different simulation model a few years ago, which is why I created a new model to see if it solved the issue. I think that this may be just an issue of bugs. Do you think uninstalling and reinstalling Matlab 2020b will help?

[Ari Biswas]

It could be that your model was saved in a bad state. If you face any issues with multi-agent training please feel free to let us know.