Can I run multiple instances (say, 2) of matlab independently on the same computer, all using parfor and (ideally) different cores?

183 views (last 30 days)
I have many cores (say 20), but some scripts only require a fraction (say 6). Does it make sense to run multiple scripts in different matlab instances? What do I need to look out for so they do not bog each other down?

Accepted Answer

Walter Roberson
Walter Roberson on 15 Jun 2021
Yes, this is something that can be reasonable to do.
What you would want to watch out for is not using more total cores than physical cores you have (with hyperthreading turned off), minus one for your OS to do all its background tasks like checking email. (And if you are using Windows, minus another one to pay the Microsoft Tax.)
Splitting a system with a number of cores to do different independent tasks is the basis of running "supercomputer" servers with Distributed Computing; https://www.mathworks.com/products/matlab-parallel-server.html . But since it is all local to you, you might at most want to create multiple parallel profiles.
  2 Comments
Florian
Florian on 14 Jun 2023
Thank you Walter!
If I might tag onto this question: I have a local computer with 28 cores that I want to use simultaneously with a colleague. We both have individual academic licences.
If we both run parallel pools in separate windows user accounts (and separate Matlab instances) I run into problems where workers cannot locate files to write into using fprintf.
However, if I run two parallel pools in separate Matlab instances in the same windows user account, this problem does not occur.
Do you have any insights into why this might be happening? To put it amateurishly, does Matlab "not see" parallel pools created by other user accounts?

Sign in to comment.

More Answers (1)

Matt J
Matt J on 15 Jun 2021
Edited: Matt J on 15 Jun 2021
Does it make sense to run multiple scripts in different matlab instances?
No, that would be a reinvention of what the Parallel Computing Toolbox already does for you, but requiring more manual work on your part. Just run all the scripts in the same Matlab instance, but in parallel, e.g., using parFeval. Let the toolbox worry about scheduling the cores.
  2 Comments
Alexander
Alexander on 15 Jun 2021
But they are different problems. Think optimizations where the derivative is done in parallel, but there are only 6-8 parameters. Obviously, I cannot run them in parallel. But it also seems a pity having idle resources.
Alexander
Alexander on 15 Jun 2021
The thing is they are really completely independent tasks that require some oversight and (possibly) interventions. So, I calibrate the same model to different parameters, and since I do not have a good idea about starting values I first calibrate some moments to get into the ballpark. Then I tell the optimizer to stop, and include more moments that I would like to match. etc. I do not want to stop one optimization just because another one is good enough to go to the next stage.

Sign in to comment.

Categories

Find more on Parallel Computing Fundamentals in Help Center and File Exchange

Products


Release

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!