Is there a way to start a parellel pool with maximum available workers as the cores on the machine?

20 views (last 30 days)
Leos Pohl
Leos Pohl on 24 Aug 2021
Answered: Raymond Norris on 24 Aug 2021
Is there a way to start a parellel pool with maximum available workers as the cores on the machine?
I am using a compiled code which then is being run on a cluster, each cluster node can have a different number of available cores. Is there a way to start parpool with the maximum possible amount of workers? If I start pool = parpool('local'); I only get 12 workers even though on the node, where such code is run, I have 32 cores allocated.

Answers (1)

Raymond Norris
Raymond Norris on 24 Aug 2021
pool = parpool('local',maxNumCompThreads);
Alternatively, if you're running this through a scheduler (e.g. PBS), you could query the scheduler first.
% Query for available cores (assume either Slurm or PBS)
sz = str2num([getenv('SLURM_CPUS_PER_TASK') getenv('PBS_NP')]); %#ok<ST2NM>
if isempty(sz)
% Not running through a scheduler, get default size
sz = maxNumCompThreads; end
pool = parpool('local',sz);

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!