27 views (last 30 days)

Show older comments

I want to batch process the following for loop in my script (which contains a call to a function I wrote called 'interp'). I need the loop to be repeated 120 times but split across 4 workers, and the function needs to call the correct data layer from 3D arrays [e.g. squeeze(bearing_rec(a,:,:))]. I would want the data to put produced at the end of each loop and saved as a .mat file.

Would anyone be able to help me with how to code this?

for a = 1:length(sourcelat) % sourcelat = 120

integral_area_size = squeeze(integral_area(1,:,:)); % integral area = 5396 x 5396

[integral_rec, Hc_rec] = interp(integral_area_size, rng, squeeze(bearing_rec(a,:,:)), squeeze(range_rec(a,:,:)), squeeze(H3(a,:,:)), squeeze(min_H(a,:,:)));

% Need to print output for each here

save(['F:/Outputs/Output_' num2str(a) '.mat'],'integral_rec', 'Hc_rec')

end

Stephane Dauvillier
on 15 Jul 2019

You need to use the batch function:

j = batch('yourScriptName','Pool',4);

And replace the for loop by a parfor loop.

Be careful: if you want your batch to be perform on 4 worker you will need 5 core as one is dedicated to launch the job.

Be careful²: if you have already strated a MATLAB pool before, then you will have fewer available worker for your batch job. Suppose you have 12 total worker and you already open a pool on 8 workers then your batch can only have 12-8 = 4 worker and since once will be dedicated to launch the job you have only 3 available worker

Be Careful^3/ save function is not allowed in parfor loop

Stephane Dauvillier
on 16 Jul 2019

With the batch function, your script will be executed on your PC as it is. The onlyu difference is your current MATLAB session won't be freeze.

If your script don't use parallel, your batch won't.

To launch the execute script on a pool of 3 workers (+1 for creating the job)

job=batch("execute","Pool",3);

if you want to see the state of your job

job.State

When the answer is finished that mean you can retrieve the result

Result = load(job)

N = Result.N ;

foo = Result.foo;

If the file execute is the following

N=10;

% Use cell array in order to save your results

foo = cell(N,1);

parfor i=1:N

foo{i} = peaks(i) ;

end

So if you want to save your result in a mat file, just do the following (at the end of your script of after retrieving the result

for i = 1:N

data = foo{i} ;

save(['mySaved',num2str(i,'%02i')],'data')

end

Note that the job (and its result will still be in memory) even when closing MATLAB. In order tio clear it from memory

delete(job)

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!