(par)for k=1:N end; select for or parfor loop without code replication

I like to use for-loop or parfor-loop with function input flag. One way to do that is using if-else with code replication. I wonder is there a cleverer way?

% solution based on code replication
function foo(parflag)
if parflg==true
    parfor k=1:3
      % processing codes
    end
else 
    for k=1:3
        % copy of processing codes
    end
end
end


%% wondering how to make this idea work?
function hoo(parflag)
if parflag==true
    forString = str2func('parfor')
else
    forString = str2func('for')
end

forString k=1:3
    % processing codes
end
end

Answer 1

[Matt J]

You can avoid replicating the loop code as follows.

numWorkers={}; % or whatever
if ~parflag
    numWorkers={0};
end

parfor (k=1:3, numWorkers{:})
  % processing codes
end

Be mindful, though, that parfor loops have stronger requirements on the code structure, and you would be limiting yourself to that, even when running serially.

[Bruno Luong]

does it start the parallel pool, etc... even for simple for-loop?

[Matt J]

No, it doesn't.

[Simon]

That would work. Parfor with zero worket would be a bit slower than for loop, but it won't affect much. I think this is an acceptable answer.

[Matt J]

Parfor with zero worket would be a bit slower than for loop

Shouldn't be, according to other posts from @Edric Ellis.

[Bruno Luong]

Same question for to parfeval? Can I set something for mypool so that it runs without parallel pool?

mypool = backgroundPool;
Future(1) = parfeval(mypool,@(x)sin(x), 1, pi);
% ...

[Matt J]

Same question for to parfeval

That I don't know, but it's less of a problem, because parfeval has a functional form, unlike parfor. You could therefore make a wrapper for parfeval that will handle the pool-free case.

[Sam Marshalik]

To add on to Matt's great comments, you can pass an empty pool flag to parfeval and it will run in serial in the background. Here is an example of what I mean:

delete(gcp); % Ensure no pool is open
p = gcp('nocreate');
% No pool exists, so p is empty
for i = 1:3
    f(i) = parfeval(p,@pause, 0);
end

p = parpool("Threads");
% p exists and will be used to run parfeval
for i = 1:3
    f(i) = parfeval(p,@pause, 0);
end

In short, you can pass the pool object over to parfeval and if it is empty, it won't be used.

[Bruno Luong]

@Sam Marshalik great thank you !

I just tested it in my real project and it seems working well. Now I have one base code for both parallel and non-parallel calculation.

This is also a convenient way to debug the parallel code, just switch to empty pool and voilà.

[Simon]

% why using cell?
numWorkers = {};
if ~parflag
numWorkers = {0};
end

% this is simpler, isn't it?
if ~parflag
    numWorkers = 0;
end

% Both of them work.

[Bruno Luong]

Good question, further more passing cell is not documented as from parfor doc

"parfor (loopvar = initval:endval, M); statements; end executes statements in a loop using a maximum of M workers or threads, where M is a nonnegative integer."

[Simon]

Actually, if we adpot numWorkers as a scalar number, we don't need if-else to make the selection.

function foo(x, numWorkers)
    parfor (1:length(x), numWorkers)
        % processing codes
    end
end

foo(x, 0) % for loop
foo(x, 3) % parfor loop using 3 workers

[Bruno Luong]

But then you have to manually set numWorkers

[Simon]

Yes, that flexibility works for me. For example, when I am not doing anything but programming Matlab, I can set it to the maximum number of workers my compouter has. But when I heavily multitask, I can set numWorkers, say, 2 or 3, and let other cores to work on non-Matlab tasks. (I guess that's how my Mac would work.)