Is cellfun multithreaded or does it do the operation one cell at a time?

35 views (last 30 days)
Does the cellfun function perform the function on all the cells simultaneously, using multiple cores if available, or does it just do one cell at a time? It seems like the process takes much longer than it should, and I don't think it's using multiple cores.
Is there some way to make cellfun multithreaded? I have the Parallel computing toolbox, although I'm not using it in this code. I'd hate to have to write everything in parfor loops just to get parallel processing.
  4 Comments
Shinya
Shinya on 19 Nov 2020
Using parfor just has a larger overhead. If the benefit of parallelization can overcome it, it will still be useful. Please see my answer below. In that implementation, my function that uses parfor is faster if the function spends roughly 100ms.

Sign in to comment.

Answers (3)

Walter Roberson
Walter Roberson on 16 Feb 2011
cellfun(@disp,num2cell(1:10))
gives a perfect 1 through 10 listing for me. Mind you I don't have a pool opened. It's a simple enough test to make though.
I would not expect cellfun to be multithreaded at the current time.
  2 Comments
Kenneth Eaton
Kenneth Eaton on 18 Feb 2011
Although, the documentation for CELLFUN does say this: "The order in which cellfun computes elements of A is not specified and should not be relied upon." So I guess the behavior you see is not guaranteed.
Matt Tearle
Matt Tearle on 18 Feb 2011
IANA developer, so this is wild speculation, but I think Walter's observation is guaranteed, because nothing in disp could change any elements of the cell. I suspect that cellfun may take the cell elements in some non-monotonic order behind the scenes, but then compile the results (again, behind the scenes). With something like parfor, OTOH, you see things in the order they actually happened.

Sign in to comment.


Matt Tearle
Matt Tearle on 16 Feb 2011
Currently cellfun is not multithreaded.

Shinya
Shinya on 19 Nov 2020
Edited: Shinya on 19 Nov 2020
As others noted, 'cellfun' is single threaded, but you can write a parallel version of 'cellfun' relatively easily.
function result = cellfunp(func, c, varargin)
% Parallel version of cellfun that uses parfor inside
p = inputParser;
addParameter(p, 'UniformOutput', 1, @isscalar);
parse(p, varargin{:});
result = cell(size(c));
parfor i = 1:numel(c)
result{i} = func(c{i});
end
if p.Results.UniformOutput % uniform
result = cell2mat(result);
end
This method has relatively large overhead (~0.1s on my computer), but will be useful if 'func' is time consuming and the benefit overcomes this overhead. Make sure that func is a function handle.
  4 Comments
Walter Roberson
Walter Roberson on 1 Mar 2023
c = arrayfun(@(varargin) randi([-9 9]), 1:20, 'uniform', 0); %just SOME cell
result = cellfunp(@(X) X.^2 - X.^3, c)
Starting parallel pool (parpool) using the 'Processes' profile ... Connected to the parallel pool (number of workers: 2).
result = 1×20
-180 -48 0 576 810 -180 392 810 -294 0 -294 -48 392 2 810 810 0 -4 36 -648
function result = cellfunp(func, c, varargin)
% Parallel version of cellfun that uses parfor inside
p = inputParser;
addParameter(p, 'UniformOutput', 1, @isscalar);
parse(p, varargin{:});
result = cell(size(c));
parfor i = 1:numel(c)
result{i} = func(c{i});
end
if p.Results.UniformOutput % uniform
result = cell2mat(result);
end
end

Sign in to comment.

Categories

Find more on Loops and Conditional Statements in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!