reduction variables and evaluation order

9 views (last 30 days)
This the doc on can read
"A reduction variable accumulates a value that depends on all the iterations together, but is independent of the iteration order. MATLAB allows reduction variables in parfor-loops."
Among the operations one have
% X = X * expr
And we all know for matrix, multiplcation order matters. So why X itcan be considered as valid reduction variable using "*"?
However when I test it it looks like parfor magically multiply in the right order as showed in this code:
A=rand(2,2,10);
% B = A(:,:,1)*A(:,:,2)*...A(:,:,10)
B=1;
for k=1:size(A,3)
B = B*A(:,:,k);
end
B
B = 2×2
0.1810 0.3914 0.1442 0.3118
C=1;
parfor k=1:size(A,3)
pause(0.1*rand()); % This will make the order of each worker more or less random
C = C*A(:,:,k);
end
Starting parallel pool (parpool) using the 'local' profile ... Connected to the parallel pool (number of workers: 2).
C
C = 2×2
0.1810 0.3914 0.1442 0.3118
norm(B-C)/norm(B)
ans = 3.3254e-16
Do I missread the doc? There is a descrption in the doc about what works are done by worker and what by client and it is not totally clear to me. Is there any joint+fork occurs when such variable is updated? Order matter or not? Can someone shed a light?

Accepted Answer

Matt J
Matt J on 8 Aug 2022
Edited: Matt J on 8 Aug 2022
First remember that each worker is assigned some consecutive subset of loop iterations. The worker will execute the iterations within its assigned subset in the original, consecutive order. Therefore, each partial sequence of reduction operations will have a predictable result.
Then, once all iterations are complete, the partial reductions are post-consolidated. I suspect that the order in which reduction variables are post-consolidated is also done to preserve the original ordering (and why not, since the processing is sequential anyway at this point).
Note therefore that concatenation works, too, even though we can clearly see that the timing of each iteration's execution is all over the place:
C=[];
parfor k=1:10
pause(0.1*rand());
disp("k="+k)
C=[C,k];
end
Starting parallel pool (parpool) using the 'local' profile ... Connected to the parallel pool (number of workers: 2). k=1 k=3 k=4 k=7 k=10 k=2 k=5 k=6 k=8 k=9
C
C = 1×10
1 2 3 4 5 6 7 8 9 10
  8 Comments
Raymond Norris
Raymond Norris on 12 Aug 2022
When the client sends the instructions/data out, it recalls the order for assignment back to the variable on the lefthand side. This doesn't mean the values will be sorted (as in the case of the clock example). It means that MATLAB knew when the 4th value returned, where to put it. Even if it was assigned after the 5th (which could have been the 1st iteration to evaluate in the 2nd group).

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!