MATLAB Answers

0

Codistributed arrays taking too long to run

Asked by Daniel Jaló on 13 Feb 2013
Hello fellow programmers.
I've been exploring the distributed arrays functionality on Matlab. I tried doing a simple matrix multiplication, where both matrices have the same dimension, and comparing the time it takes the serial and parallel execution to run.
Here's the code, not "parallelized" (is that a word?)
N = 5000;
A = eye(N);
B = magic(N);
tic
C = A*B;
toc
I try to run it in parallel by distributing the array of the matrices between the 4 workers I have available.
N = 5000;
A = eye(N);
B = magic(N);
spmd
A = codistributed(A,codistributor1d());
B = codistributed(B,codistributor1d());
end
tic
spmd
C = A*B;
end
toc
However when I run this code it takes a lot of time to distribute the matrices between the workers (with the codistributed() function) and the matrix multiplication takes significantly longer (about 5/6 times longer).
If anyone could tell me what I'm doing wrong I'd appreciate. Am I not understanding how distributed arrays should be used?
Cheers Daniel

  0 Comments

Sign in to comment.

2 Answers

Answer by Jill Reese
on 14 Feb 2013
 Accepted Answer

Here is an example of how to properly benchmark an operation on distributed arrays:
You are comparing the implicit parallelism (multithreading) of core MATLAB to the explicit parallelism (MPI communication between MATLAB workers that are themselves running singlethreaded) of the Parallel Computing Toolbox distributed arrays. If the input matrices A and B fit in memory on a single machine with enough memory left over to perform the operation and store the output, it's often pretty hard to beat the performance of the multithreaded operation. Distributed arrays provide a clear benefit when the problem exceeds the memory available on a single machine.
If you set up a benchmark for mtimes (*) following the example that I've linked, you can investigate the performance you can expect from your own machine. I think that the codistributor you have chosen is not unreasonable for this operation.

  0 Comments

Sign in to comment.


Answer by José-Luis
on 13 Feb 2013
Edited by José-Luis
on 13 Feb 2013

It's because even though you distribute your array amongst the workers, matrix multiplication might not be efficient, depending on how you distribute the array. In order to compute it you will need to access data in the other arrays, considerably slowing your code due to communication overhead.

  0 Comments

Sign in to comment.