Parallel and serial computation

Hello, everybody. Myself Anuj Kaushik, a research scholar at IIIT Allahabad, India.

I’ve a computer with 128 GB RAM, 40 cores, CentOS. I am using matlab for a job, which basically does addition, multiplication, fft and ifft of 2D matrices (4096x4096).

First I used for loop (sequentially). I saw that all the cores were busy/working. Why? I expected that only one core would be busy.

Then I used parfor loop (parallel computing). As expected 40 cores were running.

The time taken by my program for 5 different configurations in case of parallel computing were 747s, 801s, 788s, 836s, 1156s and 24s, 22s, 23s, 24s, 28s for serial computing, respectively. My question is when it is performing in parallel computation, why still taking so much time?

Answer 1

[Edric Ellis]

MATLAB uses built-in parallelism on a single machine for many large matrix operations. See this page: https://uk.mathworks.com/discovery/matlab-multicore.html . Parallel Computing Toolbox workers run in a single-threaded mode (by default, but this can be configured by changing the NumThreads property in your cluster profile), so each computation on a worker will be slower, but you can do more individual computations at a single time.

The general rule is that: if MATLAB's built-in parallelism applies to your problem, it will be faster than using PCT to solve the same problem since it does not have to copy data to/from the worker processes.