MATLAB Answers

In parfor-loop, can I call a multi-threaded mex and get some speed-up?

75 views (last 30 days)
I learned the concept of multi-threaded mex from undocumentedmatlab. (It seems this website is unaccessible now ...)
I am wondering if I can call a multi-threaded mex in parfor-loop.
My current code looks like
parfor k=1:1e6
result(k) = mex_wrapper(data(k));
end
mex_wrapper.c looks like
double calculate()
{
int N=50;
for (i=0;i<N;i++)
{
//...
}
}
void mexFunction()
{
calculate();
}
The iterations inside calculate() are independent, so I want to change the sub-routine calculate() to support multi-thread.
Although I am running parfor in process-based-environments, I am not sure if multi-threaded mex would confict with parfor.
So can I use multi-threaded mex in parfor? And would I get some speed-up by doing so?

Accepted Answer

Edric Ellis
Edric Ellis on 4 Jan 2021
You should be able to run a multi-threaded MEX file correctly inside a parfor loop. However, you will be oversubscribing your machine. For example, if your machine has 6 cores, your parfor loop will run 6 copies of your MEX function simultaneously. If each of those uses 6 threads each, you will have 36 threads active on your machine. This should work, but it will probably be less efficient than having a single-threaded MEX function. (A multithreaded MEX function inside parfor can be more useful when you have a cluster of machines - there, you might run single worker process per machine, and have each machine run the multithreaded MEX function).
  3 Comments
Xingwang Yong
Xingwang Yong on 4 Jan 2021
Thanks, Edric, I got your points. I tried to use maxNumCompThreads() in parfor-loop, it returns nothing. So this is infeasible.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!