MATLAB Answers

parfor inside for consuming different run times

5 views (last 30 days)
AA on 29 Jan 2018
Commented: AA on 29 Jan 2018
As it has been already advised, parfor within for isn't really recommended, but I am somewhat bounded by the intention of my code. I am running an optimization algorithm, and within each iteration of the algorithm I need to update around ~1k variables.
So, my code (please bear with me for the rough pseudo-code) -
for i=1:MaxIters
% I calculate some prerequisites here
parfor (i=1:1000, 10)
% I perform some independent calculations here, as the 1k
% variables are independent of each other.
% However, I need to load and save them (which I do using
% a function call)
% I update my optimization variable here, and then go back to
% the for loop for another round of optimization
If I replace the parfor loop with a for loop, I see that one inner iteration takes roughly around ~30mins. I managed to get it under control using a parfor loop, and I happily see a reduction in simulation time to ~6-7mins. However, I am observing that this simulation time keeps on increasing as it goes for another round of optimization (next outer iteration), and it keeps on happening.
On some further checks, I checked that for the first outer loop, my cores/workers are nearly running at ~100%, but for the subsequent ones, they run at around ~20-60%, which I think is contributing to the increment. However, I am not able to figure out what is the problem, as the inner process is nearly the same for every outer iteration.
I have even tried parfevalOnAll('clear', 0, 'all') to clear the workers' data and delete(gcp('nocreate')) methods to even shut down the parpool after the inner loop completes, but nothing seems to be stopping the increment. Can somebody please help me out??
Thanks so much!
AA on 29 Jan 2018
Thanks for you response. I am running my code on MatlabR2016a. I have attached my code for your kind reference(sorry for not posting it here, as it is ~700 lines long). If there is anything else I can provide, I would be more than happy to.
Thanks again!

Sign in to comment.

Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!