Why doesn't a parfor loop run properly with a custom object with saveobj and loadobj methods?

I'm having an issue with a parfor loop that uses custom objects. The loop runs properly if it was for loop instead of parfor. However, with parfor it issues warning as follows. I think this is strange in that it's not an error despite an index exceeding matrix dimensions. The output of the parfor loop contained empty objects, which later resulted in an error outside of the parfor loop.

Warning: While loading an object of
class 'MarkerChan':
Index exceeds matrix dimensions. 
> In parallel.internal.pool.deserialize (line 29)
  In distcomp.remoteparfor/getCompleteIntervals (line 257)
  In parallel_function>distributed_execution (line 823)
  In parallel_function (line 590)
  In main_GABA_util>local_prepareC (line 571)
  In main_GABA_util.prep (line 48) 
Warning: While loading an object of
class 'Record':
Index exceeds matrix dimensions. 
> In parallel.internal.pool.deserialize (line 29)
  In distcomp.remoteparfor/getCompleteIntervals (line 257)
  In parallel_function>distributed_execution (line 823)
  In parallel_function (line 590)
  In main_GABA_util>local_prepareC (line 571)
  In main_GABA_util.prep (line 48)

The same parfor loop runs properly when saveobj and loadobj methods of the custom class are commented out. Related to this finding, I found the following description in the Documentation.

> If you are passing objects into or out of a parfor-loop, the objects must properly facilitate being saved and loaded. https://uk.mathworks.com/help/distcomp/objects-and-handles-in-parfor-loops.html

As far as I know, the saveobj and loadobj methods have been running properly; when tested, the original object and a saved and loaded object were identical.

methods

    function s = saveobj(obj)

        s.ChanInfo_ = obj.ChanInfo_;
        s.Data_ = obj.Data_;
        s.Length_ = obj.Length_;
        s.MarkerCodes = obj.MarkerCodes;
        s.MarkerFilter = obj.MarkerFilter;
        s.MarkerName = obj.MarkerName;
        s.TextMark = obj.TextMark;

    end
end

methods (Static)
  function obj = loadobj(s)

        obj = MarkerChan;
        obj.ChanInfo_ = s.ChanInfo_;
        obj.Data_ = s.Data_;
        obj.Length_ = s.Length_;
        obj.MarkerCodes = s.MarkerCodes;
        obj.MarkerFilter = s.MarkerFilter;
        obj.MarkerName = s.MarkerName;
        obj.TextMark = s.TextMark;

    end
end

Can I anyone tell me what's the real problem here?

Thanks, Kouichi

Answer 1

[Edric Ellis]

You can use the undocumented flag 'debug' to a parfor loop to diagnose this sort of problem (normally the loadobj function is invoked on the workers, so you cannot debug into it). For example

parfor (idx = 1:10, 'debug')
  % do stuff
end

This forces the parfor machinery to use the serialization mechanism, but runs everything on the client. This enables you to set breakpoints in your loadobj implementation. (Note there's a slight weirdness in using the 'debug' flag - it gets ignored if you run too few iterations in your loop).

[Kouichi C. Nakamura]

Wow, thank you for the hidden tips. I was very close to give up this issue, but I will investigate further.

Answer 2

[Walter Roberson]

When I look at https://www.mathworks.com/help/matlab/matlab_oop/understanding-the-save-and-load-process.html I would read it as indicating that loadobj is only called if there were errors in the load process, and in that case what gets passed to loadobj is a struct with the fields that were retrieved from the file. In such a case, the fields might be incomplete -- you should not count on any of them existing (because loadobj is the mechanism for dealing with upgrades and changes to classes.)

If no fields were retrieved from the file, then it is not immediately obvious whether s would be an empty array, or if it would be an empty struct with no fields, or if it would be a non-empty struct with no fields.

The error message you are seeing hints that s might be an empty struct, leading to s.ChanInfo_ possibly being interpreted as s(1).ChanInfo_ at a time when s(1) does not exist.

[Kouichi C. Nakamura]

Thank you for the comment! In the parfor loop (it's lengthy to show here), I'm not trying to load saved custom objects that are in question. So, I can't really see why it's "loading" the object in the first place.

Now I've come up with an idea. I can put a breakpoint for the loadobj and run the loop as normal for, and then I should be able to catch the moment when loadobj is called, if any.

[Kouichi C. Nakamura]

To my surprise, loadobj was never called (execution didn't stop at the breakpoint in loadobj) during the for loop.

[Walter Roberson]

Right, because if I interpret correctly, loadobj() is called only if something failed in the process.

You are not explicitly loading saved custom objects, but they are being transferred into the workers for the purpose of your calculations, and the way that is handled is for the client to serialize the objects and send them to the worker and the worker deserializes them, effectively loading them.

I suggest in your loadobj check for empty s. You already need to test for fields potentially missing (I figure). I have not yet seen what should be done if the object is just not salvageable.

[Kouichi C. Nakamura]

Thanks again. This is a great advice and it explains why only with parfor object is apparently loaded.

[Edric Ellis]

loadobj is always called when loading an object. It's unfortunate that the documentation implies otherwise, but I think it's explicit in the loadobj reference page.