Does mxDestroyArray() recursively de-allocate elements of structs and cells?

The question is in the title. Is is about the C matrix library interface.

To explain in more detail, suppose that I create a 1 by 1 cell using mxCreateCellArray(), then create a numeric matrix using mxCreateNumericArray() and set it as the only element of the cell. Now will calling mxDestroyArray() on the cell destroy the numeric array as well, in one go? Or do I need to call it separately for the elements of the cell, then just the cell? I am hoping for the latter, as this is more reasonable for complex manipulations.

The documentation is ambiguous on this point.

[Szabolcs]

Note: cross posted on StackOverflow.

[Jan]

@Syabolcs: Thanks for mentioning the cross-posting. This is a good example for others. +1

You can omit the memset(), because the memory is initialized already.

Answer 1

[Jan]

Yes, mxDestroyArray() recursively de-allocates elements of structs and cells. Otherwise you could observe a memory leak.

[EDITED]

The documentation of mxDestroyArray explains clearly (e.g. in R2009a):

mxDestroyArray not only deallocates the memory occupied by the mxArray's characteristics fields [...], but also deallocates all the mxArray's associated data arrays, such as [...], fields of structure arrays, and cells of cell arrays.

And a small C-mex test function (call it test_mxDestroy.c and compile it):

#include "mex.h"
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
  mxArray *C;
  C = mxCreateCellMatrix(1, 1);
  mxSetCell(C, 0, mxCreateDoubleMatrix(1, 1000000, mxREAL));
  mxDestroyArray(C);
}

If the contents of the created cell is not freed implicitly, 8MB memory would be leaked in each call. Now inspect the operating systems memory manager while running:

for k = 1:1e6, test_mxDestroy; end

You will see, that the memory is not exhausted.

As James has explained already, your example does not crash accidentally only. When you try to use mat after mxDestroyArray(cell), you will encounter a crash soon. ATTENTION: Crashing the Matlab session can destroy data. So keep care, and even better keep a backup.

[Szabolcs]

Is there an easy way not to, i.e. only deallocate as much as mxCreate*() functions allocate? I don't expect there is, as I don't see anything like that in the docs ... but it would be much more convenient for me if this were possible.

[Szabolcs]

Also, do you perhaps have any reference for this? Something in the docs I missed? It's not something that's easy to test. If I do it incorrectly, my program may or may not crash, but that's all...

[Szabolcs]

Jan, I have to ask again, how do you know that this is the case? I saw your edit, but "observing a memory leak" is not a trivial thing. What you see in your process manager depends on many things, including how the memory allocator in use behaves exactly.

Please look at the test code I included in my main question. The result suggests (does not guarantee) that mxDestroyArray does not do recursive deallocation.

I would like to know if your answer was just a guess or you know this for a fact.

[James Tursa]

I am quite certain that Jan is correct unless something very strange is going on in Engine applications vs mex routines that I am unaware of. You can test it with a very large mat instead of a scalar. Look at the memory usage before and after destroying cell and you will see the large memory used by mat is released.

As to why your example program does not crash, I would say you are just lucky. After calling mxDestroyArray(cell), the pointer value contained in mat is no longer valid because that mxArray has just been destroyed. Subsequently calling mxDestroyArray(mat) is not valid and will produce unpredictable results (dereferencing an invalid pointer and everything bad that can result from that). Maybe nothing bad happens because mat is small instead of large, or maybe because you don't do anything much downstream in your code, or maybe because you don't have many mxArray variables in existence that could be corrupted, or ...??? Who knows? But it is definitely invalid.

[Szabolcs]

Thanks for looking at this @James. Yes, you are right that it can also be just luck that the example program doesn't crash, that is why it's not a proper test.

I did as you suggested (please see the edit to the example program), and I inserted pauses in the program after the creation of the matrix, after the destruction of the cell and after the destruction of the matrix. I was watching the memory usage at the pauses, as shown by Activity Monitor (I'm on OS X). Memory usage goes down only after mxDestroyArray(mat), not at mxDestroyArray(cell). This again suggests that destroying 'mat' separately is necessary (which is incidentally convenient for my application).

But I can't ignore that both of you said that mxDestroyArray(cell) is sufficient and mxDestroyArray(mat) is crash-prone, so I would like to ask you to please take another look at this.

Yes, this is an engine application, not mex. But the data structure manipulation functions are the same for both, aren't they?

[Jan]

See [EDITED]

[Szabolcs]

@Jan You were running your example code from within MATLAB, right? I ran my example as an engine application, i.e. in a separate process from MATLAB. Now I also tried running these from within MATLAB as you did, and it turns out that the behaviour is different:

Here's something interesting to try: change your code to allocate a lot more memory in one go than 8 MB (I went straight for 800 MB to make things very clear). The remove all mxDestroyArray calls from it. Now memory should definitely grow significantly with every call to test_mxDestroy, right? But it doesn't. If I watch the task manager carefully, I sometimes notice a momentary increase in memory by a few hundred MB, but it drops right down. I can even put test_mxDestroy in a loop as you did, and it does not cause any memory increase even though I removed all mxDestroyArray calls.

Note that this only happens when running inside matlab, not when running as an engine application.

The only explanation I can find is that:

1. mxDestroyArray is not recursive
2. But MATLAB has some sort of garbage collection mechanism that takes care of freeing unreferenced data. This garbage collector is available within MATLAB, but not in libeng (engine applications).

[Szabolcs]

I'll leave the comment above intact because I think it's still interesting that when running within MATLAB, memory gets de-allocated automatically, but it turns out that your answer is correct, and mxDestroyArray is recursive. It was a small indexing mistake in my test program that led me to believe otherwise ...

[James Tursa]

mxDestroyArray frees the memory recursively back to the MATLAB memory manager. Internal to the MATLAB memory manager it may be holding on to it (so the operating system still sees it as being used) for potential downstream use (e.g., waiting to see if you turn right around and allocate another big variable of that size). My guess is something like this may be happening with your test. But putting this all inside a loop should convince you that the memory is indeed released (at least back to the MATLAB memory manager) and available for other allocations downstream in your code (e.g., the next iteration of the loop).

[Jan]

Matlab's memory manager uses a garbage collector, which automatically calls mxDestroyArray for all mxArray's, which are not referenced in plhs[] and not marked as persistent. This is documented in the chapter "Advanced topics :: Creating C Language MEX-Files -> Automatic Cleanup of Temporary Arrays", or e.g. http://www.mathworks.com/help/matlab/matlab_external/memory-management.html.

But this automatic cleanup does not work in a stand-alone program. This is documented also, but I cannot find a link currently.

[Szabolcs]

Thanks for the link Jan. Anyway, I have the answer now and sorry about the confusion ... I also posted a short answer and a link back here to StackOverflow.

[James Tursa]

@Jan: Technical side note ... for a normal return from a mex routine, shared data copies of the plhs[] variables are what are actually returned, not the plhs[] variables themselves. Then everything on the garbage collection list is destroyed, including the plhs[] variables which are in fact on the garbage collection list (except for persistent variables such as prhs[] or using mexMakeArrayPersistent). For an error return, the only difference is that shared data copies of the plhs[] variables are not made, only the garbage collection (including the plhs[] variables) takes place.

[Jan]

Thanks, James, to explain this detail, which is surprising for me.