Setting the Output of a MEX Function as One of the Inputs After In Place Manipulation

I have simple MEX function.

It gets 2 arrays and output one array.

I want to manipulate the input arrays in place within the MEX function and then chain one of the arrays as the output.

Basically something like:

```

// Get Data Pointers
double * mI = (double *)mxGetData(prhs[0]);
double * mO = (double *)mxGetData(prhs[1]);

// Work on data
MyFun(mI, mO); // Manipulate mO and mI in place

// Set output
plhs[0] = mxCreateNumericMatrix(0, 0, mxDOUBLE_CLASS, mxREAL);
mxSetDoubles(plhs[0], mxGetDoubles(prhs[1]));
mxSetM(plhs[0], mxGetM(prhs[1]));
mxSetN(plhs[0], mxGetN(prhs[1]));
```

Yet my code keep crashing (Eve when I comment my own function `MyFun` so basically all the code does is set the output to prhs[1]).

Any idea why? Is this not allowed?

Answer 1

[James Tursa]

A couple of problems.

1) First, your crash is caused by this:

mxSetDoubles(plhs[0], mxGetDoubles(prhs[1]));

You cannot attach the data pointer of one mxArray to another mxArray like this. That is sharing data pointers illegally. The MATLAB Memory Manager essentially knows nothing about this data sharing, so when one of these variables gets destroyed downstream (either inside your mex function or in MATLAB) it will free the memory behind the data pointer because it doesn't know it is sharing this with another variable. Then when you access that other variable and that invalid data pointer gets used ... CRASH!

So, how can you legally share data pointers in a mex routine? You can't. MATLAB doesn't give you any official functions to do this. The only way to do it is to use undocumented hacks which I won't go into here.

2) Then there is the issue of modifying input variables in place. This also is illegal. Those input variables might be sharing data pointers with other workspace variables (MATLAB does this in the background automatically). So altering them in place will also alter those other variables. There are no official methods available for you to figure out when this might be the case. So it is best to not do it at all, particularly if you are a beginner.

Probably what you will need to do is to make deep copies of the variables you wish to alter inside the mex routine, then alter them, then return them back to MATLAB as output variables and assign them to the input variables at the MATLAB level. E.g., something like this at the MATLAB level:

[A,B] = myMexFunction(A,B);
C = B; % This will be a shared data copy, at least initially

Inside the mex function, A and B (prhs[0] and prhs[1]) are deep copied with mxDuplicateArray( ) into plhs[0] and plhs[1], and these deep copies are the ones you alter. At the MATLAB level these outputs get reassigned back into A and B. Then there is the C variable which you want to be equal to B, which I have done here at the MATLAB level but you could also make a deep copy of plhs[1] into plhs[2] inside the mex routine and return a third variable as C.

[Royi Avital]

Thanks for the through answer.

The issue is my main motivation for MEX is optimization of memory allocation.

What you say is that I can't optimize that. Which is a pitty.

I wish for by reference mode of MATLAB.

[Royi Avital]

What I currently do is pre allocate the ouput, send it to MEX and update it in place where teh MEX function does not return any variable. Not elegant, yet I guess the only sane way.

[James Tursa]

IF you are very careful to NEVER do anything at the MATLAB level that causes data sharing, you can get away with modifying those variables in-place in a mex routine. The problem is you have no direct control over whether a variable is sharing data pointers with another variable. You can only coax MATLAB into not doing this. E.g.,

X = nan(1,3);  % Small variables are probably shared in the background
V = X; V(1) = V(1); % Altered variables are probably NOT shared
Y = nan(1000,3000); % Large variables are probably NOT shared in the background
Z = some_other_variable; % Directly assigned variables are definitely shared

In the above, neither X or Z would be safe to modify in-place in a mex routine. V would probably be safe to modify in-place in a mex routine (but not absolutely guaranteed) because it was altered subsequent to initially sharing with X. Y probably would be safe to modify in-place in a mex routine (but not absolutely guaranteed) because it is large and you haven't done anything with it prior to calling the mex routine. And, if the only thing you do with V or Y is call the mex routine with it as an input to modify it in-place, you can probably still get away with it. But if you ever do anything with it like assign it to another variable like below, it will no longer be safe to modify in-place in a mex routine:

C = Y;  % Y and C are now sharing data pointers

Now neither Y or C would be safe to modify in-place in a mex routine because they are sharing data pointers.

See this link for examples of the pitfalls that await you if you modify variables in-place in mex routines:

https://www.mathworks.com/matlabcentral/answers/383188-mex-inplace-change-problems-r2015b-and-later?s_tid=ta_ans_results

Best practice, if you can manage it, is to do variable allocations inside the mex routine. That way you know for sure it isn't sharing with any other variables.

[Walter Roberson]

I don't know if it still happens, but there was at least one MATLAB release in which initializing two variables with nearly exactly the same text resulted in the variables sharing storage, provided the variables were small enough.

For example,

V = 1:3;
W = 1:3;
X = 1:3 ;

V and W might share storage, but X with the slightly different initialization text might not share storage.

So it is (was?) not always enough for variables to be initialized distinctly: sometimes even with distinct initialization storage is (was?) shared.

[James Tursa]

@Walter Roberson This is discussed in the link I provided above. "Small" variables get reference copies created in the background that are sharing data pointers with the user variable. Then when the same syntax is encountered later on, a reference copy is made from the background reference copy of the original, rather than creating a newly allocated variable on the fly. But none of this behavior is officially published (how small is "small"? how close does the downstream syntax have to match? etc.), and one must presume that the rules for this can change from release to release. If I were writing code that modified variables in-place in a mex routine, I think I would always do something like X(1) = X(1) just prior to the mex call to give myself a fighting chance that the variable is not shared. Bottom line is that this practice will always carry some risk no matter how careful you are.