Segmentation violation at start of Mex-File leads to termination

Hello,

my problem is that MATLAB is terminated as soon as i start a MEX-file (Segmentation-Algorithm similar to kmeans) with the reason being a segmentation violation. The used compiler is selected from Microsoft Visual C++ 2010. I've tried to debug the function in Visual Studio and setting the first breakpoint in the first line of the mexFunction. The strange thing is that it terminates right here or to be precise:

void * __cdecl _malloc_base (size_t size) { void *res = NULL;

    //  validate size
    if (size <= _HEAP_MAXREQ) {
        for (;;) {

            //  allocate memory block
            res = _heap_alloc(size);  //MATLAB terminates here

            //  if successful allocation, return pointer to memory
            //  if new handling turned off altogether, return NULL

            if (res != NULL)
            {
                break;
            }
            if (_newmode == 0)
            {
                errno = ENOMEM;
                break;

Since the function should segment large images my first thought was that there is not enough memory, but i also used a 10x10 double matrix for input with the same result. Therefor I don't think that the problem is the function itself (the first version as a regular MATLAB-functions works fine), so i'm sending only the mexFunction part:

#include "mex.h"

#include<math.h>

#include<vector>

... //several other functions ....

void mexFunction( int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) {

    int itermax=10;
    double *img;
    double *start;
    double *z;
    size_t r,c,elements;

    r=mxGetM(prhs[0]);
    c=mxGetN(prhs[0]);

    //Input: Image, Start-vector
    img=mxGetPr(prhs[0]); //Image
    start=mxGetPr(prhs[1]); //Start-vector

    elements=mxGetNumberOfElements(prhs[0]);

    //Output:segmented Image
    plhs[0]=mxCreateDoubleMatrix((mwSize)r,(mwSize)c,mxREAL);

    z=mxGetPr(plhs[0]);

    fcmCallMEX(z,img,itermax,start,(mwSize)elements,(mwSize)r,(mwSize)c);

}

The first part of the crash file:

Configuration:

Crash Decoding     : Disabled

Default Encoding   : windows-1252

MATLAB Architecture: win32

MATLAB Root        : C:\Users\Stephan\Documents\MATLAB\R2013a Student

MATLAB Version     : 8.1.0.604 (R2013a)

Operating System   : Microsoft Windows 7

Processor ID       : x86 Family 6 Model 42 Stepping 7, GenuineIntel

Virtual Machine    : Java 1.6.0_17-b04 with Sun Microsystems Inc. Java

HotSpot™ Client VM mixed mode

Window System      : Version 6.1 (Build 7601: Service Pack 1)

Fault Count: 1

Abnormal termination: Segmentation violation

Register State (from fault): EAX = 3b9c1860 EBX = 3b916a34 ECX = 00000000 EDX = 00000000 ESP = 00c2a9e4 EBP = 00c2aa78 ESI = 00000000 EDI = 3b916a30

EIP = 234d18ec  EFL = 00010202

   CS = 00000023   DS = 0000002b   SS = 0000002b
   ES = 0000002b   FS = 00000053   GS = 0000002b

I hope the information is sufficient for someone to help. Thanks!

Answer 1

[Dekun Pei]

Your debugger stopped in the malloc function, I suppose this is in the Visual Studio library, so it does not really provide too much information regarding which line in your mex file errored.

To troubleshoot this, I would:

1. Compile the mex file with the "-g" to enable debugging:

mex <filname>.c -g

2. Attach the Visual Studio debugger to MATLAB and open your mex function source code in Visual Studio. Set a debugging point in the first executable line in your mex source file, as debugging points cannot be set on a variable declaration for C source code.

Without knowing the exact line in the mex source code the seg fault occurred and how you are calling this mex file, I am merely pointing out some suspects here:

1. You called mxGetN, mxGetPr and such on prhs (the array storing the pointers to the inputs you are calling the mex function with), are you providing at least two inputs to this mex file call? As the mex source code stands right now, it will seg-faults if you call it with no input or one input. In general, you would do some error checking for the number of inputs given. The variable nrhs and nlhs will tell how many inputs and outputs the mex function is being called with. As C/C++ is not forgiving like MATLAB is in terms of auto-allocation for variables. Also check out the yprime.c example in matlab_installation\extern\examples\mex for some basic idea of input checking.

2. You call "plhs[0]=mxCreateDoubleMatrix((mwSize)r,(mwSize)c,mxREAL);", but the variables r and c are never defined in your source code. The behavior will be undefined and compiler dependent, they could become 0, random, or some negative values.

Answer 2

[Stephan]

I compiled it with Visula Studio befor so i could see where it terminates. I've also defined the variables r and c as size_t, but that wasn't the problem. There where some minor flaws in the actual functions and i've changed the inputs from matrices to vectors, so now it works, but thanks for your answer!