Why is malloc_trim freeing memory back to the OS, when matlab is not?
2 views (last 30 days)
Show older comments
Below I have a code sample one of our developers made that exhibits a "leaky" behavior matlab has that doesn't release memory back to the OS (CentOS 7) until malloc_trim() is called in a mex file:
Main:
%% READ ME
% Open htop and watch memory consumption as well as running the script. In
% matlab's eyes it appears all is well, and memory allocation and clearing
% are occuring in equal parts, but htop tells us matlab (or some
% subprocess) is consuming more. Also, malloc is able to clear this memory
% when matlab clear is not able to
%% Create Structure Array
% Pre allocate for a struct array, and then fill one field with one double
fprintf('Initial Memory Consumption: %dMB\n', logMem());
lenStruct = 1e7;
lenRand = 1;
fprintf('Creating of Structure Array\n');
x = repmat(struct('q',[]),1,lenStruct);
for ix = 1:lenStruct
x(ix).q = rand(1,lenRand);
perccount(ix,lenStruct);
end
fprintf('Memory Consumption After Struct Creation: %dMB\n', logMem());
%% Matlab Clear
% Try to use clear to wipe memory consumption
clear x
sleep(10); % wait for a bit to allow for clear to work on it?
fprintf('Memory After Clear: %dMB\n', logMem());
%% C stdlib malloc trim
% use malloc trim in a mex file
demalloc();
fprintf('Memory After Malloc Trim (mex call): %dMB\n', logMem());
% Both numbers reported by logMem (uses pmap) will be the same, but when
% visually watching htop the more memory is consumed than reported through
% logMem and more memory is freed with malloc than with clear
demalloc.c
#include "mex.h"
#include <stdlib.h>
/* The gateway function */
void mexFunction( int nlhs, mxArray *plhs[],
int nrhs, const mxArray *prhs[])
{
malloc_trim(0);
}
logMem.m
function memMB = logMem()
pid = feature('getpid');
[~,memStr] = system(sprintf('pmap %d | tail -n 1',pid));
idx = strfind(memStr, ' ');
assert(~isempty(idx));
idx = idx(end) + 1;
memMB = str2double(memStr(idx:end-2))/1024;
end
perccount.m
function perccount(jj,maxjj)
%Reports the percentage of the job done to the screen.
%
% PERCOUNT(I,Imax)
% I is the current iteration between 1 and Imax
% Imax is the maximum number of iterations
%
% Do not print anything to the screen between calls of this function!
%
% title - s cmspike/perccount vr - 1.2
% author - bodc/alead date - 2006may16
% Updated 2009May14
% At suggestion of John D'Errico renamed internal variable "max" to
% "maxjj"
% Also following D'Errico's suggestions the following functionality has
% been added:
% 1. An invocation check - checks that two input arguments are
% supplied
persistent lastCall;
if(nargin == 2)
if(lastCall ~= floor(((jj-1)/maxjj) * 100))
if(jj ~= 1)
fprintf(1,'\b\b\b');
else
fprintf(1,'\tPercentage complete: ');
end
pc_done = num2str(floor(((jj-1)/maxjj) * 100));
if(length(pc_done) == 1)
pc_done(2) = pc_done(1);
pc_done(1) = '0';
end
fprintf(1,'%s%%',pc_done);
end
lastCall = floor(((jj-1)/maxjj) * 100);
if(jj == maxjj)
fprintf(1,'\n\n');
end
else
error('Error: PERCCOUNT needs two input arguments...');
end
When all of the above files are ran together, it can be seen that matlab does not return memory to the os until malloc_trim is called. On a larger scale, this extends to the point of where matlab runs the system out of swap memory because it doesn't trim off memory which should be freed up, and instead just gobbles up more memory.
Does anyone have intuition as to the cause of this behavior?
0 Comments
Answers (0)
See Also
Categories
Find more on Write C Functions Callable from MATLAB (MEX Files) in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!