How can I monitor how much memory MATLAB is using?

I have a program that is memory intensive and I want to monitor how much memory MATLAB is using so that if it goes above a certain threshold, I can stop the program.

Answer 1

[MathWorks Support Team]

Unfortunately, there is not a convenient way to monitor memory usage in MATLAB. However, the attached function can help monitor memory usage as a workaround.

The function, 'monitor_memory_whos.m' operates by using the WHOS command and evaluating it within the 'base' workspace. Each variable's memory usage is summed up and converted into megabytes. This function can be run in the background without displaying data to the MATLAB command prompt. However, the memory usage of the workspace is not the only memory used by MATLAB. Typically, the program starts up using approximately 500 MB of memory.

To run the 'monitor_memory_whos.m' function please type the following the MATLAB command prompt:

A = magic(1000);
B = phantom(500);
C = peaks(250);
in_use = monitor_memory_whos

The memory function can also be used to monitor memory usage on Windows systems. For more information on memory management, please refer to the following link:

https://www.mathworks.com/help/matlab/performance-and-memory.html

[Thanigaivel Raja T]

The monitor_memory_whos.m gives memory usage in units of? And what memory elements does it take into account?

[Kevin Gleason]

Short Answer: MB, and values in the workspace.

The monitor_memory_whos.m relies on evaluating the "whos" function in the base workspace.

Try executing "whos" in your command window. It displays the bytes of all elements in the workspace. The monitor_memory_whos script add those byte values and divides by 2^20 to display workspace memory in MB.

Answer 2

[Jan]

Note that "memory usage" is not well defined. When the code let an array grow iteratively, it requests new memory in each iteration:

v = [];
for k = 1:1e6
  v(k)= rand;
end

Although the final array uses 8MB only, Matlab requests sum(1:1e6)*8 = 500GB of memory from the OS. Of course Matlab releases the memory, but the OS waits until it finds time to clear the memory and overwrite it with zeros. Therefore even this cute loop can exhaust the RAM. But do you consider the released and not yet cleared RAM as "occupied by Matlab" or not?

If you open a lot of files simultaneously in Matlab, the memory for caching reserved by the OS can grow to a remarkable size. This is caused by Matlab, but the memory does not belong to Matlab in the taskmanager. Matlab can store data on the graphic card also.

If several threads, e.g. in parfor, access data in the same cache-line (usually a 64 byte size block of memory), the total performance can degrade drastically. This could be considered as "memory intensive task" also, although it concerns a tiny memory block only.

In consequence "memory usage" is as vague as "processor time" in modern operating systems.

[Michael Werth]

Okay, I can understand the logic here, but is MATLAB really not clearing the size-1 array after creating the size-2 array? That's nuts! Is there any reason for Matlab to hold onto that memory when those earlier-iteration arrays each become immediately inaccessible on the next iteration? I knew that growing an array was a bad idea, but apparently it's an even worse idea than I thought.

[Walter Roberson]

MATLAB keeps a "small block" memory pool and will clean it out as needed. I do not know if it will re-use the same location over and over or if it uses a more round-robin strategy.

Answer 3

[Mark T]

Hi, here is one option for Windows that seems to work:

function memory_kb=get_process_memory1(process)

if ~exist('process','var')
    process='Matlab.exe';
end

command=sprintf('tasklist /nh /fi "imagename eq %s"',process);
[status, result] = dos(command);
result=textscan(result,'%s');

m=result{1}{5};

% Deal with the separators - sigh...
% German OS
local_dec_sep=',';
local_1000_sep='.';

i_1000=find(m==local_1000_sep);
i_dec=find(m==local_dec_sep);

if ~isempty(i_dec),
    m_dec=m(i_dec+1:end);
    m=m(1:i_dec-1);
else
    m_dec='';
end

m(i_1000)='';
m=[m '.' m_dec];

memory_kb=str2num(m);

You will need to switch the ',' and '.' for English OS. Could easily upgrade to handle multiple processes with the same name. A shame it takes 100 ms on my machine...

Answer 4

[Qifan]

Task Manager, memory usage are shown under the process section.

Answer 5

[Samuel Gray]

"I have a program that is memory intensive and I want to monitor how much memory MATLAB is using so that if it goes above a certain threshold, I can stop the program."

Try setting the MATLAB workspace preferences, the array size-limit might be suitable for use as a control.

Though this will stop the script before the limit actually is violated, e.g. if you tell it to create an NxM array in a script and given the class this would require say 900GB of ram in a system with < 900GB of ram, with the array size limit active, not using tall arrays etc., Matlab will throw an exception and stop the script without actually creating the array. It will incrementally increase an existing array in size until the limit is threatened by the next instruction to be executed. It will allow the total memory used by Matlab to exceed that limit, though, which can be wrapped in a try..catch loop for elegant resolution. This might be a functional workaround which requires minimal additional coding.

Of course you could code a test for this on your own and insert it before any step that you feel might be a problem.

e.g.

Mlim=20;Nlim=200;
M=1;N=1;
while M*N=50e9 && M< Mlim && N < Nlim
    a=rand(M,N);
    M=M+1;N=N+1;
end

not going to resolve a problem with intermediate calculations that would trip the limit,

like mean(max(sum(a*a')))

but at least it would be a check on the final result...

if you want a check on the intermediate calculations, just use a try-catch Me loop and examine the error returned

M=1e2;
for n=1:5;
    N=n*10e6;
    try,
        m=mean(max(sum(rand(M,N)*rand(N,M)))),
        fprintf('success!\n'),
        clear Me
    catch Me,
        fprintf('fail!\n'),Me,
    end;
    pause(.1);
end

...this is enough to put my system in a confused state...it will take the available memory down to 30Mb or so...on a 16GB i5...

Task Manager (and the processes that use it) will never show that intermediate memory level triggered failure because the ram required to trigger the failure won't actually get used if the code would violate the limit in trying to use it. But if not, then Matlab will happily execute the code...even if the result is that 99.999% of available ram gets used...

The command does not fail because the required trigger memory never gets used and so it does not crashes the system. MATLAB will screen the code for the potential problem and abort before actually executing such code. The point is to keep that condition from happening in the first place, but that is not the only condition that is a problem. Any machine-state with low responsiveness is a problem unless you are willing to go have a cup of coffee or something. Before MATLAB was "improved", it would happily create the requested array even if it meant using "disk memory" (from a swapfile, the opposite of a ramdisk) and you'd have to reboot your system to clear the RAM and get it to respond...MATLAB wouldn't actually crash the system as long as there was sufficient "disk memory" available to complete the command but the system was so slow to do anything that you'd want it to crash. That is why the above limit is based on installed physical ram, not "available ram". This still will not resolve the problem completely as it won't track either non-Matlab use of system memory or "available ram" (with or without a swapfile). And if you have a swapfile (or as Win10 likes to call it, a "paged pool") then you're toast because then the OS will happily swap application and even OS memory out to disk in order to free-up ram for Matlab. That is more "robust" but it just changes what has been swapped-out by the OS, not the fact that the OS had to resort to a swap-out in the first place. The ideal solution of course is to be aware of how much ram (code and data) an instruction will require to complete before actually asking MATLAB to run the command, and to simply adjust the parameters (and kill-off any unnecessary background apps) before calling an instruction so that it will run comfortably in the available memory space. Failing that, double the amount of ram in your system until this is no longer a problem. If it needs 6GB and you only have 4, going to 8 will solve the problem.

[Walter Roberson]

The preference is for the size of one individual array. You can easily create a cell or struct or table that would exceed the preference, in which case it is up to the operating system to handle the situation.

[Samuel Gray]

I hadn't known that, I'd assumed that it would apply to any type of array, including cells & structs, unless they were designed to exceed installed memory, like tall arrays or some type of array that is distributed across many workstations.

Let's just call this a good introduction to application memory management.