Maximum memory of MATLAB for 512GB workstation

If I have a workstation with 512GB for running MATLAB, can MATLAB utilize large amount of the memory, such as >400GB, to run the scripts?

[Mohammad Sami]

Yes 64bit version of matlab can utilise the memory.

https://www.mathworks.com/matlabcentral/answers/92518-what-are-the-benefits-of-64-bit-matlab-versus-32-bit-matlab

[Zijian Deng]

Thanks for your answer and the link!

Answer 1

[Walter Roberson]

Yes. The limit for 64 bit MATLAB is 262144 gigabytes per variable (really 262144 gigabyte total between all of the variables.)

This is 2^48 bytes.

The reason 2^48 bytes was chosen as the limit is that current versions of the x64 architecture only have at most 48 address lines; the architecture would have to be revised for more than 2^48 bytes.

[Zijian Deng]

Thanks a lot for your detailed answer! Sometimes I used my own computer (64GB) to run the MATLAB for large matrix calculation, the MATLAB would show "out of memory", but I checked the RAM and it was not exhausted. Could you please show a bit more details about how MATLAB apply memory? We worry that even we have 512GB (we are planning to buy a workstation to run calculation), the MATLAB may not apply sufficient memory from RAM to run huge matrix calculation.

[Walter Roberson]

MATLAB shows "out of memory" at the time it is asked to allocate memory. It sends a request to the operating system for a single chunk of memory of a given size, and the operating system returns a failure indication and does not allocate what was available to MATLAB. In some cases the operating system will try to allocate the memory, go through and swap to disk, and so on, and other cases the operating system will simply look at the size of the request and say that it cannot possibly allocate that much.

On Windows PCs, the operating system examines the size of the configured swap space to decide whether swap space could be grown large enough to meet the request.

Also, it depends on what you have set in your MATLAB Preferences for checking memory allocations. You can put a limit on how big of a chunk of memory that MATLAB is willing to ask the operating system for; you can also disable that limit so that it becomes up to the operating system to respond.

In each case, by the time you receive the "out of memory" error message, either MATLAB did not even try to allocate the memory (because you hit the limit you configured in Preferences), or else MATLAB asked the operating system for the memory and the operating system was not able to provide the memory and told MATLAB so without leaving any of the memory allocated to MATLAB. So by the time you get the message, the task monitor would already not show any of it allocated to MATLAB, because either it was not tried or the operating system already reclaimed it.

[Walter Roberson]

If you are monitoring memory usage right at the time of the memory allocation request, you might see a blip. In some cases there might even be a long delay as the operating system attempts to allocate swap space on disk.

[Zijian Deng]

Thanks a lot for your answer! it is very helpful for us to understand how MATLAB allocate memory. Really appreciate it.

[Guillaume]

"We worry that even we have 512GB [...] the MATLAB may not apply sufficient memory from RAM to run huge matrix calculation"

If you're considering matrices large enough that you need such a large amount of memory, my main worry would be how long it would take to run the calculation. Most matlab core functions can take advantage of multiple processors/threads but generally your m code will not.

[Zijian Deng]

Thansk for your attention. Yes, we usually spent more than 10, sometimes dozens of hours, to run this calculation. Therefore, we are now also trying to find a way to speed up the calculation, such as incorporating with the GPU acceleration.

[Walter Roberson]

If you are going to use GPU, then Linux is recommended. The Windows interface has memory management overhead that slows down GPU use, and also the Windows interface reserves about 1 gigabyte of memory on the GPU. Also volta drivers allocate more memory than expected (even on Linux)

https://www.mathworks.com/help/parallel-computing/gpu-support-by-release.html

https://www.mathworks.com/matlabcentral/answers/378937-titan-v-using-huge-amount-of-memory

You would also want to be using a GPU that can be put into TCC mode -- which implies that you have some other graphics card as well to handle the display.

The RTX 2080 has only 8 Gigabytes of memory, and with various intermediate copies of data needed, if your output array is the same size as your input array, then your input array can be approximately 1/3 of your GPU RAM before running into memory problems. With your arrays being so large, you would need to use an algorithm that split your computation into sections.

If your algorithm uses double precision and you cannot switch to single precision, then you need to be quite careful in selecting GPU, as they differ quite a bit in double precision performance. The fastest single precision performance does not always translate to the fasted double precision performance, as NVIDIA has at least 3 different double precision implementations that I know of (I think there was a 4th one as well in the early days.)

[Zijian Deng]

Thanks a lot for such detailed introduction! It would be very useful for us to proceed with GPU. Really appreciate it.

[Walter Roberson]

In your situation it might be worth using the GPU Coder product for further performance increase.

It might also be worth using multiple GPUs. However you should only use multiple GPUs if you also use multiple workers such as parfor: there is a lot of overhead if a process accesses more than one GPU, or a GPU is accessed by more than one process.

https://www.nvidia.com/en-us/deep-learning-ai/products/titan-rtx/ Titan RTX Professional has 24 Gigabytes of memory; also the Quadro RTX 6000. The non-graphics GV100 supports 32 Gigabytes of memory, but it is much much more expensive.

[Zijian Deng]

Sorry for the late response, and thanks a lot for your suggestions on GPU! We are now still looking for a best solution for our requirements, and your suggestions would be very useful for us. Really appreciate it.