matlab Out of memory . Type HELP MEMORY for your options

I use pcacov command in matlab with pcacov(20000*20000 matrix) input parameters, but matlab can't handel it with memory and show error:

pcacov matlab Out of memory. Type HELP MEMORY for your options.

How can i solve this problem with coding or setting in the matlab and with out add or change any hardware memory or change pc.

Answer 1

[John D'Errico]

Well, it seems clear. You cannot use pcacov. You need more memory, and you refuse to do that.

You could learn about how to perform the analysis yourself, perhaps using a tool like eigs to get only a few eigenvalues and eigenvectors, instead of all of them. It should even be doable using even as simple a tool as the power method, combined with deflation to kill off the largest eigenvalues. A virtue of the power method is you automatically know when to stop looking for another eigenvalue, since you only need the largest few eigenvalues. A problem is potentially very poor convergence if two eigenvalues are close to each other.

But it should be quite doable. You will need to learn how to do the computations though, and I won't write the code for you.

Answer 2

[Jan]

A 20000*20000 matrix uses 3.2GB of RAM - not only free RAM, but this block must be available in one contiguous block. I do not know what happens inside pcacov, but clearly it requires some memory for the computations also. You can reduce the chance, that the memory is exhausted in this function, if you provide as much free RAM as possible:

• Install a 64 bit version of Matlab and OS
• Close other applications (no internet connection, no USB sticks ==> No virus scanner)
• Reduce the e.g. the shared memory used by the graphics board
• Clear unused variables in Matlab
• Use single instead of double, if the precision is enough for your data
• But most of all the only and best solution is to install more RAM. Nothing can beat a hardware upgrade.

I've read that you do not want to install more RAM. But think twice: You have a problem, that requires a lot of RAM. Then refusing to install a lot of RAM is not a scientific way of working with a computer. There is no magic "do it with the half RAM" trick.

You can hire a server for the required number of minutes or hours, which has enough RAM. Because all such services are commercial, I will not post the links to them here - but your favorite internet search engine will find them soon.

I was successful with asking a colleague, if I can use his 128 GB/16 core machine for half an hour.

[John D'Errico]

A quick test shows that even my 8 GB, 4 core machine is sufficient to compute the few necessary eigenvalues and eigenvectors using tools like eigs.