Clear Filters
Clear Filters

Multiplication of gigantic matrices

1 view (last 30 days)
carlos g
carlos g on 6 Mar 2023
Commented: Steven Lord on 7 Mar 2023
Hi,
I am trying to multiply two gigantic matrices: [1585152 90]*[90 1585152]. MATLAB runs out of memory. Is there any workaround for this?
  5 Comments
John D'Errico
John D'Errico on 6 Mar 2023
Do you have the memory? The size of your result will be:
1585152*1585152
elements. Multiply that by 8, assuming doubles. Converting to rough gigabytes:
1585152*1585152*8/2^30
So only 18700 gigabytes, or 18.7 terabytes of RAM.
Again, do you have that much RAM? I'm not talking about disk space, where even that would be seriously big. Even in these days of large chunks of cheap RAM, 18 terabytes is flat out significantly big.
Unless your arrays are seriously sparse matrices, even converting them to sparse will not help much.
Far better is to do as has been suggested, to NOT do this. A little linear algebra will show you how to do much with the original arrays. And, worse, when you do that multiplication, you actually make things numerically pooly behaved. There are many good reasons to not do what you want to do. And one good reason why you CANNOT do it, because of the memory requirenemt.
Just because you have a formula that shows that product of matrices, does not mean it is a good idea to write code following that formula.
Steven Lord
Steven Lord on 7 Mar 2023
What were you hoping to do with this extremely large matrix? There may be alternate approaches that don't require nearly so much memory. For example, if you were hoping to solve a system using the normal equations don't do that. Use mldivide, \ or one of the iterative methods for solving a system.

Sign in to comment.

Answers (1)

Allen
Allen on 7 Mar 2023
You can use datastores and tall arrays to work with large amounts of data. See references to both.
  1 Comment
Bruno Luong
Bruno Luong on 7 Mar 2023
See John's comment, it need 18.7 Tb to store it, without looking to time to work with huge matrix.
Explicit multiplication is a bad idea that will lead pratically to nowhere.

Sign in to comment.

Categories

Find more on Linear Algebra in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!