Inverse huge matrix with tall array

Hi,

I am trying to find x in my system Ax=b. A is a huge matrix, it is a size of 10 billion *9, and out of my memory, b is 10 billion *1 vector.

Therefore, I am using tall array, while I don know how to compute the inverse of A? (pseudo inverse)

One answer is here, while it is incorrect: https://uk.mathworks.com/matlabcentral/answers/530823-taking-the-inverse-of-a-tall-array-to-solve-a-linear-system-of-equations

A = tall(ds);
b_t = tall(ones(1,1));
SOL = gather(A\b);

This code works with SOL is 9*1 vector, however, the vector b_t has all elements equal to 1.

If I use my verctor b, it shows an error with incompatible tall array argument in tall/qrLeftSolve.

Any one can help this case?

Thanks in advance.

Answer 1

[halleyhit]

Hi chen, I am writing some lines code and find they are working fine. You can directly solve equeation AX=B or get x_inv by tall array. Would you please share more information, like MATLAB version and full error message?

A=tall(rand(100,100));
B1=tall(rand(100,1));
X=A\B1; % solve X in equation A*X=B1
B2=tall(eye(100));
X_inv=A\B2; % solve inv(X)

X=gather(X)

Evaluating tall expression using the Local MATLAB Session:
- Pass 1 of 1: 0% complete
- Pass 1 of 1: Completed in 0.45 sec
Evaluation completed in 0.79 sec

X = 100x1

0.3758 0.0945 1.6328 0.5906 -0.6462 0.2529 1.8940 1.8789 -0.1144 0.0478

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

X_inv=gather(X_inv)

Evaluating tall expression using the Local MATLAB Session:
- Pass 1 of 1: 0% complete
- Pass 1 of 1: Completed in 0.12 sec
Evaluation completed in 0.28 sec

X_inv = 100x100

0.7743 -0.0693 0.8170 0.8577 -0.3118 0.3562 -0.9650 -0.7043 1.7218 -1.4824 -0.3475 -0.1169 0.8369 0.6821 0.7398 0.2457 -0.4201 0.1088 0.1201 -0.2784 -0.0559 -1.2019 -0.5790 -0.3116 0.0530 -0.0006 0.3367 0.9886 -1.0026 -0.1638 -0.0558 0.4031 0.0743 -0.3376 0.2502 -0.0059 -0.4564 0.2059 -0.6678 -0.0947 -0.0655 0.4893 -0.3262 -0.0874 -0.4319 -0.5594 0.1062 0.0800 -0.5376 0.1450 -0.1239 0.2719 0.1130 -0.3080 -0.2385 0.8645 -0.9948 -0.2167 0.1881 0.0734 -0.0784 0.1249 0.2214 -0.3601 0.2519 -0.1688 -0.2442 -0.2114 -0.6418 0.4962 0.1870 -0.0858 -0.1424 -0.2332 -0.6112 -0.1980 -0.3017 0.5306 -0.4729 -0.0702 -0.1993 -0.3099 -0.0528 -0.1944 -0.1935 0.5906 -0.3473 -0.4316 0.2758 0.0678 0.1676 -0.4320 0.4946 0.1798 -0.4843 -0.1374 0.0295 -0.1248 0.8595 -0.2044 -0.2073 -0.3594 0.5226 -0.0816 0.6570 0.5772 -0.1445 0.1138 0.3007 -0.0707 -0.0532 -0.2033 -0.3197 -0.0642 0.2278 -0.7270 0.5173 0.4822 -0.1869 -0.0182 -1.0476 0.2718 -1.6123 -0.8449 0.4432 -0.3391 2.0997 1.0481 -1.5932 1.9536 0.6653 0.2580 -0.8852 -0.7632 -1.1088 -0.2306 0.3132 -0.4577 -0.2821 0.2983 0.3981 1.8093 0.9012 1.0211 0.2953 -0.9724 -0.1735 -1.0113 0.8892 0.0465 -0.1037 0.5654 -0.1123 -0.3934 0.4696 0.1766 -0.2697 0.2524 -0.6914 0.1512 0.0448 0.3207 -0.4046 -0.0659 -0.6561 -0.4781 -0.1712 -0.1590 -0.2617 0.0846 0.1668 0.3875 0.2468 -0.0771 -0.4338 0.5762 -0.6102 -0.3239 0.3225 0.1269 -0.0470 0.2204 -0.9893 -0.5586 0.7420 -0.8195 1.3611 0.7751 -1.1705 1.5683 0.4142 0.4074 -0.6538 -0.4146 -0.7123 -0.2845 -0.1614 0.2445 -0.9458 0.3083 0.2387 1.3137 0.4728 0.6883 0.0753 0.0331 -0.7331 -0.5261 -0.0115 0.4966 0.6588 -0.6303 1.2606 0.5455 -0.4444 0.0983 -1.5520 -1.2083 1.4617 -1.1599 -0.4837 -0.3701 0.6419 0.6479 0.8795 0.4380 -0.6280 0.5090 0.2999 -0.4201 -0.3082 -1.9395 -0.8663 -0.8497 -0.1915 0.5109 0.5848 0.8014 -0.6520 -0.0784 -1.1458 0.2776 -0.8118 -0.8092 0.5263 -0.2181 0.5056 0.2499 -1.7848 1.4072 0.7064 0.0356 -0.8161 -0.5013 -1.4115 -0.3823 -0.0601 0.0124 -0.2746 0.0059 -0.2659 0.3024 0.4954 0.3484 -0.1463 0.3013 -0.2698 -1.1395 0.9805 -0.3267 -0.5385 0.7469 -0.4072 -0.7453 0.7073 0.1720 -0.2972 0.1257 -1.7257 0.7864 0.7099 0.4504 -0.7276 -0.0966 -1.5364 -0.8592 -0.3177 0.2224 -0.6054 -0.1317 -0.1317 0.0955 0.4519 -0.2018 -0.4185 0.9981 -0.6300 -0.9723 0.7656 -0.1510

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

[chen zhe]

Hi halleyhit,

Thanks for your answer. It works well for me.

But I have another issue in pinv, if I need a Pseudo inverse.

X=pinv(A)*B1; % solve X in equation A*X=B1

X=gather(X);

The error shows incorrect number or type for function diag().

Do we have any good solution for this pinv()?

Thanks in advance.

[halleyhit]

as dos shows, pinv does not support tall array

Answer 2

[Ayush Modi]

Hi Chen,

"If I use my verctor b, it shows an error with incompatible tall array argument in tall/qrLeftSolve."

Operator '\' requires both the inputs to be of same type. If either one is tall array and the other one is not, this error will be thrown. To resolve the error, you would need to convert b vector into a tall array.

Here is the sample code for your reference:

% Defining two random matrix
A = rand(100,100)

A = 100x100

0.9947 0.4626 0.6908 0.9572 0.3930 0.2770 0.4175 0.1024 0.9190 0.9325 0.8319 0.8676 0.5394 0.6506 0.8565 0.8859 0.9501 0.9520 0.2930 0.4328 0.5271 0.1691 0.2264 0.2029 0.5648 0.0718 0.9155 0.5690 0.0737 0.3573 0.0923 0.2488 0.7940 0.2614 0.8866 0.7586 0.4054 0.6688 0.5740 0.3765 0.7618 0.2080 0.1918 0.5318 0.7129 0.9550 0.4707 0.9276 0.0998 0.5540 0.7819 0.4010 0.8166 0.4177 0.1151 0.4744 0.3886 0.6050 0.6691 0.9335 0.0192 0.5947 0.6057 0.9769 0.7346 0.8659 0.5971 0.8688 0.4415 0.4081 0.3207 0.3461 0.0195 0.1080 0.9258 0.0734 0.2192 0.3056 0.8795 0.5699 0.1733 0.7057 0.6939 0.8609 0.3452 0.2789 0.9620 0.2709 0.1687 0.0543 0.0302 0.7422 0.7112 0.3936 0.1013 0.8690 0.9021 0.8979 0.7237 0.5793 0.3438 0.3597 0.6744 0.1376 0.2294 0.5965 0.6801 0.2358 0.0972 0.6335 0.5269 0.8279 0.6645 0.2513 0.0440 0.1707 0.9959 0.6991 0.8024 0.3426 0.5075 0.7772 0.1627 0.6762 0.6031 0.6545 0.7705 0.3404 0.4068 0.9728 0.8714 0.3318 0.0450 0.7205 0.8281 0.0297 0.5191 0.5438 0.6343 0.3165 0.5878 0.9132 0.9126 0.8596 0.8247 0.0339 0.4097 0.7741 0.0685 0.4046 0.0702 0.9067 0.2362 0.9370 0.2169 0.0860 0.8906 0.7138 0.4679 0.5370 0.5455 0.1653 0.3492 0.2454 0.8324 0.1050 0.2765 0.9222 0.0056 0.3227 0.7864 0.9552 0.3336 0.6613 0.0659 0.7436 0.2887 0.8740 0.7749 0.9455 0.5397 0.2529 0.7490 0.6075 0.7590 0.4994 0.8931 0.9234 0.2500 0.1606 0.8784 0.5149 0.1548 0.2430 0.5835 0.8846 0.0499 0.2962 0.8501 0.6615 0.2866 0.0072 0.9357 0.1078 0.1873 0.4558 0.5208 0.8972 0.9301 0.1669 0.5393 0.8417 0.4028 0.8963 0.9119 0.8722 0.5371 0.8855 0.2501 0.0766 0.1744 0.5684 0.1131 0.2407 0.8461 0.0656 0.9170 0.9014 0.1651 0.3962 0.1488 0.3718 0.7011 0.1383 0.8359 0.8441 0.6202 0.0033 0.6855 0.6126 0.1507 0.4941 0.0794 0.4621 0.2253 0.4462 0.3381 0.9387 0.4232 0.8856 0.9501 0.0953 0.7915 0.1792 0.6617 0.6468 0.9575 0.4384 0.1039 0.6810 0.3056 0.1640 0.5208 0.2375 0.3266 0.8771 0.6450 0.7747 0.9686 0.1680 0.7484 0.2843 0.4546 0.6945 0.6782 0.4360 0.6349 0.4398 0.5671 0.7586 0.3504 0.3579 0.2416 0.3331 0.8744 0.4183 0.3727 0.0479 0.9523 0.9842 0.7074 0.0672 0.3116 0.6295 0.6847 0.3250 0.4385 0.5122 0.3505 0.7533

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

b = rand(100,1)

b = 100x1

0.5972 0.5849 0.6938 0.5226 0.1276 0.1076 0.8850 0.6889 0.6440 0.6508

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

% Converting them to tall arrays
tallA = tall(A)

tallA =

  100x100 tall double matrix

  Columns 1 through 18

    0.9947    0.4626    0.6908    0.9572    0.3930    0.2770    0.4175    0.1024    0.9190    0.9325    0.8319    0.8676    0.5394    0.6506    0.8565    0.8859    0.9501    0.9520
    0.0923    0.2488    0.7940    0.2614    0.8866    0.7586    0.4054    0.6688    0.5740    0.3765    0.7618    0.2080    0.1918    0.5318    0.7129    0.9550    0.4707    0.9276
    0.0192    0.5947    0.6057    0.9769    0.7346    0.8659    0.5971    0.8688    0.4415    0.4081    0.3207    0.3461    0.0195    0.1080    0.9258    0.0734    0.2192    0.3056
    0.0302    0.7422    0.7112    0.3936    0.1013    0.8690    0.9021    0.8979    0.7237    0.5793    0.3438    0.3597    0.6744    0.1376    0.2294    0.5965    0.6801    0.2358
    0.5075    0.7772    0.1627    0.6762    0.6031    0.6545    0.7705    0.3404    0.4068    0.9728    0.8714    0.3318    0.0450    0.7205    0.8281    0.0297    0.5191    0.5438
    0.0702    0.9067    0.2362    0.9370    0.2169    0.0860    0.8906    0.7138    0.4679    0.5370    0.5455    0.1653    0.3492    0.2454    0.8324    0.1050    0.2765    0.9222
    0.5397    0.2529    0.7490    0.6075    0.7590    0.4994    0.8931    0.9234    0.2500    0.1606    0.8784    0.5149    0.1548    0.2430    0.5835    0.8846    0.0499    0.2962
    0.5393    0.8417    0.4028    0.8963    0.9119    0.8722    0.5371    0.8855    0.2501    0.0766    0.1744    0.5684    0.1131    0.2407    0.8461    0.0656    0.9170    0.9014
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :

  Columns 19 through 36

    0.2930    0.4328    0.5271    0.1691    0.2264    0.2029    0.5648    0.0718    0.9155    0.5690    0.0737    0.3573    0.6437    0.8474    0.7992    0.8280    0.0741    0.1939
    0.0998    0.5540    0.7819    0.4010    0.8166    0.4177    0.1151    0.4744    0.3886    0.6050    0.6691    0.9335    0.4819    0.5598    0.4953    0.4115    0.4782    0.3217
    0.8795    0.5699    0.1733    0.7057    0.6939    0.8609    0.3452    0.2789    0.9620    0.2709    0.1687    0.0543    0.0242    0.3062    0.0516    0.6704    0.0306    0.7953
    0.0972    0.6335    0.5269    0.8279    0.6645    0.2513    0.0440    0.1707    0.9959    0.6991    0.8024    0.3426    0.4167    0.6541    0.2446    0.0995    0.2396    0.4477
    0.6343    0.3165    0.5878    0.9132    0.9126    0.8596    0.8247    0.0339    0.4097    0.7741    0.0685    0.4046    0.8334    0.2130    0.6875    0.6769    0.3877    0.4635
    0.0056    0.3227    0.7864    0.9552    0.3336    0.6613    0.0659    0.7436    0.2887    0.8740    0.7749    0.9455    0.5906    0.3272    0.5108    0.3575    0.0480    0.4427
    0.8501    0.6615    0.2866    0.0072    0.9357    0.1078    0.1873    0.4558    0.5208    0.8972    0.9301    0.1669    0.3334    0.5827    0.3716    0.8463    0.3661    0.2470
    0.1651    0.3962    0.1488    0.3718    0.7011    0.1383    0.8359    0.8441    0.6202    0.0033    0.6855    0.6126    0.3333    0.6425    0.1980    0.4854    0.8663    0.5186
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :

  Columns 37 through 54

    0.5355    0.0628    0.5854    0.7285    0.3470    0.4697    0.0952    0.2652    0.5367    0.3317    0.5564    0.9495    0.7961    0.8552    0.7955    0.6598    0.0790    0.2758
    0.1317    0.1247    0.5925    0.3688    0.3156    0.2509    0.2378    0.4523    0.1257    0.3662    0.2784    0.7689    0.7046    0.9002    0.3912    0.0351    0.6458    0.8515
    0.0865    0.6577    0.8280    0.9986    0.2409    0.1537    0.6579    0.1930    0.0249    0.4879    0.7781    0.6314    0.9368    0.0240    0.0143    0.2097    0.7062    0.7341
    0.9884    0.2811    0.9029    0.6874    0.0348    0.4320    0.6434    0.8410    0.5180    0.4068    0.9110    0.1273    0.4360    0.5249    0.4062    0.9801    0.6685    0.9761
    0.0288    0.8181    0.5334    0.4100    0.8383    0.0385    0.5114    0.2457    0.0255    0.4325    0.5701    0.6968    0.2131    0.8715    0.9579    0.5394    0.1259    0.3427
    0.6658    0.1941    0.3923    0.7473    0.8542    0.1991    0.5978    0.0734    0.1934    0.6157    0.8895    0.7805    0.9647    0.7126    0.5998    0.1779    0.3588    0.6053
    0.4122    0.1348    0.8091    0.4853    0.9496    0.1586    0.8800    0.0572    0.1071    0.4716    0.7755    0.0318    0.7086    0.1122    0.3572    0.6548    0.3940    0.6840
    0.7034    0.2817    0.6467    0.2609    0.2060    0.7199    0.8099    0.2952    0.0169    0.8177    0.3036    0.2856    0.1215    0.2466    0.9243    0.0432    0.3923    0.5641
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :

  Columns 55 through 72

    0.1232    0.8125    0.3342    0.3624    0.6900    0.7983    0.9980    0.8447    0.7832    0.7575    0.0067    0.1952    0.2662    0.5321    0.0664    0.9825    0.9733    0.8893
    0.0374    0.8396    0.6496    0.5587    0.9625    0.6297    0.3435    0.6432    0.2192    0.5261    0.8477    0.8670    0.6865    0.4448    0.0893    0.0777    0.3255    0.2446
    0.7908    0.6795    0.1366    0.4217    0.7251    0.2381    0.5144    0.5793    0.6458    0.7338    0.7580    0.5616    0.4369    0.6664    0.6267    0.3804    0.3380    0.2379
    0.5076    0.2598    0.8806    0.6885    0.9630    0.9466    0.9165    0.0427    0.4639    0.2303    0.8713    0.6334    0.5789    0.5165    0.1985    0.1758    0.5908    0.5495
    0.9266    0.6116    0.9526    0.6330    0.8698    0.4270    0.3496    0.9524    0.4305    0.7161    0.9226    0.2925    0.0856    0.7509    0.1776    0.9825    0.3673    0.4231
    0.3078    0.9878    0.5069    0.3448    0.8062    0.8439    0.0511    0.2587    0.2708    0.8813    0.3075    0.0586    0.4917    0.8142    0.1890    0.0054    0.9303    0.4693
    0.9174    0.2580    0.3145    0.4633    0.1351    0.6166    0.6503    0.9601    0.2325    0.5454    0.5390    0.4688    0.0713    0.5822    0.1256    0.0251    0.5389    0.3943
    0.8867    0.6729    0.7401    0.9365    0.5946    0.3014    0.0888    0.9111    0.3976    0.8435    0.8404    0.6172    0.3288    0.5484    0.8531    0.1393    0.3151    0.2739
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :

  Columns 73 through 90

    0.1418    0.4728    0.5474    0.4756    0.7492    0.7575    0.9610    0.9560    0.3065    0.2035    0.2640    0.2454    0.1366    0.5136    0.2233    0.1296    0.6005    0.8043
    0.1893    0.4862    0.4133    0.2853    0.9044    0.8362    0.7122    0.3697    0.4223    0.9724    0.4888    0.3134    0.2218    0.4102    0.4233    0.1391    0.1185    0.7081
    0.0672    0.4929    0.4596    0.5012    0.8577    0.7600    0.2484    0.3413    0.3446    0.0896    0.4449    0.8577    0.8031    0.8912    0.0473    0.3659    0.5192    0.9020
    0.8129    0.0201    0.7949    0.7919    0.3383    0.9086    0.1702    0.7679    0.5813    0.3368    0.8705    0.3670    0.9006    0.9876    0.4270    0.3445    0.5984    0.8060
    0.5353    0.4185    0.3806    0.4322    0.0480    0.7821    0.2039    0.3469    0.5749    0.2550    0.5728    0.0580    0.8129    0.2231    0.5661    0.7490    0.9359    0.3340
    0.4876    0.9417    0.9716    0.1828    0.2480    0.4886    0.1627    0.5514    0.1248    0.7397    0.3727    0.1695    0.9214    0.4645    0.9173    0.5430    0.6221    0.0176
    0.5162    0.1587    0.7328    0.5016    0.3008    0.5404    0.9609    0.4512    0.3910    0.8881    0.9625    0.0213    0.6435    0.8747    0.2997    0.8007    0.4106    0.3149
    0.8028    0.6989    0.2706    0.2003    0.8490    0.4253    0.8486    0.8889    0.2723    0.4114    0.3909    0.1060    0.2967    0.8794    0.5920    0.0155    0.5504    0.6084
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :
      :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :         :

  Columns 91 through 100

    0.3253    0.8026    0.1349    0.1779    0.5445    0.7229    0.8003    0.1008    0.3925    0.6121
    0.3294    0.3681    0.5884    0.7623    0.0899    0.5957    0.0224    0.3019    0.7777    0.6986
    0.3077    0.1976    0.9783    0.5364    0.0945    0.2875    0.4918    0.1006    0.7761    0.0023
    0.6713    0.1236    0.1948    0.5170    0.1872    0.9505    0.3202    0.6634    0.0846    0.9351
    0.6131    0.7419    0.9973    0.6152    0.3016    0.2948    0.8884    0.6059    0.3350    0.1469
    0.5383    0.3988    0.6944    0.4074    0.4004    0.8078    0.5827    0.2220    0.3726    0.4620
    0.8897    0.4469    0.8567    0.9446    0.6511    0.4413    0.7349    0.8421    0.3107    0.8229
    0.5451    0.0544    0.0301    0.8424    0.8300    0.1674    0.7776    0.5627    0.4716    0.4457
      :         :         :         :         :         :         :         :         :         :
      :         :         :         :         :         :         :         :         :         :

tallb = tall(b)

tallb =

  100x1 tall double column vector

    0.5972
    0.5849
    0.6938
    0.5226
    0.1276
    0.1076
    0.8850
    0.6889
      :
      :

% We can do operation A\b
A\b

ans = 100x1

-2.1302 -0.6963 2.2138 -4.1181 3.3345 -0.1259 2.6496 -4.5379 4.1526 -0.1002

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

% We can also do tallA\tallb
gather(tallA\tallb)

Evaluating tall expression using the Local MATLAB Session:
- Pass 1 of 1: 0% complete
- Pass 1 of 1: Completed in 0.16 sec
Evaluation completed in 0.3 sec

ans = 100x1

-2.1302 -0.6963 2.2138 -4.1181 3.3345 -0.1259 2.6496 -4.5379 4.1526 -0.1002

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

% ERROR - if they are different
gather(tallA\b)  % Note b is matrix, not tall array 

Error using tall/qrLeftSolve (line 16)
Incompatible non-scalar tall array arguments. Each of the tall arrays must be the same size in the first dimension, must be derived from a single tall array, and must not have been indexed differently in the first dimension (indexing operations include functions such as VERTCAT, SPLITAPPLY, SORT, CELL2MAT, SYNCHRONIZE, RETIME and so on).

Error in \ (line 29)
[~,Z] = qrLeftSolve(X,Y);

[chen zhe]

This much clear. Thanks Ayush.

My another issue is pinv, if I need a Pseudo inverse.

X=pinv(A)*B1; % solve X in equation A*X=B1

X=gather(X);

The error shows incorrect number or type for function diag().

Do we have any good solution for this pinv()?

Thanks in advance.