Create a multidimensional matrix from very large set data?

Hello Matlab Community,

I am kind of learning about multidimensional matrix, and I would like to ask you how I can repeat the next routine five times, by every column of the data that is attached to this message (file 'data_2.mat'), and after that, append every new matrix to a multidimensional matrix with the next dimensions: 41, 55, 5.

Col=data_2(:,1);
Mtx_1 = reshape(Col, 55, []);  % To (55 x 41)
Mtx_2 = Mtx_1';     % To (41 x 55)
Mtx_3 = flipud(Mtx_2);   % desired data order

I think I need a for cycle, and I have done a great effort in order to find the answer, but I can't be able to solve this issue. On the other hand, I also would like to comment that it is not desired any kind of interpolation, because of the sensitivity of data interpretation (difference of decimals is important).

I really appreciate your support, and your workaround would be applicate to a large set o data (actually, my real data has 2255 rows by 30000 columns).

Best,

Miguel

Answer 1

[per isakson]

My steps

• Put your code inside a loop
• Converted to function, because I like functions better. Easier to debug. Doesn't litter my base workspace.
• Pre-allocateded memory for the result. Important for speed. nan because it makes it easy to check that the loop assigns values to all "positions".
• Replaced the blip by permute, because it is easier to read.
• And run the function

>> multi = cssm();
>> whos multi
  Name        Size              Bytes  Class     Attributes
   multi      41x55x5            90200  double              

>> any(isnan(multi(:)))
ans =
     0

where

function    multi = cssm()

      multi = nan( 41, 55, 5 );   % pre-allocate memory

      S = load('data_2.mat');     

      for jj = 1 : size( multi, 3 )
          Col = S.data_2( :, jj );
          Mtx_1 = reshape( Col, 55, [] );     % To (55 x 41)
          Mtx_2 = permute( Mtx_1, [2,1] );    % To (41 x 55)
          Mtx_3 = flipud( Mtx_2 );            % desired data order  
          multi( :, :, jj ) = Mtx_3;
      end
  end

"applicate to a large set o data (actually, my real data has 2255 rows by 30000 columns)." My estimate is that the execution time will be a few seconds.

Below is a parameterized version of the function, which I used to estimate the execution time

>>  multi = cssm( 'data_2.mat', 55 );

where

function    multi = cssm( filespec, d2 )

      S   = load( filespec ); 
      sz  = size( S.data_2 );
      dim = [ sz(1)/d2, d2, sz(2) ];

      multi = nan( dim );     % pre-allocate memory

      for jj = 1 : size( multi, 3 )
          Col = S.data_2( :, jj );
          Mtx_1 = reshape( Col, dim(2), [] );  
          Mtx_2 = permute( Mtx_1, [2,1] );     
          Mtx_3 = flipud( Mtx_2 );              
          multi( :, :, jj ) = Mtx_3;
      end
  end

[Luis Mz]

Hi isakson,

I thank you so much. It is what I was looking for.

Best,

Miguel