create a tall array from .mat files

Hey,

I have 29 arrays of size approximately 150k x 13 stored in the form of .mat labeled 1-29. I want to add an extra column to each row and then store them in a single array. I figure that the data is too large to store in memory, so I want to use a tall array. ultimately the data is to be taken from the tall array, sorted and plotted as a surface plot. I want to know if a tall array is the best way to do this or if there is an eaiser way like appending the .mats together without loading them into memory and adding the 14th coloum to them without loading into memory.

Thanks

[Ben Morrow]

to clarify, it is an array, datak (the same name in every .mat) contained in mat1.mat to mat29.mat

Answer 1

[Nanda Gupta]

I understand that you would like to add a column for each row(150) in each of your MAT files(29 files). Based on the description your entire data should be around ~450Mb.

150,000 * 13 * 8 Bytes = 15600000 Bytes = 15.6 MB

15.6 MB * 29 = 452.4 MB

In this case, you will likely be able to store all this at the same time in the MATLAB workspace, unless MATLAB is explicitly showing an error for insufficient memory.

So, an easier way would be to try using arrays before Tall arrays.

[Ben Morrow]

I wanted to have multiple plots available of similar data sets, I got some kind of memory error. I resolved it by creating smaller .matfiles by cutting out every other data point.