How to correctly use dlmread to read a large matrix in m-file?

I saved a 11-by-100001 matrix as a m-file with dlmwrite. Then I read the m-file with dlmread, but the matrix in Matlab became a column vector that had the length of 11*100001=1100011.

How can I correctly read the matrix back in Matlab? My Matlab version is 7.10.0. Thank you!

If you would like to see how the error happens, try the following code:

M = rand(11,100001);
dlmwrite('matrix_m.m',M)
LM = dlmread('matrix_m.m');

You'll see LM in workspace is a 1100011x1 column vector.

[dpb]

Show your code -- that doesn't sound right. dlmread should determine the number of columns automagically.

Oh, and maybe just a few lines of the input file and perhaps the line that wrote the file. You didn't do something like put a ':' in the argument for the array when you wrote it by any chance???

[Walter Roberson]

How about if you write out the transpose of the matrix?

First guess: limit might be about 65535 columns.

[Yu-Yun]

I just tried

M = rand(11,100000);
dlmwrite('matrix_m.m',M)
LM = dlmread('matrix_m.m');

LM is then correct. The column number must be smaller than 100001.

[dpb]

I'm not sure how dlmread ascertains the number of columns -- I reverted to an older copy here since you're using 7.10 to see if anything peculiar happened...

W/ the original sample code,

>> LM = dlmread('matrix_m.m');
>> whos LM
Name      Size         Bytes  Class

LM        4x100001   3200032  double array

Grand total is 400004 elements using 3200032 bytes

it only read a subset for some reason.

What I find peculiar is why you're using .m as an extension for a delimited file -- while it shouldn't make any difference on the content, I think dlmread uses the extension as a clue as to the content which is why I would guess it returned a single column.

I rewrote the array using .csv as the extension --

>> dlmwrite('matrix_m.csv',M);
>> clear LM
>> LM = dlmread('matrix_m.csv');
>> whos LM
Name      Size         Bytes  Class

LM       11x100001   8800088  double array

and voila!!! joy ensues.

Well, not sure why it made any difference, but there ya' go...try there and see if it cures your ills. This was R12.1

ADDENDUM:

Just for grins decided might as well try on later release as well -- copied the two files over to other workspace and

>> LM = dlmread('matrix_m.m');
>> whos LM
Name           Size              Bytes  Class     Attributes

LM        354111x1             2832888  double              

>> clear LM
>> LM = dlmread('matrix_m.csv');
>> whos LM
Name            Size              Bytes  Class     Attributes

LM        1100011x1             8800088  double

>>

So, there's something funky about using an m-file extension...

This latter is R2012b

ADDENDUM 2:

And, of course, one would get much better performance as well as full precision if one were to use a .mat file and save/load. Of course, the sample could be just pedagogical not real, but if there's some idea of using this for sizable files, I'd suggest rethinking is in order...

[Yu-Yun]

I used .m as the extension because I didn't know .csv is better for saving matrix. :P

I just tried .csv. Unfortunately, it doesn't work for Matlab 7.10.0. I think I should start using save to save my data. Anyway, thanks a lot for the help!

[dpb]

In Matlab, the 'm-file' is the storage for source files -- naming data files with the extension makes them visible to the parser as if they were source files so, amongst other unexpected results is that you could cause failure of a function or script by aliasing a real function of the same name.

Specifically what caused the difference here is unclear to me as it didn't occur this morning when I tried to document a case to send to TMW as a bug report.

I got the one-column "solution" on both cases -- peculiar, indeed.

I did look at the two files on disk--they're identical and do have the newline character after the rows so ML should be able to determine the shape from the data and other than the namesearch problem above the file extension should not have any bearing.

It's a bug IMO, whether TMW will think it serious-enough given the length of the record to take under consideration I don't know.