readData() function error message when using mapreduce

4 views (last 30 days)
I have a very large file (100's of millions of data points) and I am using mapreduce and datastore to get access to specifc chunks of data.
I get the following error (below) when I run it and I do not know how to go around it. lAny suggestions on how to deal with this?
Error using matlab.io.datastore.TabularTextDatastore/readData (line 81)
Mismatch between file and format character vector.
Trouble reading 'Numeric' field from file (row number 1136, field number 15) ==>
AAC,A,0,116,COM,7.63,25,24597\n
Error in matlab.io.datastore.TabularDatastore/read (line 120)
[t, info] = readData(ds);
Error in matlab.io.datastore.internal.shim.read (line 7)
[varargout{1 : nargout}] = read(ds);
Error in mapreduce (line 116)
outds = execMapReduce(mrcer, ds, mapfun, reducefun, parsedStruct);
Caused by:
Reading the variable name 'ticker' using format '%f' from file

Accepted Answer

Jeremy Hughes
Jeremy Hughes on 5 Aug 2019
ds.TextscanFormats{15} = '%q'
That will at least get you past the one line that's causing the problem here. For a more robust answer, I'd have to look at the file. It could be that there's an error in a line somewhere that's throwing off the parsing.
  3 Comments
Jeremy Hughes
Jeremy Hughes on 6 Aug 2019
Nope, %f is trying to read numbers, and if there isn't a number, it errors. %q reads (possibly double quoted) strings--it cannot really error.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!