Data store only importing 2 columns
5 views (last 30 days)
Show older comments
Hey,
I am trying to do a large data analysis using datastore but when trying to import multiple .csv files instead of splitting the data into multiple columns for each variable it is only making 2 columns. The first column being the first variable and second colum being all the other data sperated by commas. How do I tell the data store to seperate the other data into columns? I have attatched two example .csv.
the script I am using is simply;
ds = datastore('FileLocation\*.csv')
Thanka for the help!
0 Comments
Accepted Answer
dpb
on 6 Jul 2020
DateTimeUTC,AIMHeading,AccX,AccY,AccZ,Surge,Sway,Heave,Roll,Pitch,Inclination,Yaw,HeaveAtRadar,RadarValue,CompensatedRadarValue,AvgWaveHeight,MaxWaveHeight,SignWaveHeight,SignWavePeriod,MeanWavePeriod,PeakPeriod,AvgWaveDir,AvgWaveSpread
2019-08-09 07.01.01,228.9000,0.3864,0.3124,0.0544,0.2421,0.3638,0.0059,-1.5691,-0.3287,1.6032,0.1163,0.0090,-7.5910,7.6000,0.4000,0.9500,0.5900,8.1000,6.6000,5.8000,0.0000,0.0000
2019-08-09 07.01.02,228.9000,-0.1636,-0.0300,-0.0195,-0.18 ...
The problem is the one see so often -- the semi-broke .csv file with embedded blanks in a field without quoted strings.
However, looks like
ds=datastore('FileLocation\*.csv','delimiter',{','});
does the trick...at least for one file; I didn't download both.
>> ds=datastore('201908090801AIM1_1H.csv',"Delimiter",{','})
ds =
TabularTextDatastore with properties:
Files: {
'C:\Users\Duane\Documents\MATLAB\Work\201908090801AIM1_1H.csv'
}
FileEncoding: 'UTF-8'
AlternateFileSystemRoots: {}
PreserveVariableNames: false
ReadVariableNames: true
VariableNames: {'DateTimeUTC', 'AIMHeading', 'AccX' ... and 20 more}
DatetimeLocale: en_US
Text Format Properties:
NumHeaderLines: 0
Delimiter: {','}
RowDelimiter: '\r\n'
TreatAsMissing: ''
MissingValue: NaN
Advanced Text Format Properties:
TextscanFormats: {'%q', '%f', '%f' ... and 20 more}
TextType: 'char'
ExponentCharacters: 'eEdD'
CommentStyle: ''
Whitespace: ' \b\t'
MultipleDelimitersAsOne: false
Properties that control the table returned by preview, read, readall:
SelectedVariableNames: {'DateTimeUTC', 'AIMHeading', 'AccX' ... and 20 more}
SelectedFormats: {'%q', '%f', '%f' ... and 20 more}
ReadSize: 20000 rows
>> preview(ds)
ans =
8×23 table
DateTimeUTC AIMHeading AccX AccY AccZ Surge Sway Heave Roll Pitch Inclination Yaw HeaveAtRadar RadarValue CompensatedRadarValue AvgWaveHeight MaxWaveHeight SignWaveHeight SignWavePeriod MeanWavePeriod PeakPeriod AvgWaveDir AvgWaveSpread
_______________________ __________ _______ _______ _______ _______ _______ _______ _______ _______ ___________ _______ ____________ __________ _____________________ _____________ _____________ ______________ ______________ ______________ __________ __________ _____________
{'2019-08-09 07.01.01'} 228.9 0.3864 0.3124 0.0544 0.2421 0.3638 0.0059 -1.5691 -0.3287 1.6032 0.1163 0.009 -7.591 7.6 0.4 0.95 0.59 8.1 6.6 5.8 0 0
{'2019-08-09 07.01.02'} 228.9 -0.1636 -0.03 -0.0195 -0.1808 0.1131 -0.0466 -1.9137 0.0113 1.9137 0.1565 0.124 -7.592 7.716 0.4 0.95 0.59 8.1 6.6 5.8 0 0
{'2019-08-09 07.01.03'} 229.1 -0.2974 -0.2663 -0.0237 -0.3529 -0.0107 -0.0645 -1.299 -1.0392 1.6635 -0.0378 0.097 -7.592 7.689 0.4 0.95 0.59 8.1 6.6 5.8 0 0
{'2019-08-09 07.01.04'} 229.3 -0.1462 -0.1184 0.04 -0.2197 0.1167 -0.0527 -0.6783 -2.0269 2.1373 0.0396 -0.096 -7.591 7.495 0.4 0.95 0.59 8.1 6.6 5.8 0 0
{'2019-08-09 07.01.05'} 229.4 0.0183 0.2937 0.0223 -0.029 0.2389 -0.0537 -1.0576 -2.0764 2.3301 0.1158 -0.268 -7.591 7.323 0.4 0.95 0.6 8.4 6.6 5.8 0 0
{'2019-08-09 07.01.06'} 229.4 0.3044 0.0866 -0.0578 -0.0044 0.0772 -0.0475 -1.7729 -1.1997 2.1406 0.0002 -0.206 -7.591 7.385 0.4 0.95 0.59 8.4 6.6 5.8 0 0
{'2019-08-09 07.01.07'} 229.4 0.0308 -0.1888 -0.0367 -0.2413 -0.1377 0.008 -1.6663 -0.7888 1.8435 -0.1285 0.013 -7.59 7.603 0.4 0.95 0.59 8.1 6.5 5.8 0 0
{'2019-08-09 07.01.08'} 229.3 -0.3493 -0.1879 0.0397 -0.4157 -0.1401 0.076 -0.9687 -1.3592 1.669 -0.0366 0.167 -7.591 7.758 0.4 0.95 0.59 8.2 6.6 5.8 0 0
>>
2 Comments
dpb
on 6 Jul 2020
I'll note that that file was "only" 3600 lines -- that's easily handled in memory so not sure you need the datastore at all here -- unless going to read in the multiple files together. Not sure if that's how it works or not...never used enough to know.
More Answers (0)
See Also
Categories
Find more on Spreadsheets in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!