Clear Filters
Clear Filters

Import CSV data into Matlab

6 views (last 30 days)
Agnes Palit
Agnes Palit on 4 Jul 2018
Edited: Agnes Palit on 16 Jul 2018

I am trying to import data CSV using MATLAB, with patient_1(1).Data, as a following:

to import I used this code:

filedir = '/Users/';
files1 = dir(fullfile(filedir, '*.csv'));
numFiles1 = length(files1);
for i = 1 : numFiles1
  patient_1(i).Name = files1(i).name;
  patient_1(i).Data = readtable(fullfile(filedir,files1(i).name),'ReadVariableNames',false);   % loads data
end

But I got this warning and following error:

Warning: Unable to determine the format of the DURATION data.
Try adding a format to the DURATION specifier. e.g. '%{hh:mm:ss}T'. 
> In table/readTextFile>textscanReadData (line 554)
In table/readTextFile (line 225)
In table.readFromFile (line 39)
In readtable (line 197)
In patient1 (line 23)
Error using readtable (line 197)
Unable to determine the format of the DURATION data.
Try adding a format to the DURATION specifier. e.g. '%{hh:mm:ss}T'.
Note: readtable detected the following parameters:
'Delimiter', ';', 'HeaderLines', 0, 'Format', '%q%q%T%q%q%T%T%f%f%q%q'
Error in patient1 (line 23)
  patient_1(i).Data =    readtable(fullfile(filedir,files1(i).name),'ReadVariableNames',false);   %    loads data

Can anyone help me to solve this problem?

  4 Comments
Walter Roberson
Walter Roberson on 4 Jul 2018
patient_1(i).Data = readtable(fullfile(filedir,files1(i).name), 'ReadVariableNames', true, 'HeaderLines', 1);
A small number of lines for the test file would be fine. I extracted some of the data from the image you posted but I am not encountering the message you are getting.
Agnes Palit
Agnes Palit on 4 Jul 2018
Edited: Agnes Palit on 4 Jul 2018
This is the sample of file:
I have tried the code. And I got several lines with the same the warning such as below:
Warning: Variable names were modified to make them valid MATLAB identifiers. The original names are saved in the
VariableDescriptions property.
As you can see on the screenshot above, column number 3,6,7,8,9,11, maybe already show appropriate type. But the remain column such as col 10 I think it still string/char. Do you have idea how to solve this?
And how to check/display each column already have appropriate type? (i.e.: col 1 = day, col 2 = date, etc)
And then, as you can see on the screenshot, the first row disappears. It show from 2nd row as header.

Sign in to comment.

Accepted Answer

Walter Roberson
Walter Roberson on 4 Jul 2018
opt = detectImportOptions('EE_moderate.csv');
opt = setvartype(opt, [2 5], 'datetime');
opt = setvaropts(opt, [2 5], 'InputFormat', 'dd-MMM yyyy', 'DatetimeFormat', 'eee uuuu-MM-dd HH:mm:ss' );
...
T = readtable(fullfile(filedir,files1(i).name), opt);
T.date = T.date + T.time;
T.date_1 = T.date_1 + T.time_1;
patient_1(i).Data = T(:,[2 5 7:end]);
...
  19 Comments
Walter Roberson
Walter Roberson on 16 Jul 2018
Not all of your files have a field #10.
In the original file, field #10 contained percentages complete with the % character. detectImportOptions sees the % and figures that the entire field must be a character vector rather than a number. Doing the setvartype() is to force it to treat as a number, with the % character taken care of by way of the Suffixes option.
I think I asked you earlier what other special characters might occur in fields that you want treated as numeric.
Agnes Palit
Agnes Palit on 16 Jul 2018
Edited: Agnes Palit on 16 Jul 2018
I see thank you for the idea. Yes, I think that is the problem. The each some of the data has same field, but some of them does not have uniform field. When I saw the data, there are 6 files. that has this similar header. Among these 6 data, there is 1 file which only has 8 field/header.
day; date; time; day1; date1; time1; duration; minutes;
the minutes is double.
Is it possible to put "opt" as an array? so that I can save all input data inside opt. Because when I run the code which process 6 files, the "output" will show the latest file.

Sign in to comment.

More Answers (0)

Categories

Find more on Data Type Identification in Help Center and File Exchange

Products


Release

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!