What can I use instead of eval() to change the name of a variable in a Table?

My main goal is to create a general script that can handle a data set with unknown number of columns. The data set I generally work with contain 9 constant columns (time, battery, heading, pitch, roll, pressure, temperature, AnalogIn1 and AnalogIn2) and then a certain number of columns which gives you both the direction and the speed for each cell. This number changes depending on the data set.

I have created a code that can read the data (which is often in the format .xlsx) independent on number of columns, and I have been able to create a cell array structure called ExampleMatSpdDir (see code under) that has the name and the data for each of the variables the data contains.

My question to you is how can this be done without having to use eval() for the unknown number of columns? The reason why I ask is that I've seen several answers on here that you should not use eval(), due to its many issues. A simple solution would be to just not give each of the variable a name, such as Speed1, Dir1, Speed2, Dir2, .., Speedn, Dirn (where n is the number of cells). However, this is something that is needed for later coding.

Any feedback or tips would be highly appreciated. Mari

Here is part of my current code, where the use of eval() is found at the end of the code:

%%Import data

disp('Select the data file to import'); 

[filename,pathname] = uigetfile({'*.*';'*.txt';'*.csv';'*.xlsx'});

% Initialize variables.

delimiter = ';';
startRow = 2;

% Open the text file.

fileID = fopen(fullfile(pathname,filename),'r');

% Finds number of columns 

hdr=textscan(fgetl(fileID),'%q','Delimiter',delimiter)';

% Creating a formatSpec that contains all the columns

formatSpec=[repmat('%q',1,numel(hdr{1,1})),'%[^\n\r]'];

% Read columns of data according to the format.

dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'TextType', 'string', 'HeaderLines' ,startRow-1, 'ReturnOnError', false, 'EndOfLine', '\r\n');

% Close the text file.

fclose(fileID);

% Convert the contents of columns containing numeric text to numbers.
% Replace non-numeric text with NaN.

raw = repmat({''},length(dataArray{1}),length(dataArray)-1);

for col=1:length(dataArray)-1

      raw(1:length(dataArray{col}),col) = mat2cell(dataArray{col}, ones(length(dataArray{col}), 1));

end

numericData = NaN(size(dataArray{1},1),size(dataArray,2));

for col=[2:1:numel(dataArray)-1] 

      % Converts text in the input cell array to numbers. Replaced non-numeric
      % text with NaN.

      rawData = dataArray{col};

      for row=1:size(rawData, 1)

          % Create a regular expression to detect and remove non-numeric prefixes and
          % suffixes.

          regexstr = '(?<prefix>.*?)(?<numbers>([-]*(\d+[\,]*)+[\.]{0,1}\d*[eEdD]{0,1}[-+]*\d*[i]{0,1})|([-]*(\d+[\,]*)*[\.]{1,1}\d+[eEdD]{0,1}[-+]*\d*[i]{0,1}))(?<suffix>.*)';

          try

              result = regexp(rawData(row), regexstr, 'names');
              numbers = result.numbers;

              % Detected commas in non-thousand locations.
              invalidThousandsSeparator = false;
              if numbers.contains(',')
                  thousandsRegExp = '^\d+?(\,\d{3})*\.{0,1}\d*$';
                  if isempty(regexp(numbers, thousandsRegExp, 'once'))
                      numbers = NaN;
                      invalidThousandsSeparator = true;
                  end
              end
              % Convert numeric text to numbers.
              if ~invalidThousandsSeparator
                  numbers = textscan(char(strrep(numbers, ',', '')), '%f');
                  numericData(row, col) = numbers{1};
                  raw{row, col} = numbers{1};
              end
          catch
              raw{row, col} = rawData{row};
          end
      end
  end

% Convert the contents of columns with dates to MATLAB datetimes using the
% specified date format.

try

      dates{1} = datetime(dataArray{1}, 'Format', 'dd/MM/yyyy HH:mm:ss', 'InputFormat', 'dd/MM/yyyy HH:mm:ss');

catch

      try

          % Handle dates surrounded by quotes

          dataArray{1} = cellfun(@(x) x(2:end-1), dataArray{1}, 'UniformOutput', false);
          dates{1} = datetime(dataArray{1}, 'Format', 'dd/MM/yyyy HH:mm:ss', 'InputFormat', 'dd/MM/yyyy HH:mm:ss');

      catch

          dates{1} = repmat(datetime([NaN NaN NaN]), size(dataArray{1}));

      end
  end

dates = dates(:,1);

% Split data into numeric and string columns.

rawNumericColumns = raw(:, [2:1:numel(dataArray)-1]); 

% Replace non-numeric cells with NaN

R = cellfun(@(x) ~isnumeric(x) && ~islogical(x),rawNumericColumns); % Find non-numeric cells

rawNumericColumns(R) = {NaN}; % Replace non-numeric cells

% Create output variable

ExampleMatSpdDir = table;

ExampleMatSpdDir.DateTime = dates{:, 1};

ExampleMatSpdDir.Battery = cell2mat(rawNumericColumns(:, 1));

ExampleMatSpdDir.Heading = cell2mat(rawNumericColumns(:, 2));

ExampleMatSpdDir.Pitch = cell2mat(rawNumericColumns(:, 3));

ExampleMatSpdDir.Roll = cell2mat(rawNumericColumns(:, 4));

ExampleMatSpdDir.Pressure = cell2mat(rawNumericColumns(:, 5));

ExampleMatSpdDir.Temperature = cell2mat(rawNumericColumns(:, 6));

ExampleMatSpdDir.AnalogIn1 = cell2mat(rawNumericColumns(:, 7));

ExampleMatSpdDir.AnalogIn2 = cell2mat(rawNumericColumns(:, 8));

% Loop to evaluate the cells direction (Dir) and speed (Speed), 
% and then set the right column of data to each cell.

NumberCell=1; 

for i=9:2:(numel(dataArray)-2);

      eval(sprintf('ExampleMatSpdDir.Speed%d = cell2mat(rawNumericColumns(:, %d));',NumberCell,i));

      eval(sprintf('ExampleMatSpdDir.Dir%d = cell2mat(rawNumericColumns(:, %d));',NumberCell,i+1));

      NumberCell = NumberCell + 1;
  end

[Stephen23]

@Mari Sandvik: the variable you are accessing using eval, ExampleMatSpdDir, is a table. So why does the title ask about "a Cell Array structure" (whatever that is) ?

Why do you generate the table by writing so much code? Surely array2table would be much simpler?

" how can this be done without having to use eval() for the unknown number of columns?"

By simply reading the MATLAB documentation:

https://www.mathworks.com/help/matlab/matlab_prog/access-data-in-a-table.html

Using eval is not required at all. Why do you use it?

"Any feedback or tips would be highly appreciated. Mari"

Edit your question and upload some sample data. Then we could actually help you find an efficient way to import your data.

[Mari Sandvik]

Thank you for both your answers. You are right, it is a table so Cell Array structure is probably the wrong description of it. I will therefore change that.

When it comes to the function array2table, it is probably much easier and faster to use. However, from what I have just read about it looks like I will have to define the VariableNames manually. This is of course easy to do if the number of columns would be the same for each data file I would use the code on. Unfortunately this is not the case. Another issue is that I need to know which cell is being used for further analysis, and it would be great if I didn't have to check this manually.

I appreciate all your help, especially since I'm not an expert in coding (which often results in quite complicated and long codes).

[Stephen23]

" it is a table so Cell Array structure is probably the wrong description of it"

Not just "probably", it is the wrong description. It does not help you to attract volunteers who have experience with tables if you use entirely the wrong description in the title and body of your question.

"from what I have just read about it looks like I will have to define the VariableNames manually"

Where are the names defined? In the source data file? If so, then you can read them automatically and pass them to array2table, no manual naming required.

"which often results in quite complicated and long codes"

Yes, it does. Deciding to use eval rather than reading the MATLAB documentation on how to access data in tables also does not help you.

If you really want help then forget about your code: tell us what you are trying to do, show us what the expected output should be, and upload some sample data.

[Guillaume]

Note: I've edited the title of the question to replace cell array structure by table

I've also edited the question to format the code as code. @Mari, in the future, when asking questions, select all your code then press the {}Code button to make your post a lot more readable.

Answer 1

[Guillaume]

You say that your data is often in the format of an Excel xlsx file. Your code as it is will never be able to read such files. It will only work on text files.

Rather than writing your own parser, which tries to figure out the format of the text file, you should use readtable. In most cases, readtable is able to figure out the format of the file on its own (using pretty much the same method as you're using), and if it is not able you can override the detection. It also converts dates to datetime automatically, etc. Plus it already loads everything in a table. Finally, as an added bonus, it also read excel files.

If the column header is included in the file, readtable will automatically use it to name the columns. If not, it is trivial to rename the column, and as Stephen said, does not require eval:

ExampleMatSpdDir = readtable(fullfile(pathname,filename));  %possibly all that is needed!

%rename columns:
basenames = {'DateTime', 'Battery', 'Heading', 'Pitch', 'Roll', 'Pressure', 'Temperature', 'AnalogIn1', 'AnalogIn2'};  %names of first 9 columns are always the same
numadditionalcolumns = width(ExampleMatSpdDir) - 9;
assert(mod(numadditionalcolumns, 2) == 0, 'Number of additional columns is not even');
additionalcolumns = compose({'Speed%d'; 'Dir%d'}, 1:numadditionalcolumns/2);
ExampleMatSpdDir.Properties.VariableNames = [basenames(:); additadditionalcolumns(:)];