MATLAB Answers

How to load in a large number of data files with numbered names that skip numbers

7 views (last 30 days)
Devin Mair
Devin Mair on 6 Dec 2014
Commented: Image Analyst on 6 Dec 2014
I have about 600 data files that I need to extract a few variables from and manipulate. They are titled "experiment-n.m" where n is a number ranging from 1-900. I already have the code to manipulate them, but I was hoping to find an easier way to read in these files than individually. The hard part is that the numbering is rather sporadic, and can be experiment-200 than skip to experiment-697.
I was considering making a for loop along the lines of:
for step = 1:900
str = sprintf('experiment-%d',step)
force = load(str,magf);
area = load(str, fn_area);
end
I have yet to test this code so there may be some errors in it. However, this would not work with the data due to skipping experiment numbers (like I said, the files will be experiments 1-100, skip to 150, skip to 257, etc.)
Is there some way to have this code process an error received when the file name is not found and make it skip to the next file name?
Thank you.

  0 Comments

Sign in to comment.

Answers (2)

Image Analyst
Image Analyst on 6 Dec 2014
See the FAQ for how to use dir() to read files that actually exist:
myFolder = pwd; % or whatever....
if ~isdir(myFolder)
errorMessage = sprintf('Error: The following folder does not exist:\n%s', myFolder);
uiwait(warndlg(errorMessage));
return;
end
filePattern = fullfile(myFolder, 'experiment*.*');
baseFileNames = dir(filePattern);
for k = 1:length(baseFileNames)
% Get this filename.
baseFileName = baseFileNames(k).name;
fullFileName = fullfile(myFolder, baseFileName);
fprintf(1, 'Now reading %s\n', fullFileName);
% Load file into a structure.
s = load(fullFileName);
% Extract fields from structure into individual arrays.
force(k) = s.magf;
area(k) = s.fn_area;
end

  1 Comment

Image Analyst
Image Analyst on 6 Dec 2014
Note, my solution
  1. does not overwrite the reserved keyword "path",
  2. it also checks that the folder actually exists,
  3. it uses fullfile (the recommended way to construct full pathnames),
  4. it only calls load() once instead of twice, and
  5. it gives you arrays for force and area (rather than overwriting scalars every iteration).

Sign in to comment.


Matt Raum
Matt Raum on 6 Dec 2014
How about something like this:
path = 'C:\yourdir\';
filelist = dir([path 'experiment*']);
for k=1:numel(filelist)
fname = [path filelist(k).name];
force = load(fname,magf);
area = load(fname,fn_area);
end
I'm guessing that the "load" function your calling is your own file reader which takes the "magf" and "fn_area" arguments your passing?
In any case, the code above should allow you to only read the files that actually exist.

  0 Comments

Sign in to comment.