for loop does not work to iterate through 600+ csv files

I am trying to write code to iterate over a series of csv files, each that represents a county in the United States, that are saved with the countynumber.csv (ex: 1003.csv).
I have 631 counties ranging from 1003 to 55139 (not each number in between though), and I want to calculate some aspects (maximum values and their locations, as shown in my code) and display the results for each of the 631 counties in a table, which is the last line of my code.
How can I iterate through the 631 counties and display their results in one 631x6 table? Why is my for loop not working?
Code so far:
whichBin = {2012:2013};
binType = 'annual';
binList = {2012:2013};
geoRegion = 'FIPS';
whichYears = [2012:2013];
whichDay = 'wkend';
randStr = 'gbqo';
for geoCode=1003:55139
csvData = ['tweetograms/tweetograms_' binType '_' geoRegion '_2012_2013_' randStr '/' binType '/' whichDay '/' num2str(geoCode) '.csv'];
if ~exist(csvData,'file'),continue,end
tweetogramData = csvread(csvData);
tweetogramSmooth = smooth(tweetogramData)
[lunchpk, loc1] = max(tweetogramSmooth(44:60));
[dinnerpk, loc2] = max(tweetogramSmooth(68:92));
lunchloc = loc1 + 43;
dinnerloc = loc2 + 67;
outTable(geoCode,:) = table(geoCode, whichDay, lunchloc, lunchpk, dinnerloc, dinnerpk);
end

6 Comments

why is my loop not working?
This is usually indicated in the error message.
Can you also attach some files so that we can help you debug the code?
This is the error message: Error using csvread (line 35) File not found.
Error in Test1 (line 25) tweetogramData = csvread(csvData);
However, when I write: for geoCode=1003 (just one geoCode, rather than multiple), the code works. It might be a number to string issue, since the I am using the geoCodes as strings, not numbers. I'm not sure what files I would attach that could make it more useful since this is a complicated project - any help would be appreciated.
your loop runs from 1003 to 55139 so if only 631 files exist your loop tries to open roughly 53400 files that do not exist? it will throw an error and stop at the first non-existant file.
You could include the following line of code (before the call to csvread), however that would not prevent you from creating a mostly empty table.
if ~exist(csvData,'file'),continue,end
Also, don't use clear all; close all; clc; outside of debugging situations. Especially the clear all should be avoided. Keep your workspace clean by using functions. You can use clearvars if you really have to.
You may be much better off by using dir to find the list of files and determining the geo-code based on that.
As a final remark, you should format your code correctly. You can easily do so by editing your post, selecting the code and clicking the {}Code button.
@Mili Shah: you should follow Rik Wisselink's very good advice. Also read this:
It would be much simpler if you used dir.
Thank you everyone for your help! I used @Rik Wisselink's advice, and looked into @Stephen Cobeldick's linked article. My code now saves all the data in one table, but there are zeros for each geoCode with no data - would using
dir
be the best way to create a table with only my data? or is there a way to get rid of the zeros in the table? This is my code for the table.
outTable(geoCode,:) = table(geoCode, whichDay, lunchloc, lunchpk, dinnerloc, dinnerpk);
Thanks.

Sign in to comment.

Answers (0)

Asked:

on 16 Jul 2018

Edited:

on 17 Jul 2018

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!