Simultaneously Merging and Editing multiple text files

I need to do two simple tasks, but am having trouble combining them. First, I need to combine multiple text files into a single file. Second, I need to add a column to each of the text files prior to combining. For example, I have 3 different subjects, all of whom have nearly identical but separate text files. The files are all a few hundred rows long, but only 4 columns wide. The exact same column names (i.e. variables) are recorded for each subject/files. But, because the contents of the text files are identical, before I combine them into a single file, I need to add a variable/column to identify which file is which in the biglist.

I'm stuck at two points. 1) When I run the loop to read the data into MatLab (in order to edit and re-paste it all together), each subsequent text file is overriding the previous so I only end up with the final data file entered. I've tried to rename the data on each loop and store it to later concatenate, but it doesn't like me. See below

startSubj = 1;
endSubj = 3;
for a = startSubj:endSubj
 textFileName = ['Subject' num2str(a) '.txt.']
 fid = fopen(textFileName, 'r');
 data = textscan(fid, '%n%n%n%n', 'headerlines', 1);
 fclose(fid)
end

The second point I'm stuck is editing each data file to add a column before merging. Part of the problem is that I am not well versed in matlab and thus, unable to edit anything that isn't already a matrix (which the fopen command doesn't appear to give).

Help on any/all of the question is appreciated!

Answer 1

[Michael Haderlein]

This will merge all text files (no matter what the content is) and add the file number.

files=dir('part*.txt');
fileout='merged.txt';
fout=fopen(fileout,'w');
for cntfiles=1:length(files)
  fin=fopen(files(cntfiles).name);
  while ~feof(fin)
    fprintf(fout,'%s %d\n',fgetl(fin),cntfiles);
  end
  fclose(fin);
end
fclose(fout);

[Sarah]

Thanks, Michael. This appears to do what I need. Two small differences/addendums, however. First, could I be able to alter the cntfiles variable to instead, read the current filename and add that instead of just the sequential order/number of the files? Specifically, sometimes I may have Subject 3, 7, and 9 to combine and it would be ideal to have their 3rd column say 3, 7, and 9 respectively instead of 1,2, and 3. Also, this column would ideally have a heading included as well

Second, a more minor glitch, is that the resulting file saves the data all in a single row in the text files instead of nice columns like it was imported as. Correcting this would be a simple issue of reassigning the files in a different format when they are read in, correct?

[Sarah]

I quick correction - the files does indeed correctly transfer as multiple rows and columns (my mistake in viewing the output too cursory), and the new additional column does have a heading that is identical to the number that fills the following rows. The only slight glitch is that the the additional column is separated by a space, instead of a tab (like the prior trials), but this is a quick change of the fprint code or a quick edit in excel after the fact.

Answer 2

[Hikaru]

I did a similar task using Excel file. My approach was to make an output file and read the last row of that output file and tell MATLAB to write in the next empty row.

function compile(filename,output)
M = xlsread(filename);

a = xlsread(output); % store data of file in variable a
if isempty(a) == 1
    c = 'A2';
else
    nRows = (size(a,1)) +2; % last row with data + 2 to write in the next empty row
    b = num2str(nRows); % convert number to string
    c = strcat('A', b); % make concat strings
end

xlswrite(output, M,'Sheet1',c);

I had to add 2 to nRows because my file contains header. I think you can use the same approach with textscan.

[Sarah]

Thanks, Hikaru. But, I need to add a column, not a row to each dataset and that column needs to be filled with a number that differentiates the datasets.