How can I move to a specific position in a txt file and write a value there? Target can be a specific word or bites from origin

Hello, I've written a Matlab script that runs some calculation and that outputs the numerical/text results inside a txt file (generated on purpose) according to a specific layout (columns, comments, ...). The script works like a charm, but there is actually something more I would like to achieve: I need to write the result of a math operation in a specific position of the same txt file (which is close to the top/beginning of the txt file) but unfortunately that value to be written is calculated only at the bottom/end of the script. So the variable is not existing when the script should write it in the txt file (in the right position); it is calculated only later, once the script is over and I've reached the end of txt file.

I am thus looking for a function that can be placed at the end of my script code and that - once the script is over - moves the cursor from the bottom/top to a specific position of the txt file, gets a known value from my workspace of variables and writes it down in that specific position. The specific position can be either identified by a word or by 'X' bites from origin of file.

I was optmistic about "fseek", but either I am messing up or it is not the function I should use.

Please advise !

[dpb]

No can do...sequential files are, well, "sequential" and while you can in theory use fseek() and find a location within the file and modify it, at most all you can do is replace one set of bytes with another set of identically the same length; doing even this has to be done with fread/fwrite in stream mode, however, not via fprintf; as soon as you do that you mark the current location as the end of the file and all else past that point is now toast.

To do what you want with a sequential text file, the only basic way for the novice is to copy the file to the point of insertion to a new file, write the new information to that file, then copy the remainder of the old file to the new from that point on. Then, close both files and if desired copy the new onto the namespace of the original or delete the original and rename the new.

Other than that, one needs random-access files with capability of record insertion/deletion which Matlab doesn't support.

[Walter Roberson]

fprintf does not truncate files. MATLAB does not provide any truncate operation, which is a problem. MATLAB also disobeys Posix about the effects of seeking past the end of file and writing new data, which is a problem.

[per isakson]

One approach is

1. write the file with a placeholder for the "result of a math operation"
2. use memmap, Map memory for low-level memory read and write operations to replace the placeholder by the "result of a math operation"

[Maximilian Arpaio]

(2) seems interesting. Any hint on the way to properly use the function within my script ?

[per isakson]

Sorry, I should have linked to: memmapfile, Create memory map to a file. memmap is over-kill.

Hint:

ffs = 'c:\tmp\test.txt';
[fid,msg] = fopen( ffs, 'w' );
fprintf( fid, '%s', '1234567890----------1234567890' );
fclose( fid );
mmf = memmapfile( ffs, 'Writable',true );
reshape( mmf.Data, 1,[] )
mmf.Data(11:20) = uint8('abcdefghij');
clear('mmf')
type(ffs)

outputs

ans =
     49   50   51   52   53   54   55   56   57   48   ...
  1234567890abcdefghij1234567890

I'm a bit confused by the two bytes per character in Matlab vis-a-vis the one in the file. However, the above seems to work with "us-ascii".

Answer 1

[Walter Roberson]

You indicated that this is a text file. Formally speaking, there is no operation for seeking by offset in a text file. The only supported operation (other than beginning or end of file) is to ftell() to get a token representing the current position, after which you can fseek to the token to go back there.

fseek by absolute or relative position is only supported for binary files, not for text files.

The main difference between binary and text files is in the treatment of bytes that match the line terminator. Mostly this has to do with whether carriage return is dropped on input and automatically emitted on output when char(10) is output... Which can happen even for fwrite() if you are in text mode. But operating systems are permitted to do more complicated things in text mode. The specifications were written assuming that you might be writing a text file in the operating system of the WordStar devices that used to exist.

Still, using a text file it would be valid to open in w+ mode and to ftell() right before emitting the placeholder for the missing variable, and then later fseek back and fwrite exactly the same number of bytes as the placeholder making sure that you do emit any newlines when you do.

Answer 2

[Maximilian Arpaio]

Thanks to all for the different perspectives.

Actually, I still believe there is a means to do what I am trying to achieve. As an example, I've drafted the simple code below - that basically does what my script aims to do:

clearvars;
fid = fopen('testTriangle.txt','wt');
fprintf(fid,'#THIS IS MY SCRIPT\n');
fprintf(fid,'#IT CALCULATES THE PERIMETER OF A TRIANGLE \n\n');
fprintf(fid,'#THAT HAS AN AREA OF: %4.2f m^2\n', areaT);
A=5.3;
B=6.5;
C=7.1;
Perimeter=A+B+C;
areaT=A*C/2;
fprintf(fid,'#THE PERIMETER OF THE TRIANGLE IS: %4.2f m\n', Perimeter);
%here the "fseek" or similar function should be placed, so as to move the
%cursor to line 5 and provide to the varialbe areaT the info calculated
%only in line 10.
fclose(fid);

Is the above example clarifying the matter ? I can either move the cursor to a specific line or according to a specific word, I'am quite flexible about this. I can also close the file and open it with a different fid if this helps.

Please advise !

[dpb]

THAT specific thing you can do, yes..within some pretty restrictive limitations--that's the one case that if you carefully create the file to leave the room to fill in you can do so...but you must write the file as shown above with a specific format as shown and a default value for the area as fill--and you must leave sufficient room for anything you can possibly write and fill all those bytes...

Also note, the above format string doesn't do what you need...

>> for i=0:3,fprintf('%4.2f\n',pi*10^i),end
3.14
31.42
314.16
3141.59
>>

so that even though you asked for a specific field width, that by itself doesn't control how many characters are going to be written--you've got to pad the location with sufficient space to hold as many characters as you're EVER going to need. The C-language field width specifier, unlike Fortran FORMAT, specifies, NOT a fixed-width field but the minimum field width; it will expand to fit the numeric value. You can use some arbitrarily wide width that will (hopefully) cover every possible value you may ever encounter which will look something like

>> for i=0:3,fprintf('#THAT HAS AN AREA OF: %14.2f m^2\n', pi*10^i),end
#THAT HAS AN AREA OF:           3.14 m^2
#THAT HAS AN AREA OF:          31.42 m^2
#THAT HAS AN AREA OF:         314.16 m^2
#THAT HAS AN AREA OF:        3141.59 m^2
>>

All in all, as outlined earlier, while it is theoretically possible within limitations, overall it just isn't practical way to run a railroad; you'd be better off caching the results and writing the file sequentially (since it is, after all, a "sequential" file) or using the aforementioned technique to rewrite the file piecewise inserting the new information in place; the grief in taking care of all the nuances the proposed way is just way more effort that what can be gained.

$0.02, imo, ymmv, etc., etc., etc., ...

[Maximilian Arpaio]

Thanks dpb for the answer.

I do understand the "width" matter and I will do my best to take account for it. I also agree with you that there might be better way to do this rather than going back to line 5 from line 10, but I have no other way to do it so since my file as a specific format. This might also help me though...

So, what is now not so clear to me is the way to achieve my task by means of the fseek function and if this is the function to actually use. For my purpose, I can even arrange the code as follow:

clearvars;
fid = fopen('testTriangle.txt','wt');
fprintf(fid,'#THIS IS MY SCRIPT\n');
fprintf(fid,'#IT CALCULATES THE PERIMETER OF A TRIANGLE \n\n');
fprintf(fid,'#THAT HAS AN AREA OF: ');

and then I will tell my function to move the curson from the bottom of the txt file up to line 5 and right after "of: " so as to place the value inside the variable areaT. Then I can close the file and save it. How should I do it ?

[dpb]

If you're adamant in going this direction, what you have to do is to be able to control exactly how many characters have been written up to the point you're interested in...that means that not only is it the length of the field you want to fill in that matters but every one that came before has the same issue. Is the above example exactly the form of your file in that the particular element is the first variable field in it? IFF that is the case, then just count the number of characters to the beginning of the field and hardcode that into fseek and write the n characters at that point.

How big is the actual file? Is this the extent of what you're actually trying to do or is this just an example in a huge file like a log file or somesuch? If the file is on the order of MB or less, a simpler way code-wise would be to write a unique pattern into the file at the location of interest; complete the remainder of the task and close the file. Then read the file into memory and do a character substitution of the formatted numeric value into the magic pattern location in memory and rewrite the file.

I still think you're barking up the wrong tree here, though, and the solution is to rearrange the computation/writing of the information by caching the results in memory until it's time to commit them to disk...again, unless this is going to be some huge GB-sized file.

[dpb]

"...there might be better way to do this rather than going back to line 5 from line 10, but I have no other way to do it."

Of course there's a better way and it's easy enough to do...simply save whatever other information is required to be written in memory during the script until the point at which you have all you need to write the desired data--in this case the area. THEN and only then write the file once you already know the answer.

PS Attach your m-file that you have so far so we can see it; then almost certainly can solve your problem in far more direct and robust manner.

[Maximilian Arpaio]

the point is that '#' denotes a comment and the first five lines of the txt file are a comment to the following calculation. The purpose of the comment is to summarise the main results that can be found with more extend ....calculated further down, so that I don't need to go through the whole file to get what I want, but I can peek into the comments to get main results. So when I write the file, the comments come first, then my calculation follows on. Once the calculation is over I have all the variables I need to fill in the fields in the comments, but unfortunately they are further up.

Even if I re-arrange the script, I would still face the fact the the comments must stay at the top of the file (like a kind of header) but the data only stay at the bottom. That's the reason why I see no other way in my script that to leave it as it is and find a way to move the cursor to - let's say - line 5 and fill in the key comments with the data I need. I would not be in a position to fill the fields in at the time the comments are written, because the data don't exist. Either with fseek or not, how should I close this headache ?

EDIT: DEMO script here attached.

[Stephen23]

"how should I close this headache ?"

By avoiding it entirely. What you are trying to to is write hack code that messes around counting bytes written in files, something that is ugly and likely to fall apart when someone sneezes. However if you simply move all of the fprintf to the end of the script then this entire problem disappears. Calculate all values, then print to file. Why make it more complex than that?

"Even if I re-arrange the script, I would still face the fact the the comments must stay at the top of the file (like a kind of header) but the data only stay at the bottom. That's the reason why I see no other way in my script that to leave it as it is..."

The order of calculation is totally unrelated to the order that that data can be printed to file. MATLAB does not restrict you to printing the data in the same order that it is created. Create some values, do some calculations, store the values in variables. Print them all when you are finished, in whatever order you wish. No hacking around with file byte counting is required.

[dpb]

+1

Answer 3

[dpb]

>> maxim
>> type testTriangle.txt
#THIS IS MY SCRIPT
#IT CALCULATES THE PERIMETER OF A TRIANGLE OF AREA: 18.81 m^2
#THE PERIMETER OF THE TRIANGLE IS: 18.90 m
>> 

 >> type maxim
  % compute what is wanted/needed...
  A=5.3;
  B=6.5;
  C=7.1;
  Perimeter=A+B+C;
  areaT=A*C/2;

% NOW write out the results in the desired order
fid = fopen('testTriangle.txt','wt');
fprintf(fid,'#THIS IS MY SCRIPT\n');
fprintf(fid,'#IT CALCULATES THE PERIMETER OF A TRIANGLE OF AREA: %4.2f m^2\n', areaT);
fprintf(fid,'#THE PERIMETER OF THE TRIANGLE IS: %4.2f m\n', Perimeter);
fclose(fid);
>>

Simply rearrange when you do things; wait until it's time; and time is when you have the results required to do the task.

NB:

If the "extended calculations" are indeed of tremendous size and create a huge file and there really is some nontrivial reason can't wait to write earlier results before some final answer drops out at the bottom, the more efficient way to do so is to write that portion of the file independently of the header and when the header information is ready, write a separate header file containing that information that belongs therein.

Then, as the last operation before quitting the script, one simply copies the larger calculations file onto the tail of the header file; iow the two files are combined together to create the final file. This is done most simply by just passing the appropriate command to the operating system.

[dpb]

It's almost certainly the most wrongheaded way to possibly solve the problem, but...

...
fid = fopen('testTriangle.txt','w');
n=fprintf(fid,'#THIS IS MY SCRIPT\n');
n=n+fprintf(fid,'#IT CALCULATES THE PERIMETER OF A TRIANGLE OF AREA: ');
fprintf(fid,'%4.2f m^2\n', 99.99);
fprintf(fid,'#THE PERIMETER OF THE TRIANGLE IS: %4.2f m\n', Perimeter);
% Absolutely wrongheaded, but...
fseek(fid,n,'bof');
fprintf(fid,'%4.2f',areaT);
fclose(fid);

will overwrite the '99.99' placeholder value; all the preceding issues exist in trying to make this work in general, but, yes, you can write on top of a sequential file at any point within it but it is terribly fraught with "there be dragons!" and as the opening comment says...

[Maximilian Arpaio]

Thanks dpb,

I've tried your code but the comment is added at the bottom of my (generated) file. My fault, but I forgot to highlight the fact that the script calls a "for" cycle and writes some values in the file. So when I place at the bottom of my file your code:

n=fprintf(fid,'#THIS IS MY SCRIPT\n');
n=n+fprintf(fid,'#IT CALCULATES THE PERIMETER OF A TRIANGLE OF AREA: ');
fprintf(fid,'%4.2f m^2\n', 99.99);
fprintf(fid,'#THE PERIMETER OF THE TRIANGLE IS: %4.2f m\n', Perimeter);

text is place at the bottom, which is something that I don't want. Furthermore, this proves that I cannot rearrange this time....

Answer 4

[Walter Roberson]

width_for_placeholder = 20;
fid = fopen('testTriangle.txt','w+t');   %notice change
fprintf(fid,'#THIS IS MY SCRIPT\n');
fprintf(fid,'#IT CALCULATES THE PERIMETER OF A TRIANGLE \n\n');
fprintf(fid,'#THAT HAS AN AREA OF: '); % I want to place here the value of the variable
cursor = ftell(fid);
fwrite(fid, uint8(blanks(width_for_placeholder)));  %reserved for "areaT"
fprintf(fid, '\n');
A=5.3;
B=6.5;
C=7.1;
Perimeter=A+B+C;
areaT=A*C/2;
fprintf(fid,'#THE PERIMETER OF THE TRIANGLE IS: %4.2f m\n', Perimeter);
fseek(fid, cursor, 'bof');
fmt_areat = uint8( sprintf('%*.10g', width_for_placeholder, areaT) );  %g format is good about respecting widths no matter what the range of values
%it is important that the output be exactly the reserved length!
if length(fmt_areat) ~= width_for_placeholder
  fmt_areat = repmat(uint8('*'), 1, width_for_placeholder);
  fprintf('Warning: area value overflowed available width, replaced with **');
end
fwrite(fid, fmt_areat);
%do NOT output a newline here!!
fclose(fid);

The uint8() part is so that we are reserving an exact number of bytes no matter what the character encoding.

Note: if the character encoding is UTF-16 then the area will not be written out in UTF-16. It would be possible to do so, by adding a couple of extra lines.

Answer 5

[Maximilian Arpaio]

Thanks to all. I will go trough all the different advices and test your lines with my final code. As soon as I have a feedback, I will come back to you in the next hours. Cheers

[SHAHMUSTAFA MUJAWAR]

hi Maximilllian, did you get any idea about your problem?

[Walter Roberson]

We gave precise code. After, that is, plenty of warnings as to why this should be avoided.