Why is matlab's fopen so slow?

In our codebase, we want to log strings to a file. I use a very simple function for this:

function log(logstring)

fid = fopen("logging.log","A");

fwrite(fid,logstring);

fclose(fid);

end

Problem is that this is very slow (and I'm already using "A", as recommended for speed).

I also have pyton configured on my pc, which opens up the following alternative way to do the same thing:

function log_python(logstring)

filename = "logging.log";

code = ["with open(filename, 'a',encoding='utf-8',newline='') as f:";

" f.write(data)"];

pyrun(code,data=logstring,filename=filename);

end

This method turns out to be about 10x faster than the matlab version. How is this possible?

[Dyuman Joshi]

I will just point out that log is a built-in function (with many different implementations) in MATLAB, best to use a different name for your function.

which log -all

built-in (/MATLAB/toolbox/matlab/elfun/log)
built-in (/MATLAB/toolbox/matlab/elfun/@double/log)                                                            % double method
built-in (/MATLAB/toolbox/matlab/elfun/@single/log)                                                            % single method
/MATLAB/toolbox/matlab/datatypes/tabular/@tabular/log.m                                                        % tabular method
/MATLAB/toolbox/matlab/connector2/logger/+connector/+internal/Logger.p                                         % connector.internal.Logger method
log is a Java method                                                                                           % com.mathworks.matlabserver.connector.api.Connector method
/MATLAB/toolbox/matlab/testframework/unittest/core/+matlab/+unittest/+internal/Loggable.m                      % matlab.unittest.TestCase method
/MATLAB/toolbox/matlab/testframework/unittest/core/+matlab/+unittest/+fixtures/Fixture.m                       % matlab.unittest.fixtures.EmptyFixture method
log is a built-in method                                                                                       % matlab.automation.Verbosity method
log is a built-in method                                                                                       % matlab.unittest.internal.fixtures.FolderScope method
/MATLAB/toolbox/matlab/external/interfaces/webservices/restful/+matlab/+internal/+webservices/HTTPConnector.m  % matlab.internal.webservices.HTTPConnector method
built-in                                                                                                       % gpuArray method
/MATLAB/toolbox/coder/half/@bfloat16/log.p                                                                     % bfloat16 method
/MATLAB/toolbox/coder/half/@half/log.p                                                                         % half method
/MATLAB/toolbox/nnet/deep/deep/@dlarray/log.m                                                                  % dlarray method
/MATLAB/toolbox/parallel/array/distributed/@codistributed/log.m                                                % codistributed method
/MATLAB/toolbox/parallel/gpu/gpu/@gpuArray/log.m                                                               % gpuArray method
/MATLAB/toolbox/symbolic/symbolic/@sym/log.m                                                                   % sym method

[dpb]

Where in the python version is the file closed? I don't know nuthin' about python so not sure just what the "with open(filename, ..." sequence does, but I wonder if it isn't using the already opened file on subsequent passes as I don't see anything that looks like it would close it unless it's hidden in the write method which one wouldn't think would be so.

My penchant for doing similar is to also open the file at the beginning and then close it when done instead of every time if it's in a tight loop.

Also, is the time required dependent upon the size of the file -- I've not tried testing for it, but the fopen() with the append access requires a move of the file position pointer to the end of the file; wonder if that is taking up more time with longer file.

[Adam]

The with statement in python creates a context manager which will close the file correctly after the code inside has run. These two codes just do the same thing.

It is not always possible to keep the file open in between writes, especially when logging in a large codebase which might crash and then leave some file unclosed. So I prefer, for robustness of my codebase do open and close the file each time, something that apparently can be done quickly in python but not in Matlab

[dpb]

Create an onCleanup function to deal with that if crashes or user Ctrl-C's or the like.

Probably the difference between MATLAB and Python is in system buffering of when data are actually written to the file system. I'm pretty sure MATLAB flushes the buffer every time rather than caching and writing only larger blocks.

Can you verify if the code does crash that all logging data is saved both ways?