fopen and fprintf do not work within parfor or smpd running on grid

Hello everyone,

I met a problem to use fopen and fprintf within parfor or smpd when I run my codes on grid. Here are my codes:

    spmd

fid = fopen(sprintf( 'worker_%d.csv', labindex ),'w');

fprintf(fid,'%s\r\n',vector1);

fclose(fid);

...

end

OR

parfor i=1:100

t = getCurrentTask(); 

fid = fopen(sprintf( 'worker_%d.csv', t.ID ),'w');

fprintf(fid,'%s\r\n',vector1);

fclose(fid);

...

end

Neither one would work, and both of them will generated the similar error messages:

spmd:

??? Error using ==> spmd_feval at 8
Error detected on lab(s) 2 15 16

Error in ==> BruteForce2>(spmd) at 103
spmd

Error in ==> BruteForce2 at 103
spmd

Caused by:

      Invalid file identifier.  Use fopen to generate a valid file identifier.

      Error stack:
      BruteForce2>(spmd body) at 105

      Invalid file identifier.  Use fopen to generate a valid file identifier.

      Error stack:
      BruteForce2>(spmd body) at 105

      Invalid file identifier.  Use fopen to generate a valid file identifier.

      Error stack:
      BruteForce2>(spmd body) at 105

PARFOR:

??? Error using ==> parallel_function at 598
Error in ==> BruteForce2>(parfor body) at 146
Invalid file identifier.  Use fopen to generate a valid file identifier.

Error in ==> BruteForce2 at 105
parfor i = 1:g

The problem is my fopen, fid returns -1. The codes are running ok with local workers. So would it be some writing permission issues? I would like to know what the problem is, and where our admin should change on his side. Could anyone give me any suggestion? Thanks a lot!

Rui

Answer 1

[Walter Roberson]

fname = sprintf( 'worker_%d.csv', t.ID );
[fid, message] = fopen(fname, 'w');
if fid < 0;
 fprintf(2, 'failed to open "%s" because "%s"\n', fname, message);
 %and here, get out gracefully
end

[Rui]

Hi Walter, thanks for helping. I currently don't have access to server. But I know that fid = -1, and message is probably "permission denied..."

Rui

[Walter Roberson]

If the message is "permission denied" then I suggest you print out which directory you are in.

[Rui]

I think I have the written permission to that directory. I could create folder manually there. I tried spmd which... end or spmd type... end, both of them were working all right. Is that what you mean?

[Walter Roberson]

I speculate that it has finding one of the files already open. Print out each filename as it is to be opened, and when the open fails, say which one.

[Rui]

Walter, sorry...but could you give me the codes to print out failing open? Thanks!

[Walter Roberson]

The code here in this Answer detects the failure and prints out a message.

[Rui]

Walter, here is the error message:

Lab 13: failed to open "worker_13.csv" because "Permission denied"

Lab 14: failed to open "worker_14.csv" because "Permission denied"

Lab 15: failed to open "worker_15.csv" because "Permission denied"

Lab 16: failed to open "worker_16.csv" because "Permission denied"

my codes are

spmd
fname = sprintf( 'worker_%d.csv', labindex );
[fid, message] = fopen(fname, 'w');
if fid < 0;
 fprintf(2, 'failed to open "%s" because "%s"\n', fname, message);
 %and here, get out gracefully
end
end

[Walter Roberson]

Are all the fopen() failing, or only some of them? Try something like,

fname = sprintf( 'worker_%d.csv', t.ID );
fprintf(2, 'About to open "%s"\n', fname);
[fid, message] = fopen(fname, 'w');
if fid < 0;
 fprintf(2, 'failed to open "%s" because "%s"\n', fname, message);
 %and here, get out gracefully
else
   fprintf(2, 'Success opening "%s"\n', fname);
   fprintf(fid,'%s\r\n',vector1);
   fclose(fid);
   fprintf(2,, 'Closed file "%s"\n', fname);
end

[Rui]

Walter, Just some of them. I have 16 workers. With help of our local unix expert, we find that if I login into Permission-denied machine manually using ssh, then I do not have this problem anymore. So it looks like a configuration problem on server side. We have sent them emails. Thanks a lot for your help!

        Lab  1 to 12: 
          About to open "worker_6.csv"
          Success opening "worker_6.csv"
          Closed file "worker_6.csv"
        Lab 13: 
          About to open "worker_13.csv"
          failed to open "worker_13.csv" because "Permission denied"

Answer 2

[Jason Ross]

It's possible that the workers are running under a different user name than yours. You haven't provided the specifics of the operating systems and what schedulers you are using, but you can get this information using

spmd
system('whoami')
end

And then make sure that username also has access to the directories where you want to open the files.

[Rui]

Jason, thanks for help! username is all right, we just figure out that this is possibly due to a configuration issue on server. Manually logging into machines using ssh could solve my problem. There may be something wrong with the interface between matlab and grid machines.

Answer 3

[Jan]

• Do you have write permissions to the current directory?
• Is the file open already from a former crashing call? See fopen all.
• Is the file opened from another application?

Using absolute filenames is an important strategy especially for parallel processing and multi-threading. Checking the success of FOPEN is a good programming practize also.

[Rui]

Hi Jan,

thanks for the help! Yes, I do have write permission to the directory, I can manually create folders under that directory. Matlab can create files there, too, if not using workers from grid. I closed matlab, then opened it again. It should release the all files, right? But the problem was still there. I do not think any other application will need it on server.

I could do a double check on everything on Monday. It seems to me that I have enough permission to do anything, and also matlab local workers, but not matlab grid workers... however, if I try spmd which... end or spmd type... end, they both work with no error report.

is it ever possible? what may lead to that?

Rui

[Rui]

Jan,

Just checked everything again. fopen all shows that nothing is open. I do have the permission, but not the grid workers.