502 views (last 30 days)

I would like to save some variables to MAT files from inside a PARFOR loop. However I get an error:

??? Error using ==> parallel_function at 598

Error in ==> parallel_function>make_general_channel/channel_general at 894

Transparency violation error.

See Parallel Computing Toolbox documentation about Transparency.

MathWorks Support Team
on 24 Feb 2020

Edited: MathWorks Support Team
on 24 Feb 2020

Transparency is violated by the SAVE command because in general MATLAB cannot determine which variables from the workspace will be saved to a file.

The solution is to move the call to SAVE to a separate function and to call that function from inside the PARFOR loop. Pass any variables that are to be saved as arguments to the function. That way MATLAB can determine which ones will be saved and transparency is not violated.

For example:

Save the following as "parsave.m":

function parsave(fname, x,y)

save(fname, 'x', 'y')

end

Then run it with:

parfor ii = 1:4

x = rand(10,10);

y = ones(1,3);

parsave(sprintf('output%d.mat', ii), x, y);

end

Similarly, for loading several files with enumerated file names, we can use the following code.

function [output] = par_load(i)

eval(['load file_' num2str(i) '.mat']);

output = time;

end

Which can be run with:

parfor i=1:4

data = par_load(i);

a(i) = 10 + data(1);

end

Daniel
on 14 Mar 2016

A nice alternative to implement your own save function is using the built-in matfile function:

parfor ii = 1:4

m=matfile(sprintf('output%d.mat', ii),'writable',true)

x = rand(10,10);

y = ones(1,3);

m.x=x;

m.y=y;

end

Here the parser has no issue understanding which variables are used.

Tihomir Kostadinov
on 22 Aug 2014

Edited: Tihomir Kostadinov
on 22 Aug 2014

I have enhanced the function that allows saving within a parfor loop as follows. This should allow giving it a variable number of arguments and the function recognizes their names. Let me know if there are problems with this implementation.

function save_for_parfor(fname,numvars,varargin)

for i = 1:numvars

eval([inputname(i+2),' = varargin{i};']);

end

save('-mat',fname,inputname(3));

for i = 2:numvars

save('-mat',fname,inputname(i+2),'-append');

end

This function would be called as follows:

>> x = 3; y = 4; z = 5; g = 'tester_parfor';

>> save_for_parfor('tester.mat',4,x,y,z,g)

Walter Roberson
on 13 Mar 2016

Sign in to comment.

Qifeng Chen
on 30 Nov 2014

This is a better implementation based on Tihomir. The code here does not require the input of "numvars".

function parsave(fname,varargin)

numvars=numel(varargin);

for i=1:numvars

eval([inputname(i+1),'=varargin{i};']);

end

save('-mat',fname,inputname(2));

for i = 2:numvars

save('-mat',fname,inputname(i+1),'-append');

end

end

Linus Schumacher
on 23 May 2017

Erik's solution still give me a transparency violation error in Line 6 (savevar...)

Myrtle42
on 16 Jun 2017

This still gives an error in Matlab R2016a: the variable I'm trying to save cannot be classified. Same with Alex Shaw's version below. Any ideas? Here is my code:

function test

var_to_save = NaN(10,1);

parfor ii = 1:10

var_to_save(ii,:) = rand;

parsave('test.mat',var_to_save)

end

end

function parsave(varargin)

savefile = varargin{1}; % first input argument

for i = 2:nargin

savevar.(inputname(i)) = varargin{i}; % other input arguments

end

save(savefile,'-struct','savevar')

end

Sign in to comment.

Alex Shaw
on 9 Dec 2015

Edited: Alex Shaw
on 9 Dec 2015

I encountered this issue, and used the function proposed by Qifeng Chen.

This worked very well in 2015a.

However, since upgrading to 2015b my code reports a transparancy violation again. The release notes state that transparancy is now enforced more strictly, however all variables that I attempt to save either appear as inputs in expression in the parfor loop, or are the results of calculation in the parfor loop so I think this may be a bug.

Further to this, the affected function seems to be inputname(). The simplest (and probably fastest) way to work round this is to add all variables to a single structure variable, and then save that with a fixed name:

function parsave_simple(fname,C )

save('-mat',fname,'C');

end

however, this would have been a lot of surgery in my case, plus other code expects the variables not to be in structure format. So the 'minimal surgery' option in my case is simply to pass the names of variables into the function via a cell array:

function parsave_named(fname____,vnames___,varargin)

numvars___=numel(varargin);

for ii___=1:numvars___

eval([vnames___{ii___},'=varargin{ii___};']);

end

save('-mat',fname____,vnames___{1});

for ii___ = 2:numvars___

save('-mat',fname____,vnames___{ii___},'-append');

end

end

This method seems a bit messy, there is a performance overhead to calling 'eval' and some very hard to trace bugs will arise if names do not match the actual variables. Also note the use of strange var names to ensure eval does not overwrite any internal logic! But on the other hand, it only needs one extra line in the calling loop to populate the names variable.

echo zhang
on 11 Apr 2019

This works for me, and a working example below:

parfor ii = 1:n

x=1;

y=2;

z=3;

var_name = ["x","y","z"];

parsave('whatever.mat',var_name,x,y,z);

end

Sign in to comment.

Anand
on 13 Mar 2016

Edited: Anand
on 13 Mar 2016

Another addition to various versions of doing this. This one does not use a structure. It, IMO, combines the best features of all answers above.

function parfor_save(varargin)

fname=varargin{1};

for i=2:nargin

eval([inputname(i),'=varargin{i};']);

if i==2

save('-mat',fname,inputname(i));

else

save('-mat',fname,inputname(i),'-append');

end

end

Walter Roberson
on 13 Mar 2016

Sign in to comment.

CrossEntropy
on 3 Dec 2019

Sign in to answer this question.

Opportunities for recent engineering grads.

Apply Today
## 4 Comments

## Direct link to this comment

https://au.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox#comment_362678

⋮## Direct link to this comment

https://au.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox#comment_362678

## Direct link to this comment

https://au.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox#comment_362703

⋮## Direct link to this comment

https://au.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox#comment_362703

## Direct link to this comment

https://au.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox#comment_413376

⋮## Direct link to this comment

https://au.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox#comment_413376

## Direct link to this comment

https://au.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox#comment_653370

⋮## Direct link to this comment

https://au.mathworks.com/matlabcentral/answers/135285-how-do-i-use-save-with-a-parfor-loop-using-parallel-computing-toolbox#comment_653370

Sign in to comment.