Readmatrix in struct using parfor
5 views (last 30 days)
Show older comments
Hi,
My for-loop that reads multiple .csv files (1 file per iteration) takes too much time. Therefore, I would like to increase the performance using parfor.
Using parfor and structs seems not to be straightforward. How can I adjust the following code to use a parfor?
Thanks.
% load the data from each file
% parfor i = 1:length(Files2Read)
for i = 1:length(Files2Read)
% create fieldname for each file to read
fld = ['File#', num2str(i) ];
% load data
Data.(fld) = readmatrix(Files2Read{i});
% store additional information:
Data.General.FieldName{i} = fld;
end
1 Comment
Stephen23
on 27 Jun 2022
" How can I adjust the following code to use a parfor?"
Use actual indexing into a structure array rather than forcing a pseudo-index into the fieldname.
Accepted Answer
Jan
on 27 Jun 2022
Edited: Jan
on 27 Jun 2022
Fields names cannot contain the charcater '#' .
fld = sprintf('File_%d', i);
Data.(fld) = readmatrix(Files2Read{i});
This should work.
AS Stephen told already, this would be nicer:
Data(i).Value = readmatrix(Files2Read{i});
or
Data.File(i) = readmatrix(Files2Read{i}); % Or File{i}
By the way, if the disk access is the bottleneck, parfor cannot help. Please post, if you code runs faster than with for.
3 Comments
Jan
on 28 Jun 2022
Starting the parallel pool takes some time, but remember, that reading the files might store their contents in a RAM cache also. Then optimizing the code for repeated reading has much less effects in real applications.
More Answers (0)
See Also
Categories
Find more on Matrices and Arrays in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!