function who seems slow

5 views (last 30 days)
Kenny Nona
Kenny Nona on 16 Mar 2021
Commented: Kenny Nona on 2 Apr 2021
Hello,
I am using the MATLAB function 'who' to assess which variables are present in a .mat file. Basically something like this:
fileInfo = who('-file','filename.mat');
To return if the variable is inside the file or not, I use something like this:
ismember('my_variable',fileInfo)
My .mat file is about 11Gb in size and the 'who' function takes a very long time to evaluate (about 5min). Is there a faster way to get the variable names from a .mat file?
The .mat files are v7.3.
Regards,
Kenny

Accepted Answer

David Saidman
David Saidman on 1 Apr 2021
Edited: David Saidman on 1 Apr 2021
So its a bit messy, but I just tried this and its pretty much instant regardless of size using low level H5 libraries. Hope it works for you. I've tried on windows 2017b, havent tried in linux or other releases.
Note: This solution uses low level HDF5 libraries that are already built into matlab, so this method assumes your mat file is HDF5 (-v7.3). Otherwise it will not work.
I've tried checking 15 variables in a 3Gb file, worked basically instantly (didnt time it, fast enough for my needs).
You can be sure is a valid hdf5 file by doing this:
isValidHDF = H5F.is_hdf5('my_file.mat');
Then, if ValidHDF is true, check the variable is in the file by:
isThere = false; %Initialize as default value of false
fid = H5F.open('myfile.mat') % Use low level H5F builtin to open
try % Never use try/catch when possible but this is a good for when its ok, maybe someone can suggest alternative?
% Try to open the h5 group.
% Will error and catch to report back false if the variable isnt there
% Otherwise the variable exists
gid = H5G.open(fid,'/data_info'); % Note the "/". It required for H5 syntax and its OS independent (never "\" even in windows)
% I think this makes sure the variable isnt empty if the group opened successfully,
% you might not need this bit but has been working for me. Otherwise
% can just do if gid > 0
hInfo = H5G.get_info(gid);
isThere = hInfo.nlinks > 0;
H5G.close(gid);
end
H5F.close(fid);
  1 Comment
Kenny Nona
Kenny Nona on 2 Apr 2021
Works like a charm! With 'who' I got an execution time of 32sec, while with this code, using a for loop over all variables to check: 0.03sec! Thank you!

Sign in to comment.

More Answers (1)

Cameron B
Cameron B on 16 Mar 2021
Not sure, but maybe one of these is faster
load('filename.mat')
exist('my_variable')
or
load('filename.mat','my_variable')
  1 Comment
Kenny Nona
Kenny Nona on 16 Mar 2021
I did a quick test on a (smaller) file. Running the 'who' command:
tic
variableInfo = who('-file', 'filename.mat');
toc
Elapsed time is 47.116613 seconds.
While with load:
tic
load('filename.mat','my_variable')
toc
Elapsed time is 46.814521 seconds.
So, the load command is indeed (a bit) faster in my case, but still not nearly as fast as when you click on a mat file in the folder browser and within seconds you see its contents.

Sign in to comment.

Tags

Products


Release

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!