Extracting subsets of structs from HDF5 files

Hi there,

I've just recently started working with HDF5 functions in MATLAB, and I am having trouble accessing some of my data in a particular way. Specifically, I am trying to access a subset of data in an HDF5 file, but this data is contained within a struct (which contains more structs, which then contain the actual data).

The example I'm showing here only contains a few records in each of the structs that contain real data, but I will soon be working with files that are much, much larger and are updating live. Because of the nature of our software requirements (very fast updating), we can't re-load the entire file each time--so I need to figure out a way to select a subset of the data inside the structs.

A cursory search on the MathWorks forums shows people having some difficulty with this even as late as 2011, and I was wondering if MATLAB has since implemented being able to read a subset of a struct inside an HDF5 file, or if a workaround has been devised for this, or if perhaps I'm just overlooking or misunderstanding the problem. (I am running MATLAB 2012a, but could potentially upgrade to 2013a if this version has the capabilities I require.)

If it helps, here is the layout of the kind of file I am dealing with (although the real one will have a much larger size, soon enough):

>> h5disp('demo.h5')
HDF5 demo.h5 
Group '/' 
    Dataset 'FDA_1' 
        Size:  8
        MaxSize:  Inf
        Datatype:   H5T_COMPOUND
            Member 'time_':  H5T_IEEE_F32LE (single)
            Member 'tempF_':  H5T_IEEE_F32LE (single)
        ChunkSize:  1000
        Filters:  none
        FillValue:  H5T_COMPOUND
    Dataset 'FDA_2' 
        Size:  13
        MaxSize:  Inf
        Datatype:   H5T_COMPOUND
            Member 'S1__WeatherReading':  H5T_COMPOUND
                Member 'time_':  H5T_IEEE_F32LE (single)
                Member 'temp_':  H5T_STD_I32LE (int32)
                Member 'pressure_':  H5T_STD_U32LE (uint32)
        ChunkSize:  1000
        Filters:  none
        FillValue:  H5T_COMPOUND

>> h5info('demo.h5')

ans = 

        Filename: 'demo.h5'
            Name: '/'
          Groups: []
        Datasets: [2x1 struct]
       Datatypes: []
           Links: []
      Attributes: []

To clarify, I would be happy if I could figure out a MATLAB function that would allow me to simply extract part of the contents of one of the sub-structs (like the last 50 records of "time_" or "temp_") without loading the rest of the file into a variable.

I'm not opposed to using low-level HDF5 functions, I just don't know if they can do what I need them to do in MATLAB.

Any guidance would be immensely appreciated!

Thanks, -Steve O.

Answer 1

[Ashish Uthama]

Steve,

You would have to use the low-level functions. I believe it can be done, but its a bit more complex than what I could put together an example for you on the fly.

To get you oriented, you could try following the reading compound datatype example first, this will show you have to get a "id" to the variable/field, and then you could combine it with this approach for reading hyper slabs.

You could also post to the HDF5 mailing list. Any low-level C code they show you or point you to could be converted to the low-level MATLAB API easily (there is a close 1:1 mapping).