(Not recommended) Read data from custom mini-batch datastore
read method of
matlab.io.datastore.MiniBatchable is not recommended. For more
information, see Compatibility
data — Output data
Output data, returned as a table with
MiniBatchSize number of rows. For the last
mini-batch of data in the datastore, if
NumObservations is not evenly divisible by
contain the remaining observations in the datastore (a partial batch smaller
The table should have two columns, with predictors in the first column and responses in the second column.
info — Information about read data
Information about read data, returned as a structure array.
To learn about attributes of methods, see Method Attributes.
Version HistoryIntroduced in R2018a
read is not recommended
Before R2018a, to perform custom image preprocessing for training deep learning
networks, you had to specify a custom read function using the
readFcn property of
However, reading files using a custom read function was slow because
imageDatastore did not prefetch files.
In R2018a, four classes including
matlab.io.datastore.MiniBatchable were introduced as a
solution to perform custom image preprocessing with support for prefetching,
shuffling, and parallel training. Implementing a custom mini-batch datastore using
matlab.io.datastore.MiniBatchable has several challenges and limitations.
In addition to specifying the preprocessing operations, you must also define properties and methods to support reading data in batches, reading data by index, and partitioning and shuffling data.
You must specify a value for the
NumObservationsproperty, but this value may be ill-defined or difficult to define in real-world applications.
Custom mini-batch datastores are not flexible enough to support common deep learning workflows, such as deployed workflows using GPU Coder™.
Starting in R2019a, built-in datastores natively support prefetch, shuffling, and
parallel training when reading batches of data. The
transform function is the preferred way to perform custom data
preprocessing, or transformations. The
combine function is the preferred way to concatenate read data from
multiple datastores, including transformed datastores. Concatenated data can serve
as the network inputs and expected responses for training deep learning networks.
combine functions have
several advantages over
The functions enable data preprocessing and concatenation for all types of datastores, including
transformfunction only requires you to define the data processing pipeline.
When used on a deterministic datastore, the functions support
talldata types and MapReduce.
The functions support deployed workflows.
The recommended solution to transform data with basic image preprocessing
operations, including resizing, rotation, and reflection, is
augmentedImageDatastore. For more information, see Preprocess Images for Deep Learning.
There are no plans to remove the
read method of
matlab.io.datastore.MiniBatchable at this time.