Error forming mini-batch of targets for network output. Layer output format is not consistent with the input data.

I want to classify images into one of 5 categories by trying to train a network using trainnet, and I have both numeric and image data. The image data is RGB images of type double and size 61x61x3. Every image is associated with a numeric vector 1x5 and a categorical label (5 classes in total). I have created three CombinedDatastores (for train, validation and testing) by combining imageDatastores and arrayDatastores. The command:

read(dsTrain)

gives a 1x2 cell array of a 61x61x3 double (the image) and a 1x5 vector (the numeric data).

I have defined the options as follows:

options = trainingOptions("sgdm", ...
    InitialLearnRate=0.01, ...
    MaxEpochs=15, ...
    Shuffle="every-epoch", ...
    ValidationData=dsValidation, ...
    ValidationFrequency=30, ...
    Plots="training-progress", ...
    Metrics="accuracy", ...
    Verbose=true, ...
    MiniBatchSize=64, ...
    LearnRateSchedule="piecewise", ...
    L2Regularization=1e-2);

The network architecture looks like this (sorry for the bad quality):

When trying to run trainnet, I get the following error:

Error using trainnet

Error forming mini-batch of targets for network output "softmax". Layer output format "BC" is not consistent with the input data.

Caused by: Index exceeds the number of array elements. Index must not exceed 2.

Is it possible that I have done something wrong then combining the datastores, or have I constructed the network wrong? Using Matlab 2023b.

All help is deeply appreciated,

[mad_de]

I managed to solve it by adding the labels as a second arrayDatastore when combining. Can anyone tell me why this solved it? :D

Answer 1

[Pratyush Swain]

Hi mad_de,

This error suggests there was a mismatch between the network's expected output format and the format of the training labels/targets being provided.CombinedDatastore in your case should consist of Datastores of images,numeric vectors (predictors) and also labels (responses)

You can refer to the example here:https://www.mathworks.com/help/deeplearning/ug/train-network-on-image-and-feature-data.html

The data loading and combining workflow in the above example can be analyzed as follows:

% Load the digits images, labels, and clockwise rotation angles.
load DigitsDataTrain

% To train a network with multiple inputs using the trainnet function, 
% create a single datastore that contains the training predictors and responses.

dsX1Train = arrayDatastore(XTrain,IterationDimension=4);
dsX2Train = arrayDatastore(anglesTrain);
dsTTrain = arrayDatastore(labelsTrain);

% Combining images, clockwise rotation angles (numeric vector) &
% label(categorical)
dsTrain = combine(dsX1Train,dsX2Train,dsTTrain);

Hence by adding the labels as a arrayDatastore and combining it with the original CombinedDatastore, you effectively aligned the training targets with the network's expectations.

Hope this helps.