Multi-input imagedatastore

50 views (last 30 days)
OJ27
OJ27 on 31 Aug 2020
Commented: Mohammad Sami on 21 Oct 2020
I am trying to train a network with N inputs to perform binary classification. Each image is multispectral (more than 3 channels rgb). I read in a comment from mathworks staff which creates data and stores it in a datastore to train a network with two inputs: image and vector data.
I have also looked at the example Image Classification using CNN with Multi Input, which uses two inuts to classify digits.
Both of these examples do not use imageDatastore. I was wondering if we could use
combine
to combine the imageDatastore for each data sample. I read in Datastores for Deep Learning that the datastore must be a combined or transformed datastore that returns a cell array with (numInputs+1) columns containing the predictors and the responses, where numInputs is the number of network inputs and numResponses is the number of responses.
My question is the following. Can I have multiple imageDatastore objects and combine them? If so, how do I store the label column? I tried the code below and got an error.
imds1 = imageDatastore(...);
imds2 = imageDatastore(...);
labels = ???
datastore = combine(imds1, imds2, labels);
Usually, we can assign the label through imds1.Labels. I also want to know if once combined, data augmentation can be done on the images and random split for training and validation.

Answers (1)

Mohammad Sami
Mohammad Sami on 5 Sep 2020
Edited: Mohammad Sami on 5 Sep 2020
You can save the labels as a text file and then create a tabularTextDatastore to read them back.
Something like this could work.
read_size = 4; % readsize must be fixed otherwise concatenation will fail
digitDatasetPath = fullfile(matlabroot,'toolbox','nnet', ...
'nndemos','nndatasets','DigitDataset');
imds = imageDatastore(digitDatasetPath, ...
'IncludeSubfolders',true, ...
'LabelSource','foldernames');
imds.ReadSize = read_size;
labels = imds.Labels; % your labels here
% the order of your images and label must be the same
writematrix(labels,'labels.txt');
% %C = categorical
labelStore = tabularTextDatastore('labels.txt','TextscanFormats','%C',"ReadVariableNames",false);
labelStore.ReadSize = read_size;
labelStoreCell = transform(labelStore,@setcat_and_table_to_cell);
finalStore = combine(imds,labelStoreCell);
% test read
finalStore.read
You will also need to correct the categorical array categories during transformation.
function [dataout] = setcat_and_table_to_cell(datain)
validcats = string(0:9); % define valid labels for categorical array
datain.(1) = setcats(datain.(1),validcats);
dataout = table2cell(datain);
end
  9 Comments
Mohammad Sami
Mohammad Sami on 21 Oct 2020
It's because we set the 10 categories in the setcat_and_table_to_cell.
You should change the variable validcats to the appropriate 7 categories for your case.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!