The CNN model only predicts a single class out of three?

Hello,

I am working on training a CNN model for pavement distress identification using image classification. I have three categories for defects (Alligator Cracks, Potholes and Rutting). When I run my model it only predicts a single class out of three rendering my model accuracy to 33.33%. I tried changing the training parameters/ options but the result is same. I was hoping if someone can help me figure out why. The code I am using is given below. Thanks in advance.

_______________________________________________________________________________________________________________________________________

rootFolder = fullfile ('CNN Input');

categories = {'Alligator Cracks','Potholes','Rutting'};

imds = imageDatastore (fullfile (rootFolder,categories), 'LabelSource', 'Foldernames');

tbl = countEachLabel (imds);

minSetCount = min(tbl{:,2});

countEachLabel(imds);

AlligatorCracks = find(imds.Labels == 'Alligator Cracks',1);

Potholes = find(imds.Labels == 'Potholes',1);

Rutting = find(imds.Labels == 'Rutting',1);

net = alexnet ();

rng ('default');

net.Layers(1);

net.Layers(end);

numel(net.Layers(end).ClassNames);

[trainingSet, testSet, validationSet] = splitEachLabel(imds,0.75,0.15,'randomize');

imageSize = net.Layers(1).InputSize;

augmentedTrainingSet = augmentedImageDatastore(imageSize,trainingSet,'ColorPreprocessing','gray2rgb');

augmentedTestSet = augmentedImageDatastore(imageSize,testSet,'ColorPreprocessing','gray2rgb');

augmentedValidationSet = augmentedImageDatastore(imageSize,validationSet,'ColorPreprocessing','gray2rgb');

w1 = net.Layers(2).Weights;

w1 = mat2gray(w1);

featurelayer = 'drop7';

trainingFeatures = activations(net, augmentedTrainingSet, featurelayer,'MiniBatchSize', 32,'OutputAs', 'columns');

options = trainingOptions('sgdm', ...

'LearnRateSchedule','piecewise',...

'LearnRateDropFactor',0.2,...

'learnRateDropPeriod',1,...

'MaxEpochs',8,...

'MiniBatchSize',16,...

'Plots','training-progress',...

'Shuffle','once');

numClasses = 3;

layersTransfer = net.Layers(1:end-3);

layers = [

layersTransfer

fullyConnectedLayer(numClasses,'WeightLearnRateFactor',20,'BiasLearnRateFactor',20)

softmaxLayer

classificationLayer];

trainedNet = trainNetwork(augmentedTrainingSet,layers,options);

trainingLabels = trainingSet.Labels;

classifier = fitcecoc(trainingFeatures, trainingLabels, 'Learner', 'Linear', 'Coding', 'onevsall','observationsIn','columns');

testFeatures = activations(trainedNet,augmentedTestSet, featurelayer,'MiniBatchSize', 32,'OutputAs', 'columns');

validationFeatures = activations(trainedNet,augmentedValidationSet, featurelayer,'MiniBatchSize', 32,'OutputAs', 'columns');

predictLabels = predict(classifier,testFeatures,'ObservationsIn','columns');

predictLabelss = predict(classifier,validationFeatures,'ObservationsIn','columns');

testLables = testSet.Labels;

validationLables = validationSet.Labels;

confMat = confusionmat(testLables, predictLabels);

confMatt = confusionmat(validationLables, predictLabelss);

confMat = bsxfun(@rdivide, confMat, sum(confMat,2));

confMatt = bsxfun(@rdivide, confMatt, sum(confMatt,2));

mean(diag(confMat));

mean(diag(confMatt));

Answer 1

[Shishir Singhal]

Hi, You facing this problem may be because of class imbalance.

Just check the number of datapoints you have in each class.

To handle the class imbalance, you can use data augmentation techniques.

For data augmentation in MATLAB, you can refer to this link: https://www.mathworks.com/help/deeplearning/ref/imagedataaugmenter.html

[Vinay Chawla]

Thank you for responding. I am very new to MATLAB, so please bear with me.

The datapoints you are referring to are the number of images, right? I used an equal number of images for each class. As for the augmnentation, I did use the data augmnentation for each of the three groups i.e, training, test and validation set. Are you saying I should augment the data multiple times?

I appreciate you answering the question, but if you can explain it to me in easier terms that would be great.

Thanks once again.

[Shishir Singhal]

Yes, do data augmentation multiple times.

You can refer to these links to know more about data augmentation techniques:

https://blogs.mathworks.com/deep-learning/2019/08/22/data-augmentation-for-image-classification-applications-using-deep-learning/

https://in.mathworks.com/matlabcentral/fileexchange/68728-image-classification-using-data-augmentation

In your case, it also might happened an unusual split.

For example, it may happen that our training set consist more examples for a particular class. Please check the distribution of each class in train, test and validation set.

Moreover, first apply data augmentation technique in your data. And then do stratified split. Because, if there is an unsual split, and you are applying data augmentation on top of it, then you have almost similiar kind of images in your training set which makes your traning model bias towards a particular class.

[Vinay Chawla]

Thank you for answering. I will try multiple augmnetation.

I believe the images in eact class are the same number but I will look into it and see how the data is distributed. Can you please provide me with the line code to do that or any link that would help me in doing that. As I mentioned earlier, I do not know MATLAB very well and I just started with learning CNN.

Your help is really appreciated.

Thank you very much.

Answer 2

[Vinay Chawla]

I found my model working by changing the training options. One can try a different combination of base learning rate and mini-batch size along with number of epochs. It really is a matter of choice but you have to find the combination that works for your problem.