CNN implementation error #while training the model #error in input

I am doing a bearing faults classification project with CWRU datset using CNN. I have seperated the zero load datas in a csv file. Its size is 2782629x2 and contains 14 kind of faults. I have done the segmation as given below. The data is 11mb, so i have attached a link to data ' https://drive.google.com/file/d/13pdJDMBb0L3M4UXZ8gP5HeC6ry0cxDax/view?usp=sharing ' . The code is as follows. The error is mentioned below

df = readtable("all_faults.csv")
%Segmentation
win_len = 784;
stride = 300;
X = [];
Y = [];

faults = unique(df.fault);
for k = 1:numel(faults)
    df_temp_2 = df(strcmp(df.fault, faults{k}), :);    
    for i = 1:stride:(size(df_temp_2, 1) - win_len + 1)
        temp = df_temp_2{i:(i + win_len - 1), 1:end-1}; % Extract features, excluding the last column (labels)
        temp = temp(:)'; % Reshape to a row vector        
        X = [X; temp];       
        % Append the label for the last time step in the window
        Y = [Y; df_temp_2{i + win_len - 1, end}];
    end
end

% Now, 'X' contains the segmented and reshaped feature data, and 'Y' contains corresponding labels

Y = grp2idx(Y)
Y =(categorical(Y))

%Y_OHE = onehotencode(Y,2)
%[tried using onehotencoding also, the error in
%this case is also mentioned below. Please help me understand the correct
%usage of onehotencoding if i am wrong here]

%splitting data training

testProportion = 0.5;
% Split the data into training and testing sets
cv = cvpartition(Y, 'HoldOut', testProportion);
idxTrain = training(cv);
idxTest = test(cv);
%Reshaping the X for CNN purpose
X = reshape(X, [size(X, 1), 28, 28, 1]);


Y_train = Y(idxTrain)
Y_test = Y(idxTest)
X_train = X(idxTrain,:,:)
X_test = X(idxTest,:,:)

incase of using Onehotencoding i edited the above four lines as this

Y_train = Y_OHE(idxTrain,:)

Y_test = Y_OHE(idxTest,:)

X_train = X(idxTrain,:,:)

X_test = X(idxTest,:,:)

no_classes = numel(unique(df.fault)); % Number of classes

layers = [
    imageInputLayer([28 28 1])    
    convolution2dLayer([20 3], 32, 'Padding', 'same')
    reluLayer    
    maxPooling2dLayer([20 2], 'Stride', [5 5], 'Padding', 'same')    
    convolution2dLayer([10 3], 64, 'Padding', 'same')
    reluLayer    
    maxPooling2dLayer([10 2], 'Stride', [3 3], 'Padding', 'same')    
    fullyConnectedLayer(128)
    reluLayer    
    fullyConnectedLayer(no_classes)
    softmaxLayer    
    classificationLayer
];
options = trainingOptions('adam', ...
    'MaxEpochs', 50, ...
    'MiniBatchSize', 500, ...
    'Shuffle', 'every-epoch', ...
    'ValidationFrequency', 10, ...
    'ValidationData', {X_test, Y_test}, ...
    'Verbose', true, ...
    'Plots', 'training-progress', ...
    'ExecutionEnvironment', 'auto'); % or 'gpu' if available
% Create and compile the model
cnn_model = trainNetwork(X_train,Y_train , layers, options);

This is the error i get from above code

Error using trainNetwork

Number of observations in X and Y disagree.

While using onehotencoding i am getting this error

Error using trainNetwork

Invalid training data. For image, sequence-to-label, and feature classification tasks, responses must be categorical.

Please help me run the model. As I am new in this field , it would be helpfull if someone could help me clarrify the concepts.

Answer 1

[Milan Bansal]

Hi,

I understand that you are facing an error while training the Convolution Neural Network using "trainNetwork" function.

While using the "trainNetwork" function with images as "numeric array", the dimensions of "X " and "Y" are required to be :

X – 4-D – h × w × c × N

Y – 1-D – N × 1

As per the given code, "X" is a reshaped data with dimensions (9246 × 28 × 28 × 1) and "Y" has the dimension (9246× 1). The number of observations in "X" i.e. 9246 must be the fourth dimension.

In order to resolve the issue, reshape "X" such that its fourth dimension contains the value 9246. Refer to the code given below.

X = reshape(X, [28, 28, 1,size(X, 1)]);

Correspondingly, change the splitting of "X_train" and "X_test" such that they achieve the required dimension. Refer to the code given below.

X_train = X(:,:,:,idxTrain)
X_test = X(:,:,:,idxTest)

Refer to the documention link to learn more about the data formats in "trainNetwork" function.

https://www.mathworks.com/help/deeplearning/ref/trainnetwork.html#bu6sn4c-2

Hope it helps!