CNN and LSTM error with input size

Hi,

I am building the following network

layers = [

sequenceInputLayer([13, 13,1],'Name','input')

sequenceFoldingLayer

convolution2dLayer(5,8)

batchNormalizationLayer

reluLayer

convolution2dLayer(5,16)

batchNormalizationLayer

reluLayer

convolution2dLayer(5,32)

batchNormalizationLayer

reluLayer

convolution2dLayer(5,64)

batchNormalizationLayer

reluLayer

sequenceUnfoldingLayer

flattenLayer

bilstmLayer(vars.numHiddenUnits,'OutputMode', 'sequence')

fullyConnectedLayer(100)

dropoutLayer(0.4)

fullyConnectedLayer(1)

regressionLayer];

I train the network with X and Y of sizes

size(X):

13 13 1 52

size(Y):

52 1

Ans I get the errors "Invalid training data. X and Y must have the same number of observations.". I have backtracted the error and it fails in file "NetworkDataValidator.m" line 858 because ismatrix(x) is False. Given that x is a 4D matrix.

I can't understand what I am doing wrong. Any help is greatly appreciated.

Best regards,

Annalisa

Answer 1

[Srivardhan Gadila]

Latest Edit:

In order to model the LSTM regression networks with 2-D data, the input should be a Nx1 cell, N being the number of observations. Each cell entry should then comprise a HxWxCxS array, where H = height, W=width, C=channels and S= sequence length. The responses, Y, can be a NxR matrix, where N = observations, R = number of responses (or output of the network) for sequence-to-one problems or a Nx1 cell array of RxS responses for sequence-to-sequence problems.

For more info refer to the sequences Input Argument of the function trainNetwork.

If in the above script S=54 and it's a sequence-to-sequence task with N=1, then the fix is simple:

X = {rand(13,11,1,54)};

Y = {rand(1,54)};

net = trainNetwork(X,Y,lgraph,options);

If N=54, and it's a sequence-to-one task, then I see two issues:

1. I don't see the S dimension. Assuming N=54 is the number of observations, I see only 54 arrays of 13x11x1 dimensions but I don't see a sequence of these arrays.

2. The 'OutputMode' of the bilstmLayer is set to 'sequence'. This would correspond to a sequence-to-sequence problem but the responses are only Nx1, so I assume the 'OutputMode' needs to be changed to 'last' to correspond to a sequence-to-one task.

For example, if you change the input and response to (where S=10):

X = arrayfun(@(x)rand(13,11,1,10),1:54,'UniformOutput',false)';

Y = rand(54,1);

Where, H=13,W=11,C=1,S=10,N=54,R=1, and change the OutputMode to 'last', then you can train the network.

bilstmLayer(100,'OutputMode', 'last','Name', 'bilstm')

Earlier suggestion:

Your layers itself has some errors, refer to the attached image for the errors. Use analyzeNetwork to check for the errors in your network as follows:

analyzeNetwork(layers)

I would suggest you to fix the errors in your network and then train the network. Since I don't have the value of the variable vars.numHiddenUnits, I used the value 200 for the bilstmLayer.

Refer to the example Train Convolutional Neural Network for Regression, and check the size of XTrain & YTrain to get an idea on what should be the dimension of the input data and the target data.

[Annalisa Riccardi]

Thanks for the reply. I have missed to include the lgraph part that connects the fold/unfold layers.

I have managed to reproduce a small example (here attached) that has the same dimension-mismatch error as in my implemenation. For sensitivity reason I cannot share my dataset so I added few lines to generate a matrix of random number instead.

If you run the example you can see the error "Invalid training data. X and Y must have the same number of observations." where

>> size(X)

ans =

13 11 1 54

>> size(Y)

ans =

1 54

And I am training with Y transpose so the dimension are correct

[Srivardhan Gadila]

@Annalisa Riccardi, I have updated the answer with necessary information which might help you.

[Annalisa Riccardi]

Sorry but that's not very helpful...

your example works as

>> [XTrain,~,YTrain] = digitTrain4DArrayData;

size(XTrain)

size(YTrain)

ans =

28 28 1 5000

ans =

5000 1

Mine as

>> size(X)

ans =

13 11 1 54

>> size(Y)

ans =

1 54

If you see in the attached file in my previous post I am training with the transpose of Y:

net = trainNetwork(X,Y',lgraph,options);

So the dimensions are exactly as in the example you have provided.

[Srivardhan Gadila]

@Annalisa Riccardi, I have updated the answer with information which might help you.

[HONGYI LI]

For N=54 and it's a sequence-to-one task，only 54 arrays of [13 13 1] is correct if S=1.