LSTM Video classification Matlab official example issue "unconnected output. each layer output must .."

Question (a): The Matlab example for Video classification might have an error included. It can be found here under "Create Network for Video Classification" https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.sequenceinputlayer.html#mw_d1a76588-f98f-4d19-9130-d4411b4a5ee9

The analyzeNetwork gives an error "unconnected output. each layer output must be connected to the input of another layer". I would like to get video classification using an RNN (LSTM) to work.

Can someone advise on how to properly connect the sequenceFoldingLayer('Name','fold') and the sequenceUnfoldingLayer('Name','unfold')?

Question (b): My final input would be consist of multiple RGB such as: inputSize = [28 28 3]. Would that change anything in the sequenceFoldingLayer('Name','fold') layer?

I appreciate your help in advance. Best regard.

 inputSize = [28 28 1];
    filterSize = [5 5];
    numFilters = 20;
    numHiddenUnits = 200;
    numClasses  = numel(unique(round(y_reg_vec_train)));
    clearvars layers
    layers = [ ...
        sequenceInputLayer(inputSize,'Name','input')        
        sequenceFoldingLayer('Name','fold')        
        convolution2dLayer(filterSize,numFilters,'Name','conv')
        batchNormalizationLayer('Name','bn')
        reluLayer('Name','relu')        
        sequenceUnfoldingLayer('Name','unfold')
        flattenLayer('Name','flatten')        
        lstmLayer(numHiddenUnits,'OutputMode','last','Name','lstm')        
        fullyConnectedLayer(numClasses, 'Name','fc')
        softmaxLayer('Name','softmax')
        classificationLayer('Name','classification')];

    
    lgraph = layerGraph(layers);
    lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');
    figure
    plot(lgraph)
    analyzeNetwork(layers)
    

Answer 1

[Antoni Woss]

Question (a): The layer array has disconnected sequence folding and unfolding layers as indicated in the network analyzer. In your snippet, the lgraph variable connects these sequence folding and unfolding layers. That is this snippet of code:

lgraph = connectLayers(lgraph,'fold/miniBatchSize','unfold/miniBatchSize');

You should run network analyzer on the layer graph, lgraph, to see these layers connected:

analyzeNetwork(lgraph)

Question (b): Regarding the input data, you would need to change the input size to the network to accommodate your 3 input channels, i.e. inputSize = [28 28 3] but do not need to change anything regarding the sequence folding and unfolding aspects of the network. These operate in the batch and time dimension only, the sequence folding collecting batch and time dimensions into a batch dimension (treating time as a batch label) and the unfolding undoing this transformation. Changes to the spatial or channel dimensions would not necessitate any changes to sequence folding and unfolding here.