Deep Learning Toolbox - How to get Video Inputs in the Network, and use Videos in training?

Hello, I am a new user to the Deep Learning Toolbox.

I am looking through the software and I have only seen the ability to make a network with a image input, and in the training area, it hasn't let me use videos for training. Any help would be appreciated.

[Lars]

If there is no way to do this in the Deep Learning Toolbox, where should I look next?

Answer 1

[Shreeya]

Hi

To use video data for training a model using in the Deep Learning Toolbox, the `VideoReader` function in MATLAB to read video files frame by frame can be used. You can refer to the documentation linked below for understanding the implementation:

https://www.mathworks.com/help/matlab/ref/videoreader.html

Below is a sample code to read the video files and use them as training data

v = VideoReader(videoFile);
frames = [];
while hasFrame(v)
    %Preprocess and store the data as per the requirements
end

% Sample network architecture
layers = [
    imageInputLayer([224 224 3])
    convolution2dLayer(3, 8, 'Padding', 'same')
    batchNormalizationLayer
    reluLayer
    fullyConnectedLayer(2)
    softmaxLayer
    classificationLayer];

%Train the Network
options = trainingOptions('sgdm', ...
    'MaxEpochs', 10, ...
    'InitialLearnRate', 0.01, ...
    'Verbose', false, ...
    'Plots', 'training-progress');
trainNetwork(frames, labels, layers, options);

Let me know if this helps!