Which are the training examples in a sequence classification?

I am implementing a Sequence-to-Sequence Classification. As an example here it is the documentation from mathworks:

https://it.mathworks.com/help/deeplearning/examples/sequence-to-sequence-classification-using-deep-learning.html

Exploring the various fields of the function trainingOptions i have a doubt about the definition of the 'MiniBatchSize' and the training sample definition. If i have 8 sequences of 2000 time samples each which are considered training samples for the mini-batch?

• the 8 sequences, so if i define a minibatch of 1 i will feed the networks with 2000 samples each training iteration.
• the 2000x8 samples, so if i define a minibatch of 100 i will feed the networks with 100 time samples at each iteration.

Answer 1

[Raunak Gupta]

Hi,

The 'MiniBatchSize' creates batches in the observation dimension, not the time dimension. For example, having 8 sequences with 2000 length each if you want to have 2 batches for training in each epoch the MiniBatchSize will be 4.

If you want to create batches across time dimension you can use ‘SequenceLength’ parameter. The default 'SequenceLength' parameter is ‘longest’, which means that the batch sequence length is determined by the longest sequence in the batch. So, if you want to create a minibatch of 100 samples setting the ‘SequenceLength’ to 100 will create 20 small sequence for each sequence and based on MinBatchSize the overall iterations will be calculated.

So, to correctly state if you define 'SequenceLength' of 100 and ‘MiniBatchSize’ of 1, then the trainNetwork will feed the network with 100 time samples at each iteration.

Hope this clarifies.