Why is the CNN predict function faster when running a set of images from ImageDataStore compared to running each image individually?
3 views (last 30 days)
I am trying this example code
One thing I notice is that running the Classify function with the imageDataStore (imdsValidation)is much faster than running one image multiple times. Is this some sort of batch process that is using Matlab vectorization to speed things up or is it inherent to a CNN?
YPred = classify(net,imdsValidation);
And a related question, when doing codegen or CNNcodegen, is the resulting C++ code able to also run multiple images like this? I cannot see a way to do it with the C++ output code.
Vineet Joshi on 20 Jul 2021
As you can see in the documentation page classify - MiniBatchSize, larger mini-batches can result in faster prediction but requires more memory and hence it is not something specific to the CNN network.
As for your second question, the CNNcodegen function only generates the codes for the network, how you inference it depends on your choice. You can write the code to sequencially inference the network and get the C++ code, or use other techniques like multiple workers and parallel computing to make it faster in a batch setting.
Hope this was helpful.