Can anyone explain why CNN model trained using Deep Learning Toolbox in Matlab predicts much faster than one built in Pytorch or TensorFlow?

Hi! I have been noticing that CNN models trained using Deep Learning Toolbox in Matlab execute prediction much faster than a similar model trained in Pytorch or TensorFlow. For example, if I train an Inception model of certain depth in Matlab and then train a similar model in Pytorch or TensorFlow, I find that running my test set through the trained model ( saved .mat file on Matlab or .h5 file from Keras), takes much longer with the h5 file than the ,mat file from Matlab. Any idea why the Matlab saved model executes much faster?

Answer 1

[David Willingham]

Hi Cat-22,

Thanks for bringing this to our attention! It would be great to look into this benchmark you've raised. Would you be able to provide a little more information? I.e.:

• what versions of Tensorflow, PyTorch and MATLAB you were using?
• are you able to share the timings?
• what was the depth of the inception model were you training?

David

Deep Learning Product Manager

[Cat-22]

Doesn't really matter much. Whichever pre-trained model you may choose, the performance difference persists. For example, take ResNet18 model from Deep Learning tool and in Keras Tensorflow. Modify just the top and bottom layers for your purpose in Matlab and Tensorflow. Train it with your examples. The training and validation values are different with the same hyperparameters with Matlab providing a better performance for the same number of epochs. Not sure if it has to do with initialization of weights. I use the default that's provided by Matlab and by Keras.

In addition, the trained model can be saved in Matlab (as .mat file) and in Keras (as .h5) file. If you run test examples through these files, Matlab provides classification much faster than when run through .h5 file in Python/Tensorflow environment with the same hardware available.