estimateNetworkMetrics

Estimate network metrics for specific layers of a neural network

Since R2022a

Syntax

dataTable = estimateNetworkMetrics(net)

[dataTable1,dataTable2,…,dataTablen] = estimateNetworkMetrics(net1,net2,…,netn)

Description

dataTable = estimateNetworkMetrics(net) returns a table containing these estimated layer-wise metrics for a deep neural network:

• LayerName — Name of layer

• LayerType — Type of layer

• NumberOfLearnables — Number of learnable parameters (weights and biases) in the network

• NumberOfOperations — Total number of multiplications and additions

• ParameterMemory (MB) — Memory required to store all of the learnable parameters

• NumberOfMACs — Number of multiply-accumulate operations

• ArithmeticIntensity — Amount of data reuse of data that is being fetched from memory. A high arithmetic intensity indicates more data reuse.

This function estimates network metrics for learnable layers, which have weights and bias, in the network. Estimated metrics are provided for 2-D convolutional layers, 2-D grouped convolutional layers, and fully connected layers.

[dataTable1,dataTable2,…,dataTablen] = estimateNetworkMetrics(net1,net2,…,netn) returns metrics for multiple networks.

This function requires Deep Learning Toolbox Model Quantization Library. To learn about the products required to quantize a deep neural network, see Quantization Workflow Prerequisites.

Examples

Estimate Metrics for Neural Network Layers

This example shows how to estimate layer-wise metrics for a neural network.

Load the pretrained network. net is a SqueezeNet convolutional neural network that has been retrained using transfer learning to classify images in the MerchData data set.

load squeezenetmerch
net

net = 
  DAGNetwork with properties:

         Layers: [68×1 nnet.cnn.layer.Layer]
    Connections: [75×2 table]
     InputNames: {'data'}
    OutputNames: {'new_classoutput'}

Use the estimateNetworkMetrics function to estimate metrics for the 2-D convolutional layers, 2-D grouped convolutional layers, and fully connected layers in your network.

estNet = estimateNetworkMetrics(net)

estNet=26×7 table
        LayerName           LayerType      NumberOfLearnables    NumberOfOperations    ParameterMemory (MB)    NumberOfMACs    ArithmeticIntensity
    __________________    _____________    __________________    __________________    ____________________    ____________    ___________________

    "conv1"               "Convolution"           1792                4.413e+07             0.0068359           2.2065e+07           25.739       
    "fire2-squeeze1x1"    "Convolution"           1040               6.4225e+06             0.0039673           3.2113e+06           12.748       
    "fire2-expand1x1"     "Convolution"           1088               6.4225e+06             0.0041504           3.2113e+06           12.748       
    "fire2-expand3x3"     "Convolution"           9280               5.7803e+07                0.0354           2.8901e+07           111.12       
    "fire3-squeeze1x1"    "Convolution"           2064               1.2845e+07             0.0078735           6.4225e+06           14.158       
    "fire3-expand1x1"     "Convolution"           1088               6.4225e+06             0.0041504           3.2113e+06           12.748       
    "fire3-expand3x3"     "Convolution"           9280               5.7803e+07                0.0354           2.8901e+07           111.12       
    "fire4-squeeze1x1"    "Convolution"           4128               6.4225e+06              0.015747           3.2113e+06           24.791       
    "fire4-expand1x1"     "Convolution"           4224               6.4225e+06              0.016113           3.2113e+06           24.791       
    "fire4-expand3x3"     "Convolution"          36992               5.7803e+07               0.14111           2.8901e+07           178.07       
    "fire5-squeeze1x1"    "Convolution"           8224               1.2845e+07              0.031372           6.4225e+06           27.449       
    "fire5-expand1x1"     "Convolution"           4224               6.4225e+06              0.016113           3.2113e+06           24.791       
    "fire5-expand3x3"     "Convolution"          36992               5.7803e+07               0.14111           2.8901e+07           178.07       
    "fire6-squeeze1x1"    "Convolution"          12336               4.8169e+06              0.047058           2.4084e+06            33.51       
    "fire6-expand1x1"     "Convolution"           9408               3.6127e+06              0.035889           1.8063e+06           32.109       
    "fire6-expand3x3"     "Convolution"          83136               3.2514e+07               0.31714           1.6257e+07           125.07       
      ⋮

Compare Metrics for Floating-Point and Quantized Neural Network

This example shows how to estimate the metrics for a floating-point and quantized neural network.

Load the pretrained network. net is a SqueezeNet convolutional neural network that has been retrained using transfer learning to classify images in the MerchData data set.

load squeezenetmerch
net

net = 
  DAGNetwork with properties:

         Layers: [68×1 nnet.cnn.layer.Layer]
    Connections: [75×2 table]
     InputNames: {'data'}
    OutputNames: {'new_classoutput'}

Unzip and load the MerchData images as an image datastore. Define an augmentedImageDatastore object to resize the data for the network, and split the data into calibration and validation data sets to use for quantization.

unzip('MerchData.zip');
imds = imageDatastore('MerchData', ...
    'IncludeSubfolders',true, ...
    'LabelSource','foldernames');
[calData, valData] = splitEachLabel(imds, 0.7,'randomized');
aug_calData = augmentedImageDatastore([227 227],calData);
aug_valData = augmentedImageDatastore([227 227],valData);

Create a dlquantizer object and specify the network to quantize. Set the execution environment to MATLAB.

quantObj = dlquantizer(net,'ExecutionEnvironment','MATLAB');

Use the calibrate function to exercise the network with sample inputs and collect range information.

calResults = calibrate(quantObj,aug_calData);

Attempt to calibrate with host GPU errored with the message: 
Unable to find a supported GPU device. For more information on GPU support, see GPU Support by Release. 
Reverting to use host CPU. 

Use the quantize method to quantize the network object and return a simulatable quantized network.

qNet = quantize(quantObj)

qNet = 
Quantized DAGNetwork with properties:

         Layers: [68×1 nnet.cnn.layer.Layer]
    Connections: [75×2 table]
     InputNames: {'data'}
    OutputNames: {'new_classoutput'}

Use the quantizationDetails method to extract quantization details.

Use the estimateNetworkMetrics function to compare metrics for the floating-point and quantized networks.

[dataTableFloat,dataTableQuantized] = estimateNetworkMetrics(net,qNet)

dataTableFloat=26×7 table
        LayerName           LayerType      NumberOfLearnables    NumberOfOperations    ParameterMemory (MB)    NumberOfMACs    ArithmeticIntensity
    __________________    _____________    __________________    __________________    ____________________    ____________    ___________________

    "conv1"               "Convolution"           1792                4.413e+07             0.0068359           2.2065e+07           25.739       
    "fire2-squeeze1x1"    "Convolution"           1040               6.4225e+06             0.0039673           3.2113e+06           12.748       
    "fire2-expand1x1"     "Convolution"           1088               6.4225e+06             0.0041504           3.2113e+06           12.748       
    "fire2-expand3x3"     "Convolution"           9280               5.7803e+07                0.0354           2.8901e+07           111.12       
    "fire3-squeeze1x1"    "Convolution"           2064               1.2845e+07             0.0078735           6.4225e+06           14.158       
    "fire3-expand1x1"     "Convolution"           1088               6.4225e+06             0.0041504           3.2113e+06           12.748       
    "fire3-expand3x3"     "Convolution"           9280               5.7803e+07                0.0354           2.8901e+07           111.12       
    "fire4-squeeze1x1"    "Convolution"           4128               6.4225e+06              0.015747           3.2113e+06           24.791       
    "fire4-expand1x1"     "Convolution"           4224               6.4225e+06              0.016113           3.2113e+06           24.791       
    "fire4-expand3x3"     "Convolution"          36992               5.7803e+07               0.14111           2.8901e+07           178.07       
    "fire5-squeeze1x1"    "Convolution"           8224               1.2845e+07              0.031372           6.4225e+06           27.449       
    "fire5-expand1x1"     "Convolution"           4224               6.4225e+06              0.016113           3.2113e+06           24.791       
    "fire5-expand3x3"     "Convolution"          36992               5.7803e+07               0.14111           2.8901e+07           178.07       
    "fire6-squeeze1x1"    "Convolution"          12336               4.8169e+06              0.047058           2.4084e+06            33.51       
    "fire6-expand1x1"     "Convolution"           9408               3.6127e+06              0.035889           1.8063e+06           32.109       
    "fire6-expand3x3"     "Convolution"          83136               3.2514e+07               0.31714           1.6257e+07           125.07       
      ⋮

dataTableQuantized=26×7 table
        LayerName           LayerType      NumberOfLearnables    NumberOfOperations    ParameterMemory (MB)    NumberOfMACs    ArithmeticIntensity
    __________________    _____________    __________________    __________________    ____________________    ____________    ___________________

    "conv1"               "Convolution"           1792                4.413e+07               0.001709          2.2065e+07           25.739       
    "fire2-squeeze1x1"    "Convolution"           1040               6.4225e+06             0.00099182          3.2113e+06           12.748       
    "fire2-expand1x1"     "Convolution"           1088               6.4225e+06              0.0010376          3.2113e+06           12.748       
    "fire2-expand3x3"     "Convolution"           9280               5.7803e+07              0.0088501          2.8901e+07           111.12       
    "fire3-squeeze1x1"    "Convolution"           2064               1.2845e+07              0.0019684          6.4225e+06           14.158       
    "fire3-expand1x1"     "Convolution"           1088               6.4225e+06              0.0010376          3.2113e+06           12.748       
    "fire3-expand3x3"     "Convolution"           9280               5.7803e+07              0.0088501          2.8901e+07           111.12       
    "fire4-squeeze1x1"    "Convolution"           4128               6.4225e+06              0.0039368          3.2113e+06           24.791       
    "fire4-expand1x1"     "Convolution"           4224               6.4225e+06              0.0040283          3.2113e+06           24.791       
    "fire4-expand3x3"     "Convolution"          36992               5.7803e+07               0.035278          2.8901e+07           178.07       
    "fire5-squeeze1x1"    "Convolution"           8224               1.2845e+07               0.007843          6.4225e+06           27.449       
    "fire5-expand1x1"     "Convolution"           4224               6.4225e+06              0.0040283          3.2113e+06           24.791       
    "fire5-expand3x3"     "Convolution"          36992               5.7803e+07               0.035278          2.8901e+07           178.07       
    "fire6-squeeze1x1"    "Convolution"          12336               4.8169e+06               0.011765          2.4084e+06            33.51       
    "fire6-expand1x1"     "Convolution"           9408               3.6127e+06              0.0089722          1.8063e+06           32.109       
    "fire6-expand3x3"     "Convolution"          83136               3.2514e+07               0.079285          1.6257e+07           125.07       
      ⋮

The quantized network has significantly lower parameter memory requirements than the floating-point version of the network.

Input Arguments

net — Neural network

DAGNetwork object | SeriesNetwork object | dlnetwork object

Neural network, specified as a DAGNetwork object, SeriesNetwork object, or dlnetwork object.

net1,net2,…,netn — Neural networks

DAGNetwork object | SeriesNetwork object | dlnetwork object

Neural networks, specified as a comma-separated list of DAGNetwork objects, SeriesNetwork objects, or dlnetwork objects.

Version History

Introduced in R2022a