Main Content

Instance normalization layer

An instance normalization layer normalizes a mini-batch of data across each channel for each observation independently. To improve the convergence of training the convolutional neural network and reduce the sensitivity to network hyperparameters, use instance normalization layers between convolutional layers and nonlinearities, such as ReLU layers.

After normalization, the layer scales the input with a learnable scale factor
*γ* and shifts by a learnable offset *β*.

`layer = instanceNormalizationLayer`

creates an instance
normalization layer.

`layer = instanceNormalizationLayer(Name,Value)`

creates an
instance normalization layer and sets the optional `Epsilon`

, Parameters and Initialization, Learn Rate and Regularization, and `Name`

properties using one or more name-value arguments. You can
specify multiple name-value arguments. Enclose each property name in quotes.

`instanceNormalizationLayer('Name','instancenorm')`

creates
an instance normalization layer with the name
`'instancenorm'`

The instance normalization operation normalizes the elements
*x _{i}* of the input by first calculating the
mean

$$\widehat{{x}_{i}}=\frac{{x}_{i}-{\mu}_{I}}{\sqrt{{\sigma}_{I}^{2}+\u03f5}},$$

where *ϵ* is a constant that improves numerical
stability when the variance is very small.

To allow for the possibility that inputs with zero mean and unit variance are not optimal for the operations that follow instance normalization, the instance normalization operation further shifts and scales the activations using the transformation

$${y}_{i}=\gamma {\widehat{x}}_{i}+\beta ,$$

where the offset *β* and scale factor
*γ* are learnable parameters that are updated during network
training.

`batchNormalizationLayer`

| `convolution2dLayer`

| `fullyConnectedLayer`

| `groupNormalizationLayer`

| `layerNormalizationLayer`

| `reluLayer`

| `trainingOptions`

| `trainNetwork`