addPix2PixHDLocalEnhancer
Syntax
Description
adds a local enhancer network to a pix2pixHD generator network, netWithEnhancer
= addPix2PixHDLocalEnhancer(net
)net
.
For more information about the network architecture, see pix2pixHD Local Enhancer Network.
This function requires Deep Learning Toolbox™.
controls aspects of the local enhancer network creation using name-value arguments.netWithEnhancer
= addPix2PixHDLocalEnhancer(net
,Name=Value
)
Examples
Add Local Enhancer to Pix2pixHD Generator Network
Specify the network input size for 32-channel data of size 512-by-1024.
inputSize = [512 1024 32];
Create a pix2pixHD global generator network.
pix2pixHD = pix2pixHDGlobalGenerator(inputSize)
pix2pixHD = dlnetwork with properties: Layers: [84x1 nnet.cnn.layer.Layer] Connections: [92x2 table] Learnables: [110x3 table] State: [0x3 table] InputNames: {'GlobalGenerator_inputLayer'} OutputNames: {'GlobalGenerator_fActivation'} Initialized: 1 View summary with summary.
Add a local enhancer network to the pix2pixHD network.
pix2pixHDEnhanced = addPix2PixHDLocalEnhancer(pix2pixHD)
pix2pixHDEnhanced = dlnetwork with properties: Layers: [113x1 nnet.cnn.layer.Layer] Connections: [124x2 table] Learnables: [146x3 table] State: [0x3 table] InputNames: {'LocalEnhancer_inputLayer' 'GlobalGenerator_inputLayer'} OutputNames: {'LocalEnhancer_fActivation'} Initialized: 1 View summary with summary.
Display the network with the local enhancer.
analyzeNetwork(pix2pixHDEnhanced)
Input Arguments
net
— pix2pixHD generator network
dlnetwork
object
Pix2pixHD generator network, specified as a dlnetwork
(Deep Learning Toolbox) object. You can create a
pix2pixHD generator network using the pix2pixHDGlobalGenerator
function.
Name-Value Arguments
Specify optional pairs of arguments as
Name1=Value1,...,NameN=ValueN
, where Name
is
the argument name and Value
is the corresponding value.
Name-value arguments must appear after other arguments, but the order of the
pairs does not matter.
Example: netWithEnhancer =
addPix2PixHDLocalEnhancer(net,FilterSizeInFirstAndLastBlocks=[5 7])
adds a local
enhancer whose first and last convolution layers have a size of 5-by-7.
Before R2021a, use commas to separate each name and value, and enclose
Name
in quotes.
Example: netWithEnhancer =
addPix2PixHDLocalEnhancer(net,"FilterSizeInFirstAndLastBlocks",[5 7])
adds a
local enhancer whose first and last convolution layers have a size of 5-by-7.
FilterSizeInFirstAndLastBlocks
— Filter size in first and last convolution layers
7
(default) | positive odd integer | 2-element vector of positive odd integers
Filter size in the first and last convolution layers of the local enhancer network, specified as a positive odd integer or 2-element vector of positive odd integers of the form [height width]. When you specify the filter size as a scalar, the filter has equal height and width.
FilterSizeInIntermediateBlocks
— Filter size in intermediate convolution layers
3
(default) | 2-element vector of positive odd integers | positive odd integer
Filter size in intermediate convolution layers in the local enhancer network, specified as a positive odd integer or 2-element vector of positive odd integers of the form [height width]. The intermediate convolution layers are the convolution layers excluding the first and last convolution layer. When you specify the filter size as a scalar, the filter has identical height and width. Typical values are between 3 and 7.
NumResidualBlocks
— Number of residual blocks
3
(default) | positive integer
Number of residual blocks in the local enhancer network, specified as a positive integer. Each residual block consists of a set of convolution, normalization and nonlinear layers with skip connections between every block.
ConvolutionPaddingValue
— Style of padding
"symmetric-exclude-edge"
(default) | "symmetric-include-edge"
| "replicate"
| numeric scalar
Style of padding used in the local enhancer network, specified as one of these values.
PaddingValue | Description | Example |
---|---|---|
Numeric scalar | Pad with the specified numeric value |
|
"symmetric-include-edge" | Pad using mirrored values of the input, including the edge values |
|
"symmetric-exclude-edge" | Pad using mirrored values of the input, excluding the edge values |
|
"replicate" | Pad using repeated border elements of the input |
|
UpsampleMethod
— Method used to upsample activations
"transposedConv"
(default) | "bilinearResize"
| "pixelShuffle"
Method used to upsample activations in the local enhancer network, specified as one of these values:
"transposedConv"
— Use atransposedConv2dLayer
(Deep Learning Toolbox) with a stride of [2 2]"bilinearResize"
— Use aconvolution2dLayer
(Deep Learning Toolbox) with a stride of [1 1] followed by aresize2dLayer
with a scale of [2 2]"pixelShuffle"
— Use aconvolution2dLayer
(Deep Learning Toolbox) with a stride of [1 1] followed by adepthToSpace2dLayer
with a block size of [2 2]
Data Types: char
| string
ConvolutionWeightsInitializer
— Weight initialization used in convolution layers
"narrow-normal"
(default) | "glorot"
| "he"
| function
Weight initialization used in convolution layers of the local enhancer network,
specified as "glorot"
, "he"
,
"narrow-normal"
, or a function handle. For more information, see
Specify Custom Weight Initialization Function (Deep Learning Toolbox).
ActivationLayer
— Activation function
"relu"
(default) | "leakyRelu"
| "elu"
| layer object
Activation function to use in the local enhancer network, specified as one of these values. For more information and a list of available layers, see Activation Layers (Deep Learning Toolbox).
"relu"
— Use areluLayer
(Deep Learning Toolbox)"leakyRelu"
— Use aleakyReluLayer
(Deep Learning Toolbox) with a scale factor of 0.2"elu"
— Use aneluLayer
(Deep Learning Toolbox)A layer object
NormalizationLayer
— Normalization operation
"instance"
(default) | "none"
| "batch"
| layer object
Normalization operation to use after each convolution in the local enhancer network, specified as one of these values. For more information and a list of available layers, see Normalization Layers (Deep Learning Toolbox).
"instance"
— Use aninstanceNormalizationLayer
(Deep Learning Toolbox)"batch"
— Use abatchNormalizationLayer
(Deep Learning Toolbox)"none"
— Do not use a normalization layerA layer object
Dropout
— Probability of dropout
0
(default) | number in the range [0, 1]
Probability of dropout in the local enhancer network, specified as a number in the
range [0, 1]. If you specify a value of 0
, then the network does
not include dropout layers. If you specify a value greater than 0
,
then the network includes a dropoutLayer
(Deep Learning Toolbox)
in each residual block.
NamePrefix
— Prefix to all layer names
"LocalEnhancer_"
(default) | string | character vector
Prefix to all layer names in the local enhancer network, specified as a string or character vector.
Data Types: char
| string
Output Arguments
netWithEnhancer
— pix2pixHD generator network with local enhancer
dlnetwork
object
Pix2pixHD generator network with local enhancer, returned as a dlnetwork
(Deep Learning Toolbox) object.
More About
pix2pixHD Local Enhancer Network
The addPix2PixHDLocalEnhancer
function performs
these operations to add a local enhancer network to a pix2pixHD global generator network.
The default enhanced network follows the architecture proposed by Wang et. al. References.
The local enhancer network has an initial block of layers that accepts images of size [2*H 2*W C], where H is the height, W is the width, and C is the number of channels of the input to the global generator network,
net
. Whennet
has multiple image input layers, the input image size of the local enhancer network is twice the input size with the maximum resolution.After the initial block, the local enhancer network has a single downsampling block that downsamples the data by a factor of two. Therefore, the output after downsampling has size [H W 2*C].
The
addPix2PixHDLocalEnhancer
function trims the final block from the global generator network. The function then adds the output of the last upsampling block in the global generator network to the output of the downsampled data from the enhancer network using anadditionLayer
(Deep Learning Toolbox).The output of the addition then passes through
NumResidualBlocks
residual blocks from the local enhancer.The residual blocks are followed by a single upsampling block that upsamples data to size [2*H 2*W C].
The
addPix2PixHDLocalEnhancer
function adds a final block to the enhanced network. The convolution layer has properties specified by the arguments ofaddPix2PixHDLocalEnhancer
. If the global generator network has a final activation layer, then the function adds the same type of activation layer to the enhanced network.
The table describes the blocks of layers that comprise the local enhancer network.
Block Type | Layers | Diagram of Default Block |
---|---|---|
Initial block |
|
|
Downsampling block |
|
|
Residual block |
|
|
Upsampling block |
|
|
Final block |
|
|
References
[1] Wang, Ting-Chun, Ming-Yu Liu, Jun-Yan Zhu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. "High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs." In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 8798–8807. Salt Lake City, UT, USA: IEEE, 2018. https://doi.org/10.1109/CVPR.2018.00917.
Version History
Introduced in R2021a
See Also
Topics
MATLAB Command
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
You can also select a web site from the following list:
How to Get Best Site Performance
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)