GAN intermediate layer for image multiplication
1 view (last 30 days)
Show older comments
So I have created a GAN that uses a regular noise input for image synthesis. However, I want to multiply or get the product of the generated image with an outside image to speed up training/ training convergence. Here is the basic idea:![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1304225/image.jpeg)
![](https://www.mathworks.com/matlabcentral/answers/uploaded_files/1304225/image.jpeg)
%This is the layer setup I use for my GAN
numFilters = 128;
numLatentInputs = 256;
inputSize = [128 128 3];
projectionSize = [4 4 3];
layersGenerator = [
featureInputLayer(numLatentInputs)
projectAndReshapeLayer(projectionSize)
resize2dLayer('OutputSize',[4 4],'Method','bilinear')
batchNormalizationLayer
leakyReluLayer
resize2dLayer('OutputSize',[16 16],'Method','bilinear')
batchNormalizationLayer
leakyReluLayer
resize2dLayer('OutputSize',[32 32],'Method','bilinear')
batchNormalizationLayer
leakyReluLayer
resize2dLayer('OutputSize',[64 64],'Method','bilinear')
batchNormalizationLayer
leakyReluLayer
resize2dLayer('OutputSize',[128 128],'Method','bilinear')
batchNormalizationLayer
tanhLayer];
layersDiscriminator = [
imageInputLayer(inputSize,Normalization="none")
dropoutLayer(dropoutProb)
resize2dLayer('OutputSize',[128 128],'Method','bilinear')
convolution2dLayer(filterSize,numFilters,Stride=2,Padding="same")
leakyReluLayer(scale)
resize2dLayer('OutputSize',[64 64],'Method','bilinear')
convolution2dLayer(filterSize,2*numFilters,Stride=2,Padding="same")
batchNormalizationLayer
leakyReluLayer(scale)
resize2dLayer('OutputSize',[32 32],'Method','bilinear')
convolution2dLayer(filterSize,4*numFilters,Stride=2,Padding="same")
batchNormalizationLayer
leakyReluLayer(scale)
resize2dLayer('OutputSize',[16 16],'Method','bilinear')
convolution2dLayer(filterSize,8*numFilters,Stride=2,Padding="same")
batchNormalizationLayer
leakyReluLayer(scale)
resize2dLayer('OutputSize',[8 8],'Method','bilinear')
convolution2dLayer(8,1)
sigmoidLayer];
numEpochs = 2000;
miniBatchSize = 16;
learnRate = 0.0003;
gradientDecayFactor = 0.5;
squaredGradientDecayFactor = 0.999;
flipProb = 0.75;
validationFrequency = 5;
%The custom layer that I have already made:
classdef imageProduct < nnet.layer.Layer
properties (Constant)
ImageinputT = imread("image.jpg");
end
methods
function layer = imageProduct()
layer.Type = "Image Product";
layer.Description = "Combine Images";
layer.OutputNames = "out";
end
function [Z] = predict(~,X)
%gets the image size so the input image can be resized. Basically the target scale
[WidthX, HeightX] = size(X);
%resizes the imported image
GinputIm_scaled = imresize(crossAttention.ImageinputT, [WidthX HeightX]);
%converts images to double
GinputIm_scaled_single = single(GinputIm_scaled);
Xsingle = single(X);
%Gets the tensor product of the two images
Xnew = tensorprod(GinputIm_scaled_single, Xsingle);
Z = Xnew; %output image
end
end
end
1 Comment
Answers (0)
See Also
Categories
Find more on Image Data Workflows in Help Center and File Exchange
Products
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!