How to use transfer learning on U-net created in Matlab?

29 views (last 30 days)
Hridayi on 20 Feb 2020
Commented: sab kara on 9 Aug 2020
I'm interested in using a U-net that I have trained in Matlab on one set of data and then applying transfer learning on the last couple layers with another set of data (as shown in )
How would this process work with a network created using U-net layers in Matlab? Also, is the U-net layer in Matlab already pre-trained using some other set of data when it's used in the script?
Thank you
I'm attempting to change the final convolutional layer using this code:
load SimNet1.mat
Net2 = SimNet1; %renaming U-Net
layers = Net2.Layers
layers(56) = convolution2dLayer([1 1], 2,'NumChannels',64); %Should I specify weights and bias?
% Directories of new training images (not used in pre-trained network)
segDir = fullfile('images location');
USDir = fullfile('images location'); % Labels
imds = imageDatastore(USDir); %DataStore of input training images - ultrasound images
classNames = ["bone","background"]; %labels
labelIDs = [1 0];
pxds = pixelLabelDatastore(segDir,classNames,labelIDs); %Has Ground Truth pixel data on training images - Seg images are pixel labeled (only have 255/0)
options = trainingOptions('adam','InitialLearnRate', 3e-4, ...
'MaxEpochs',100,'MiniBatchSize',15, ...
ds = pixelLabelImageDatastore(imds,pxds) %returns a datastore based on input image data(imds - US images)
%and pxds (required network output - segmentations)
TLNet1 = trainNetwork(ds,layers,options)
save TLNet1
But I am recieving an error that says:
Shouldn't these layers already be connected from the pre-trained network?

Answers (1)

Daniel Vieira
Daniel Vieira on 21 Feb 2020
what you made works only for sequential models, where each layer is necessarily connected to the next in the same order that they are stacked in "Layers". The uNet is not a sequential model, some layers have multiple outputs, some have multiple inputs, a "early" layer may be connedted to a "late" layer. When you just pick the layers and pass it to trainNetwork, you are ignoring the connections.
Instead of passing the layers, you should pass the lgraph object. And before passing, you need to change it with the methods addLayers, removeLayers, connectLayers, disconnectLayers. You mustn't leave any layer input or output unassigned in the Connections property of lgraph.
Hridayi on 16 Mar 2020
Hello, I had a follow up question on this.
If I wanted to use transfer learning on the U-net itself, do I need to only replace the last 3 layers(Final convolution, pixelclassification, softmax)? What are the correct layers to replace? I also had difficulties with replacing other layers in the network because of input size not matching for the U-net (Asked on MATLAB Answers)
Thanks for your help.

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!