MATLAB Answers

Self Organizing Map training question

13 views (last 30 days)
Darin McCoy
Darin McCoy on 12 Jul 2013
Edited: Simon Nunn on 13 Jul 2017
Hi,
I have a difficult question about using Matlab's neural network toolbox. I would like to train a SOM neural network with a data set; however, my data set is quite large. Because of this, I need to split the data into sections and train it individually. Here's my code now
%%Combination method
%IN THIS EXAMPLE - ITS POSSIBLE BECAUSE ITS A SMALL DATASET. IT IS NOT POSSIBLE FOR MY ACTUAL DATA
%Load and combine the data
data1 = [1:10:400;1:20:800]';
data2 = [400:1:440;800:1:840]';
combined = [data1;data2]';
% Create a Self-Organizing Map
dimension1 = 5;
dimension2 = 5;
net = selforgmap([dimension1 dimension2]);
% Train the Network
[net,tr] = train(net,combined);
%Plot combined results
plotsomhits(net,combined);
plotsomhits(net,data1');
plotsomhits(net,data2');
%%Iterative METHOD
%This is what I actually want to use to train the network
% Create a Self-Organizing Map
dimension1 = 5;
dimension2 = 5;
net = selforgmap([dimension1 dimension2]);
% Train the Network
data1 = [1:10:400;1:20:800]';
[net,tr] = train(net,data1');
data2 = [400:1:440;800:1:840]';
[net,tr] = train(net,data2');
% View the Network
combined = [data1;data2]';
plotsomhits(net,combined);
plotsomhits(net,data1');
plotsomhits(net,data2');
As you can tell - the results are skewed significantly because the data is trained twice. Is there anyway to limit the bias when you are training the second time?

  1 Comment

Simon Nunn
Simon Nunn on 13 Jul 2017
I am also having a similar issue to this.
First, I can see that the redefinition of the network with the second use of selforgmap() will completely discard any training performed by the first call to train().
However removing this redefinition of your network is not sufficient to solve the issue.
Having taken the time to read the manual I also experimented with the use of adapt() instead of train() this is supposed to perform one step of the training process for the purpose of not batch processing data, which works fine with NARX and other feedforward MLP, but with SOM it seems to reset the network in the same way that train() does.
Digging even deeper I started to experiment with calling the learning function learnsomb() directly with the intend of manually applying the delta weight matrix that it returns, but I've struggled to find a suitable input for A.
After some reverse engineering of the MATLAB code I've finally run out of steam and I've come here to find answers.
Here the code I have so far:
% lifted from debugging MATLAB code
% feval(learnFcn,net.IW{i,j}, ...
% PD{i,j,ts},
% IWZ{i,j},
% N{i},
% Ac{i,ts+numLayerDelays},
% t,
% e,
% gIW{i,j},
% gA{i},
% net.layers{i}.distances,
% net.inputWeights{i,j}.learnParam,
% IWLS{i,j});
w = net.IW;
d = net.layers{1}.distances;
a = net(P); % this is not right :(
LP = net.inputWeights{1}.learnParam;
[dW,ls] = learnsomb(w,P,[],[],a,[],[],[],[],d,LP,[]);

Sign in to comment.

Accepted Answer

Greg Heath
Greg Heath on 13 Jul 2013
This is a well known NN training phenomenon simply referred to as forgetting. See comp.ai.neural-nets posts and FAQ.
The only way to mitigate forgetting is to make sure that the salient characteristics of the 1st training set are reinforced during the later learning. Typically, these characteristics are represented by a subset of first set samples or cluster centers.
Hope this helps.
Thank you for formally accepting my answer
Greg

  1 Comment

Darin McCoy
Darin McCoy on 15 Jul 2013
Thanks Greg,
For others that struggle with this issue - I recommend reading this white paper...its pretty good!

Sign in to comment.

More Answers (0)

Sign in to answer this question.