Problem: feed-forward neural network - the connection between the hidden layer and output layer is removed.

Hi everybody.

I am facing a strange problem with Matlab and, in particular, with the training of a feed-forward neural network.

In practice, I set the network, which is formed by an input layer, a hidden layer and an output layer. But, when I call the train function, the connection between the hidden layer and the output layer is removed and I do not understand why. I hope someone can help me.

The following is the simple code I use:

if true
load fisheriris
feedforwardNetwork = feedforwardnet(10);
feedforwardNetwork.divideFcn = 'dividetrain';
feedforwardNetwork.trainFcn = 'traingd';
feedforwardNetwork.trainParam.epochs = 10;
feedforwardNetwork = train(feedforwardNetwork, meas');
end

Gianni.

[Brendan Hamm]

You have no targets defined so there is no training which occurs. What is it you are trying to train?

The following works just fine:

targets = strcmp('setosa',species');
feedforwardNetwork = feedforwardnet(10);
feedforwardNetwork.divideFcn = 'dividetrain';
feedforwardNetwork.trainFcn = 'traingd';
feedforwardNetwork.trainParam.epochs = 10;
feedforwardNetwork = train(feedforwardNetwork, meas',targets );

Answer 1

[Greg Heath]

% Hi Greg and Brendan. Thanks for your reply. % % Well, after struggling reading the Matlab documentation, % I think I understood what the problem was. % % The code I posted was just a dummy example to explain the % issue I was facing. My real problem is the following: I am % trying to solve an anomaly detection problem and, in % particular, reading sensor data, I am trying to detect when % there is an anomaly behavior. % % In order to do so, I am using different machine learning % algorithms and evaluating their performance. So far, I have % used the nearest neighbor algorithm, the self-organizing maps % and the support vector machines. Another "instrument" I would % like to use is that of neural networks.

 Your problem is that you did not do the following:

1. Identify the problem as one of the following

   a. regression/curvefitting
   b. classification/patternrecognition
   c. clustering
   d. time-series

 2.  Search both NEWSGROUP and ANSWERS using 
   a. classification 
   b. pattern-recognition

   to identify 
   a. classification/pattern-recognition functions 
      (e.g., patternnet)
   b. example classification/pattern-recognition code 
      and data examples 

 3. Practice using one or more of the MATLAB classification/...
    pattern-recognition example data obtained from 

   help nndata
   doc  nndata

5. Apply what is learned above on your dataset.

% My idea was to train the neural network with normal data % (so, a one-class data set) and use the net to compute a sort % of anomaly score. But, if I got it right, it has no sense to % train a neural network having just one output neuron with % data belonging to just one class. Neural networks are very % variegated and represent a vast subject. I am slowly learning % them.

 No. Create an input and target sets for 2 classes.

% As regard the default settings, I have just modified the % algorithm in order to have the classic gradient descent.

 No. Always begin using as many defaults as possible. Then 
consider (one-at-a-time) replacing default settins.

% P.s. Little question: when there is no target data defined, % isn't the problem an unsupervised machine learning problem? % Theorically, I could do it (given that my data set is not a % one-class data set).

 Your best bet is to create a 2-class dataset.

Hope this helps.

Greg

Answer 2

[Greg Heath]

 Hi everybody.

 I am facing a strange problem with Matlab and, in particular, the training of 
a feed-forward neural network.

 In practice, I set the network, which is formed by an input layer, a hidden 
layer and an output layer. But, when I call the train function, the connection between 
the hidden layer and the output layer is removed and I do not understand why. 

 I hope someone can help me.

========================================================================

 close all, clear all, clc

 load fisheriris
whos
% Name     Size    Bytes  Class 
% meas     150x4   4800   double              
% species  150x1   19300  cell 

 % NOTE: FISHERIRIS IS A PATTERN-RECOGNITION/CLASSIFICATION DATA SET. The 
appropriate training function to use is PATTERNNET. See the appropriate 
help and doc documentation:

 help patternnet

 % This yields a script which I don't think is particularly helpful.

However, you should probably read it anyway just to see what MATLAB thinks is useful

 [x,t] = iris_dataset;     % Dimensions of input x and target t?
 net   = patternnet(10);

% The default of 10 hidden nodes is OK for first getting the feel of the problem. However, in the final result I like to use as few hidden nodes as possible. If successful, I think it is the best way to design a net which does not have an excess number of degrees of freedom that could rein havoc on unseen data via the dreaded phenomenon of "OVERTRAINING AN OVERFIT NET".

 net = train(net,x,t);   %RNG seed for weight initialization??
 view(net)
 y = net(x);             % Posterior probability estimates
 perf = perform(net,t,y);% What performance criterion?
 classes = vec2ind(y); % Predicted classes. Class error rates??

 doc patternnet  % Yields the same sample code.

 % Now consider your code:

 clear all, close all, clc      % LET'S START FRESH
 load fisheriris                % WHAT IS LOADED? DIMENSIONS?
 net = feedforwardnet(10);      % DEFAULT
 net.divideFcn = 'dividetrain'; % WHY NO VALIDATION AND TEST ??
 net.trainFcn = 'traingd';      % PROBABLY OK. BUT WHY?
 net.trainParam.epochs = 10;    % ??? DEAD IN THE WATER!!!
 net = train(net, meas');       % NEED INPUT & TARGET !!

%Your biggest problems are

 1. Ignoring the sample code in the help and doc documentation
 2. Not beginning by using as many defaults as possible
 3. Not realizing that designs are a trial and error process 
   best begun by using as many defaults as possible.
 4. Not searching in the NEWSGROUP and ANSWERS for previous 
   posts

Hope this helps.

Greg

[Giovanni]

Hi Greg and Brendan. Thanks for your reply.

Well, after struggling reading the Matlab documentation, I think I understood what the problem was.

The code I posted was just a dummy example to explain the issue I was facing. My real problem is the following: I am trying to solve an anomaly detection problem and, in particular, reading sensor data, I am trying to detect when there is an anomaly behavior.

In order to do so, I am using different machine learning algorithms and evaluating their performance. So far, I have used the nearest neighbor algorithm, the self-organizing maps and the support vector machines. Another "instrument" I would like to use is that of neural networks.

My idea was to train the neural network with normal data (so, a one-class data set) and use the net to compute a sort of anomaly score. But, if I got it right, it has no sense to train a neural network having just one output neuron with data belonging to just one class. Neural networks are very variegated and represent a vast subject. I am slowly learning them.

As regard the default settings, I have just modified the learning algorithm in order to have the classic gradient descent.

P.s. Little question: when there is no target data defined, isn't the problem an unsupervised machine learning problem? Theorically, I could do it (given that my data set is not a one-class data set).

[Giovanni]

Moreover:

- Why no validation and test? Because I have already divided my entire data set into training set, validation set and test set. The first phase is to train the neural network on the training set. The second is to perform validation and test. I do so in order to have a coherent comparison with the other algorithms.

- 'traingd': as I have already said, it was just to use the classic gradient descent algorithm.

- epochs: it was a random choice; no specific reason behind this =D