how can i evaluate my artificial neural network when the targets' range is very short and mse doesnt work?

1 view (last 30 days)
I'm working with a.n.n in matlab and I want to fit a network to my experimental datas. Totally, most of MLPs I train can work easily, and no matter how many neurons I have used, most of networks have a mse less than 0.001 which is acceptable.
but the problem is regression, in the regression plots for test data, result is not accurate enough. and when I check the network manually with my datas, i dont get what I need.
what shoud I do? remembering that my targets have a rang of : 0.1000 to 0.1800 .
is there any other parameter for evaluating the network accuracy?

Answers (1)

Greg Heath
Greg Heath on 28 Mar 2015
Yes, the normalized MSE
NMSE = mse(t-y)/mean(var(t',1))
and corresponding coefficient of determination or Rsquared
R2 = 1 - NMSE
============================================================
NEWSGROUP and ANSWERS search info:
greg NMSE
greg R2
SEARCH ENGINE INFO:
Coefficient of determination - Wikipedia, the free encyclopedia
en.wikipedia.org/wiki/Coefficient_of_determination
R2 - Wikipedia, the free encyclopedia
en.wikipedia.org/wiki/R2
Thank you for formally accepting my answer
Greg
  2 Comments
Poooneh mmm
Poooneh mmm on 9 Apr 2015
Dear prof. Heath thanks for your detailed answer.
after you introduced normalized mse, i searched about it on the intenet. and I found some equations explaning nrmse and nmse (which were totally diffrent. )
any way, I still dont know how to compare results with nmse. which nmse is better? how can I find optimume number of neurons and layers?
I guesse a normalizing between target datas is needed, but i dont know how . and also I think when matlab itself normalizes datas( by 'mapminmax') are there better ways of annalyzing datas?
Greg Heath
Greg Heath on 9 Apr 2015
It sounds like you didn't concentrate on searching my posts in
1. The NEWSGROUP
2. ANSWERS
With targets standardized to zero-mean/unit-variance, NMSE = MSE.
In short I usually recommend
1. Standardizing inputs and outputs to zero-mean/unit-variance (zscore or mapstd)
2. MSEgoal <= 0.01 (Rsquare >= 0.99)
3. MinGrad <= MSEgoal/100
4. Minimize the number of hidden nodes using a double loop search:
a. outer loop over Hmin:dH:Hmax
b. inner loop over 1:Ntrials initial random number states (determines random trn/val/tst division and initial weights)
I have posted zillions of examples.
Greg

Sign in to comment.

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!