Why do i obtain good results but poor input error cross correlation plot?

Hi,

May i know why do i obtain a low MSE and good predicted outputs but poor input error cross correlation plot?The correlations do not fall within the confidence limit. I am using neural networks time series tool. Why does the confidence limit varies for different MSE results? Currently mine my confidence limit is at +- 2000. Other MSE results can give me confidence limit of +- 4000 but with higher MSE. May i know the reason for this?

Thank you

Answer 1

[Greg Heath]

If the input/error cross-correlation is large, then a different set of weights could use that correlation to further improve performance.

The best result would be an error that is not at all correlated with either input or output.

Hope this helps.

Thank you for formally accepting my answer

Greg

[WT]

Thank you for replying.

What do you mean by a different set of weights? Is it the ratio for training validation and testing?

Can i still use my saved network since it gives me good predicted values?

Thank you

[Greg Heath]

How are you obtaining the MSE confidence limit?

Why not use normalized MSE values (divide by average target variance, MSE00) to remove the obscuration of target scaling?

Different weights associated with a different design.

 a. divideblock instead of dividerand 
 b. Better choice of delays (See my correlation function method of choosing delays). Search NEWSGROUP and ANSWERS using

 greg ncorr

 c. Better combination of hidden nodes and random initial weights See my double for loop method of optimizing number of hidden nodes and initial weights):

 greg narxnet Ntrials

Hope this helps.

Thank you for formally accepting my answer

Greg

[Greg Heath]

To be clear:

Training results are biased because the data used to obtain performance summaries are the same data used to design the model. However, if the number of training equations is larger than the number of unknown weights, a degree of freedom correction is useful.

Validation results are also biased because the data is used to determine when to stop the design. However, the bias is much smaller. Therefore, the val performance should be used to choose among competing designs.

Therefore, test performance which, in no way is involved in design, is the only UNBIASED prediction of performance on unseen data.

Hope this helps.

Greg