3 views (last 30 days)

Hello everyone,

I have been exploring the Neural Network capabilities of Matlab concerning fitting/interpolation, using the nftool.

So far I have been using a vector of:

- [N x 2] as an input
- [N x 1] as an output

where N is ~40000. Training is rather fast (say within 10 minutes) and with hiddenLayerSize of [4,10,20] I get very good agreement when testing the network; effectively the R parameter is unity (see test.png).

Training parameters are:

net.divideParam.valRatio = 5/100;

net.divideParam.testRatio = 5/100;

net.divideParam.trainRatio = 1 - net.divideParam.valRatio - net.divideParam.testRatio;

using Bayesian regularisation.

Now, I need to repeat the same process with an input vector of [N x 3] and output again [N x 1], but now N is in the order of 4*10^6 (there is the possibility to expand even beyond that, with N x 6 input and even larger Ns). It is still possible to get a good training by increasing No of layers (I have obtained good performance with hiddenLayerSize = [20, 5, 10, 20];), though it is much more time consuming (takes around a day to train) and whereas the match is good (again R~1), there are clearly some points that do not get captured as shown in the regression plot and there is a bit of noise when the trained network is used; this I dont think is from over-fitting, as performance tends to improve with increasing layer sizes.

Is there any way to improve the performance of training? Can deep learning networks or convolutional networks help here? It is a bit unclear if it is possible to use them for fitting applications, as they are shown in Matlab examples..

Any idea is welcome

Thanks

Srivardhan Gadila
on 27 Nov 2020

The following are some suggestions based on my knowledge:

In general the dataset is splitted as follows: 70% for training, 15% for validation & 15% for testing. Also I think that shallow neural networks are good enough for this problem, even though you can try regression using deep neural networks. Also refer to the following documentation pages Improve Shallow Neural Network Generalization and Avoid Overfitting, Divide Data for Optimal Neural Network Training, Workflow for Neural Network Design for shallow neural networks and Deep Learning Tips and Tricks, Deep Learning Tuning and Visualization in case of Deep neural networks.

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!
## 0 Comments

Sign in to comment.