L2 Regularization Hyperparameter in trainingOptions

37 views (last 30 days)
Hello,
I want to start training my neural network without L2 regularization.
By default, trainingOptionstrainingOptions() set the L2 regularization parameters to 1e-4, which means that it adds some penalities to the weights.
Would it be possible to train by setting L2Regularization to 0? Which is the range of values suggested for this parameter of the deepNN library?
Is there any correlation inside the library with other parameters that might be carefully tuned?
Thank you in advance for your help.
All the best.

Accepted Answer

Jyothis Gireesh
Jyothis Gireesh on 10 Feb 2020
The most common values of the regularization parameter are often on a logarithmic scale between 0 and 0.1, such as 0.1, 0.001, 0.00001 etc.
Setting the regularization parameter to zero may cause the network to overfit to the training data and reduces the generalizing capability of the network. Changing the regularization parameter as such doesn’t affect other carefully tuned parameters within the model. But it’s effects may be observed during the convergence of the loss function.

More Answers (0)

Categories

Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange

Products


Release

R2019b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!