Why is the learning rate for the bias usually twice as large as the the LR for the weights?

Why is the learning rate for the bias usually twice as large as the the LR for the weights?

2 Comments

on the other hand if you check, for example alexnet layers you will find that net.Layers(N, 1).BiasLearnRateFactor=2 and net.Layers(N, 1).WeightLearnRateFactor = 1

Sign in to comment.

Answers (0)

Categories

Find more on Deep Learning Toolbox in Help Center and File Exchange

Asked:

on 3 Jan 2018

Edited:

on 5 Jan 2018

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!