Adaboost Learning Rate in Matlab Documentation

Hey,

the description of Adaboost allows to set a learn rate.

https://de.mathworks.com/help/stats/fitcensemble.html

However, the learn rate typically refers to Gradient Boosting. Could anyone explain to me what Matlab is doing when AdaboostM1 is applied.

Thanks for your help!

Answer 1

[Raunak Gupta]

Hi,

The LearnRate option in AdaBoostM1 tells about the learning rate of shrinkage which is essentially the shrink in contribution of each new base-model learned in the ensemble. This parameter controls how much the new model contributes to existing one. Normally if the LearnRate is too small it will require more iterations to get trained and will be more accurate. AdaBoostM1 is used for binary classification problem only.

[Dario Walter]

Thanks for the reply Raunak. The way in which LearnRate acts is clear to me. However, the original AdaBoost M1 algorithm does not include a LearnRate Parameter.

Neither in this link (https://de.mathworks.com/help/stats/ensemble-algorithms.html#bsw8aue) nor in any other pseudocode the LearnRate is mentioned. Where in the pseudocode would I have to add the LearnRate?

[Raunak Gupta]

Hi Dario,

The LearnRate parameter is included while calculate the weights of the weak hypothesis in the ensemble that is () in the Algorithm mentioned here. It is not explained in the original AdaBoost algorithm but is used widely in almost all of the application of AdaBoost because it provides a way to tinker the actual contribution of subsequent weak learners. So when any is calculated for a learner it is multiplied with LearnRate to diminish or enhance its contribution (based on value being < 1 or > 1).

You can see in this example, the influence of different LearnRate.

[Dario Walter]

Thank you Raunak. This helps me a lot.