Fit ensemble of learners for regression
returns the trained regression ensemble model object (Mdl
= fitrensemble(Tbl
,ResponseVarName
)Mdl
)
that contains the results of boosting 100 regression trees using LSBoost and the
predictor and response data in the table Tbl
.
ResponseVarName
is the name of the response variable in
Tbl
.
applies Mdl
= fitrensemble(Tbl
,formula
)formula
to fit the model to the predictor and
response data in the table Tbl
. formula
is
an explanatory model of the response and a subset of predictor variables in
Tbl
used to fit Mdl
. For example,
'Y~X1+X2+X3'
fits the response variable
Tbl.Y
as a function of the predictor variables
Tbl.X1
, Tbl.X2
, and
Tbl.X3
.
uses additional options specified by one or more Mdl
= fitrensemble(___,Name,Value
)Name,Value
pair arguments and any of the input arguments in the previous syntaxes. For
example, you can specify the number of learning cycles, the ensemble aggregation
method, or to implement 10-fold cross-validation.
Create a regression ensemble that predicts the fuel economy of a car given the number of cylinders, volume displaced by the cylinders, horsepower, and weight. Then, train another ensemble using fewer predictors. Compare the in-sample predictive accuracies of the ensembles.
Load the carsmall
data set. Store the variables to be used in training in a table.
load carsmall
Tbl = table(Cylinders,Displacement,Horsepower,Weight,MPG);
Train a regression ensemble.
Mdl1 = fitrensemble(Tbl,'MPG');
Mdl1
is a RegressionEnsemble
model. Some notable characteristics of Mdl1
are:
The ensemble aggregation algorithm is 'LSBoost'
.
Because the ensemble aggregation method is a boosting algorithm, regression trees that allow a maximum of 10 splits compose the ensemble.
One hundred trees compose the ensemble.
Because MPG
is a variable in the MATLAB® Workspace, you can obtain the same result by entering
Mdl1 = fitrensemble(Tbl,MPG);
Use the trained regression ensemble to predict the fuel economy for a four-cylinder car with a 200-cubic inch displacement, 150 horsepower, and weighing 3000 lbs.
pMPG = predict(Mdl1,[4 200 150 3000])
pMPG = 25.6467
Train a new ensemble using all predictors in Tbl
except Displacement
.
formula = 'MPG ~ Cylinders + Horsepower + Weight';
Mdl2 = fitrensemble(Tbl,formula);
Compare the resubstitution MSEs between Mdl1
and Mdl2
.
mse1 = resubLoss(Mdl1)
mse1 = 0.3096
mse2 = resubLoss(Mdl2)
mse2 = 0.5861
The in-sample MSE for the ensemble that trains on all predictors is lower.
Train an ensemble of boosted regression trees by using fitrensemble
. Reduce training time by specifying the 'NumBins'
name-value pair argument to bin numeric predictors. After training, you can reproduce binned predictor data by using the BinEdges
property of the trained model and the discretize
function.
Generate a sample data set.
rng('default') % For reproducibility N = 1e6; X1 = randi([-1,5],[N,1]); X2 = randi([5,10],[N,1]); X3 = randi([0,5],[N,1]); X4 = randi([1,10],[N,1]); X = [X1 X2 X3 X4]; y = X1 + X2 + X3 + X4 + normrnd(0,1,[N,1]);
Train an ensemble of boosted regression trees using least-squares boosting (LSBoost
, the default value). Time the function for comparison purposes.
tic Mdl1 = fitrensemble(X,y); toc
Elapsed time is 78.662954 seconds.
Speed up training by using the 'NumBins'
name-value pair argument. If you specify the 'NumBins'
value as a positive integer scalar, then the software bins every numeric predictor into a specified number of equiprobable bins, and then grows trees on the bin indices instead of the original data. The software does not bin categorical predictors.
tic
Mdl2 = fitrensemble(X,y,'NumBins',50);
toc
Elapsed time is 43.353208 seconds.
The process is about two times faster when you use binned data instead of the original data. Note that the elapsed time can vary depending on your operating system.
Compare the regression errors by resubstitution.
rsLoss = resubLoss(Mdl1)
rsLoss = 1.0134
rsLoss2 = resubLoss(Mdl2)
rsLoss2 = 1.0133
In this example, binning predictor values reduces training time without a significant loss of accuracy. In general, when you have a large data set like the one in this example, using the binning option speeds up training but causes a potential decrease in accuracy. If you want to reduce training time further, specify a smaller number of bins.
Reproduce binned predictor data by using the BinEdges
property of the trained model and the discretize
function.
X = Mdl2.X; % Predictor data Xbinned = zeros(size(X)); edges = Mdl2.BinEdges; % Find indices of binned predictors. idxNumeric = find(~cellfun(@isempty,edges)); if iscolumn(idxNumeric) idxNumeric = idxNumeric'; end for j = idxNumeric x = X(:,j); % Convert x to array if x is a table. if istable(x) x = table2array(x); end % Group x into bins by using the discretize function. xbinned = discretize(x,[-inf; edges{j}; inf]); Xbinned(:,j) = xbinned; end
Xbinned
contains the bin indices, ranging from 1 to the number of bins, for numeric predictors. Xbinned
values are 0
for categorical predictors. If X
contains NaN
s, then the corresponding Xbinned
values are NaN
s.
Estimate the generalization error of an ensemble of boosted regression trees.
Load the carsmall
data set. Choose the number of cylinders, volume displaced by the cylinders, horsepower, and weight as predictors of fuel economy.
load carsmall
X = [Cylinders Displacement Horsepower Weight];
Cross-validate an ensemble of regression trees using 10-fold cross-validation. Using a decision tree template, specify that each tree should be a split once only.
rng(1); % For reproducibility t = templateTree('MaxNumSplits',1); Mdl = fitrensemble(X,MPG,'Learners',t,'CrossVal','on');
Mdl
is a RegressionPartitionedEnsemble
model.
Plot the cumulative, 10-fold cross-validated, mean-squared error (MSE). Display the estimated generalization error of the ensemble.
kflc = kfoldLoss(Mdl,'Mode','cumulative'); figure; plot(kflc); ylabel('10-fold cross-validated MSE'); xlabel('Learning cycle');
estGenError = kflc(end)
estGenError = 26.2356
kfoldLoss
returns the generalization error by default. However, plotting the cumulative loss allows you to monitor how the loss changes as weak learners accumulate in the ensemble.
The ensemble achieves an MSE of around 23.5 after accumulating about 30 weak learners.
If you are satisfied with the generalization error of the ensemble, then, to create a predictive model, train the ensemble again using all of the settings except cross-validation. However, it is good practice to tune hyperparameters such as the maximum number of decision splits per tree and the number of learning cycles..
This example shows how to optimize hyperparameters automatically using fitrensemble
. The example uses the carsmall
data.
Load the data.
load carsmall
You can find hyperparameters that minimize five-fold cross-validation loss by using automatic hyperparameter optimization.
Mdl = fitrensemble([Horsepower,Weight],MPG,'OptimizeHyperparameters','auto')
In this example, for reproducibility, set the random seed and use the 'expected-improvement-plus'
acquisition function. Also, for reproducibility of random forest algorithm, specify the 'Reproducible'
name-value pair argument as true
for tree learners.
rng('default') t = templateTree('Reproducible',true); Mdl = fitrensemble([Horsepower,Weight],MPG,'OptimizeHyperparameters','auto','Learners',t, ... 'HyperparameterOptimizationOptions',struct('AcquisitionFunctionName','expected-improvement-plus'))
|===================================================================================================================================| | Iter | Eval | Objective: | Objective | BestSoFar | BestSoFar | Method | NumLearningC-| LearnRate | MinLeafSize | | | result | log(1+loss) | runtime | (observed) | (estim.) | | ycles | | | |===================================================================================================================================| | 1 | Best | 2.9726 | 16.854 | 2.9726 | 2.9726 | Bag | 413 | - | 1 | | 2 | Accept | 6.2619 | 1.6222 | 2.9726 | 3.6133 | LSBoost | 57 | 0.0016067 | 6 | | 3 | Accept | 2.9975 | 0.8919 | 2.9726 | 2.9852 | Bag | 32 | - | 2 | | 4 | Accept | 4.1897 | 1.2422 | 2.9726 | 2.972 | Bag | 55 | - | 40 | | 5 | Accept | 6.3321 | 2.4277 | 2.9726 | 2.9715 | LSBoost | 55 | 0.001005 | 2 | | 6 | Best | 2.9714 | 0.97789 | 2.9714 | 2.9715 | Bag | 39 | - | 1 | | 7 | Best | 2.9615 | 1.2797 | 2.9615 | 2.9681 | Bag | 55 | - | 1 | | 8 | Accept | 3.0499 | 0.28101 | 2.9615 | 2.9873 | Bag | 10 | - | 1 | | 9 | Accept | 2.9855 | 11.252 | 2.9615 | 2.9633 | Bag | 500 | - | 1 | | 10 | Best | 2.928 | 6.2075 | 2.928 | 2.9317 | Bag | 282 | - | 2 | | 11 | Accept | 2.9362 | 6.7497 | 2.928 | 2.9336 | Bag | 304 | - | 2 | | 12 | Accept | 2.9316 | 5.3101 | 2.928 | 2.9327 | Bag | 247 | - | 2 | | 13 | Best | 2.9215 | 5.0977 | 2.9215 | 2.9299 | Bag | 242 | - | 2 | | 14 | Accept | 4.1882 | 13.082 | 2.9215 | 2.9298 | LSBoost | 498 | 0.011265 | 50 | | 15 | Accept | 4.1881 | 11.798 | 2.9215 | 2.9297 | LSBoost | 497 | 0.075987 | 50 | | 16 | Accept | 3.6293 | 0.65533 | 2.9215 | 2.9297 | LSBoost | 24 | 0.95396 | 1 | | 17 | Accept | 4.1881 | 2.3163 | 2.9215 | 2.9296 | LSBoost | 92 | 0.95228 | 49 | | 18 | Accept | 3.3804 | 0.37449 | 2.9215 | 2.9296 | LSBoost | 12 | 0.16163 | 1 | | 19 | Accept | 3.5064 | 11.651 | 2.9215 | 2.9296 | LSBoost | 473 | 0.043212 | 1 | | 20 | Accept | 3.5342 | 11.479 | 2.9215 | 2.9296 | LSBoost | 487 | 0.24602 | 1 | |===================================================================================================================================| | Iter | Eval | Objective: | Objective | BestSoFar | BestSoFar | Method | NumLearningC-| LearnRate | MinLeafSize | | | result | log(1+loss) | runtime | (observed) | (estim.) | | ycles | | | |===================================================================================================================================| | 21 | Accept | 2.9413 | 5.0514 | 2.9215 | 2.9319 | Bag | 222 | - | 2 | | 22 | Accept | 2.9313 | 10.662 | 2.9215 | 2.931 | Bag | 500 | - | 2 | | 23 | Accept | 2.9496 | 8.5612 | 2.9215 | 2.9332 | Bag | 395 | - | 2 | | 24 | Accept | 6.2871 | 0.45327 | 2.9215 | 2.9333 | LSBoost | 10 | 0.0077899 | 1 | | 25 | Accept | 3.5075 | 11.683 | 2.9215 | 2.9333 | LSBoost | 488 | 0.092689 | 1 | | 26 | Accept | 3.1057 | 0.39062 | 2.9215 | 2.9332 | LSBoost | 11 | 0.37151 | 6 | | 27 | Accept | 3.3708 | 0.78724 | 2.9215 | 2.9332 | LSBoost | 10 | 0.18122 | 6 | | 28 | Accept | 3.3523 | 0.36302 | 2.9215 | 2.9333 | LSBoost | 10 | 0.40692 | 2 | | 29 | Accept | 3.6144 | 11.843 | 2.9215 | 2.9331 | LSBoost | 497 | 0.44774 | 7 | | 30 | Accept | 3.2239 | 0.33549 | 2.9215 | 2.9331 | LSBoost | 10 | 0.31373 | 36 |
__________________________________________________________ Optimization completed. MaxObjectiveEvaluations of 30 reached. Total function evaluations: 30 Total elapsed time: 204.6093 seconds Total objective function evaluation time: 161.6798 Best observed feasible point: Method NumLearningCycles LearnRate MinLeafSize ______ _________________ _________ ___________ Bag 242 NaN 2 Observed objective function value = 2.9215 Estimated objective function value = 2.9329 Function evaluation time = 5.0977 Best estimated feasible point (according to models): Method NumLearningCycles LearnRate MinLeafSize ______ _________________ _________ ___________ Bag 282 NaN 2 Estimated objective function value = 2.9331 Estimated function evaluation time = 6.635
Mdl = RegressionBaggedEnsemble ResponseName: 'Y' CategoricalPredictors: [] ResponseTransform: 'none' NumObservations: 94 HyperparameterOptimizationResults: [1x1 BayesianOptimization] NumTrained: 282 Method: 'Bag' LearnerNames: {'Tree'} ReasonForTermination: 'Terminated normally after completing the requested number of training cycles.' FitInfo: [] FitInfoDescription: 'None' Regularization: [] FResample: 1 Replace: 1 UseObsForLearner: [94x282 logical] Properties, Methods
The optimization searched over the methods for regression (Bag
and LSBoost
), over NumLearningCycles
, over the LearnRate
for LSBoost
, and over the tree learner MinLeafSize
. The output is the ensemble regression with the minimum estimated cross-validation loss.
One way to create an ensemble of boosted regression trees that has satisfactory predictive performance is to tune the decision tree complexity level using cross-validation. While searching for an optimal complexity level, tune the learning rate to minimize the number of learning cycles as well.
This example manually finds optimal parameters by using the cross-validation option (the 'KFold'
name-value pair argument) and the kfoldLoss
function. Alternatively, you can use the 'OptimizeHyperparameters'
name-value pair argument to optimize hyperparameters automatically. See Optimize Regression Ensemble.
Load the carsmall
data set. Choose the number of cylinders, volume displaced by the cylinders, horsepower, and weight as predictors of fuel economy.
load carsmall
Tbl = table(Cylinders,Displacement,Horsepower,Weight,MPG);
The default values of the tree depth controllers for boosting regression trees are:
10
for MaxNumSplits
.
5
for MinLeafSize
10
for MinParentSize
To search for the optimal tree-complexity level:
Cross-validate a set of ensembles. Exponentially increase the tree-complexity level for subsequent ensembles from decision stump (one split) to at most n - 1 splits. n is the sample size. Also, vary the learning rate for each ensemble between 0.1 to 1.
Estimate the cross-validated mean-squared error (MSE) for each ensemble.
For tree-complexity level , , compare the cumulative, cross-validated MSE of the ensembles by plotting them against number of learning cycles. Plot separate curves for each learning rate on the same figure.
Choose the curve that achieves the minimal MSE, and note the corresponding learning cycle and learning rate.
Cross-validate a deep regression tree and a stump. Because the data contain missing values, use surrogate splits. These regression trees serve as benchmarks.
rng(1) % For reproducibility MdlDeep = fitrtree(Tbl,'MPG','CrossVal','on','MergeLeaves','off', ... 'MinParentSize',1,'Surrogate','on'); MdlStump = fitrtree(Tbl,'MPG','MaxNumSplits',1,'CrossVal','on', ... 'Surrogate','on');
Cross-validate an ensemble of 150 boosted regression trees using 5-fold cross-validation. Using a tree template:
Vary the maximum number of splits using the values in the sequence . m is such that is no greater than n - 1.
Turn on surrogate splits.
For each variant, adjust the learning rate using each value in the set {0.1, 0.25, 0.5, 1}.
n = size(Tbl,1); m = floor(log2(n - 1)); learnRate = [0.1 0.25 0.5 1]; numLR = numel(learnRate); maxNumSplits = 2.^(0:m); numMNS = numel(maxNumSplits); numTrees = 150; Mdl = cell(numMNS,numLR); for k = 1:numLR for j = 1:numMNS t = templateTree('MaxNumSplits',maxNumSplits(j),'Surrogate','on'); Mdl{j,k} = fitrensemble(Tbl,'MPG','NumLearningCycles',numTrees, ... 'Learners',t,'KFold',5,'LearnRate',learnRate(k)); end end
Estimate the cumulative, cross-validated MSE of each ensemble.
kflAll = @(x)kfoldLoss(x,'Mode','cumulative'); errorCell = cellfun(kflAll,Mdl,'Uniform',false); error = reshape(cell2mat(errorCell),[numTrees numel(maxNumSplits) numel(learnRate)]); errorDeep = kfoldLoss(MdlDeep); errorStump = kfoldLoss(MdlStump);
Plot how the cross-validated MSE behaves as the number of trees in the ensemble increases. Plot the curves with respect to learning rate on the same plot, and plot separate plots for varying tree-complexity levels. Choose a subset of tree complexity levels to plot.
mnsPlot = [1 round(numel(maxNumSplits)/2) numel(maxNumSplits)]; figure; for k = 1:3 subplot(2,2,k) plot(squeeze(error(:,mnsPlot(k),:)),'LineWidth',2) axis tight hold on h = gca; plot(h.XLim,[errorDeep errorDeep],'-.b','LineWidth',2) plot(h.XLim,[errorStump errorStump],'-.r','LineWidth',2) plot(h.XLim,min(min(error(:,mnsPlot(k),:))).*[1 1],'--k') h.YLim = [10 50]; xlabel('Number of trees') ylabel('Cross-validated MSE') title(sprintf('MaxNumSplits = %0.3g', maxNumSplits(mnsPlot(k)))) hold off end hL = legend([cellstr(num2str(learnRate','Learning Rate = %0.2f')); ... 'Deep Tree';'Stump';'Min. MSE']); hL.Position(1) = 0.6;
Each curve contains a minimum cross-validated MSE occurring at the optimal number of trees in the ensemble.
Identify the maximum number of splits, number of trees, and learning rate that yields the lowest MSE overall.
[minErr,minErrIdxLin] = min(error(:));
[idxNumTrees,idxMNS,idxLR] = ind2sub(size(error),minErrIdxLin);
fprintf('\nMin. MSE = %0.5f',minErr)
Min. MSE = 16.77593
fprintf('\nOptimal Parameter Values:\nNum. Trees = %d',idxNumTrees);
Optimal Parameter Values: Num. Trees = 78
fprintf('\nMaxNumSplits = %d\nLearning Rate = %0.2f\n',... maxNumSplits(idxMNS),learnRate(idxLR))
MaxNumSplits = 1 Learning Rate = 0.25
Create a predictive ensemble based on the optimal hyperparameters and the entire training set.
tFinal = templateTree('MaxNumSplits',maxNumSplits(idxMNS),'Surrogate','on'); MdlFinal = fitrensemble(Tbl,'MPG','NumLearningCycles',idxNumTrees, ... 'Learners',tFinal,'LearnRate',learnRate(idxLR))
MdlFinal = RegressionEnsemble PredictorNames: {1x4 cell} ResponseName: 'MPG' CategoricalPredictors: [] ResponseTransform: 'none' NumObservations: 94 NumTrained: 78 Method: 'LSBoost' LearnerNames: {'Tree'} ReasonForTermination: 'Terminated normally after completing the requested number of training cycles.' FitInfo: [78x1 double] FitInfoDescription: {2x1 cell} Regularization: [] Properties, Methods
MdlFinal
is a RegressionEnsemble
. To predict the fuel economy of a car given its number of cylinders, volume displaced by the cylinders, horsepower, and weight, you can pass the predictor data and MdlFinal
to predict
.
Instead of searching optimal values manually by using the cross-validation option ('KFold'
) and the kfoldLoss
function, you can use the 'OptimizeHyperparameters'
name-value pair argument. When you specify 'OptimizeHyperparameters'
, the software finds optimal parameters automatically using Bayesian optimization. The optimal values obtained by using 'OptimizeHyperparameters'
can be different from those obtained using manual search.
t = templateTree('Surrogate','on'); mdl = fitrensemble(Tbl,'MPG','Learners',t, ... 'OptimizeHyperparameters',{'NumLearningCycles','LearnRate','MaxNumSplits'})
|====================================================================================================================| | Iter | Eval | Objective: | Objective | BestSoFar | BestSoFar | NumLearningC-| LearnRate | MaxNumSplits | | | result | log(1+loss) | runtime | (observed) | (estim.) | ycles | | | |====================================================================================================================| | 1 | Best | 3.3955 | 1.3296 | 3.3955 | 3.3955 | 26 | 0.072054 | 3 | | 2 | Accept | 6.0976 | 6.3773 | 3.3955 | 3.5549 | 170 | 0.0010295 | 70 | | 3 | Best | 3.2914 | 9.4601 | 3.2914 | 3.2917 | 273 | 0.61026 | 6 | | 4 | Accept | 6.1839 | 2.6633 | 3.2914 | 3.2915 | 80 | 0.0016871 | 1 | | 5 | Best | 3.0379 | 0.8248 | 3.0379 | 3.0384 | 18 | 0.21288 | 31 | | 6 | Accept | 3.3628 | 0.68604 | 3.0379 | 3.1888 | 10 | 0.17826 | 5 | | 7 | Best | 3.0192 | 0.56051 | 3.0192 | 3.0146 | 10 | 0.27711 | 59 | | 8 | Accept | 4.3148 | 0.58189 | 3.0192 | 3.0191 | 11 | 0.099523 | 99 | | 9 | Accept | 3.1939 | 0.6954 | 3.0192 | 3.2463 | 10 | 0.8556 | 62 | | 10 | Accept | 3.4117 | 0.65447 | 3.0192 | 3.0193 | 10 | 0.97894 | 97 | | 11 | Accept | 3.0556 | 0.49599 | 3.0192 | 3.0262 | 10 | 0.40405 | 27 | | 12 | Accept | 3.1137 | 0.68894 | 3.0192 | 3.0196 | 10 | 0.42996 | 89 | | 13 | Accept | 3.4358 | 0.68729 | 3.0192 | 3.0184 | 10 | 0.98766 | 16 | | 14 | Accept | 3.0444 | 0.46698 | 3.0192 | 3.0211 | 10 | 0.3072 | 28 | | 15 | Accept | 3.1599 | 0.56523 | 3.0192 | 3.0226 | 10 | 0.21933 | 1 | | 16 | Accept | 5.7086 | 0.6417 | 3.0192 | 3.0324 | 10 | 0.036906 | 26 | | 17 | Accept | 3.0827 | 1.8004 | 3.0192 | 3.0324 | 47 | 0.14064 | 19 | | 18 | Accept | 3.233 | 0.93856 | 3.0192 | 3.0327 | 20 | 0.57027 | 25 | | 19 | Best | 2.9344 | 1.9532 | 2.9344 | 2.9348 | 57 | 0.06688 | 1 | | 20 | Best | 2.9301 | 1.7625 | 2.9301 | 2.9298 | 49 | 0.085566 | 6 | |====================================================================================================================| | Iter | Eval | Objective: | Objective | BestSoFar | BestSoFar | NumLearningC-| LearnRate | MaxNumSplits | | | result | log(1+loss) | runtime | (observed) | (estim.) | ycles | | | |====================================================================================================================| | 21 | Accept | 3.0949 | 3.8916 | 2.9301 | 2.9298 | 109 | 0.086821 | 15 | | 22 | Accept | 2.9938 | 2.0892 | 2.9301 | 2.9312 | 60 | 0.34565 | 2 | | 23 | Accept | 3.1667 | 1.306 | 2.9301 | 2.931 | 28 | 0.28864 | 79 | | 24 | Accept | 3.2671 | 2.5289 | 2.9301 | 2.9246 | 79 | 0.60876 | 4 | | 25 | Best | 2.918 | 1.9716 | 2.918 | 2.9268 | 53 | 0.11995 | 1 | | 26 | Accept | 2.9193 | 3.9779 | 2.918 | 2.9305 | 118 | 0.26486 | 1 | | 27 | Accept | 2.9259 | 1.9028 | 2.918 | 2.9058 | 57 | 0.089008 | 1 | | 28 | Best | 2.8857 | 3.7852 | 2.8857 | 2.905 | 101 | 0.3349 | 1 | | 29 | Accept | 2.97 | 3.5924 | 2.8857 | 2.8928 | 110 | 0.030579 | 1 | | 30 | Accept | 2.9271 | 6.8808 | 2.8857 | 2.8931 | 209 | 0.032758 | 1 |
__________________________________________________________ Optimization completed. MaxObjectiveEvaluations of 30 reached. Total function evaluations: 30 Total elapsed time: 127.0126 seconds Total objective function evaluation time: 65.7605 Best observed feasible point: NumLearningCycles LearnRate MaxNumSplits _________________ _________ ____________ 101 0.3349 1 Observed objective function value = 2.8857 Estimated objective function value = 2.8931 Function evaluation time = 3.7852 Best estimated feasible point (according to models): NumLearningCycles LearnRate MaxNumSplits _________________ _________ ____________ 101 0.3349 1 Estimated objective function value = 2.8931 Estimated function evaluation time = 3.4655
mdl = RegressionEnsemble PredictorNames: {1x4 cell} ResponseName: 'MPG' CategoricalPredictors: [] ResponseTransform: 'none' NumObservations: 94 HyperparameterOptimizationResults: [1x1 BayesianOptimization] NumTrained: 101 Method: 'LSBoost' LearnerNames: {'Tree'} ReasonForTermination: 'Terminated normally after completing the requested number of training cycles.' FitInfo: [101x1 double] FitInfoDescription: {2x1 cell} Regularization: [] Properties, Methods
Tbl
— Sample dataSample data used to train the model, specified as a table. Each
row of Tbl
corresponds to one observation, and
each column corresponds to one predictor variable. Tbl
can
contain one additional column for the response variable. Multicolumn
variables and cell arrays other than cell arrays of character vectors
are not allowed.
If Tbl
contains the response variable
and you want to use all remaining variables as predictors, then specify
the response variable using ResponseVarName
.
If Tbl
contains the response
variable, and you want to use a subset of the remaining variables
only as predictors, then specify a formula using formula
.
If Tbl
does not contain the response
variable, then specify the response data using Y
.
The length of response variable and the number of rows of Tbl
must
be equal.
Note
To save memory and execution time, supply X
and Y
instead
of Tbl
.
Data Types: table
ResponseVarName
— Response variable nameTbl
Response variable name, specified as the name of the response variable in
Tbl
.
You must specify ResponseVarName
as a character
vector or string scalar. For example, if Tbl.Y
is the
response variable, then specify ResponseVarName
as
'Y'
. Otherwise, fitrensemble
treats all columns of Tbl
as predictor
variables.
Data Types: char
| string
formula
— Explanatory model of response variable and subset of predictor variablesExplanatory model of the response variable and a subset of the predictor variables,
specified as a character vector or string scalar in the form
'Y~x1+x2+x3'
. In this form, Y
represents the
response variable, and x1
, x2
, and
x3
represent the predictor variables.
To specify a subset of variables in Tbl
as predictors for
training the model, use a formula. If you specify a formula, then the software does not
use any variables in Tbl
that do not appear in
formula
.
The variable names in the formula must be both variable names in Tbl
(Tbl.Properties.VariableNames
) and valid MATLAB® identifiers. You can verify the variable names in Tbl
by
using the isvarname
function. If the variable names
are not valid, then you can convert them by using the matlab.lang.makeValidName
function.
Data Types: char
| string
X
— Predictor dataPredictor data, specified as numeric matrix.
Each row corresponds to one observation, and each column corresponds to one predictor variable.
The length of Y
and the number of rows
of X
must be equal.
To specify the names of the predictors in the order of their
appearance in X
, use the PredictorNames
name-value
pair argument.
Data Types: single
| double
Specify optional
comma-separated pairs of Name,Value
arguments. Name
is
the argument name and Value
is the corresponding value.
Name
must appear inside quotes. You can specify several name and value
pair arguments in any order as
Name1,Value1,...,NameN,ValueN
.
'NumLearningCycles',500,'Method','Bag','Learners',templateTree(),'CrossVal','on'
cross-validates an ensemble of 500 bagged regression trees using 10-fold
cross-validation.Note
You cannot use any cross-validation name-value pair argument along with the
'OptimizeHyperparameters'
name-value pair argument. You can modify
the cross-validation for 'OptimizeHyperparameters'
only by using the
'HyperparameterOptimizationOptions'
name-value pair
argument.
'Method'
— Ensemble aggregation method'LSBoost'
(default) | 'Bag'
Ensemble aggregation method, specified as the comma-separated pair
consisting of 'Method'
and
'LSBoost'
or
'Bag'
.
Value | Method | Notes |
---|---|---|
'LSBoost' | Least-squares boosting (LSBoost) | You can specify the learning rate for shrinkage
by using the 'LearnRate'
name-value pair argument. |
'Bag' | Bootstrap aggregation (bagging, for example, random forest[2]) | fitrensemble uses bagging
with random predictor selections at each split
(random forest) by default. To use bagging without
the random selections, use tree learners whose
'NumVariablesToSample' value is
'all' . |
For details about ensemble aggregation algorithms and examples, see Algorithms, Ensemble Algorithms, and Choose an Applicable Ensemble Aggregation Method.
Example: 'Method','Bag'
'NumLearningCycles'
— Number of ensemble learning cycles100
(default) | positive integerNumber of ensemble learning cycles, specified as the comma-separated
pair consisting of 'NumLearningCycles'
and a positive
integer. At every learning cycle, the software trains one weak learner
for every template object in Learners
.
Consequently, the software trains
NumLearningCycles*numel(Learners)
learners.
The software composes the ensemble using all trained learners and
stores them in Mdl.Trained
.
For more details, see Tips.
Example: 'NumLearningCycles',500
Data Types: single
| double
'Learners'
— Weak learners to use in ensemble'tree'
(default) | tree template object | cell vector of tree template objectsWeak learners to use in the ensemble, specified as the comma-separated
pair consisting of 'Learners'
and
'tree'
, a tree template object, or a cell vector
of tree template objects.
'tree'
(default) —
fitrensemble
uses default
regression tree learners, which is the same as using
templateTree()
. The default values of
templateTree()
depend on the value of
'Method'
.
For bagged decision trees, the maximum number
of decision splits ('MaxNumSplits'
) is
n–1
, where n
is the number of observations. The number of
predictors to select at random for each split
('NumVariablesToSample'
) is one third
of the number of predictors. Therefore,
fitrensemble
grows deep
decision trees. You can grow shallower trees to
reduce model complexity or computation
time.
For boosted decision trees,
'MaxNumSplits'
is 10 and
'NumVariablesToSample'
is
'all'
. Therefore,
fitrensemble
grows shallow
decision trees. You can grow deeper trees for
better accuracy.
See templateTree
for the default settings of a weak learner.
Tree template object — fitrensemble
uses the tree template object created by templateTree
.
Use the name-value pair arguments of
templateTree
to specify settings of
the tree learners.
Cell vector of m tree template objects
— fitrensemble
grows
m regression trees per learning cycle
(see NumLearningCycles
). For example,
for an ensemble composed of two types of regression trees,
supply {t1 t2}
, where
t1
and t2
are
regression tree template objects returned by
templateTree
.
To obtain reproducible results, you must specify the 'Reproducible'
name-value pair argument of
templateTree
as true
if
'NumVariablesToSample'
is not
'all'
.
For details on the number of learners to train, see
NumLearningCycles
and Tips.
Example: 'Learners',templateTree('MaxNumSplits',5)
'NPrint'
— Printout frequency'off'
(default) | positive integerPrintout frequency, specified as the comma-separated pair consisting
of 'NPrint'
and a positive integer or 'off'
.
To track the number of weak learners or folds that
fitrensemble
trained so far, specify a positive integer. That
is, if you specify the positive integer m:
Without also specifying any cross-validation option
(for example, CrossVal
), then fitrensemble
displays
a message to the command line every time it completes training m weak
learners.
And a cross-validation option, then fitrensemble
displays
a message to the command line every time it finishes training m folds.
If you specify 'off'
, then fitrensemble
does
not display a message when it completes training weak learners.
Tip
For fastest training of some boosted decision trees, set NPrint
to the
default value 'off'
. This tip holds when the classification
Method
is 'AdaBoostM1'
,
'AdaBoostM2'
, 'GentleBoost'
, or
'LogitBoost'
, or when the regression Method
is
'LSBoost'
.
Example: 'NPrint',5
Data Types: single
| double
| char
| string
'NumBins'
— Number of bins for numeric predictors[]
(empty) (default) | positive integer scalarNumber of bins for numeric predictors, specified as the comma-separated pair
consisting of 'NumBins'
and a positive integer scalar.
If the 'NumBins'
value is empty (default), then
fitrensemble
does not bin any predictors.
If you specify the 'NumBins'
value as a positive integer scalar
(numBins
), then fitrensemble
bins every
numeric predictor into at most numBins
equiprobable bins, and
then grows trees on the bin indices instead of the original data.
The number of bins can be less than numBins
if a
predictor has fewer than numBins
unique
values.
fitrensemble
does not bin categorical
predictors.
When you use a large training data set, this binning option speeds up training but might cause
a potential decrease in accuracy. You can try 'NumBins',50
first, and
then change the value depending on the accuracy and training speed.
A trained model stores the bin edges in the BinEdges
property.
Example: 'NumBins',50
Data Types: single
| double
'CategoricalPredictors'
— Categorical predictors list'all'
Categorical predictors list, specified as one of the values in this table.
Value | Description |
---|---|
Vector of positive integers |
Each entry in the vector is an index value corresponding to the column of the predictor data that contains a categorical variable. The index values are between 1 and If |
Logical vector |
A |
Character matrix | Each row of the matrix is the name of a predictor variable. The names must match the entries in PredictorNames . Pad the names with extra blanks so each row of the character matrix has the same length. |
String array or cell array of character vectors | Each element in the array is the name of a predictor variable. The names must match the entries in PredictorNames . |
'all' | All predictors are categorical. |
By default, if the predictor data is in a table
(Tbl
), fitrensemble
assumes that a variable is
categorical if it is a logical vector, unordered categorical vector, character array, string
array, or cell array of character vectors. If the predictor data is a matrix
(X
), fitrensemble
assumes that all predictors are
continuous. To identify any other predictors as categorical predictors, specify them by using
the 'CategoricalPredictors'
name-value argument.
Example: 'CategoricalPredictors','all'
Data Types: single
| double
| logical
| char
| string
| cell
'PredictorNames'
— Predictor variable namesPredictor variable names, specified as a string array of unique names or cell array of unique
character vectors. The functionality of PredictorNames
depends on the
way you supply the training data.
If you supply X
and Y
, then you
can use PredictorNames
to assign names to the predictor
variables in X
.
The order of the names in PredictorNames
must correspond to the column order of X
.
That is, PredictorNames{1}
is the name of
X(:,1)
,
PredictorNames{2}
is the name of
X(:,2)
, and so on. Also,
size(X,2)
and
numel(PredictorNames)
must be
equal.
By default, PredictorNames
is
{'x1','x2',...}
.
If you supply Tbl
, then you can use
PredictorNames
to choose which predictor variables to
use in training. That is, fitrensemble
uses only the
predictor variables in PredictorNames
and the response
variable during training.
PredictorNames
must be a subset of
Tbl.Properties.VariableNames
and cannot
include the name of the response variable.
By default, PredictorNames
contains the
names of all predictor variables.
A good practice is to specify the predictors for training
using either 'PredictorNames'
or
formula
, but not both.
Example: 'PredictorNames',{'SepalLength','SepalWidth','PetalLength','PetalWidth'}
Data Types: string
| cell
'ResponseName'
— Response variable name'Y'
(default) | character vector | string scalarResponse variable name, specified as a character vector or string scalar.
If you supply Y
, then you can
use 'ResponseName'
to specify a name for the response
variable.
If you supply ResponseVarName
or formula
,
then you cannot use 'ResponseName'
.
Example: 'ResponseName','response'
Data Types: char
| string
'ResponseTransform'
— Response transformation'none'
(default) | function handleResponse transformation, specified as either 'none'
or a function
handle. The default is 'none'
, which means @(y)y
,
or no transformation. For a MATLAB function or a function you define, use its function handle for the
response transformation. The function handle must accept a vector (the original response
values) and return a vector of the same size (the transformed response values).
Example: Suppose you create a function handle that applies an exponential
transformation to an input vector by using myfunction = @(y)exp(y)
.
Then, you can specify the response transformation as
'ResponseTransform',myfunction
.
Data Types: char
| string
| function_handle
'Options'
— Options for computing in parallel and setting random numbersOptions for computing in parallel and setting random numbers, specified as a structure. Create
the Options
structure with statset
.
Note
You need Parallel Computing Toolbox™ to compute in parallel.
This table lists the option fields and their values.
Field Name | Value | Default |
---|---|---|
UseParallel | Set this value to | false |
UseSubstreams | Set this value to To compute reproducibly, set
| false |
Streams | Specify this value as a RandStream object or cell array of such objects. Use a single object except when the UseParallel value is true and the UseSubstreams value is false . In that case, use a cell array that has the same size as the parallel pool. | If you do not specify Streams , then fitrensemble uses the default stream or streams. |
For an example using reproducible parallel training, see Train Classification Ensemble in Parallel.
For dual-core systems and above, fitrensemble
parallelizes
training using Intel® Threading Building Blocks (TBB). Therefore, specifying the
UseParallel
option as true
might not provide a
significant speedup on a single computer. For details on Intel TBB, see https://software.intel.com/en-us/intel-tbb.
Example: 'Options',statset('UseParallel',true)
Data Types: struct
'CrossVal'
— Cross-validation flag'off'
(default) | 'on'
Cross-validation flag, specified as the comma-separated pair
consisting of 'Crossval'
and 'on'
or 'off'
.
If you specify 'on'
, then the software implements
10-fold cross-validation.
To override this cross-validation setting, use one of these
name-value pair arguments: CVPartition
, Holdout
, KFold
,
or Leaveout
. To create a cross-validated model,
you can use one cross-validation name-value pair argument at a time
only.
Alternatively, cross-validate later by passing Mdl
to crossval
or crossval
.
Example: 'Crossval','on'
'CVPartition'
— Cross-validation partition[]
(default) | cvpartition
partition objectCross-validation partition, specified as a cvpartition
partition object
created by cvpartition
. The partition object
specifies the type of cross-validation and the indexing for the training and validation
sets.
To create a cross-validated model, you can specify only one of these four name-value
arguments: CVPartition
, Holdout
,
KFold
, or Leaveout
.
Example: Suppose you create a random partition for 5-fold cross-validation on 500
observations by using cvp = cvpartition(500,'KFold',5)
. Then, you can
specify the cross-validated model by using
'CVPartition',cvp
.
'Holdout'
— Fraction of data for holdout validationFraction of the data used for holdout validation, specified as a scalar value in the range
(0,1). If you specify 'Holdout',p
, then the software completes these
steps:
Randomly select and reserve p*100
% of the data as
validation data, and train the model using the rest of the data.
Store the compact, trained model in the Trained
property of the cross-validated model.
To create a cross-validated model, you can specify only one of these four name-value
arguments: CVPartition
, Holdout
,
KFold
, or Leaveout
.
Example: 'Holdout',0.1
Data Types: double
| single
'KFold'
— Number of folds10
(default) | positive integer value greater than 1Number of folds to use in a cross-validated model, specified as a positive integer value
greater than 1. If you specify 'KFold',k
, then the software completes
these steps:
Randomly partition the data into k
sets.
For each set, reserve the set as validation data, and train the model
using the other k
– 1 sets.
Store the k
compact, trained models in a
k
-by-1 cell vector in the Trained
property of the cross-validated model.
To create a cross-validated model, you can specify only one of these four name-value
arguments: CVPartition
, Holdout
,
KFold
, or Leaveout
.
Example: 'KFold',5
Data Types: single
| double
'Leaveout'
— Leave-one-out cross-validation flag'off'
(default) | 'on'
Leave-one-out cross-validation flag, specified as 'on'
or
'off'
. If you specify 'Leaveout','on'
, then
for each of the n observations (where n is the
number of observations, excluding missing observations, specified in the
NumObservations
property of the model), the software completes
these steps:
Reserve the one observation as validation data, and train the model using the other n – 1 observations.
Store the n compact, trained models in an
n-by-1 cell vector in the Trained
property of the cross-validated model.
To create a cross-validated model, you can specify only one of these four name-value
arguments: CVPartition
, Holdout
,
KFold
, or Leaveout
.
Example: 'Leaveout','on'
'Weights'
— Observation weightsTbl
Observation weights, specified as the comma-separated pair consisting
of 'Weights'
and a numeric vector of positive values
or name of a variable in Tbl
. The software weighs
the observations in each row of X
or
Tbl
with the corresponding value in
Weights
. The size of Weights
must equal the number of rows of X
or
Tbl
.
If you specify the input data as a table Tbl
,
then Weights
can be the name of a variable in
Tbl
that contains a numeric vector. In this
case, you must specify Weights
as a character
vector or string scalar. For example, if the weights vector
W
is stored as Tbl.W
, then
specify it as 'W'
. Otherwise, the software treats all
columns of Tbl
, including W
, as
predictors or the response when training the model.
The software normalizes the values of Weights
to
sum to 1.
By default, Weights
is
ones(
, where
n
,1)n
is the number of observations in
X
or Tbl
.
Data Types: double
| single
| char
| string
'FResample'
— Fraction of training set to resample1
(default) | positive scalar in (0,1]'Replace'
— Flag indicating to sample with replacement'on'
(default) | 'off'
Flag indicating sampling with replacement, specified as the
comma-separated pair consisting of 'Replace'
and 'off'
or 'on'
.
For 'on'
, the software samples
the training observations with replacement.
For 'off'
, the software samples
the training observations without replacement. If you set Resample
to 'on'
,
then the software samples training observations assuming uniform weights.
If you also specify a boosting method, then the software boosts by
reweighting observations.
Unless you set Method
to 'bag'
or
set Resample
to 'on'
, Replace
has
no effect.
Example: 'Replace','off'
'Resample'
— Flag indicating to resample'off'
| 'on'
Flag indicating to resample, specified as the comma-separated
pair consisting of 'Resample'
and 'off'
or 'on'
.
If Method
is a boosting method, then:
'Resample','on'
specifies to sample training
observations using updated weights as the multinomial sampling
probabilities.
'Resample','off'
(default) specifies to reweight
observations at every learning iteration.
If Method
is 'bag'
,
then 'Resample'
must be 'on'
.
The software resamples a fraction of the training observations (see FResample
)
with or without replacement (see Replace
).
If you specify to resample using Resample
, then it is good
practice to resample to entire data set. That is, use the default setting of 1 for
FResample
.
'LearnRate'
— Learning rate for shrinkage1
(default) | numeric scalar in (0,1]Learning rate for shrinkage, specified as the comma-separated pair consisting of
'LearnRate'
and a numeric scalar in the interval (0,1].
To train an ensemble using shrinkage, set LearnRate
to a value less than 1
, for example, 0.1
is a popular choice. Training an ensemble using shrinkage requires more learning iterations, but often achieves better accuracy.
Example: 'LearnRate',0.1
Data Types: single
| double
'OptimizeHyperparameters'
— Parameters to optimize'none'
(default) | 'auto'
| 'all'
| string array or cell array of eligible parameter names | vector of optimizableVariable
objectsParameters to optimize, specified as the comma-separated pair
consisting of 'OptimizeHyperparameters'
and one of
the following:
'none'
— Do not optimize.
'auto'
— Use
{'Method','NumLearningCycles','LearnRate'}
along with the default parameters for the specified
Learners
:
Learners
=
'tree'
(default) —
{'MinLeafSize'}
Note
For hyperparameter optimization,
Learners
must be a single argument,
not a string array or cell array.
'all'
— Optimize all eligible
parameters.
String array or cell array of eligible parameter names
Vector of optimizableVariable
objects,
typically the output of hyperparameters
The optimization attempts to minimize the cross-validation loss
(error) for fitrensemble
by varying the parameters.
To control the cross-validation type and other aspects of the
optimization, use the
HyperparameterOptimizationOptions
name-value
pair.
Note
'OptimizeHyperparameters'
values override any values you set using
other name-value pair arguments. For example, setting
'OptimizeHyperparameters'
to 'auto'
causes the
'auto'
values to apply.
The eligible parameters for fitrensemble
are:
Method
— Eligible methods are
'Bag'
or
'LSBoost'
.
NumLearningCycles
—
fitrensemble
searches among positive
integers, by default log-scaled with range
[10,500]
.
LearnRate
—
fitrensemble
searches among positive
reals, by default log-scaled with range
[1e-3,1]
.
MinLeafSize
—
fitrensemble
searches among integers
log-scaled in the range
[1,max(2,floor(NumObservations/2))]
.
MaxNumSplits
—
fitrensemble
searches among integers
log-scaled in the range
[1,max(2,NumObservations-1)]
.
NumVariablesToSample
—
fitrensemble
searches among integers in
the range [1,max(2,NumPredictors)]
.
Set nondefault parameters by passing a vector of
optimizableVariable
objects that have nondefault
values. For example,
load carsmall params = hyperparameters('fitrensemble',[Horsepower,Weight],MPG,'Tree'); params(4).Range = [1,20];
Pass params
as the value of
OptimizeHyperparameters
.
By default, iterative display appears at the command line, and
plots appear according to the number of hyperparameters in the optimization. For the
optimization and plots, the objective function is log(1 + cross-validation loss) for regression and the misclassification rate for classification. To control
the iterative display, set the Verbose
field of the
'HyperparameterOptimizationOptions'
name-value pair argument. To
control the plots, set the ShowPlots
field of the
'HyperparameterOptimizationOptions'
name-value pair argument.
For an example, see Optimize Regression Ensemble.
Example: 'OptimizeHyperparameters',{'Method','NumLearningCycles','LearnRate','MinLeafSize','MaxNumSplits'}
'HyperparameterOptimizationOptions'
— Options for optimizationOptions for optimization, specified as the comma-separated pair consisting of
'HyperparameterOptimizationOptions'
and a structure. This
argument modifies the effect of the OptimizeHyperparameters
name-value pair argument. All fields in the structure are optional.
Field Name | Values | Default |
---|---|---|
Optimizer |
| 'bayesopt' |
AcquisitionFunctionName |
Acquisition functions whose names include
| 'expected-improvement-per-second-plus' |
MaxObjectiveEvaluations | Maximum number of objective function evaluations. | 30 for 'bayesopt' or 'randomsearch' , and the entire grid for 'gridsearch' |
MaxTime | Time limit, specified as a positive real. The time limit is in seconds, as measured by | Inf |
NumGridDivisions | For 'gridsearch' , the number of values in each dimension. The value can be
a vector of positive integers giving the number of
values for each dimension, or a scalar that
applies to all dimensions. This field is ignored
for categorical variables. | 10 |
ShowPlots | Logical value indicating whether to show plots. If true , this field plots
the best objective function value against the
iteration number. If there are one or two
optimization parameters, and if
Optimizer is
'bayesopt' , then
ShowPlots also plots a model of
the objective function against the
parameters. | true |
SaveIntermediateResults | Logical value indicating whether to save results when Optimizer is
'bayesopt' . If
true , this field overwrites a
workspace variable named
'BayesoptResults' at each
iteration. The variable is a BayesianOptimization object. | false |
Verbose | Display to the command line.
For details, see the
| 1 |
UseParallel | Logical value indicating whether to run Bayesian optimization in parallel, which requires Parallel Computing Toolbox. Due to the nonreproducibility of parallel timing, parallel Bayesian optimization does not necessarily yield reproducible results. For details, see Parallel Bayesian Optimization. | false |
Repartition | Logical value indicating whether to repartition the cross-validation at every iteration. If
| false |
Use no more than one of the following three field names. | ||
CVPartition | A cvpartition object, as created by cvpartition . | 'Kfold',5 if you do not specify any cross-validation
field |
Holdout | A scalar in the range (0,1) representing the holdout fraction. | |
Kfold | An integer greater than 1. |
Example: 'HyperparameterOptimizationOptions',struct('MaxObjectiveEvaluations',60)
Data Types: struct
Mdl
— Trained regression ensemble modelRegressionBaggedEnsemble
model object | RegressionEnsemble
model object | RegressionPartitionedEnsemble
cross-validated model
objectTrained ensemble model, returned as one of the model objects in this table.
Model Object | Specify Any Cross-Validation Options? | Method
Setting | Resample
Setting |
---|---|---|---|
RegressionBaggedEnsemble | No | 'Bag' | 'on' |
RegressionEnsemble | No | 'LSBoost' | 'off' |
RegressionPartitionedEnsemble | Yes | 'LSBoost' or
'Bag' | 'off' or
'on' |
The name-value pair arguments that control cross-validation
are CrossVal
, Holdout
,
KFold
, Leaveout
, and
CVPartition
.
To reference properties of Mdl
, use dot notation. For
example, to access or display the cell vector of weak learner model objects
for an ensemble that has not been cross-validated, enter
Mdl.Trained
at the command line.
NumLearningCycles
can vary from a few dozen to a few
thousand. Usually, an ensemble with good predictive power requires from a few
hundred to a few thousand weak learners. However, you do not have to train an
ensemble for that many cycles at once. You can start by growing a few dozen
learners, inspect the ensemble performance and then, if necessary, train more
weak learners using resume
.
Ensemble performance depends on the ensemble setting and the setting of the weak learners. That is, if you specify weak learners with default parameters, then the ensemble can perform poorly. Therefore, like ensemble settings, it is good practice to adjust the parameters of the weak learners using templates, and to choose values that minimize generalization error.
If you specify to resample using Resample
, then it is
good practice to resample to entire data set. That is, use the default setting
of 1
for FResample
.
After training a model, you can generate C/C++ code that predicts responses for new data. Generating C/C++ code requires MATLAB Coder™. For details, see Introduction to Code Generation.
For details of ensemble aggregation algorithms, see Ensemble Algorithms.
If you specify 'Method','LSBoost'
, then the software grows
shallow decision trees by default. You can adjust tree depth by specifying the
MaxNumSplits
, MinLeafSize
, and
MinParentSize
name-value pair arguments using templateTree
.
For dual-core systems and above, fitrensemble
parallelizes
training using Intel Threading Building Blocks (TBB). For details on Intel TBB, see https://software.intel.com/en-us/intel-tbb.
[1] Breiman, L. “Bagging Predictors.” Machine Learning. Vol. 26, pp. 123–140, 1996.
[2] Breiman, L. “Random Forests.” Machine Learning. Vol. 45, pp. 5–32, 2001.
[3] Freund, Y. and R. E. Schapire. “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting.” J. of Computer and System Sciences, Vol. 55, pp. 119–139, 1997.
[4] Friedman, J. “Greedy function approximation: A gradient boosting machine.” Annals of Statistics, Vol. 29, No. 5, pp. 1189–1232, 2001.
[5] Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning section edition, Springer, New York, 2008.
fitrensemble
supports parallel training
using the 'Options'
name-value argument. Create options using statset
, such as options = statset('UseParallel',true)
.
Parallel ensemble training requires you to set the 'Method'
name-value
argument to 'Bag'
. Parallel training is available only for tree learners, the
default type for 'Bag'
.
To perform parallel hyperparameter optimization, use the
'HyperparameterOptimizationOptions', struct('UseParallel',true)
name-value argument in the call to this function.
For more information on parallel hyperparameter optimization, see Parallel Bayesian Optimization.
For general information about parallel computing, see Run MATLAB Functions with Automatic Parallel Support (Parallel Computing Toolbox).
predict
| RegressionBaggedEnsemble
| RegressionEnsemble
| RegressionPartitionedEnsemble
| templateTree
You have a modified version of this example. Do you want to open this example with your edits?
You clicked a link that corresponds to this MATLAB command:
Run the command by entering it in the MATLAB Command Window. Web browsers do not support MATLAB commands.
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
Select web siteYou can also select a web site from the following list:
Select the China site (in Chinese or English) for best site performance. Other MathWorks country sites are not optimized for visits from your location.