Regression Tree Ensembles

Random forests, boosted and bagged regression trees

A regression tree ensemble is a predictive model composed of a weighted combination of multiple regression trees. In general, combining multiple regression trees increases predictive performance. To boost regression trees using LSBoost, use fitrensemble. To bag regression trees or to grow a random forest, use fitrensemble or TreeBagger. To implement quantile regression using a bag of regression trees, use TreeBagger.

For classification ensembles, such as boosted or bagged classification trees, random subspace ensembles, or error-correcting output codes (ECOC) models for multiclass classification, see Classification Ensembles.

Apps

Regression Learner

Train regression models to predict data using supervised machine learning

Blocks

RegressionEnsemble Predict

Predict responses using ensemble of decision trees for regression (Since R2021a)

Functions

expand all

Boosted Ensemble of Trees

Create Regression Ensemble

`fitrensemble`	Fit ensemble of learners for regression
`compact`	Reduce size of machine learning model
`fitensemble`	Fit ensemble of learners for classification and regression

Modify Regression Ensemble

`regularize`	Find optimal weights for learners in regression ensemble
`removeLearners`	Remove members of compact regression ensemble
`resume`	Resume training of regression ensemble model
`shrink`	Prune regression ensemble

Cross-Validate Regression Ensemble

`cvshrink`	Cross-validate pruning and regularization of regression ensemble
`kfoldLoss`	Loss for cross-validated partitioned regression model
`kfoldPredict`	Predict responses for observations in cross-validated regression model
`kfoldfun`	Cross-validate function for regression

Measure Performance

`loss`	Regression error for regression ensemble model
`resubLoss`	Resubstitution loss for regression ensemble model

Classify Observations

`predict`	Predict responses using regression ensemble model
`resubPredict`	Predict response of regression ensemble by resubstitution

Gather Properties of Regression Ensemble

gather Gather properties of Statistics and Machine Learning Toolbox object from GPU

Bagged Ensemble of Trees

`fitrensemble`	Fit ensemble of learners for regression
`TreeBagger`	Ensemble of bagged decision trees
`predict`	Predict responses using ensemble of bagged decision trees
`oobPredict`	Ensemble predictions for out-of-bag observations
`quantilePredict`	Predict response quantile using bag of regression trees
`oobQuantilePredict`	Quantile predictions for out-of-bag observations from bag of regression trees

Improve Regression

crossval Cross-validate machine learning model

Interpret Regression

`lime`	Local interpretable model-agnostic explanations (LIME)
`partialDependence`	Compute partial dependence
`permutationImportance`	Predictor importance by permutation (Since R2024a)
`plotPartialDependence`	Create partial dependence plot (PDP) and individual conditional expectation (ICE) plots
`predictorImportance`	Estimates of predictor importance for regression ensemble of decision trees
`shapley`	Shapley values (Since R2021a)

Multiresponse Regression with Regression Chains

`fitrchains`	Multiresponse regression with regression chains (Since R2024b)
`compact`	Reduce size of multiresponse regression model (Since R2024b)
`loss`	Loss for multiresponse regression model (Since R2024b)
`predict`	Predict responses using multiresponse regression model (Since R2024b)

Objects

expand all

Boosted Ensemble of Trees

`RegressionEnsemble`	Ensemble regression
`CompactRegressionEnsemble`	Compact regression ensemble
`RegressionPartitionedEnsemble`	Cross-validated regression ensemble

Bagged Ensemble of Trees

`TreeBagger`	Ensemble of bagged decision trees
`CompactTreeBagger`	Compact ensemble of bagged decision trees
`RegressionBaggedEnsemble`	Regression ensemble grown by resampling

Multiresponse Regression with Regression Chains

`RegressionChainEnsemble`	Multiresponse regression model (Since R2024b)
`CompactRegressionChainEnsemble`	Compact multiresponse regression model (Since R2024b)

Topics

Ensemble Algorithms
Learn about different algorithms for ensemble learning.
Framework for Ensemble Learning
Obtain highly accurate predictions by using many weak learners.
Train Regression Ensemble
Train a simple regression ensemble.
Test Ensemble Quality
Learn methods to evaluate the predictive quality of an ensemble.
Select Predictors for Random Forests
Select split-predictors for random forests using interaction test algorithm.
Ensemble Regularization
Automatically choose fewer weak learners for an ensemble in a way that does not diminish predictive performance.
Bootstrap Aggregation (Bagging) of Regression Trees Using TreeBagger
Create a TreeBagger ensemble for regression.
Use Parallel Processing for Regression TreeBagger Workflow
Speed up computation by running TreeBagger in parallel.
Working with Quantile Regression Models
Estimate prediction intervals and create models that are robust to outliers by using quantile regression models.
Detect Outliers Using Quantile Regression
Detect outliers in data using quantile random forest.
Conditional Quantile Estimation Using Kernel Smoothing
Estimate conditional quantiles of a response given predictor data using quantile random forest and by estimating the conditional distribution function of the response using kernel smoothing.
Tune Random Forest Using Quantile Error and Bayesian Optimization
Tune quantile random forest using Bayesian optimization.
Predict Responses Using RegressionEnsemble Predict Block
Train a regression ensemble model with optimal hyperparameters, and then use the RegressionEnsemble Predict block for response prediction.
Manually Perform Time Series Forecasting Using Ensembles of Boosted Regression Trees
Manually perform single-step and multiple-step time series forecasting with ensembles of boosted regression trees.