(Not Recommended) Create generalized linear regression model by stepwise regression
GeneralizedLinearModel.stepwise is not recommended. Use
mdl = GeneralizedLinearModel.stepwise(tbl,modelspec)
mdl = GeneralizedLinearModel.stepwise(X,y,modelspec)
mdl = GeneralizedLinearModel.stepwise(...,modelspec,Name,Value)
creates a generalized linear model of a table or dataset array
mdl = GeneralizedLinearModel.stepwise(
using stepwise regression to add or remove predictors.
the starting model for the stepwise procedure.
modelspec— Starting model
'constant'(default) | character vector or string scalar naming the model | t-by-(p + 1) terms matrix | character vector or string scalar formula in the form
'Y ~ terms'
Starting model, specified as one of the following:
Character vector or string scalar specifying the type of model.
|Model contains only a constant (intercept) term.|
|Model contains an intercept and linear term for each predictor.|
|Model contains an intercept, linear term for each predictor, and all products of pairs of distinct predictors (no squared terms).|
|Model contains an intercept term and linear and squared terms for each predictor.|
|Model contains an intercept term, linear and squared terms for each predictor, and all products of pairs of distinct predictors.|
|Model is a polynomial with all terms up to degree |
t-by-(p+1) matrix, namely terms matrix, specifying terms to include in model, where t is the number of terms and p is the number of predictor variables, and plus one is for the response variable.
Character vector or string scalar representing a formula in the form
termsare in Wilkinson Notation.
comma-separated pairs of
the argument name and
Value is the corresponding value.
Name must appear inside quotes. You can specify several name and value
pair arguments in any order as
'Criterion'— Criterion to add or remove terms
Criterion to add or remove terms, specified as the comma-separated pair consisting of
'Criterion' and one of the following:
'Deviance' — p-value for F or
chi-squared test of the change in the deviance by adding or removing the term.
F-test is for testing a single model. Chi-squared test is
for comparing two different models.
'sse' — p-value for an F-test of the
change in the sum of squared error by adding or removing the term.
'aic' — Change in the value of Akaike information criterion (AIC).
'bic' — Change in the value of Bayesian information criterion (BIC).
'rsquared' — Increase in the value of R2.
'adjrsquared' — Increase in the value of adjusted R2.
'PEnter'— Threshold for criterion to add term
Threshold for the criterion to add a term, specified as the comma-separated pair
'PEnter' and a scalar value, as described in this
|0.05||If the p-value of F-statistic
or chi-squared statistic is less than |
|0.05||If the SSE of the model is less than |
|0||If the change in the AIC of the model is less than
|0||If the change in the BIC of the model is less than
|0.1||If the increase in the R-squared value of the model is greater than
|0||If the increase in the adjusted R-squared value of the model is
greater than |
For more information, see the
Criterion name-value pair
'PRemove'— Threshold for criterion to remove term
Threshold for the criterion to remove a term, specified as the comma-separated pair
'PRemove' and a scalar value, as described in this
|0.10||If the p-value of F-statistic
or chi-squared statistic is greater than |
|0.10||If the p-value of the F statistic is greater than
|0.01||If the change in the AIC of the model is greater than
|0.01||If the change in the BIC of the model is greater than
|0.05||If the increase in the R-squared value of the model is less than
|-0.05||If the increase in the adjusted R-squared value of the model is less
At each step, the
GeneralizedLinearModel.stepwise function also checks whether a term
is redundant (linearly dependent) with other terms in the current model. When any term
is linearly dependent with other terms in the current model, the
GeneralizedLinearModel.stepwise function removes the redundant term, regardless of
the criterion value.
For more information, see the
Criterion name-value pair
Create response data using just three of 20 predictors, and create a generalized linear model stepwise to see if it uses just the correct predictors.
Create data with 20 predictors, and Poisson response using just three of the predictors, plus a constant.
rng default % for reproducibility X = randn(100,20); mu = exp(X(:,[5 10 15])*[.4;.2;.3] + 1); y = poissrnd(mu);
Fit a generalized linear model using the Poisson distribution.
mdl = stepwiseglm(X,y,... 'constant','upper','linear','Distribution','poisson')
1. Adding x5, Deviance = 134.439, Chi2Stat = 52.24814, PValue = 4.891229e-13 2. Adding x15, Deviance = 106.285, Chi2Stat = 28.15393, PValue = 1.1204e-07 3. Adding x10, Deviance = 95.0207, Chi2Stat = 11.2644, PValue = 0.000790094
mdl = Generalized linear regression model: log(y) ~ 1 + x5 + x10 + x15 Distribution = Poisson Estimated Coefficients: Estimate SE tStat pValue ________ ________ ______ __________ (Intercept) 1.0115 0.064275 15.737 8.4217e-56 x5 0.39508 0.066665 5.9263 3.0977e-09 x10 0.18863 0.05534 3.4085 0.0006532 x15 0.29295 0.053269 5.4995 3.8089e-08 100 observations, 96 error degrees of freedom Dispersion: 1 Chi^2-statistic vs. constant model: 91.7, p-value = 9.61e-20
A terms matrix
T is a
t-by-(p + 1) matrix specifying terms in a model,
where t is the number of terms, p is the number of
predictor variables, and +1 accounts for the response variable. The value of
T(i,j) is the exponent of variable
j in term
For example, suppose that an input includes three predictor variables
C and the response variable
Y in the order
Y. Each row of
represents one term:
[0 0 0 0] — Constant term or intercept
[0 1 0 0] —
A^0 * B^1 * C^0
[1 0 1 0] —
[2 0 0 0] —
[0 1 2 0] —
0 at the end of each term represents the response variable. In
general, a column vector of zeros in a terms matrix represents the position of the response
variable. If you have the predictor and response variables in a matrix and column vector,
then you must include
0 for the response variable in the last column of
A formula for model specification is a character vector or string scalar of
Y is the response name.
terms represents the predictor terms in a model using
'Y ~ A + B + C' specifies a three-variable
linear model with intercept.
'Y ~ A + B + C – 1' specifies a
three-variable linear model without intercept. Note that
formulas include a constant (intercept) term by default. To
exclude a constant term from the model, you must include
–1 in the formula.
Wilkinson notation describes the terms present in a model. The notation relates to the terms present in a model, not to the multipliers (coefficients) of those terms.
Wilkinson notation uses these symbols:
+ means include the next variable.
– means do not include the next variable.
: defines an interaction, which is a product of
* defines an interaction and all lower-order terms.
^ raises the predictor to a power, exactly as in
* repeated, so
^ includes lower-order
terms as well.
() groups terms.
This table shows typical examples of Wilkinson notation.
|Wilkinson Notation||Term in Standard Notation|
|Constant (intercept) term|
|Do not include |
Statistics and Machine
Learning Toolbox™ notation always includes a constant term unless you explicitly remove the term
For more details, see Wilkinson Notation.
The generalized linear model
mdl is a standard linear model
unless you specify otherwise with the
For other methods such as
devianceTest, or properties of
GeneralizedLinearModel object, see
Stepwise regression is a systematic method
for adding and removing terms from a linear or generalized linear
model based on their statistical significance in explaining the response
variable. The method begins with an initial model, specified using
and then compares the explanatory power of incrementally larger and
GeneralizedLinearModel.stepwise function uses forward and backward stepwise regression to
determine a final model. At each step, the function searches for terms to add to the model
or remove from the model based on the value of the
The default value of
'Criterion' for a linear regression model is
'sse'. In this case,
LinearModel use the
p-value of an F-statistic to test models with and
without a potential term at each step. If a term is not currently in the model, the null
hypothesis is that the term would have a zero coefficient if added to the model. If there is
sufficient evidence to reject the null hypothesis, the function adds the term to the model.
Conversely, if a term is currently in the model, the null hypothesis is that the term has a
zero coefficient. If there is insufficient evidence to reject the null hypothesis, the
function removes the term from the model.
Stepwise regression takes these steps when
Fit the initial model.
Examine a set of available terms not in the model. If any of the terms have p-values less than an entrance tolerance (that is, if it is unlikely a term would have a zero coefficient if added to the model), add the term with the smallest p-value and repeat this step; otherwise, go to step 3.
If any of the available terms in the model have p-values greater than an exit tolerance (that is, the hypothesis of a zero coefficient cannot be rejected), remove the term with the largest p-value and return to step 2; otherwise, end the process.
At any stage, the function will not add a higher-order term if the model does not also include
all lower-order terms that are subsets of the higher-order term. For example, the function
will not try to add the term
X1:X2^2 unless both
X2^2 are already in the model. Similarly, the function will not
remove lower-order terms that are subsets of higher-order terms that remain in the model.
For example, the function will not try to remove
X1:X2^2 remains in the model.
You can specify other criteria by using the
'Criterion' name-value pair
argument. For example, you can specify the change in the value of the Akaike information
criterion, Bayesian information criterion, R-squared, or adjusted R-squared as the criterion
to add or remove terms.
Depending on the terms included in the initial model, and the order in which the function adds and removes terms, the function might build different models from the same set of potential terms. The function terminates when no single step improves the model. However, a different initial model or a different sequence of steps does not guarantee a better fit. In this sense, stepwise models are locally optimal, but might not be globally optimal.
You can also create a stepwise generalized linear model using
 Collett, D. Modeling Binary Data. New York: Chapman & Hall, 2002.
 Dobson, A. J. An Introduction to Generalized Linear Models. New York: Chapman & Hall, 1990.
 McCullagh, P., and J. A. Nelder. Generalized Linear Models. New York: Chapman & Hall, 1990.