loss

Regression loss for generalized additive model (GAM)

Since R2021a

Syntax

L = loss(Mdl,Tbl,ResponseVarName)

L = loss(Mdl,Tbl,Y)

L = loss(Mdl,X,Y)

L = loss(___,Name,Value)

Description

L = loss(Mdl,Tbl,ResponseVarName) returns the regression loss (L), a scalar representing how well the generalized additive model Mdl predicts the predictor data in Tbl compared to the true response values in Tbl.ResponseVarName.

The interpretation of L depends on the loss function ('LossFun') and weighting scheme ('Weights'). In general, better models yield smaller loss values. The default 'LossFun' value is 'mse' (mean squared error).

L = loss(Mdl,Tbl,Y) uses the predictor data in table Tbl and the true response values in Y.

L = loss(Mdl,X,Y) uses the predictor data in matrix X and the true response values in Y.

L = loss(___,Name,Value) specifies options using one or more name-value arguments in addition to any of the input argument combinations in previous syntaxes. For example, you can specify the loss function and the observation weights.

Examples

Determine Test Sample Regression Loss

Determine the test sample regression loss (mean squared error) of a generalized additive model. When you compare the same type of loss among many models, a lower loss indicates a better predictive model.

Load the patients data set.

load patients

Create a table that contains the predictor variables (Age, Diastolic, Smoker, Weight, Gender, SelfAssessedHealthStatus) and the response variable (Systolic).

tbl = table(Age,Diastolic,Smoker,Weight,Gender,SelfAssessedHealthStatus,Systolic);

Randomly partition observations into a training set and a test set. Specify a 10% holdout sample for testing.

rng('default') % For reproducibility
cv = cvpartition(size(tbl,1),'HoldOut',0.10);

Extract the training and test indices.

trainInds = training(cv);
testInds = test(cv);

Train a univariate GAM that contains the linear terms for the predictors in tbl.

Mdl = fitrgam(tbl(trainInds,:),"Systolic");

Determine how well the algorithm generalizes by estimating the test sample regression loss. By default, the loss function of RegressionGAM estimates the mean squared error.

L = loss(Mdl,tbl(testInds,:))

L = 
35.7540

Compare GAMs by Examining Regression Loss

Train a generalized additive model (GAM) that contains both linear and interaction terms for predictors, and estimate the regression loss (mean squared error, MSE) with and without interaction terms for the training data and test data. Specify whether to include interaction terms when estimating the regression loss.

Load the carbig data set, which contains measurements of cars made in the 1970s and early 1980s.

load carbig

Specify Acceleration, Displacement, Horsepower, and Weight as the predictor variables (X) and MPG as the response variable (Y).

X = [Acceleration,Displacement,Horsepower,Weight];
Y = MPG;

Partition the data set into two sets: one containing training data, and the other containing new, unobserved test data. Reserve 10 observations for the new test data set.

rng('default') % For reproducibility
n = size(X,1);
newInds = randsample(n,10);
inds = ~ismember(1:n,newInds);
XNew = X(newInds,:);
YNew = Y(newInds);

Train a generalized additive model that contains all the available linear and interaction terms in X.

Mdl = fitrgam(X(inds,:),Y(inds),'Interactions','all');

Mdl is a RegressionGAM model object.

Compute the resubstitution MSEs (that is, the in-sample MSEs) both with and without interaction terms in Mdl. To exclude interaction terms, specify 'IncludeInteractions',false.

resubl = resubLoss(Mdl)

resubl = 
0.0292

resubl_nointeraction = resubLoss(Mdl,'IncludeInteractions',false)

resubl_nointeraction = 
4.7330

Compute the regression MSEs both with and without interaction terms for the test data set. Use a memory-efficient model object for the computation.

CMdl = compact(Mdl);

CMdl is a CompactRegressionGAM model object.

l = loss(CMdl,XNew,YNew)

l = 
12.8604

l_nointeraction = loss(CMdl,XNew,YNew,'IncludeInteractions',false)

l_nointeraction = 
15.6741

Including interaction terms achieves a smaller error for the training data set and test data set.

Input Arguments

Mdl — Generalized additive model

RegressionGAM model object | CompactRegressionGAM model object

Generalized additive model, specified as a RegressionGAM or CompactRegressionGAM model object.

• If you trained Mdl using sample data contained in a table, then the input data for loss must also be in a table (Tbl).

• If you trained Mdl using sample data contained in a matrix, then the input data for loss must also be in a matrix (X).

Tbl — Sample data

table

Sample data, specified as a table. Each row of Tbl corresponds to one observation, and each column corresponds to one predictor variable. Multicolumn variables and cell arrays other than cell arrays of character vectors are not allowed.

Tbl must contain all of the predictors used to train Mdl. Optionally, Tbl can contain a column for the response variable and a column for the observation weights.

• The response variable must be a numeric vector. If the response variable in Tbl has the same name as the response variable used to train Mdl, then you do not need to specify ResponseVarName.

• The weight values must be a numeric vector. You must specify the observation weights in Tbl by using 'Weights'.

If you trained Mdl using sample data contained in a table, then the input data for loss must also be in a table.

Data Types: table

ResponseVarName — Response variable name

name of variable in Tbl

Response variable name, specified as a character vector or string scalar containing the name of the response variable in Tbl. For example, if the response variable Y is stored in Tbl.Y, then specify it as 'Y'.

Data Types: char | string

Y — Response data

numeric column vector

Response data, specified as a numeric column vector. Each entry in Y is the response to the data in the corresponding row of X or Tbl.

Data Types: single | double

X — Predictor data

numeric matrix

Predictor data, specified as a numeric matrix. Each row of X corresponds to one observation, and each column corresponds to one predictor variable.

If you trained Mdl using sample data contained in a matrix, then the input data for loss must also be in a matrix.

Data Types: single | double

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Before R2021a, use commas to separate each name and value, and enclose Name in quotes.

Example: 'IncludeInteractions',false,'Weights',w specifies to exclude interaction terms from the model and to use the observation weights w.

IncludeInteractions — Flag to include interaction terms

true | false

Flag to include interaction terms of the model, specified as true or false.

The default 'IncludeInteractions' value is true if Mdl contains interaction terms. The value must be false if the model does not contain interaction terms.

Example: 'IncludeInteractions',false

Data Types: logical

LossFun — Loss function

'mse' (default) | function handle

Loss function, specified as 'mse' or a function handle.

• 'mse' — Weighted mean squared error.

• Function handle — To specify a custom loss function, use a function handle. The function must have this form:

  lossval = lossfun(Y,YFit,W)

  • The output argument lossval is a floating-point scalar.

  • You specify the function name (lossfun).

  • Y is a length n numeric vector of observed responses, where n is the number of observations in Tbl or X.

  • YFit is a length n numeric vector of corresponding predicted responses.

  • W is an n-by-1 numeric vector of observation weights.

Example: 'LossFun',@lossfun

Data Types: char | string | function_handle

Weights — Observation weights

ones(size(X,1),1) (default) | vector of scalar values | name of variable in Tbl

Observation weights, specified as a vector of scalar values or the name of a variable in Tbl. The software weights the observations in each row of X or Tbl with the corresponding value in Weights. The size of Weights must equal the number of rows in X or Tbl.

If you specify the input data as a table Tbl, then Weights can be the name of a variable in Tbl that contains a numeric vector. In this case, you must specify Weights as a character vector or string scalar. For example, if weights vector W is stored as Tbl.W, then specify it as 'W'.

loss normalizes the values of Weights to sum to 1.

Data Types: single | double | char | string

More About

Weighted Mean Squared Error

The weighted mean squared error measures the predictive inaccuracy of regression models. When you compare the same type of loss among many models, a lower error indicates a better predictive model.

The weighted mean squared error is calculated as follows:

$$\text{mse}=\frac{{\displaystyle \sum _{j=1}^n{w}_j{\left(f\left(x_j\right)-y_j\right)}^2}}{{\displaystyle \sum _{j=1}^n{w}_j}},$$

where:

• n is the number of rows of data.

• x_j is the jth row of data.

• y_j is the true response to x_j.

• f(x_j) is the response prediction of the model Mdl to x_j.

• w is the vector of observation weights.

Version History

Introduced in R2021a