Basic Lifetime PD Model Validation

This example shows how to perform basic model validation on a lifetime probability of default (PD) model by viewing the fitted model, estimated coefficients, and p-values. For more information on model validation, see modelDiscrimination and modelCalibration.

Load Data

Load the portfolio data.

load RetailCreditPanelData.mat
data = join(data,dataMacro);
disp(head(data))

    ID    ScoreGroup    YOB    Default    Year     GDP     Market
    __    __________    ___    _______    ____    _____    ______

    1      Low Risk      1        0       1997     2.72      7.61
    1      Low Risk      2        0       1998     3.57     26.24
    1      Low Risk      3        0       1999     2.86      18.1
    1      Low Risk      4        0       2000     2.43      3.19
    1      Low Risk      5        0       2001     1.26    -10.51
    1      Low Risk      6        0       2002    -0.59    -22.95
    1      Low Risk      7        0       2003     0.63      2.78
    1      Low Risk      8        0       2004     1.85      9.48

Fit Model and Review Model Goodness of Fit

Create training and test datasets to perform a basic model validation.

nIDs = max(data.ID);
uniqueIDs = unique(data.ID);

rng('default'); % for reproducibility
c = cvpartition(nIDs,'HoldOut',0.4);

TrainIDInd = training(c);
TestIDInd = test(c);

TrainDataInd = ismember(data.ID,uniqueIDs(TrainIDInd));
TestDataInd = ismember(data.ID,uniqueIDs(TestIDInd));

Fit the model using fitLifetimePDModel for a Logistic, Probit, or Cox model.

ModelType = "probit";
pdModel = fitLifetimePDModel(data(TrainDataInd,:),ModelType,...
    'AgeVar','YOB',...
    'IDVar','ID',...
    'LoanVars','ScoreGroup',...
    'MacroVars',{'GDP','Market'},...
    'ResponseVar','Default');
disp(pdModel)

  Probit with properties:

            ModelID: "Probit"
        Description: ""
    UnderlyingModel: [1x1 classreg.regr.CompactGeneralizedLinearModel]
              IDVar: "ID"
             AgeVar: "YOB"
           LoanVars: "ScoreGroup"
          MacroVars: ["GDP"    "Market"]
        ResponseVar: "Default"
         WeightsVar: ""
       TimeInterval: 1

Display the PD model and review the fit statistics, such as the p-values.

disp(pdModel.UnderlyingModel)

Compact generalized linear regression model:
    probit(Default) ~ 1 + ScoreGroup + YOB + GDP + Market
    Distribution = Binomial

Estimated Coefficients:
                               Estimate        SE         tStat       pValue   
                              __________    _________    _______    ___________

    (Intercept)                  -1.6267      0.03811    -42.685              0
    ScoreGroup_Medium Risk      -0.26542      0.01419    -18.704     4.5503e-78
    ScoreGroup_Low Risk         -0.46794     0.016364    -28.595     7.775e-180
    YOB                         -0.11421    0.0049724    -22.969    9.6208e-117
    GDP                        -0.041537     0.014807    -2.8052      0.0050291
    Market                    -0.0029609    0.0010618    -2.7885      0.0052954


388097 observations, 388091 error degrees of freedom
Dispersion: 1
Chi^2-statistic vs. constant model: 1.85e+03, p-value = 0

pdModel.UnderlyingModel.Coefficients

ans=6×4 table
                               Estimate        SE         tStat       pValue   
                              __________    _________    _______    ___________

    (Intercept)                  -1.6267      0.03811    -42.685              0
    ScoreGroup_Medium Risk      -0.26542      0.01419    -18.704     4.5503e-78
    ScoreGroup_Low Risk         -0.46794     0.016364    -28.595     7.775e-180
    YOB                         -0.11421    0.0049724    -22.969    9.6208e-117
    GDP                        -0.041537     0.014807    -2.8052      0.0050291
    Market                    -0.0029609    0.0010618    -2.7885      0.0052954