unconditionalDE

Unconditional Du-Escanciano (DE) expected shortfall (ES) backtest

Syntax

``TestResults = unconditionalDE(ebtde)``
``[TestResults,SimTestStatistic] = unconditionalDE(___,Name,Value)``

Description

example

````TestResults = unconditionalDE(ebtde)` runs the unconditional Du-Escanciano (DE) expected shortfall (ES) backtest [1] . The unconditional test supports critical values by large-scale approximation and by finite-sample simulation.```

example

````[TestResults,SimTestStatistic] = unconditionalDE(___,Name,Value)` specifies options using one or more name-value pair arguments in addition to the input argument in the previous syntax.```

Examples

collapse all

Create an `esbacktestbyde` object for a t model with 10 degrees of freedom, and then run an `unconditionalDE` test.

```load ESBacktestDistributionData.mat rng('default'); % For reproducibility ebtde = esbacktestbyde(Returns,"t",... 'DegreesOfFreedom',T10DoF,... 'Location',T10Location,... 'Scale',T10Scale,... 'PortfolioID',"S&P",... 'VaRID',["t(10) 95%","t(10) 97.5%","t(10) 99%"],... 'VaRLevel',VaRLevel); unconditionalDE(ebtde)```
```ans=3×14 table PortfolioID VaRID VaRLevel UnconditionalDE PValue TestStatistic LowerCI UpperCI Observations CriticalValueMethod MeanLS StdLS Scenarios TestLevel ___________ _____________ ________ _______________ ________ _____________ _________ _________ ____________ ___________________ ______ _________ _________ _________ "S&P" "t(10) 95%" 0.95 accept 0.181 0.028821 0.019401 0.030599 1966 "large-sample" 0.025 0.0028565 NaN 0.95 "S&P" "t(10) 97.5%" 0.975 accept 0.086278 0.015998 0.0085028 0.016497 1966 "large-sample" 0.0125 0.0020394 NaN 0.95 "S&P" "t(10) 99%" 0.99 reject 0.016871 0.0080997 0.0024575 0.0075425 1966 "large-sample" 0.005 0.0012972 NaN 0.95 ```

Input Arguments

collapse all

`esbacktestbyde` (`ebtde`) object, which contains a copy of the data (the `PortfolioData`, `VarData`, and `ESData` properties) and all combinations of portfolio ID, VaR ID, and VaR levels to be tested. For more information on creating an `esbacktestbyde` object, see `esbacktestbyde`.

Name-Value Pair Arguments

Specify optional comma-separated pairs of `Name,Value` arguments. `Name` is the argument name and `Value` is the corresponding value. `Name` must appear inside quotes. You can specify several name and value pair arguments in any order as `Name1,Value1,...,NameN,ValueN`.

Example: ```TestResults = unconditionalDE(ebtde,'CriticalValueMethod','large-sample','TestLevel',0.99)```

Method to compute critical values, confidence intervals, and p-values, specified as the comma-separated pair consisting of `'CriticalValueMethod'` and a character vector or string with a value of `'large-sample'` or `'simulation'`.

Data Types: `char` | `string`

Test confidence level, specified as the comma-separated pair consisting of `'TestLevel'` and a numeric value between `0` and `1`.

Data Types: `double`

Output Arguments

collapse all

Results, returned as a table where the rows correspond to all combinations of portfolio ID, VaR ID, and VaR levels to be tested. The columns correspond to the following:

• `'PortfolioID'` — Portfolio ID for the given data

• `'VaRID'` — VaR ID for each of the VaR levels

• `'VaRLevel'` — VaR level

• `'UnconditionalDE'`— Categorical array with the categories `'accept'` and `'reject'`, which indicate the result of the unconditional DE test

• `'PValue'`P-value of the unconditional DE test

• `'TestStatistic'`— Unconditional DE test statistic

• `'LowerCI'`— Confidence-interval lower limit for the unconditional DE test statistic

• `'UpperCI'`— Confidence-interval upper limit for the unconditional DE test statistic

• `'Observations'`— Number of observations

• `'CriticalValueMethod'`— Method for computing confidence intervals and p-values

• `'MeanLS'`— Mean of the `large-sample` normal distribution; if `CriticalValueMethod` is `'simulation'`, `'MeanLS'` is reported as `NaN`

• `'StdLS'`— Standard deviation of the `large-sample` normal distribution; if `CriticalValueMethod` is `'simulation'`, `'StdLS'` is reported as `NaN`

• `'Scenarios'`— Number of scenarios simulated to get the p-values; if `CriticalValueMethod` is `'large-sample'`, the number of scenarios is reported as `NaN`

• `'TestLevel'`— Test confidence level

Note

For the test results, the terms `accept` and `reject` are used for convenience. Technically, a test does not accept a model; rather, a test fails to reject it.

Simulated values of the test statistics, returned as a `NumVaRs`-by-`NumScenarios` numeric array.

collapse all

Unconditional DE Test

The unconditional DE test is a two-sided test to check if the test statistic is close to an expected value of ɑ/2, where ɑ = 1- VaRLevel.

The test statistic for the unconditional DE test is

`${U}_{ES}=\frac{1}{N}{\sum }_{t=1}^{N}{H}_{t}$`

where

• Ht is the cumulative failures or violations process; Ht = (α - Ut)I(Ut < α) / α, where I(x) is the indicator function.

• Ut are the ranks or mapped returns Ut = Pt(Xt), where Pt(Xt) = P(Xt | θt) is the cumulative distribution of the portfolio outcomes or returns Xt over a given test window t = 1,...N and θt are the parameters of the distribution. For simplicity, the subindex t is both the return and the parameters, understanding that the parameters are those used on date t, even though those parameters are estimated on the previous date t-1, or even prior to that.

Significance of the Test

The test statistic UES is a random variable and a function of random return sequences:

`${U}_{ES}={U}_{ES}\left({X}_{1},...,{X}_{N}\right).$`

For returns observed in the test window 1,…,N, the test statistic attains a fixed value:

`${U}_{ES}^{obs}={U}_{ES}\left({X}_{1}^{obs},...,{X}_{N}^{obs}\right).$`

In general, for unknown returns that follow a distribution of Pt, the value of UES is uncertain and follows a cumulative distribution function:

`${P}_{U}\left(x\right)=P\left[{U}_{ES}\le x\right].$`

This distribution function computes a confidence interval and a p-value. To determine the distribution PU, the `esbacktestbyde` class supports the large-sample approximation and simulation methods. You can specify one of these methods by using the optional name-value pair argument `CriticalValueMethod`.

For the large-sample approximation method, the distribution PU is derived from an asymptotic analysis. If the number of observations N is large, the test statistic UES is distributed as

`${U}_{ES}\underset{dist}{\to }N\left(\frac{\alpha }{2},\frac{\alpha \left(1/3-\alpha /4\right)}{N}\right)={P}_{U}$`

where N(μ,σ2) is the normal distribution with mean μ and variance σ2.

Because the test statistic cannot be smaller than 0 or greater than 1, the analytical confidence interval limits are clipped to the interval [0,1]. Therefore, if the analytical value is negative, the test statistic is reset to 0, and if the analytical value is greater than 1, it is reset to 1.

The p-value is

`${p}_{value}=2\ast \mathrm{min}\left\{{P}_{U}\left({U}_{ES}^{obs}\right),1-{P}_{U}\left({U}_{ES}^{obs}\right)\right\}.$`

The test rejects if pvalue < αtest.

For the simulation method, the distribution PUis estimated as follows

1. Simulate M scenarios of returns as

2. Compute the corresponding test statistic as

3. Define PU as the empirical distribution of the simulated test statistic values as

`${P}_{U}=P\left[{U}_{ES}\le x\right]=\frac{1}{M}I\left({U}_{ES}^{s}\le x\right),$`

where I(.) is the indicator function.

In practice, simulating ranks is more efficient than simulating returns and then transforming the returns into ranks. For more information, see `simulate`.

For the empirical distribution, the value of 1-PU(x) can differ from the value of P[UESx] because the distribution may have nontrivial jumps (simulated tied values). Use the latter probability for the estimation of confidence levels and p-values.

If ɑtest = 1 - test confidence level, then the confidence intervals levels CIlower and CIupper are the values that satisfy equations:

`$\begin{array}{l}{P}_{U}\left(C{I}_{lower}\right)=P\left[C{I}_{lower}\le {U}_{ES}\right]=\frac{{\alpha }_{test}}{2},\\ P\left[{U}_{ES}\ge C{I}_{upper}\right]=\frac{{\alpha }_{test}}{2}.\end{array}$`

The reported confidence interval limits CIlower and CIupper are simulated test statistic values UsES that approximately solve the preceding equations.

The p-value is determined as

`${p}_{value}=2\ast \mathrm{min}\left\{P\left[{U}_{ES}\le {U}_{ES}^{obs}\right],P\left[{U}_{ES}\ge {U}_{ES}^{obs}\right]\right\}.$`

The test rejects if pvalue < αtest.

References

[1] Du, Z., and J. C. Escanciano. "Backtesting Expected Shortfall: Accounting for Tail Risk." Management Science. Vol. 63, Issue 4, April 2017.

[2] Basel Committee on Banking Supervision. "Minimum Capital Requirements for Market Risk". January 2016 (https://www.bis.org/bcbs/publ/d352.pdf).