## MMSE Forecasting Regression Models with ARIMA Errors

### What Are MMSE Forecasts?

An objective of time series analysis is generating forecasts for responses over a future time horizon. That is, you can generate predictions for *y*_{T + 1},*y*_{T + 2},...,*y*_{T + h} given the following:

An observed series

*y*_{1},*y*_{2},...,*y*_{T}A forecast horizon

*h*Nonstochastic predictors

*x*_{1},*x*_{2},...,*x*,...,_{T}*x*_{T + h}, where*x*is an_{k}*r*-vector containing the measurements of*r*predictors observed at time*k*A regression model with ARIMA errors

$$\begin{array}{l}{y}_{t}=c+{X}_{t}\beta +{u}_{t}\\ {\rm H}(L){u}_{t}={\rm N}(L){\epsilon}_{t},\end{array}$$

where H(

*L*) and N(*L*) are compound autoregressive and moving average lag operator polynomials (possibly containing integration), respectively.

Let $${\widehat{y}}_{t+1}$$ denote a forecast for the process at time *t* + 1, conditional on the history of the process up to time *t* (*H _{t}*), and assume that the predictors are fixed. The minimum mean square error (MMSE) forecast is the forecast $${\widehat{y}}_{t+1}$$ that minimizes expected square loss,

$$E{\left({y}_{t+1}-{\widehat{y}}_{t+1}|{H}_{t}\right)}^{2}.$$

Minimizing this loss function yields the MMSE forecast,

$${\widehat{y}}_{t+1}=E({y}_{t+1}|{H}_{t}).$$

### How forecast Generates MMSE Forecasts

`forecast`

generates MMSE forecasts recursively. When you call `forecast`

, you must specify a `regARIMA`

model (`Mdl`

) and the forecast horizon. You can also specify presample observations (`Y0`

), predictors (`X0`

), innovations (`E0`

), and conditional disturbances (`U0`

) using name-value pair arguments.

To begin forecasting *y _{t}* starting at time

*T*+ 1, use the last few observations of

*y*and

_{t}*X*as presample responses and predictors to initialize the forecast. Alternatively, you can specify presample unconditional disturbances or innovations.

_{t}However, when you specify presample data:

If you provide presample predictor data (

`X0`

), then you must also provide predictor forecasts (`XF`

). It is best practice to set`X0`

to the same predictor matrix that estimates the parameters. If you do not provide presample and future predictors, then`forecast`

ignores the regression component in the model.If the error process in

`Mdl`

contains a seasonal or nonseasonal autoregressive component, or seasonal or nonseasonal integration, then`forecast`

requires a minimum of*P*presample unconditional disturbances to initialize the forecast. The property`P`

of`Mdl`

stores*P*.If the error process in

`Mdl`

contains a seasonal or nonseasonal moving average component, then`forecast`

requires a minimum of*Q*presample innovations to initialize the forecast. The property`Q`

of`Mdl`

stores*Q*.If you provide a sufficient amount of presample unconditional disturbances, then

`forecast`

ignores`Y0`

and`X0`

. If you also do not provide`E0`

, but provide enough presample unconditional disturbances, then`forecast`

infers the required amount of presample innovations from the ARIMA error model and`U0`

.If you provide a sufficient amount of presample responses and predictors (and do not provide

`U0`

), then`forecast`

uses the regression model to infer the presample unconditional disturbances.If you do not provide presample observations, then

`forecast`

sets the required amount of presample unconditional disturbances and innovations to 0.If you provide an insufficient amount of presample observations, then

`forecast`

returns an error.

Consider generating forecasts from a regression model with ARMA(3,2) errors:

$$\begin{array}{c}{y}_{t}=c+{X}_{t}\beta +{u}_{t}\\ \left(1-{a}_{1}L-{a}_{2}{L}^{2}-{a}_{3}{L}^{3}\right){u}_{t}=\left(1+{b}_{1}L+{b}_{2}{L}^{2}\right){\epsilon}_{t}\\ \text{or}\\ a(L){u}_{t}=b(L){\epsilon}_{t},\end{array}$$

where *a*(*L*) and *B*(*L*) are lag operator polynomials. The largest AR lag is 3, the largest MA lag is 2. This model does not contain any seasonal lags nor integration. Therefore, *P* = 3 and *Q* = 2. To forecast this model, you need three presample responses and predictors, or three presample unconditional disturbances, and two presample innovations.

Given presample unconditional disturbances $$\left({u}_{T-2},{u}_{T-1},{u}_{T}\right),$$ presample innovations $$\left({\epsilon}_{T-1},{\epsilon}_{T}\right),$$ and future predictors $$\left({X}_{T+1},{X}_{T+2},\mathrm{...}\right),$$ you can forecast the model as follows:

$$\begin{array}{l}{\widehat{u}}_{T+1}={a}_{1}{u}_{T}+{a}_{2}{u}_{T-1}+{a}_{3}{u}_{T-2}+{b}_{1}{\epsilon}_{T}+{b}_{2}{\epsilon}_{T-1}\\ {\widehat{y}}_{T+1}=c+{X}_{T+1}\beta +{\widehat{u}}_{T+1}.\end{array}$$

$$\begin{array}{l}{\widehat{u}}_{T+2}={a}_{1}{\widehat{u}}_{T+1}+{a}_{2}{u}_{T}+{a}_{3}{u}_{T-1}+{b}_{2}{\epsilon}_{T}\\ {\widehat{y}}_{T+2}=c+{X}_{T+2}\beta +{\widehat{u}}_{T+2}.\end{array}$$

$$\begin{array}{l}{\widehat{u}}_{T+3}={a}_{1}{\widehat{u}}_{T+2}+{a}_{2}{\widehat{u}}_{T+1}+{a}_{3}{u}_{T}\\ {\widehat{y}}_{T+3}=c+{X}_{T+3}\beta +{\widehat{u}}_{T+3}.\end{array}$$

...

Note that:

Future innovations take on their unconditional mean, 0.

For stationary error processes, such as this one:

The forecasted unconditional disturbances converge to their unconditional mean,

$$E({u}_{t})=\frac{b(L)}{a(L)}E({\epsilon}_{t})=0.$$

*c*+*X*governs the long-term behavior of the forecasted responses._{t}β

### Forecast Error

The forecast error for an *s*-step ahead forecast of a regression model with ARIMA errors is

$$\begin{array}{c}\text{MSE}=E{\left({y}_{T+s}-{\widehat{y}}_{T+s}|{H}_{T+s-1}\right)}^{2}\\ =E{\left(c+{X}_{T+s}\beta +{u}_{T+s}-c-{X}_{t+s}\beta -{\widehat{u}}_{T+s}|{H}_{T+s-1}\right)}^{2}\\ =E{\left({u}_{T+s}-{\widehat{u}}_{T+s}|{H}_{T+s-1}\right)}^{2}\\ =\frac{{\rm N}(L)}{{\rm H}(L)}E\left({\epsilon}_{t}{}^{2}|{H}_{T+s-1}\right)\\ =\psi (L){\sigma}^{2},\end{array}$$

where the dividend *ψ*(*L*) is an infinite lag operator polynomial, and *σ*^{2} is the innovation variance.

If the error process is stationary, then the coefficients of *ψ*(*L*) are absolutely summable. Therefore, the MSE (mean square error) converges to the unconditional variance of the process [1].

If the error process is not stationary, then the MSE grows with increasing *s*.

## References

[1] Box, G. E. P., G. M. Jenkins, and G. C. Reinsel. *Time Series Analysis: Forecasting and Control*. 3rd ed. Englewood Cliffs, NJ: Prentice Hall, 1994.

## See Also

## Related Examples

- Verify Predictive Ability Robustness of a regARIMA Model
- Forecast a Regression Model with Multiplicative Seasonal ARIMA Errors
- Forecast a Regression Model with ARIMA Errors