Anda di halaman 1dari 34

Lesson 9: Autoregressive-Moving

Average (ARMA) models

Umberto Triacca
Dipartimento di Ingegneria e Scienze dellInformazione e Matematica
Universita dellAquila,
umberto.triacca@ec.univaq.it

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Introduction

We have seen that in the class of stationary, zero mean,


Gaussian processes the probabilistic structure of a stochastic
process is completly characterized by the autocovariance
function.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autocovariance function

Stationary, zero mean, Gaussian process

'$

DGP x (k)
&%
7










 ? 
x1 , ..., xT
 
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Introduction

However, in general, to know the autocovariance function


means to know a sequence composed by an infinite number of
elements.

We have to estimate a infinite number of parameters

x (0), x (1), x (2), ...,

from observed data.

This mission is impossible

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Introduction

We introduce a very important class of stochastic processes,


which autocovariance functions depend on a finite number of
unknown parameters:

the class of the AutoregRessive Moving Average (ARMA)


processes.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models
Definition. The process {xt ; t Z} is an autoregressive
moving average process of order (p, q), denoted with

xt ARMA(p, q),

if

xt 1 xt1 ... p xtp = ut + 1 ut1 + ... + q utq t Z,

where ut WN(0, u2 ), and 1 , ..., p , 1 , ..., q are p + q


constants and the polynomials

(z) = 1 1 z ... p z p

and
(z) = 1 + 1 z... + q z q
have no common factors.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models

For q = 0 the process reduces to an autoregressive process of


order p, denoted with xt AR(p),

xt 1 xt1 ... p xtp = ut t Z,

For p = 0 to a moving average process of order q, denoted


with xt MA(q)

xt = ut + 1 ut1 + ... + q utq t Z,

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


An example of Autoregressive-Moving Average
(ARMA) process

The process {xt ; t Z} defined by

xt = 0.3xt1 + ut + 0.7ut1 t Z,

where ut WN(0, u2 ), is an ARMA(1,1) process.


Here
(z) = 1 0.3z
and
(z) = 1 + 0.7z.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


An example of Autoregressive-Moving Average
(ARMA) process

A realizzation of the ARMA(1,1) process


xt = 0.3xt1 + ut + 0.7ut1 is presented in the following figure.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


An example of Autoregressive (AR) process

The process {xt ; t Z} defined by

xt = 0.7xt1 0.5xt1 + ut t Z,

where ut WN(0, u2 ), is an AR(2) process.


Here
(z) = 1 0.7z + 0.5z 2

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


An example of Autoregressive (AR) process

A realizzation of the AR(2) process


xt = 0.7xt1 0.5xt2 + ut is presented in the following figure.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


An example of Moving Average (MA) process

The process {xt ; t Z} defined by

xt = ut + 0.7ut1 t Z,

where ut WN(0, u2 ), is an MA(1) process.


Here
(z) = 1 + 0.7z

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


An example of Moving Average (MA) process

A realizzation of the MA(1) process xt = ut + 0.7ut1 is


presented in the following figure.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


An example of over-parameterization

Consider the process {xt ; t Z} defined by

xt = xt1 0.21xt2 + ut 0.7ut1 t Z,

where ut WN(0, u2 ).

This process looks like an ARMA(2,1) process but it is not an


ARMA(2,1) process.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


An example of over-parameterization
Here

(z) = 1 z + 0.21z 2 = (1 0.7z)(1 0.3z)

and
(z) = 1 0.7z
We note that both polynomials have a common factor, namely
1 0.7z. Discarding the common factor in each leaves

(z) = 1 0.3z

and
(z) = 1.
Thus the process is an AR(1) process, defined by
xt = 0.3xt1 + ut
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Causal Autoregressive-Moving Average (ARMA)
models

Definition. An ARMA(p, q) process {xt ; t Z} is causal


(strictly, a causal function of {ut ; t Z}) if there exists
constants 0 , 1 , ... such that

X
|j | <
j=0

and
X
xt = j utj t.
j=0

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

Here, it is important to clarify the meaning of equality



X
xt = j utj t
j=0

It means that
!2
n
X
lim E xt j utj = 0.
n
j=0

The equality is defined in terms of a limit in the quadratic


mean.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

The following two theorems provide, respectively, a


characterization of the of causality and stationarity of an
ARMA(p, q) process.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

Theorem. An ARMA(p, q) process {xt ; t Z} is causal if


and only if

(z) = 1 1 z ... p z p 6= 0 for all |z| 1.

Theorem. An ARMA(p, q) process {xt ; t Z} is stationary


if and only if

(z) = 1 1 z ... p z p 6= 0 for all |z| = 1.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

The causality and the stationarity of an ARMA process depend


entirely on the autoregressive parameters and not on the
moving-average ones.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

Further, we note that if an ARMA(p, q) process is causal, then


is stationary, but stationarity does not imply causality.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

Consider, for example, the following AR(1) process:

xt = 3xt1 + ut

where ut WN(0, u2 ). We have that

(z) = 1 3z 6= 0 for all |z| = 1.

and hence the process is stationary, but non causal since

(z) = 1 3z = 0 for z = 1/3.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models
An important result: There is a one-to-one correspondence
between the parameters of a causal ARMA(p,q) process and
the autocovariance function.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

It is important to underline that if we consider the set of


autocorrelation functions there is not a one-to-one
correspondence between the parameters of a causal
ARMA(p,q) process and the autocorrelation function.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models
Consider the following two MA(1) processes.

xt = ut + ut1

where ut WN(0, u2 ), with || < 1 and


1
yt = ut + ut1

where ut WN(0, u2 ).
Since
1/
= ,
1+ 2
1 + (1/)2
we have that both processes share the same autocorrelation
function. Thus it cannot be used to distinguish between the
two parametrizations.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models

This example shows that an MA(1)-process is not uniquely


determined by its autocorrelation function. There is an
identification problem with the MA(1) models.

In general, (if all roots of (z) = 0 are real) there can be 2q


different MA(q) processes with the same autocorrelation
function.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

Definition. An ARMA(p, q) process {xt ; t Z} is invertible


(strictly, an invertible function of {ut ; t Z}) if there exists
constants 0 , 1 , ... such that

X
|j | <
j=0

and
X
ut = j xtj t.
j=0

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

The following theorem provides a necessary and sufficient


condition for the invertibility.

Theorem. An ARMA(p, q) process {xt ; t Z} is invertible


if and only if

(z) = 1 + 1 z + ... + q z q 6= 0 for all |z| 1.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

We note that an AR(p) process is always invertible, even if it


is non-stationary, while an MA(q) process is always stationary,
even if it is non-invertible.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models

The invertibility can be used in order to ensure the


identifiability of MA processes.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Autoregressive-Moving Average (ARMA) models
In general, (if all roots of (z) = 0 are real) there can be 2q
different MA(q) processes with the same autocorrelation
function, but only one of these is invertible.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Conclusion
In the class of the mean-zero causal and invertible Gaussian
ARMA processes there is a one-to-one correspondence
between the family of the finite dimensional distributions of
the process and the finite parametric representation of process.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Conclusion

In the class of the mean-zero causal and invertible


Gaussian ARMA processes the probabilistic properties of the
process are completely characterized by the finite set of
parameters

1 , 2 , ..., p , 1 , 2 , ..., q , u2


Now, we have to estimate a finite number (p + q + 1) of


parameters from observed data.

This mission is possible.

Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models


Conclusion

Zero-mean causal invertible Gaussian ARMA process

'$

DGP {1 , 2 , ..., p , 1 , 2 , ..., q , u2 }


&%
7










 ? 
x1 , ..., xT
 
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models