Anda di halaman 1dari 34

# Lesson 9: Autoregressive-Moving

## Average (ARMA) models

Umberto Triacca
Dipartimento di Ingegneria e Scienze dellInformazione e Matematica
Universita dellAquila,
umberto.triacca@ec.univaq.it

Introduction

## We have seen that in the class of stationary, zero mean,

Gaussian processes the probabilistic structure of a stochastic
process is completly characterized by the autocovariance
function.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autocovariance function

## Stationary, zero mean, Gaussian process

'\$

DGP x (k)
&%
7










 ? 
x1 , ..., xT
 
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Introduction

## However, in general, to know the autocovariance function

means to know a sequence composed by an infinite number of
elements.

Introduction

## We introduce a very important class of stochastic processes,

which autocovariance functions depend on a finite number of
unknown parameters:

processes.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models
Definition. The process {xt ; t Z} is an autoregressive
moving average process of order (p, q), denoted with

xt ARMA(p, q),

if

## where ut WN(0, u2 ), and 1 , ..., p , 1 , ..., q are p + q

constants and the polynomials

(z) = 1 1 z ... p z p

and
(z) = 1 + 1 z... + q z q
have no common factors.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models

## For q = 0 the process reduces to an autoregressive process of

order p, denoted with xt AR(p),

with xt MA(q)

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

An example of Autoregressive-Moving Average
(ARMA) process

## The process {xt ; t Z} defined by

xt = 0.3xt1 + ut + 0.7ut1 t Z,

Here
(z) = 1 0.3z
and
(z) = 1 + 0.7z.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

An example of Autoregressive-Moving Average
(ARMA) process

## A realizzation of the ARMA(1,1) process

xt = 0.3xt1 + ut + 0.7ut1 is presented in the following figure.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

An example of Autoregressive (AR) process

## The process {xt ; t Z} defined by

xt = 0.7xt1 0.5xt1 + ut t Z,

## where ut WN(0, u2 ), is an AR(2) process.

Here
(z) = 1 0.7z + 0.5z 2

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

An example of Autoregressive (AR) process

## A realizzation of the AR(2) process

xt = 0.7xt1 0.5xt2 + ut is presented in the following figure.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

An example of Moving Average (MA) process

## The process {xt ; t Z} defined by

xt = ut + 0.7ut1 t Z,

Here
(z) = 1 + 0.7z

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

An example of Moving Average (MA) process

## A realizzation of the MA(1) process xt = ut + 0.7ut1 is

presented in the following figure.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

An example of over-parameterization

## xt = xt1 0.21xt2 + ut 0.7ut1 t Z,

where ut WN(0, u2 ).

## This process looks like an ARMA(2,1) process but it is not an

ARMA(2,1) process.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

An example of over-parameterization
Here

## (z) = 1 z + 0.21z 2 = (1 0.7z)(1 0.3z)

and
(z) = 1 0.7z
We note that both polynomials have a common factor, namely
1 0.7z. Discarding the common factor in each leaves

(z) = 1 0.3z

and
(z) = 1.
Thus the process is an AR(1) process, defined by
xt = 0.3xt1 + ut
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Causal Autoregressive-Moving Average (ARMA)
models

## Definition. An ARMA(p, q) process {xt ; t Z} is causal

(strictly, a causal function of {ut ; t Z}) if there exists
constants 0 , 1 , ... such that

X
|j | <
j=0

and
X
xt = j utj t.
j=0

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## Here, it is important to clarify the meaning of equality

X
xt = j utj t
j=0

It means that
!2
n
X
lim E xt j utj = 0.
n
j=0

mean.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## The following two theorems provide, respectively, a

characterization of the of causality and stationarity of an
ARMA(p, q) process.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

and only if

if and only if

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## The causality and the stationarity of an ARMA process depend

entirely on the autoregressive parameters and not on the
moving-average ones.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## Further, we note that if an ARMA(p, q) process is causal, then

is stationary, but stationarity does not imply causality.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

xt = 3xt1 + ut

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models
An important result: There is a one-to-one correspondence
between the parameters of a causal ARMA(p,q) process and
the autocovariance function.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## It is important to underline that if we consider the set of

autocorrelation functions there is not a one-to-one
correspondence between the parameters of a causal
ARMA(p,q) process and the autocorrelation function.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models
Consider the following two MA(1) processes.

xt = ut + ut1

## where ut WN(0, u2 ), with || < 1 and

1
yt = ut + ut1

where ut WN(0, u2 ).
Since
1/
= ,
1+ 2
1 + (1/)2
we have that both processes share the same autocorrelation
function. Thus it cannot be used to distinguish between the
two parametrizations.
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models
Autoregressive-Moving Average (ARMA) models

## This example shows that an MA(1)-process is not uniquely

determined by its autocorrelation function. There is an
identification problem with the MA(1) models.

## In general, (if all roots of (z) = 0 are real) there can be 2q

different MA(q) processes with the same autocorrelation
function.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## Definition. An ARMA(p, q) process {xt ; t Z} is invertible

(strictly, an invertible function of {ut ; t Z}) if there exists
constants 0 , 1 , ... such that

X
|j | <
j=0

and
X
ut = j xtj t.
j=0

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## The following theorem provides a necessary and sufficient

condition for the invertibility.

if and only if

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## We note that an AR(p) process is always invertible, even if it

is non-stationary, while an MA(q) process is always stationary,
even if it is non-invertible.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models

## The invertibility can be used in order to ensure the

identifiability of MA processes.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Autoregressive-Moving Average (ARMA) models
In general, (if all roots of (z) = 0 are real) there can be 2q
different MA(q) processes with the same autocorrelation
function, but only one of these is invertible.

## Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models

Conclusion
In the class of the mean-zero causal and invertible Gaussian
ARMA processes there is a one-to-one correspondence
between the family of the finite dimensional distributions of
the process and the finite parametric representation of process.

Conclusion

## In the class of the mean-zero causal and invertible

Gaussian ARMA processes the probabilistic properties of the
process are completely characterized by the finite set of
parameters

1 , 2 , ..., p , 1 , 2 , ..., q , u2


## Now, we have to estimate a finite number (p + q + 1) of

parameters from observed data.

Conclusion

'\$

## DGP {1 , 2 , ..., p , 1 , 2 , ..., q , u2 }

&%
7










 ? 
x1 , ..., xT
 
Umberto Triacca Lesson 9: Autoregressive-Moving Average (ARMA) models