1.

Nonlinear Univariate Times Series

1.1 Background
Example 1.1: Consider the following daily close-toclose Nasdaq composite share index values [January
3, 1989 to February 4, 2000]

Financial Time Series Analysis

and
Econometrics II

Nasdaq Composite [Jan 3, 1989 to Feb 4, 2000]

15

8
10

4
-5

-10
2

1
3.1.1989

2.1.1990

31.12.1990

30.12.1991

28.12.1992

27.12.1993

23.12.1994

22.12.1995

20.12.1996

19.12.1997

21.12.1998

Day


c Professor

Seppo Pynn
onen, Department of Mathematics and Statistics, University of Vaasa, Box
700, 65100 Vaasa, phone: (06)324 8259 email:
sjp@uwasa., URL: http://www.uwasa./sjp/
1

-15
21.12.1999

Return

Log Index

Seppo Pynn
onen,

2008

Below are autocorrelations of the log-index.

Denition 1.1: Time series yt, t = 1, . . . , T is

covariance stationary if

Obviously the persistence of autocorrelations indicate

E[yt] = , for all t

that the series is integrated (see denition below).

(1)
The autocorrelations of the return series suggest that

Cov[yt , yt+k ] = k , for all t

Var[yt] = 0 (< ), for all t

the returns are stationary with statistically signicant

Any series that are not stationary are said to be

rst order autocorrelation.

nonstationary.

Correlogram of LNSDQ

Correlogram of DNSDQ

Date: 04/01/01 Time: 14:19

Sample: 2276 5080
Included observations: 2805
Autocorrelation

Denition 1.2:Time series ut is a white noise process

if
E[ut] = , for all t

Date: 04/01/01 Time: 14:19

Sample: 2276 5080
Included observations: 2805

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.998
0.997
0.995
0.993
0.992
0.990
0.988
0.987
0.985
0.983
0.981
0.980
0.978
0.976
0.974
0.973
0.971
0.970
0.968
0.966
0.965
0.963
0.961
0.959
0.958
0.956
0.954
0.953
0.951
0.949
0.948
0.946
0.944
0.943
0.941
0.939

PAC
0.998
-0.010
0.016
0.001
0.013
0.006
-0.018
-0.001
-0.011
0.007
-0.019
0.003
-0.001
-0.003
0.005
0.007
0.016
-0.009
-0.016
0.014
0.018
-0.016
0.001
-0.025
0.005
0.001
-0.002
0.002
-0.003
-0.004
0.000
0.003
0.013
0.000
0.007
0.014

Q-Stat

Prob

2798.4
5587.9
8369.0
11142.
13906.
16663.
19411.
22151.
24882.
27605.
30319.
33025.
35722.
38410.
41090.
43762.
46426.
49081.
51728.
54367.
56998.
59621.
62236.
64842.
67440.
70030.
72611.
75185.
77750.
80306.
82855.
85395.
87927.
90451.
92967.
95476.

0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000

Autocorrelation

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.106
0.001
0.008
-0.009
-0.009
-0.012
-0.024
-0.001
-0.006
0.017
-0.004
0.049
0.058
0.009
0.017
-0.028
0.008
0.036
0.040
-0.007
-0.019
-0.046
-0.005
0.027
-0.002
0.013
0.032
0.002
0.020
0.014
0.005
0.004
0.037
-0.032
-0.037
-0.006

PAC
0.106
-0.010
0.009
-0.011
-0.007
-0.010
-0.021
0.004
-0.007
0.019
-0.009
0.051
0.048
-0.002
0.016
-0.032
0.017
0.034
0.038
-0.013
-0.017
-0.044
0.003
0.029
-0.012
0.014
0.025
-0.005
0.021
0.006
-0.002
0.000
0.041
-0.034
-0.022
-0.005

Q-Stat

Prob

31.459
31.465
31.659
31.904
32.123
32.514
34.069
34.071
34.171
35.025
35.079
41.844
51.463
51.686
52.507
54.694
54.875
58.504
63.052
63.174
64.219
70.298
70.379
72.511
72.527
73.018
75.957
75.968
77.127
77.648
77.714
77.767
81.689
84.586
88.467
88.571

0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000

(2)

Cov[ut , us ] = 0, for all t = s

Var[ut] = u2 < , for all t.

We denote ut W N (, u2 ).
Remark 1.1: Usually it is assumed in (2) that = 0.

Figure. Nasdaq Composite index autocorrelations for log levels and log differences (returns)

Remark 1.2: A WN-process is obviously stationary.

Denition 1.3: Times series yt is said to be

integrated of order 1, if it is of the form
(3)

Remark 1.3: If a time series process is of the form of

the right hand side of (4), i.e.,

(1 L)yt = + (L)ut,

(5)

denoted as yt I(1), where

where (L) satises the conditions of Def 1.3, it can

(L) = 1 + 1 L + 2 L + 3L +

such that
j=1 |j | < , (1) = 0, roots of (z) = 0
(4)

xt = + (L)ut ,

are outside the unit circle [or the polynomial (4) is of

order zero], and ut is a white noise series with mean
zero and variance u2 .

be shown that xt is stationary. We denote xt I(0),

i.e, integrated of order zero.
Remark 1.4: The assumption (1) = 0 is important.
It rules out for example trend stationary series
(6)

yt = + t + (L)ut.

Because E[yt] = + t, yt is nonstationary. However,

(7)

(1 L)yt = + (L)u
t,

where
(8)

(L)
= (1 L)(L).

Now, although, (1 L)yt is stationary, however,

(1)
= (1 1)(1) = 0,
which does not satisfy the rule in Denition 1.3, and
hence a trend stationary series is not I(1).

Example 1.2: (Example 1.1 continued) Below are re-

Table. MA(1) estimates

sults after tting an AR(1) and an MA(1) model to

Dependent Variable: DNSDQ
Method: Least Squares
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 4 iterations
Variable Coecient Std. Error t-Statistic
C
0.086153
0.022811
3.776796
MA(1)
0.107093
0.018779
5.702685

the return series

Table. AR(1) estimates.
Dependent Variable: DNSDQ
Method: Least Squares
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 2 iterations
Variable Coecient Std. Error t-Statistic
C
0.086126
0.023048
3.736845
AR(1)
0.105933
0.018782
5.640001
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
Durbin-Watson stat
Inverted AR Roots

0.011221
0.010868
1.091357
3338.542
-4224.341
1.997947
.11

R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
Durbin-Watson stat

Prob.
0.0002
0.0000

Mean dependent var

S.D. dependent var
Akaike info criterion
Schwarz criterion
F-statistic
Prob(F-statistic)

0.086119
1.097336
3.013434
3.017668
31.80961
0.000000

Inverted MA Roots

0.011323
0.010970
1.091301
3338.198
-4224.196
2.000051

Prob.
0.0002
0.0000

Mean dependent var

S.D. dependent var
Akaike info criterion
Schwarz criterion
F-statistic
Prob(F-statistic)

-.11

0.086119
1.097336
3.013331
3.017565
32.10153
0.000000

Both models give virtually equally good t, MA(1) is

just only marginally better.
The residual autocorrelations and related Q-statistics
indicate no further autocorrelation left to the series.

Because squared observations are the building blocks of the variance of the series, the
results suggest that the variation (volatility)
of the series is time dependent.
This leads to the so called ARCH-family of
models.

Correlogram of Residuals

Correlogram of Residuals Squared

Date: 04/01/01 Time: 15:27

Sample: 2276 5080
Included observations: 2805
Q-statistic probabilities adjusted for 1 ARMA term(s)

Sample: 2276 5080

Included observations: 2805
Q-statistic probabilities adjusted for 1 ARMA term(s)
Autocorrelation

Autocorrelation

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.000
0.001
0.009
-0.010
-0.007
-0.009
-0.023
0.003
-0.008
0.019
-0.011
0.044
0.054
0.001
0.020
-0.031
0.008
0.031
0.038
-0.009
-0.014
-0.045
-0.004
0.029
-0.007
0.011
0.031
-0.003
0.019
0.011
0.004
0.000
0.041
-0.033
-0.034
0.003

PAC
0.000
0.001
0.009
-0.010
-0.007
-0.009
-0.023
0.003
-0.008
0.020
-0.012
0.044
0.053
0.001
0.020
-0.031
0.010
0.032
0.042
-0.008
-0.013
-0.046
-0.005
0.030
-0.010
0.010
0.027
-0.004
0.019
0.008
0.000
-0.004
0.043
-0.027
-0.025
-0.001

Q-Stat

Prob

2.E-05
0.0009
0.2428
0.4990
0.6319
0.8445
2.3176
2.3375
2.5307
3.5945
3.9433
9.5255
17.620
17.624
18.779
21.455
21.634
24.341
28.389
28.622
29.145
34.754
34.794
37.100
37.220
37.533
40.340
40.373
41.442
41.793
41.831
41.831
46.528
49.557
52.794
52.814

0.976
0.886
0.919
0.959
0.974
0.888
0.939
0.960
0.936
0.950
0.574
0.128
0.172
0.174
0.123
0.155
0.110
0.056
0.072
0.085
0.030
0.041
0.032
0.042
0.051
0.036
0.047
0.049
0.059
0.074
0.093
0.047
0.032
0.021
0.027

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.278
0.272
0.192
0.193
0.217
0.154
0.141
0.145
0.074
0.106
0.107
0.127
0.115
0.124
0.120
0.137
0.133
0.091
0.148
0.076
0.126
0.144
0.105
0.188
0.088
0.120
0.142
0.142
0.120
0.117
0.119
0.106
0.081
0.076
0.087
0.073

PAC
0.278
0.211
0.084
0.089
0.118
0.026
0.022
0.047
-0.039
0.023
0.040
0.051
0.028
0.047
0.032
0.045
0.037
-0.022
0.063
-0.031
0.040
0.065
0.001
0.100
-0.029
0.013
0.046
0.042
-0.014
0.018
0.023
-0.007
-0.013
-0.015
0.007
-0.013

Q-Stat

Prob

216.92
425.28
529.00
633.28
765.20
832.06
888.03
947.14
962.58
994.10
1026.5
1072.0
1109.1
1152.2
1192.7
1245.3
1295.1
1318.5
1380.4
1396.7
1441.8
1500.7
1531.8
1632.3
1654.0
1694.7
1751.8
1808.9
1849.5
1888.4
1928.4
1960.4
1978.9
1995.3
2016.9
2031.9

0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000

Figure. Autocorrelations of the squared MA(1) residuals

Figure. Autocorrelations of the squared MA(1) residuals

The autocorrelations of the squared residuals strongly

suggest that there is still left time dependency into the
The

series.
The dependency, however, is nonlinear by nature.

inventor of this modeling approach is Robert

F. Engle (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of
United Kingdom ination. Econometrica, 50, 987
1008.
9

Furthermore, it is assumed that > 0, i 0

for all i and 1 + + q < 1.

1.2. Conditional Heteroscedasticity

ARCH-models

For short it is denoted ut ARCH(q).

The general setup for ARCH models is
(9)

yt = xt + ut

with xt = (x1t, x2t, . . . , xpt), = (1, 2, . . . , p),

t = 1, . . . , T , and
(10)

ut|Ft1 N (0, ht),

where Ft is the information available at time

t (usually the past values of ut; u1, . . . , ut1),
and

This reminds essentially an AR(q) process for

the squared residuals, because dening t =
u2
t ht , we can write
2
2
2
u2
t = + 1 ut1 + 2ut2 + + q utq + t .
(12)
Nevertheless, the error term t is time heteroscedastic, which implies that the conventional estimation procedure used in AR-estimation
does not produce optimal results here.

ht = Var(ut|Ft1 ) = + 1 u2t1 + 2 u2t2 + + q u2tq .

(11)

10

11

Properties of ARCH-processes

(ii) Constant variance: Using again the law

of iterated expectations, we get

Consider (for the sake of simplicity) ARCH(1)

process
(13)

ht = + u2
t1

with > 0 and 0 < 1 and ut|ut1

N (0, ht).

(i) Constant mean (zero):

E[ut] = E[Et1[ut]] = E[0] = 0.



=0

=
...
(16)

= (1 + + 2 + + n)
]
+ n+1E[u2

 tn1
n
0, as 

i
= limn n

i=0

(a) ut is white noise

(14)

Var[ut] =


2
= E Et1[ut ]
E[ht] = E[ + u2
t1]
+ E[u2
t1 ]

E[u2
t]

Note Et1[ut] = E[ut|Ft1], the conditional

expectation given information up to time t
1.

.
= 1
(iii) Autocovariances: Exercise, show that
autocovariances are zero, i.e., E[utut+k ] = 0
for all k = 0. (Hint: use the law of iterated expectations.)

The law of iterated expectations: Consider time points t < t

1
2

such that Ft1 Ft2 , then for any t > t2

(15)

Et1 [Et2 [ut ]] = E [E[ut |Ft2 ]|Ft1 ] = E[ut |Ft1 ] = Et1 [ut ].

12

13

(b) The unconditional distribution of ut is

symmetric, but nonnormal:

Because (1 2)/(1 32) > 1 we have that

(18) E[u4
t] > 3

(i) Skewness: Exercise, show that E[u3

t ] = 0.
(ii) Kurtosis: Exercise, show that under the
assumption
ut|ut1 N (0, ht), and that <

1/3, the kurtosis
(17)

E[u4
t] = 3

2
1 2

.
(1 )2 1 32

Hint: If X N (0, 2 ) then E[(X )4 ] = 3( 2 )2 = 3 4 .

14

2
= 3[Var(ut)]2,
2
(1 )

we nd that the kurtosis of the unconditional

distribution exceed that what it would be, if
ut were normally distributed.
Thus the unconditional distribution of ut is
nonnormal and has fatter tails than a normal
distribution with variance equal to Var[ut] =
/(1 ).

15

(c) Standardized variables:

Estimation of ARCH models

Write

Given the model

(19)

ut
zt =
ht

(21)

yt = xt + ut

then zt NID(0, 1), i.e., normally and independently distributed.

with ut|Ft1 N (0, ht), we have yt|{xt, Ft1}

N (xt, ht), t = 1, . . . , T .

Thus we can always write

Then the log-likelihood function becomes

(20)

u t = zt ht ,

(22)

where zt independent standard normal random variables (strict white noise).

This gives us a useful device to check after
tting an ARCH model the adequacy of the
specication: Check the autocorrelations of
the squared standardized series.

16

() =

t()

t=1

with
1
1
1

t() = log(2) log ht (yt xt)2/ht,

2
2
2
(23)
where = ( , , ).

17

The maximum likelihood (ML) estimate is

the value maximizing the likelihood function,
i.e.,
(24)

Generalized ARCH models

In practice the ARCH needs fairly many lags.

() = max
().

Usually far less lags are needed by modifying

the model to

The maximization is accomplished by numerical methods.

(25)

ht = + u2
t1 + ht1 ,

with > 0, > 0, 0, and + < 1.

Remark 1.5: OLS estimates of the regression parameters are inecient (unreliable) compared to the ML
estimates.

The model is called the Generalized ARCH

(GARCH) model.
Usually the above GARCH(1,1) is adequate
in practice.
Econometric packages call (coecient of
u2
t1 ) the ARCH parameter and (coecient
of ht1) the GARCH parameter.

18

19

Note again that dening t = u2

t ht, we can
write
2
(26) u2
t = + ( + )ut1 + t t1

Imposing additional lag terms, the model can

be extended to GARCH(r, q) model
(28)

ht = +

r

j=1

a heteroscedastic ARMA(1,1) process.

j htj +

u2
ti

i=1

[c.f. ARMA(p, q)].

Applying backward substitution, one easily
gets
(27)

+
j1u2
ht =
tj
1
j=1

Nevertheless, as noted above, in practice

GARCH(1,1) is adequate.

an ARCH() process.
Thus the GARCH term captures all the history from t 2 backwards of the shocks ut.

20

21

Example 1.3: MA(1)-GARCH(1,1) model of Nasdaq

returns. The model is
(29)

rt = + ut + ut1
ht = + u2t1 + ht1 .

Estimation results (EViews 4.0)

Correlogram of Standardized Residuals Squared

Sample: 2276 5080
Included observations: 2805
Q-statistic probabilities adjusted for 1 ARMA term(s)
Autocorrelation

Partial Correlation

Dependent Variable: DNSDQ

Method: ML - ARCH (Marquardt)
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 21 iterations
Bollerslev-Wooldrige robust standard errors & covariance
MA backcast: 2275, Variance backcast: ON
Coecient Std. Error z-Statistic Prob.
C
0.084907
0.017700
4.797124
0.0000
MA(1)
0.171620
0.020952
8.190983
0.0000
Variance Equation
C
0.027892
0.009213
3.027258
0.0000
ARCH(1)
0.121770
0.020448
5.955103
0.0000
GARCH(1) 0.857095
0.021526
39.81666
0.0000
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
Durbin-Watson stat
Inverted MA Roots

0.007104
0.005685
1.094213
3352.444
-3775.938
2.129878
-.17

Mean dependent var

S.D. dependent var
Akaike info criterion
Schwarz criterion
F-statistic
Prob(F-statistic)

0.086119
1.097336
2.695856
2.706443
5.008069
0.000507

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.004
0.035
-0.007
-0.007
-0.009
0.001
-0.021
-0.023
-0.019
-0.016
-0.024
-0.008
-0.007
-0.004
0.005
-0.004
-0.008
-0.025
0.002
-0.030
0.000
-0.016
-0.004
0.030
-0.012
-0.012
0.011
-0.015
0.021
-0.002
-0.001
-0.008
0.037
-0.016
0.009
-0.013

PAC
0.004
0.035
-0.007
-0.008
-0.008
0.001
-0.020
-0.023
-0.017
-0.014
-0.023
-0.008
-0.006
-0.005
0.004
-0.005
-0.010
-0.027
0.001
-0.030
-0.002
-0.016
-0.005
0.030
-0.014
-0.016
0.010
-0.017
0.018
-0.004
-0.004
-0.008
0.036
-0.017
0.007
-0.013

Q-Stat

Prob

0.0519
3.5358
3.6727
3.8082
4.0159
4.0179
5.2184
6.7031
7.7149
8.4002
9.9573
10.148
10.278
10.324
10.405
10.448
10.639
12.405
12.421
14.903
14.903
15.671
15.710
18.231
18.627
19.046
19.387
20.051
21.360
21.372
21.377
21.542
25.370
26.108
26.352
26.810

0.060
0.159
0.283
0.404
0.547
0.516
0.460
0.462
0.494
0.444
0.517
0.592
0.667
0.732
0.791
0.831
0.775
0.825
0.729
0.782
0.788
0.830
0.745
0.772
0.795
0.820
0.829
0.810
0.845
0.876
0.897
0.791
0.797
0.823
0.838

8
4
0
-4
8

-8

-12

0
-4
-8
-12
2500

3000

3500

Residual

4000

4500

Actual

5000

Fitted

Figure. Conditional standard deviation function

350
Series: Standardized Residuals
Sample 2276 5080
Observations 2805

300
250

Mean
Median
Maximum
Minimum
Std. Dev.
Skewness
Kurtosis

200
150

2
100
50

Jarque-Bera
Probability

0
-5.0

-2.5

0.0

-0.019280
0.050222
3.265924
-6.466048
0.999654
-0.731417
5.456912
955.6050
0.000000

2.5

Figure. Conditional standard deviation function

0
2500

3000

3500

4000

4500

5000

Figure. Conditional standard deviation function

22

23

The autocorrelations of the squared standardized residuals pass the white noise test.

The variance function can be extended by

including regressors (exogenous or predetermined variables), xt, in it

Nevertheless, the normality of the standardized resid-

(30)

uals is strongly rejected.

This is why robust standard errors are used in the
estimation of the standard errors.

ht = + u2
t1 + ht1 + xt .

Note that if xt can assume negative values, it

may be desirable to introduce absolute values
|xt| in place of xt in the conditional variance
function.
For example, with daily data a Monday dummy
could be introduced into the model to capture the non-trading over the weekends in the
volatility.

24

25

ARCH-M Model

Example 1.4: Does the daily mean return of Nasdaq

depend on the volatility level?

The regression equation may be extended

by introducing the variance function into the
equation
(31)

yt = xt + g(ht) + ut,

where ut GARCH, and g is a suitable function (usually square root or logarithm).

This is called the ARCH in Mean (ARCH-M)
model [Engle, Lilien and Robbins (1987)].
The ARCH-M model is often used in nance,
where the expected return on an asset is related to the expected asset risk.

Dependent Variable: DNSDQ

Method: ML - ARCH (Marquardt)
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 22 iterations
Bollerslev-Wooldrige robust standard errors & covariance
MA backcast: 2275, Variance backcast: ON
Coecient Std. Error z-Statistic Prob.
SQR(GARCH) 0.198064
0.074141
2.671456
0.0076
C
-0.069416
0.061432
-1.129969 0.2585
MA(1)
0.174785
0.020644
8.466806
0.0000
Variance Equation
C
0.031799
0.009301
3.419007
0.0006
ARCH(1)
0.134070
0.020974
6.392287
0.0000
GARCH(1)
0.842134
0.021350
39.44407
0.0000
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
Durbin-Watson stat
Inverted MA Roots

0.011379
0.009613
1.092049
3338.007
-3773.330
2.127550
-.17

Mean dependent var

S.D. dependent var
Akaike info criterion
Schwarz criterion
F-statistic
Prob(F-statistic)

0.086119
1.097336
2.694709
2.707413
6.443432
0.000006

The volatility term in the mean equation is statistically

signicant indicating that rather than being constant

The coecient reects the risk-return tradeo.

Econometrica,

the mean return is dependent on the level of volatility.

55, 391407.
26

27

Consequently, the data suggests that the best tting

model so far is of the form

rt = ht + ut + ut1
ht = + u2t1 + ht1 .
Below are the estimation results for the above model

Correlogram of Standardized Residuals Squared

Sample: 2276 5080
Included observations: 2805
Q-statistic probabilities adjusted for 1 ARMA term(s)
Autocorrelation

Dependent Variable: DNSDQ

Method: ML - ARCH (Marquardt)
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 16 iterations
Bollerslev-Wooldrige robust standard errors & covariance
MA backcast: 2275, Variance backcast: ON
Coecient Std. Error z-Statistic Prob.
SQR(GARCH) 0.119204
0.022944
5.195479
0.0000
MA(1)
0.174104
0.020771
8.382098
0.0000
Variance Equation
C
0.031291
0.009545
3.278211
0.0010
ARCH(1)
0.133713
0.021011
6.363810
0.0000
GARCH(1)
0.843131
0.021785
38.70279
0.0000
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
Inverted MA Roots

0.010861
0.009448
1.092141
3339.759
-3774.146
-.17

Mean dependent var

S.D. dependent var
Akaike info criterion
Schwarz criterion
Durbin-Watson stat

0.086119
1.097336
2.694578
2.705165
2.132844

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

-0.003
0.034
-0.008
-0.009
-0.009
0.001
-0.022
-0.023
-0.018
-0.015
-0.023
-0.007
-0.006
-0.001
0.005
-0.003
-0.006
-0.025
0.006
-0.030
0.003
-0.016
-0.002
0.031
-0.011
-0.011
0.013
-0.015
0.022
-0.003
0.000
-0.006
0.041
-0.016
0.009
-0.013

PAC
-0.003
0.034
-0.008
-0.010
-0.009
0.001
-0.021
-0.023
-0.017
-0.014
-0.023
-0.007
-0.005
-0.002
0.003
-0.005
-0.008
-0.027
0.004
-0.030
0.000
-0.015
-0.004
0.031
-0.013
-0.015
0.011
-0.016
0.019
-0.004
-0.002
-0.006
0.040
-0.016
0.006
-0.012

Q-Stat

Prob

0.0213
3.2371
3.4198
3.6300
3.8611
3.8636
5.1942
6.6228
7.5391
8.1998
9.7327
9.8585
9.9568
9.9594
10.023
10.055
10.163
11.907
11.994
14.490
14.511
15.193
15.207
17.937
18.265
18.602
19.073
19.737
21.147
21.173
21.173
21.293
26.022
26.794
27.013
27.472

0.072
0.181
0.304
0.425
0.569
0.519
0.469
0.480
0.514
0.464
0.543
0.620
0.697
0.761
0.816
0.858
0.806
0.848
0.754
0.804
0.813
0.853
0.761
0.790
0.816
0.833
0.842
0.819
0.853
0.882
0.904
0.763
0.769
0.797
0.814

8
4
0
-4
8

-8

-12

0
-4
-8
-12
2500

3000

3500

Residual

4000
Actual

4500

5000

Fitted

Figure. Actual and fitted series, and residuals

4
2
350

Series: Standardized Residuals

Sample 2276 5080
Observations 2805

300
250

-2

Mean
Median
Maximum
Minimum
Std. Dev.
Skewness
Kurtosis

200
150

-4

100
50

-6

Jarque-Bera
Probability

0
-5.0

-2.5

0.0

-0.056142
0.018376
3.239140
-6.557423
0.998195
-0.726040
5.409519
924.9855
0.000000

2.5

-8
2500

3000

3500

4000

4500

5000

Standardized Residuals

28

29

Looking at the standardized residuals, the distribution

and the sample statistics of the distribution, we observe that the residual distribution is obviously skewed
in addition to the leptokurtosis.
The skewness may be due to some asymmetry in the
conditional volatility which we have not yet modeled.
In nancial data the asymmetry is usually such that
downward shocks cause higher volatility in the near

============================================================
Z,Z2(-i)
Z,Z2(+i)
i
lag
lead
============================================================
****|
|
****|
| 0 -0.3916 -0.3916
|
|
*|
| 1
0.0342 -0.0782
|
|
*|
| 2 -0.0055 -0.0842
|
|
|
| 3
0.0093 -0.0373
|
|
|
| 4
0.0066 0.0315
|
|
|
| 5
0.0134 -0.0046
|
|
|
| 6 -0.0134 -0.0019
|
|
|
| 7
0.0113 -0.0004
|
|
|
| 8 -0.0019 0.0045
|
|
|
| 9 -0.0034 0.0272
|
|
|
| 10 -0.0205 0.0128
============================================================

future than the positive shocks.

The cross autocorrelations correlations are not large,

In nance this is called the leverage eect.

An obvious and simple rst hand check for the asym-

but may indicate the presence of some asymmetry.

metry is to investigate the cross autocorrelations between standardized and squared standardized GARCH
residuals.
Below are the cross autocorrelations between the standardized and squared standardized residuals of the tted MA(1)-GARCH(1,1) model.

30

31

Asymmetric ARCH: TARCH and EGARCH

The TARCH model

A kind of stylized fact in stock markets is

that downward movements are followed by
higher volatility.

Threshold ARCH, TARCH (Zakoian 1994,

Journal of Economic Dynamics and Control,
931955 , Glosten, Jagannathan and Runkle 1993, Journal of Finance, 1779-1801) is
given by [TARCH(1,1)]

EViews includes two models that allow for

asymmetric shocks to volatility.

2
(32)ht = + u2
t1 + ut1dt1 + ht1 ,

where dt = 1, if ut < 0 (bad news) and zero

otherwise.
The impact of good news is and bad news
+ .
Thus, = 0 implies asymmetry.
Leverage exists if > 0.

32

33

Example 1.5: Estimation results for the MA(1)-TARCHM model.

Furthermore, as seen below, the rst few cross auto-

Dependent Variable: DNSDQ

Method: ML - ARCH (Marquardt)
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 26 iterations
Bollerslev-Wooldrige robust standard errors & covariance
MA backcast: 2275, Variance backcast: ON
=============================================================
Coefficient Std. Error z-Statistic Prob.
=============================================================
SQR(GARCH)
0.091184
0.023097
3.947880
0.0001
MA(1)
0.184263
0.020899
8.816678
0.0000
=============================================================
Variance Equation
=============================================================
C
0.037068
0.009513
3.896566
0.0001
ARCH(1)
0.084275
0.025080
3.360240
0.0008
(RESID<0)*ARCH(1) 0.099893
0.040881
2.443502
0.0145
GARCH(1)
0.833239
0.019001
43.85202
0.0000
=============================================================
R-squared
0.009604
Mean dependent var
0.086119
Adjusted R-squared 0.007835
S.D. dependent var
1.097336
S.E. of regression 1.093029
Akaike info criterion 2.686832
Sum squared resid 3344.000
Schwarz criterion
2.699536
Log likelihood
-3762.281
Durbin-Watson stat
2.149776
=============================================================
Inverted MA Roots
-.18
=============================================================

ones.

correlations reduce to about one half of the original

They are still statistically signicant by slightly ex

ceeding the approximate 95% boundaries 2/ T =

2/ 2805 0.038.
Cross autocorrelations of the standardized and squared
standardized MA(1)-TARCH(1,1)-M model.
============================================================
Z,Z2(-i)
Z,Z2(+i)
i
lag
lead
-----------------------------------------------------------****|
|
****|
| 0 -0.3543 -0.3543
|
|
*|
| 1
0.0304 -0.0488
|
|
*|
| 2 -0.0071 -0.0537
|
|
|
| 3
0.0112 -0.0156
|
|
|*
| 4
0.0069
0.0545
|
|
|
| 5
0.0113
0.0080
============================================================

The goodness of t improves and the statistically signicant positive asymmetry parameter estimate indicates presence of leverage.
34

35

The EGARCH model

Example 1.6: MA(1)-EGARCH(1,1)-M estimation results.

Nelson (1991) (Econometrica, 347370) proposed the Exponential GARCH (EGARCH)

model for the variance function of the form
(EGARCH(1,1))

Dependent Variable: DNSDQ

Method: ML - ARCH (Marquardt)
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 28 iterations
Bollerslev-Wooldrige robust standard errors & covariance
MA backcast: 2275, Variance backcast: ON
=========================================================
Coefficient Std. Error z-Stat
Prob.
=========================================================
SQR(GARCH)
0.084631 0.022593 3.745866 0.0002
MA(1)
0.171543 0.020387 8.414399 0.0000
=========================================================
Variance Equation
=========================================================
C
-0.197193 0.023051 -8.554804 0.0000
|RES|/SQR[GARCH](1) 0.251752 0.030816 8.169621 0.0000
RES/SQR[GARCH](1)
-0.071425 0.024034 -2.971755 0.0030
EGARCH(1)
0.958125 0.010941 87.57385 0.0000
=========================================================
R-squared
0.010762 Mean dependent var 0.086119
Adjusted R-squared 0.008995 S.D. dependent var 1.097336
S.E. of regression 1.092390 Akaike info criter 2.682518
Sum squared resid 3340.093 Schwarz criterion
2.695222
Log likelihood
-3756.232 Durbin-Watson stat 2.124928
=========================================================
Inverted MA Roots
-.17
=========================================================

(33)

log ht = + log ht1 + |zt1 | + zt1 ,

where zt = ut/ ht is the standardized shock.

Again, the impact is asymmetric if = 0,
and leverage is present if < 0.

Cross autocorrelations (not shown here) are about

the same as with the TARCH model (i.e., disappear).
Thus TARCH and EGARCH capture most part of the
leverage eect.
36

37

News Impact Curve

Below is a graph for the NIC of the above estimate

EGARCH variance function, where ht1 is replaced by

The asymmetry of the conditional volatility

function can be conveniently illustrated by
the news impact curve (NIC).

the median of the estimated EGARCH series.

News Impact Curve: EGARCH

log h(t) = -.197 + .252 |z(t)| - .071 z(t) + .958 log h(t-1)

The curve is simply the graph of ht(z), where

z indicates the shocks (news).

18
16
14

ht

12
10
8
6
4
2
0
-10

-8

-6

-4

-2

38

39

The Component ARCH Model

The model is

We can write the GARCH(1,1) model as

(34) ht =
+ (u2
) + (ht1
),
t1
where

=
(35)
1
is the unconditional variance of the series.

ht qt = (u2
t1 qt1 ) + (ht1 qt1 )
qt = + (qt1 ) + (u2
t1 ht1 ).
(36)
An asymmetric version for the model is
ht qt = (u2
t1 qt1 )
+(u2
t1 qt1)dt1 + (ht1 qt1 )
qt = + (qt1 ) + (u2
t1 ht1 ).
(37)

Thus the usual GARCH has a mean reversion

tendency towards
.
A further extension is to allow this unconditional or long term volatility to vary over
time.
This lead to so called component ARCH that
allows mean reversion to a varying level qt
instead of
.

40

41

Example 1.7: Asymmetric Component ARCH of the

Nasdaq composite returns.

1.3 Regime switching models

Dependent Variable: DNSDQ

Method: ML - ARCH (Marquardt)
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 4 iterations
Bollerslev-Wooldrige robust standard errors & covariance
MA backcast: 2275, Variance backcast: ON
=================================================================
Coefficient Std. Error z-Statistic
Prob.
=================================================================
SQR(GARCH)
0.097235
0.023997
4.052002
0.0001
MA(1)
0.182908
0.027822
6.574119
0.0000
=================================================================
Variance Equation
================================================================
Perm: C
0.926329
0.080794
11.46533
0.0000
Perm: [Q-C]
0.734067
0.077885
9.425047
0.0000
Perm: [ARCH-GARCH]
0.228360
0.048519
4.706581
0.0000
Tran: [ARCH-Q]
0.037121
0.039565
0.938228
0.3481
Tran: (RES<0)*[ARCH-Q] -0.077747
0.076100 -1.021635
0.3070
Tran: [GARCH-Q]
-0.688202
0.353158 -1.948710
0.0513
=================================================================
R-squared
0.008632 Mean dependent var
0.086119
Adjusted R-squared
0.006151 S.D. dependent var
1.097336
S.E. of regression
1.093956 Akaike info criterion 2.771137
Sum squared resid
3347.282 Schwarz criterion
2.788076
Log likelihood
-3878.519 Durbin-Watson stat
2.152828
=================================================================
Inverted MA Roots
-.18
=================================================================

A potentially useful approach to model nonlinearities in time series is to assume dierent

behavior (structural break) in dierent subsamples (or regimes).
If the dates, the regimes switches have taken
place, are known, modeling can be worked
out simply with dummy variables.

This model, however, does not t well into the data.

Thus it seems that the best tting models so far are
either the TARCH or EGARCH.
42

43

Consider the following regression model

(38)

yt = xtSt + ut,

t = 1, . . . , T ,

That is, xt = (1, yt1).

where
(39)

For the sake of simplicity, consider an AR(1)

model.

2 ),
ut NID(0, S
t

(40)

St = 0(1 St) + 1St,

(41)

2 = 2 (1 S ) + 2 S ,
S
t
0
1 t
t

Usually it is assumed that the possible dierence between the regimes is a mean and/or a
volatility shift, but no change in the autoregression parameter.

and
(42)

St = 0 or 1,

That is,

(Regime 0 or 1).

Thus, under regime 1 the coecient parameter vector is 1 and error variance 12.

(43)

yt = St + 1(yt1 st1 ) + ut,

with
(44)

2 ),
ut NID(0, S
t

2 as dewhere St = 0(1 St) + 1St and S

t
ned above.

If St, t = 1, . . . , T is known a priori, then the

problem is just a usual dummy variable autoregression problem.
44

45

In practice, however, the prevailing regime is

not usually directly observable.

The joint probability density function for

yt, St, St1, given past information
Ft1 = {yt1, yt2, . . .}, is

Denote then
(45)

P (St = j|St1 = i) = pij ,

i, j = 0, 1, called transition probabilities, with

pi0 + pi1 = 1, i = 0, 1.
This kind of process, where the next state
depends only on the previous state, is called
the Markov process, and the model
a Markov switching model in the mean and
variance.

f (yt, St , St1 |Ft1) = f (yt|St, St1 , Ft1 )P (St, St1 |Ft1 ),

(46)

with

f (yt |St , St1 , Ft1 ) =

1
2S2t


exp

[yt St 1 (yt1 St1 )]2

2S2t

(47)

Thus, in this model additional parameters to

be estimated are the transition probabilities
pij .
Usually the parameters are estimated (numerically) by the ML method.
For

a detailed discussion, see Kim Chang-Jin and

Charles A. Nelson (1999).
State Space Models
with Regime Switching. Classical and Gibbs-Sampling Approaches with Applications. MIT-Press.
46

47


.

Then the log-likelihood function to be maximized with respect to the unknown parameters is
(48)

() =

t(),

Because of the Markov property

t=1

where

t () = log

1
1

f (yt|St, St1 , Ft1 )P [St, St1|Ft1 ],

St =0 St1 =0

(49)

(50)
with

p = P [St = 0|St1 = 0],

(52)

q = P [St = 1|St1 = 1],

(53)

P [St|St1, Ft1] = P [St|St1],

we can write
(54)

P [St , St1 |Ft1 ] = P [St|St1 ]P [St1 |Ft1 ],

and the problem reduces to calculating (estimating) the time dependent state probabilities, P [St1|Ft1], and weight them with
the transition probabilities to obtain the joint
probability.

= (p, q, 0, 1, 1, 02, 12),

(51)

In order to evaluate the log-likelihood function we need to dene the joint probabilities
P [St, St1|Ft1].

being the transition probabilities.

48

49

This can be achieved as follows:

First, let P [S0 = 1|F0] = P [S0 = 1] = be
given (then P [S0 = 0] = 1 ).
Then the probabilities P [St1|Ft1] and the
joint probabilities are obtained using the following two steps algorithm

Once we have the joint probability for the

time point t, we can calculate the likelihood

t().
The maximum likelihood estimates for is
then obtained iteratively maximizing the likelihood function by updating the likelihood
function at each iteration with the above algorithm.

10 Given P [St1 = i|Ft1 ], i = 0, 1, at the beginning

of time t (tth iteration),
P [St = j, St1 = i|Ft1 ] = P [St = j|St1 = i]P [St1 = i|Ft1 ],
(55)

20 Once yt is observed, we update the information

set Ft = {Ft1 , yt } and the probabilities
P [St = j, St1 = i|Ft] = P [St = j, St1 = i|Ft1 , yt]
=

f (St =i,St1 =j,yt |Ft1 )

f (yt |Ft1 )

= 1f (yt|St =j,St1 =i,Ft1)P [St =j,St1 =i|Ft1 ]

st ,st1 =0

f (yt |st ,st1 ,Ft1 )P [St =st ,St1 =st1 |Ft1 ]

(56)
with
P [St = st |Ft] =

P [St = st, St1 = st1 |Ft].

st1 =0

(57)
50

51

Steady state probabilities

Smoothed probabilities

The probabilities = P [S0 = 1|F0] is called

the steady state probability, and, given the
transition probabilities p and q, is obtained
as
1p
(58)
.
= P [S0 = 1|F0] =
2pq
Note that in the two state Markov chain
1q
.
P [S0 = 0|F0] = 1 P [S0 = 1|F0] =
2pq
(59)

Recall that the state St is unobserved.

However, once we have estimated the model,
we can make inferences on St using all the
information from the sample.
This gives us
(60)

P [St = j|FT ],

j = 0, 1,

which are called the smoothed probabilities

(for details, see Kim and Nelson 1999, pp.
6869).
Remark 1.6: In the estimation procedure we derived
P [St = j|Ft ] that are usually called the ltered probabilities.

52

53

Expected duration

Estimating the model from quarterly data for sample

period 1972I to 1996IV gives

The expected length the system is going to

stay in state j can be calculated from the
transition probabilities. Let D denote the
number of periods the system is in state j.
The probabilities are easily found to be equal
to P [D = k] = pk1
jj (1 pjj ), so that

Parameter
0
1
02
12
p (regime 1)
q (regime 0)

Estimate
2.605
-3.277
13.56
20.82
0.857
0.866

Std err
0.964
1.582
3.34
4.79
0.084
0.097

The expected length of stay in regime 0 is given by

1/(1 p) = 7.0 quarters, and in regime 1 1/(1 q) =

1
(61) E[D] =
kP [D = k] =
.
1

p
jj
k=1

7.5 quarters.

Note that in our case p00 = p and p11 = q.

Example 1.8: Are there long swings in the dollar/sterling
exchange rate?
If the exchange rate xt is RW with long swings, it can
be modeled as
xt = 0 + 1 St + t ,
so that x1 N (0 , 02 ) when St = 0 and xt
N (1 , 12), when St = 1, where 0 = 0 and 1 =
0 +1. Parameters 0 and 1 constitute two dierent
drifts (if 1 = 0) in the random walk model.

54

55

Example 1.9: Suppose we are interested whether the

The market model is

market risk of a share is dependent on the level of

yt = St + St xt + t ,

volatility on the market. In the CAPM world the mar-

where St = 0 (1 St ) + 1 St, St = 0 (1 St ) + 1 St

ket risk of a stock is measured by .

and t N (0, S2t ) with t2 = 02 (1 St ) + 12 St.

Estimating the model yields

World and Finnish Returns

10.0

Parameter
Estimate
0 (low)
-0.0068
1 (high)
0.0802
0 (low)
0.9679
1 (high)
1.8040
2
0.5225
0 (low)
1.7050
12 (high)
State Prob
P (High|High)
0.96417
P (Low|High)
0.03583
P (High|Low)
0.01728
P (Low|Low)
0.98272
P (High)
0.67471
P (Low)
0.32529
Log-likelihood -3208.438

7.5

Finnish Returns

5.0

2.5

0.0

-2.5

-5.0

-7.5

-10.0
-6

-4

-2

World Returns

Std Err
0.0178
0.0508
0.0215
0.0690
0.0198
0.0711

t-value
-0.39
1.57
45.04
26.15
26.37
23.96

p-value
0.700
0.114
0.000
0.000
0.000
0.000

The empirical results give evidence that the stocks

Consider for the sake of simplicity only the cases of
high and low volatility.

market risk depends on the level of stock volatility.

The expected duration of high volatility is 1/(1
.9642) 27 days, and for low volatility 59 days.

56

57

Market returns with high-low volatility probabilities

58