ANALYSIS OF FINANCIAL TIME SERIES

Nonlinear Univariate
and
Linear Multivariate
Time Series

Seppo Pynn
onen and
Bernd Pape, 2005

University

of Vaasa, Dep. of Mathematics and

Statistics, P.O. Box 700, 65100 Vaasa, phone:
(06) 324 8277 email:
bepa@uwasa., URL:
http://www.uwasa./bepa/

1. Nonlinear Univariate Times Series

1.1 Background
Example. Consider the following daily close-to-close
Nasdaq composite share index values [January 3, 1989
to February 4, 2000]

Nasdaq Composite [Jan 3, 1989 to Feb 4, 2000]

15

8
10

4
-5

-10
2

1
3.1.1989

2.1.1990

31.12.1990

30.12.1991

28.12.1992

27.12.1993

23.12.1994

22.12.1995

20.12.1996

19.12.1997

21.12.1998

-15
21.12.1999

Day

Return

Log Index

Below are autocorrelations of the log-index. Obviously the persistence of autocorrelations indicate that
the series is integrated. The autocorrelations of the
return series suggest that the returns are stationary
with statistically signicant rst order autocorrelation.

Correlogram of LNSDQ

Correlogram of DNSDQ

Date: 04/01/01 Time: 14:19

Sample: 2276 5080
Included observations: 2805
Autocorrelation

Date: 04/01/01 Time: 14:19

Sample: 2276 5080
Included observations: 2805

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.998
0.997
0.995
0.993
0.992
0.990
0.988
0.987
0.985
0.983
0.981
0.980
0.978
0.976
0.974
0.973
0.971
0.970
0.968
0.966
0.965
0.963
0.961
0.959
0.958
0.956
0.954
0.953
0.951
0.949
0.948
0.946
0.944
0.943
0.941
0.939

PAC
0.998
-0.010
0.016
0.001
0.013
0.006
-0.018
-0.001
-0.011
0.007
-0.019
0.003
-0.001
-0.003
0.005
0.007
0.016
-0.009
-0.016
0.014
0.018
-0.016
0.001
-0.025
0.005
0.001
-0.002
0.002
-0.003
-0.004
0.000
0.003
0.013
0.000
0.007
0.014

Q-Stat

Prob

2798.4
5587.9
8369.0
11142.
13906.
16663.
19411.
22151.
24882.
27605.
30319.
33025.
35722.
38410.
41090.
43762.
46426.
49081.
51728.
54367.
56998.
59621.
62236.
64842.
67440.
70030.
72611.
75185.
77750.
80306.
82855.
85395.
87927.
90451.
92967.
95476.

0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000

Autocorrelation

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.106
0.001
0.008
-0.009
-0.009
-0.012
-0.024
-0.001
-0.006
0.017
-0.004
0.049
0.058
0.009
0.017
-0.028
0.008
0.036
0.040
-0.007
-0.019
-0.046
-0.005
0.027
-0.002
0.013
0.032
0.002
0.020
0.014
0.005
0.004
0.037
-0.032
-0.037
-0.006

PAC
0.106
-0.010
0.009
-0.011
-0.007
-0.010
-0.021
0.004
-0.007
0.019
-0.009
0.051
0.048
-0.002
0.016
-0.032
0.017
0.034
0.038
-0.013
-0.017
-0.044
0.003
0.029
-0.012
0.014
0.025
-0.005
0.021
0.006
-0.002
0.000
0.041
-0.034
-0.022
-0.005

Q-Stat

Prob

31.459
31.465
31.659
31.904
32.123
32.514
34.069
34.071
34.171
35.025
35.079
41.844
51.463
51.686
52.507
54.694
54.875
58.504
63.052
63.174
64.219
70.298
70.379
72.511
72.527
73.018
75.957
75.968
77.127
77.648
77.714
77.767
81.689
84.586
88.467
88.571

0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000

Figure. Nasdaq Composite index autocorrelations for log levels and log differences (returns)

Denition. Time series y , t = 1, . . . , T is covariance stationary

t

if
E[yt ]

, for all t

cov[yt , yt+k ]

k , for all t

var[yt ] = 0 (< ), for all t

Any series that are not stationary are said to be nonstationary.
Denition Times series yt is said to be integrated of order d,
denoted as yt I(d), if d yt is stationary. Note that if yt is
stationary then yt = 0 yt . Thus for short a stationary series is
denoted as yt I(0), i.e., integrated of order zero.
2

Below are results after tting an AR(1) and an MA(1)

model to the return series
Table. AR(1) estimates.
Dependent Variable: DNSDQ
Method: Least Squares
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 2 iterations
Variable Coecient Std. Error t-Statistic
C
0.086126
0.023048
3.736845
AR(1)
0.105933
0.018782
5.640001
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
Durbin-Watson stat
Inverted AR Roots

0.011221
0.010868
1.091357
3338.542
-4224.341
1.997947
.11

Prob.
0.0002
0.0000

Mean dependent var

S.D. dependent var
Akaike info criterion
Schwarz criterion
F-statistic
Prob(F-statistic)

0.086119
1.097336
3.013434
3.017668
31.80961
0.000000

Table. MA(1) estimates

Dependent Variable: DNSDQ
Method: Least Squares
Sample: 2276 5080
Included observations: 2805
Convergence achieved after 4 iterations
Variable Coecient Std. Error t-Statistic
C
0.086153
0.022811
3.776796
MA(1)
0.107093
0.018779
5.702685
R-squared
Adjusted R-squared
S.E. of regression
Sum squared resid
Log likelihood
Durbin-Watson stat
Inverted MA Roots

0.011323
0.010970
1.091301
3338.198
-4224.196
2.000051

Prob.
0.0002
0.0000

Mean dependent var

S.D. dependent var
Akaike info criterion
Schwarz criterion
F-statistic
Prob(F-statistic)

-.11
3

0.086119
1.097336
3.013331
3.017565
32.10153
0.000000

Both models give virtually equally good t, MA(1)

only just marginally better. The residual autocorrelations and related Q-statistics indicate no further autocorrelation left to the series.

Correlogram of Residuals

Correlogram of Residuals Squared

Date: 04/01/01 Time: 15:27

Sample: 2276 5080
Included observations: 2805
Q-statistic probabilities adjusted for 1 ARMA term(s)

Sample: 2276 5080

Included observations: 2805
Q-statistic probabilities adjusted for 1 ARMA term(s)
Autocorrelation

Autocorrelation

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.000
0.001
0.009
-0.010
-0.007
-0.009
-0.023
0.003
-0.008
0.019
-0.011
0.044
0.054
0.001
0.020
-0.031
0.008
0.031
0.038
-0.009
-0.014
-0.045
-0.004
0.029
-0.007
0.011
0.031
-0.003
0.019
0.011
0.004
0.000
0.041
-0.033
-0.034
0.003

PAC
0.000
0.001
0.009
-0.010
-0.007
-0.009
-0.023
0.003
-0.008
0.020
-0.012
0.044
0.053
0.001
0.020
-0.031
0.010
0.032
0.042
-0.008
-0.013
-0.046
-0.005
0.030
-0.010
0.010
0.027
-0.004
0.019
0.008
0.000
-0.004
0.043
-0.027
-0.025
-0.001

Q-Stat

Prob

2.E-05
0.0009
0.2428
0.4990
0.6319
0.8445
2.3176
2.3375
2.5307
3.5945
3.9433
9.5255
17.620
17.624
18.779
21.455
21.634
24.341
28.389
28.622
29.145
34.754
34.794
37.100
37.220
37.533
40.340
40.373
41.442
41.793
41.831
41.831
46.528
49.557
52.794
52.814

0.976
0.886
0.919
0.959
0.974
0.888
0.939
0.960
0.936
0.950
0.574
0.128
0.172
0.174
0.123
0.155
0.110
0.056
0.072
0.085
0.030
0.041
0.032
0.042
0.051
0.036
0.047
0.049
0.059
0.074
0.093
0.047
0.032
0.021
0.027

Partial Correlation

AC
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

0.278
0.272
0.192
0.193
0.217
0.154
0.141
0.145
0.074
0.106
0.107
0.127
0.115
0.124
0.120
0.137
0.133
0.091
0.148
0.076
0.126
0.144
0.105
0.188
0.088
0.120
0.142
0.142
0.120
0.117
0.119
0.106
0.081
0.076
0.087
0.073

PAC
0.278
0.211
0.084
0.089
0.118
0.026
0.022
0.047
-0.039
0.023
0.040
0.051
0.028
0.047
0.032
0.045
0.037
-0.022
0.063
-0.031
0.040
0.065
0.001
0.100
-0.029
0.013
0.046
0.042
-0.014
0.018
0.023
-0.007
-0.013
-0.015
0.007
-0.013

Q-Stat

Prob

216.92
425.28
529.00
633.28
765.20
832.06
888.03
947.14
962.58
994.10
1026.5
1072.0
1109.1
1152.2
1192.7
1245.3
1295.1
1318.5
1380.4
1396.7
1441.8
1500.7
1531.8
1632.3
1654.0
1694.7
1751.8
1808.9
1849.5
1888.4
1928.4
1960.4
1978.9
1995.3
2016.9
2031.9

0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000
0.000

Figure. Autocorrelations of the squared MA(1) residuals

Figure. Autocorrelations of the squared MA(1) residuals

Recall, however, that uncorrelatedness does not imply

independence. The autocorrelations of the squared
residuals strongly suggest that there is still left time
dependence in the series. Because squared residuals
are the building blocks of the variance of the series (recall: the sample variance of n observations x1, x2, . . . xn

is s2 = ni=1(xi x
)2/(n 1)), the results suggest that
the variation (volatility) of the series is time dependent.

This leads to the so called ARCH-family of

models.
4

1.2 ARCH-models
The general setup for ARCH models is
yt = xt + ut
with xt = (x1t, x2t, . . . , xpt) , = (1, 2, . . . , p) ,
t = 1, . . . , T , and
ut|Ft1 N (0, ht),
where Ft is the information available at time
t (usually the past values of ut; u1, . . . , ut1),
and
ht = var(ut |Ft1) = + 1u2t1 + 2u2t2 + + q u2tq .

Furthermore, it is assumed that > 0 and

i 0 for all i (to ensure positive variance),
and 1 + +q < 1 (to get stationarity). For
short the model is denoted ut ARCH(q).
The

inventor of this modeling approach is Robert

F. Engle (1982). Autoregressive conditional heteroscedasticity with estimates of the variance of
United Kingdom ination. Econometrica, 50, 987{
1008.
5

Properties of ARCH-processes
Consider (for the sake of simplicity) ARCH(1)
process
ht = + u2
t1
with > 0 and 0 < 1 and ut|ut1
N (0, ht).
(a) ut is white noise
(i) Constant mean (zero):
E[ut] = E[Et1[ut]] = E[0] = 0.


,
=0

Note Et1[ut] = E[ut|Ft1], the conditional

expectation given information up to time t
1.

The law of iterated expectations: Consider time points t < t

1
2

such that Ft1 Ft2 , then for any t3 > t2

Et1 [Et2 [ut3 ]] = E [E[ut3 |Ft2 ]|Ft1 ] = E[ut3 |Ft1 ] = Et1 [ut3 ],

which impies in particular E [Et2 [ut3 ]] = E[ut3 ].

6

(ii) Constant variance: Using again the law

of iterated expectations, we get

=
2
2
2
u := var[ut] = E[ut ] = E Et1[ut ]
= E[ht] = E[ + u2
t1]
= + E[u2
t1] = + E[ht1]
= + ( + E[u2
t2])
= (1 + ) + 2E[u2
t2]

..
= (1 + + 2 + + n)

]
+ n+1E[u2

, tn11
n
p 0, as 
Q
n
i
= limn i=0

.
= 1
(iii) Autocovariances: cov(ut, ut+k ) = 0 for
all k = 0, because using again the law of
iterated expectations we get

cov(ut, ut+k ) = E[utut+k ] = E(Et+k1(utut+k ))

= E[utEt+k1(ut+k )) = E(ut 0)
= 0.
7

(b) The unconditional distribution of ut is

symmetric, but nonnormal.
(i) Skewness: Exercise, show that E[u3
t ] = 0.
(ii) Kurtosis: Exercise, show that under the
assumption ut|ut1 N (0, ht), and that <

1/3, the kurtosis
2
2

E[u4

.
t] = 3
(1 )2 1 32

Because (1 2)/(1 32) > 1 we have that

2

2 = 3 4,
=
3[var(u
)]
E[u4
t
t] > 3
u
(1 )2

we nd that the kurtosis of the unconditional

distribution exceed that what it would be,
if ut were normally distributed. Thus the
unconditional distribution of ut is nonnormal
and has fatter tails than a normal distribution
with variance equal to u2 = /(1 ).
8

(c) Analogy with AR-processes

Consider again the specication of the variance in a general ARCH(q) process:
ht = var(ut |Ft1) = + 1u2t1 + 2u2t2 + + q u2tq .

This reminds essentially of an AR(q) process

for the squared residuals, because dening
t = u2
t ht, we can write
2
2
2
u2
t = + 1ut1 + 2ut2 + + q utq + t.

The expected value of t is zero, but var(t)

is time dependent (see below), implying that
this is not a stationary process in the sense
dened above. This implies that the conventional estimation procedure in AR-estimation
does not produce optimal results here.

(d) Standardized variables

Write
ut
zt =
ht
then zt|Ft1 N(0, 1) implying zt NID(0, 1),
i.e., normally and independently distributed.
Thus we can always write
0

ut = zt ht,
where the zt are independent standard normal
random variables (strict white noise). This
gives us a useful device to check after tting
an ARCH model the adequacy of the specication: Check the autocorrelations of the
squared standardized series.
Returning to the analogy with AR-processes, we note
that using standardized variables we may write
t = u2t ht = zt2ht ht = ht (zt2 1).

The factor on the right hand side is identically and

independently distributed (iid), that is, in particular
independent of ht , with zero mean and variance 2 (because zt2 is iid 2(1)). So t has zero mean too, but
time-dependent variance, because ht is time-dependent.
10

Estimation of ARCH models

Given the model
yt = xt + ut
with ut|Ft1 N (0, ht), we have yt|{xt, Ft1}
N (xt, ht), t = 1, . . . , T with conditional pdf
1
f (yt|Ft1) =
2ht

(yt xt )2
2h
t
e

T

The log-likelihood function is then () = t=1 t ()

with
1
1
1
log ht (yt xt )2/ht,
t () = log(2)
2
2
2
where = ( , , ) .
The maximum likelihood (ML) estimate ^ is the value
maximizing the likelihood function, i.e.,
(^) = max ().

The maximization is accomplished by numerical methods.

Note: OLS estimates of the regression parameters are
inecient (unreliable) compared to the ML estimates.

11

Variance Forecasting with ARCH models

Consider again for simplicity the ARCH(1) process
with ht = + u2t1. The expected value at time t
of the variance k time steps ahead is
Et (u2t+k ) = Et Et+k1(u2t+k ) = Et (ht+k ) + Et(u2t+k1).
We apply this recursively to get:
Et (u2t+1) = + Et (u2t ) = + u2t = ht+1
Et (u2t+2) = + Et (u2t+1) = + ht+1
Et (u2t+3) = + Et (u2t+2) = + ( + ht+1)
= (1 + ) + 2ht+1
...
Et (u2t+k ) = (1 + + . . . + k2) + k1ht+1
1 k1
+ k1ht+1
=
1
k1

+ k1ht+1
=
1
1 w
W

+ k1 ht+1
=
1
1
k

= u2 + k1(ht+1 u2) u2.

We note that for long forecast horizons k, the forecasts of the conditional variance ht+k approach the
unconditional variance u2 = /(1 ).
12