Mle Testing

Basic Concepts
The Trilogy
Application: Test for Heteroskedasticity
ML Based Hypothesis Testing

Walter Sosa-Escudero
Econ 507. Econometric Analysis. Spring 2009
April 21, 2009
Basic Concepts
The Trilogy
Basic Concepts
Z f (y; ), <K .
is a vector of K parameters.
is the parameter space: set of values can take.
A hypothesis is a statement about . The goal is to learn
something about the validity of the hypothesis, based on a sample.
If H0 : 0 , the hypothesis is nested if 0 . In general,
they are restriction hypotheses (eg., = 0, = <, 0 = {0}).
Basic Concepts
The Trilogy
A test statistic is a random variable that has one particular

distribution when H0 is true and some other one when it is
false.
The alternative hypothesis, HA , is a subset of all the values
can take when H0 is false.
Example: K = 1, = <, H0 : = 0, HA : > 0.
The rejection region is the set of all value the test statistic
can take for which the null is rejected.
A test is a test statistic combined with a rejection region.
Basic Concepts
The Trilogy
The probability that the test rejects H0 when it is true (type-I

error) is the level of the test. It depends strictly on the null
hypothesis.
The power of a test is the probability that the test rejects H0 .
When H0 is false, this probability depends on HA .
The type-II error is one minus the power (do not reject when
H0 is false).
In general there is a trade-off between type I and II errors: a
test with level 0 (always accepts) tends to have zero power
(never reject).
The classical approach (Neyman-Pearson) proposes to set a
level exogeneously and find test that maximize power for some
relevant alternative hypothesis.
A test is consistent if the power tends to one when H0 is false
and the sample size tends to infinity.
Basic Concepts
The Trilogy
Non-linear Hypothesis
H0 : h() = 0, h() : <K <r ,
h() is a vector of r restrictions on the K parameters.
In this case = <k , and

0 = | h() = 0 , 0
so the hypothesis is nested.
Let D(), be an (r k) matrix with (i, j) element:
D()ij =
hi ()
j
and hi () denotes the ith element of h(), i = 1, . . . , r. We will

assume (D()) = r
Basic Concepts
The Trilogy
Example: = (1 , 2 , 3 )0 . Consider H0 : h() = 0, with

1 2
h() =
2 3 1
This is equivalent to H0 : 1 = 2 , 2 3 = 1. There are r = 2
restrictions on K parameters.
In this case:

D() =
1 1 0
0 3 2
Basic Concepts
The Trilogy
The nested testing problem implies two estimators.

The unrestricted MLE, is defined as
argmax l(; z)
and under suitable regularity conditions, the FOCs are:

z) = 0
s(;
Basic Concepts
The Trilogy
Similarly, the restricted MLE, R is defined as

argmax l(; z)
0
The lagrangean function for the restricted maximization problem is

LG(, ) = l(; z) h()
and the FOCs are:
= 0
s(R ; z) D(R )0
h(R ) = 0
Basic Concepts
The Trilogy
Three Tests
Assume Vn is a consistent estimator of J. Consider the following
test statistics for H0 : h(0 ) = 0.
1
Likelihood Ratio:
"
z) l(R ; z)
l(;
LR = n 2
n
n
2
Wald:
h
i1
n ()
1 D()
0
0 D()V
h()
W = n h()
Score (or Lagrange Multiplier):

!0
s(R
LM = n
Vn1 (R )
n
s(R
n
Basic Concepts
The Trilogy
Result: Under all the regularity assumptions, if Zi f (z; 0 ),

i = 1, 2, . . . , n, under H0 : h(0 ) = 0, LR, W and LM converge in
distribution to 2 (r) and they diverge to infinity under
HA : h(0 ) = , for any constant 6= 0.
The three tests are asymptotically equivalent.

The three tests are consistent.
Basic Concepts
The Trilogy
Intuition: is always consistent for 0 . R is consistent for 0 only

if H0 holds. Then, under H0 :
l(R )
and R are consistent, then l()
= 0 (LR).
h() = 0 (W)
The shadow price of imposing the restriction is zero:
= 0.
From the first FOC:
s(R ; z) D(R )0 = 0
Since (H()) = r, then s(R )
= 0 (score or LM).
The idea is to reject H0 when LR,W or LM are large.
Basic Concepts
The Trilogy
A graphical representation
Consider the following simplification

< (K = 1)
H0 : = 0 (a simple case).
Basic Concepts
The Trilogy
Asymptotic Distributions under H0
The plan:
1
we
We will start with the Wald test since it depends on :
have already proved its asymptotic properties, in particular
n( 0 ) N (0, J 1 )
LR and LM depend on R . We only know it is consistent.

Then first we need to establish its asymptotic behavior.
Basic Concepts
The Trilogy
Wald Test
h
i1
0 D()V
1 D()
0
h()
W = n h()
n
around 0 :
Take a mean value expansion of h()

= h(0 ) + D()
0
h()
where is a mean value between and 0 . Again, consistency
guarantees the exactness of the approximation.
Under H0 : h(0 ) = 0

= D()
n 0
n h()
Basic Concepts
The Trilogy
By asymptotic normality of

d

n h()
N 0, D(0 )J 1 D(0 )0
p
p
p
and D()

since D()
D(0 ), Vn J and 0 , by Slutzkys
theorem:

d
1 D()
0

N 0, D()V
n h()
n
Then, taking the normed quadratic form:

i1
h
d
0 D()V
1 D()
0

h()
2 (r)
W = n h()
n
Basic Concepts
The Trilogy
AN of the restrited MLE
To prove the other two cases, we need two additional results:

1
n(R 0 ) N (0, V (0 )1 )
, where V (0 )1 J 1 J 1 D0 [DJ 1 D0 ]1 DJ 1 ,
D D(0 )
2
1
s(0 ) d

' R(0 )1 DJ 1
N (0, R(0 )1 ),
n
n
with R DJ 1 D0 .
Basic Concepts
The Trilogy
Proof: from the FOCs

R ; z)
1 s(
n
R )0
1 D(
n
n h(R )
= 0
(1)
= 0
Basic Concepts
The Trilogy
Take a mean value expansion around 0 for both equations:

R 0 )
s(R ; z) = s(0 ; z) + H()(
H()
1
1
s(R ; z) = s(0 ; z) + n
(R 0 )
n
n
n
1
= s(0 ; z) n J (R 0 ) (Asymptotically, Why?) (2)
n
R 0 )
n h(0 ) + n D()(
=
n D(0 )(R 0 )
(Asymptotically, Why?) (3)
n h(R ) =
Basic Concepts
The Trilogy
Now replace (2) and (3) in (1) and use the fact that under H0
p
D(R ) D(0 ).
R 0 ) 1 D0
= 0
1 s(0 ; z
n
J
(
n
n
n D(R 0 ) = 0
Solving the system (do it as homework):
1
n (R 0 ) = A s(0 ; z)
n
with A J 1 J 1 D0 R1 DJ 1
Basic Concepts
The Trilogy
Note that by our asymptotic normality result:

Pn
s(0 ; z)
s(0 , yi ) d
1
s(0 ; z) = n
= n i=1
N (0, J)
n
n
n
Then using Slutzkys theorem,
1
d
n(R 0 ) = A s(0 ; z) N (0, A J A0 )
n
which is the desired result.
We leave as homework to prove the similar result for .
Basic Concepts
The Trilogy
LM/Score
1 0 1
s(R ) V (R ) s(R )
n
From the FOC of the restricted MLE problem:
LM =
1
1
=0
s(R ; z) D(R )0
n
n
Replacing:
0
d
D(R )V (R )1 D(R )0
' R(0 )
2 (r),
n
n
n
n
d
n
since /
N (0, R()1 ), and it is a normed quadratic form.
Basic Concepts
The Trilogy
LR Test
h
i
z) l(R ; z)
LR = 2 l(;
Consider a second order mean value expansion of (R ) around ,

under H0 :
Supress dependence on z
momentarily to simplify notation

0

+ s()
0 R + 1 R H()
R
l(R ) = l()
2

0

1
R
=
R H()
(Why?)
l(R ) l()
2
Basic Concepts
The Trilogy
Replacing and dividing and multiplying by n:

0 1

LR = n R
H()
n R
n
0

'
(Why?)
n R I(0 ) n R
Recall
s(0 ; z)
n ( 0 ) ' J 1
n
n (R 0 ) '
J 1 J 1 D0 R1 DJ 1
s(0 ; z)
Substracting both sides:

s(0 )
n R ' J 1 D0 R1 DJ 1 = J 1 D0
n
n
Basic Concepts
The Trilogy
Replacing above:
LR '
'
0
0
DJ 1 JJ 1 D0
n
n
0
0
DJ 1 D0
n
n
d
' LM 2 (r)
Basic Concepts
The Trilogy
Consistency of LR, LM and W

Now HA : h(0 ) 6= 0. Assume HA : h(0 ) = 6= 0.
h
i1
0 D()V
1 D()
0
n h()
h()
n

1
n 0 D(0 )Vn1 D(0 )0
Note that under HA , plim s(R ; z)/n 6= 0 (Why?). Then:
LM
1
s(R ; z)0 V (R )1 s(R ; z)
n
s(R ; z)0 1 s(R ; z)0
V ( R )
= n
n
n
Basic Concepts
The Trilogy
We can use a similar argument with LR

#
"
i
z) l(R ; z)
l(
;
z) l(R ; z) = n 2
LR = 2 l(;
n
n
|
{z
}
h
6=0
(Why: recall the consistency proof...)
Basic Concepts
The Trilogy
Consistency
Under H0 , W, LM and LR have asymptotic 2 (r) distribution.
For a level , the acceptance region is [0, z ], where z is the
1 quantile of 2 (r), a finite number.
Hence, under our HA , W, LM and LR , hence lie outside
the acceptance region wpt1: they are consistent.
Basic Concepts
The Trilogy
An application: A test for heteroskedasticity

Recall that the Breusch/Pagan test for heteroskedasticity is based
on the following steps:
1
2
Estimate by OLS, and save squared residuals e2i .

Regresss e2i on the Zik variables, k = 2, . . . , p and get (ESS).
The test statistic is:
1
ESS 2 (p 1) 2 (p)
2
under H0 , asymptotically. We reject if it is too large.
This is an LM based test. The goal is to derive it from basic

principles (likelihood).
Basic Concepts
The Trilogy
Setup:
yi = x0i + ui
1
xi is a non-stochastic vector of K explanatory variables,

including an intercept.
ui N (0, i2 ) (possible heteroskedasticity)
i2 = h(1 + 2 z2i + 3 z3i + . . . + p zpi ).
h(.) is any positive, twice differentiable function.
zi is a vector of p non-random variables.
Homoskedasticity: H0 : 2 = 3 = = p = 0
Basic Concepts
The Trilogy
Preliminary results I: LM tests for subvectors
In this setup
= (, 1 , 2 , . . . , p )0 .
The null of homskedasticity, H0 : 2 , . . . , p = 0 involves a
subvector of , say, all the other parameters are free.
The LM test can be simplified in this situation
Basic Concepts
The Trilogy
Remember that the null h() = 0 can be tested based on:

LM = n s(R )0 Vn1 s(R )
where Vn is consistent for J and R is the restricted MLE.
Consider now the case:
= (1 2 )0 , H0 : 2 = 20 , 2 is r 1.
Note that:
H0 :
0 Ir
1
2
{z
h()

2o = 0
}
with D() = [0 Ir ].
Basic Concepts
The Trilogy
Similarly

s1 ()
s()
,
s1 ()

Vn () =
V11 V12
V21 V22

,
Vn1 ()
V 11 V 12
V 21 V 22
Basic Concepts
The Trilogy

1
1
Result (Theil, 1971, pp. 18): V 22 = V22 V21 V11
V12 .
1
In particular, if V21 = 0 = V 22 = V22
.
FOC of the restricted MLE problem:
as:
s1 (1 , 2o )
s ( , )
2 1 2o
2
in this case, R = (1 , 2o ) so
=
=
=
2o
Basic Concepts
The Trilogy
Reeplacing and simplifying in the LM formula:

s2 (1 ; 2o )
LM = n s2 (1 ; 2o )0 V 22 ()
If V21 = 0
s2 (1 ; 2o )
LM = n s2 (1 ; 2o )0 V22 ()
Basic Concepts
The Trilogy
Preliminary results II: some OLS algebra

Y = X + u, XnK including intercept.
ESS =
(Yi Y )2 =
Yi2 N Y 2 =
X
Yi2 (1/N )(
Yi )2
In matrix form:
SCE = Y 0 Y (i0 Y )2 /N, con i = (1, 1, . . . , 1)0
But:
Y 0 Y = Y 0 X(X 0 X)1 X 0 X(X 0 X)1 X 0 Y = Y 0 X(X 0 X)1 X 0 Y
|
{z
}|
{z
}
(X 0 )
replacing: SCE = Y 0 X(X 0 X)1 X 0 Y (i0 Y )2 /N

Basic Concepts
The Trilogy
The Breusch-Pagan Test

Let
ei , OLS residuals.
P
2 = (1/N ) e2i , MLE of 2 under H0 .

gt e2i /
2.
Test (Breusch-Pagan): Under H0 :
1
SCEg,z 2 (p 1)
2
= ESS of regressing gi on zi .
LM =
ESSg,z
Basic Concepts
The Trilogy
Proof: yi N (x0i , i2 ), i2 = h(zi0 )

2
1
1X
1X
l(, ) = N ln(2)
ln i2
(1/i2 ) yi x0i
2
2
2
Note
s(; x
, z) =
and
s(; x
, z) =
X 1
(yi x0i )xi
i2
1 X h0i
1 X h0i
zi
zi
2
hi
2
h2i
with hi h(zi0 ), h0i h(zi0 )/.
Basic Concepts
The Trilogy
Let 0 and 0 denote the true values under the null. Also note
that 0 = (1 , 0, . . . , 0)0 .
From the previous result, it is easy to check that J is block
diagonal, that is
2
l
J, = E
=0
when evaluated at the true values under the null. Then, since our
H0 only involves the components of , according to our previous
result, a test can be based on
LM =
1
1
s (R ; z, x
)0 J
s (R ; z, x
)
n
Basic Concepts
The Trilogy
Let denote the MLE of under the null.
in this case?
What it
Under H0 of homoskedasticity, let

2 denote the MLE of
V (ui ) = h(1 ). By the invariance property, the MLE of 1 (
1 ) is
h
defined implicitely by
(
1 ).
It is easy to verify
s (R ) =
and

2

1 h0 (
1 ) X
ei
z
1
i
2

l(; x
, z)
1 h0 (
1 ) X 0
J =
=
zi zi
0 =R
2
Basic Concepts
The Trilogy
Replacing and simplifying:

LM =
0 X
1 X

1 X
zi fi
zi zi0
zi fi
2
with fi e2i /
2 1 = gi 1. In matrix terms:
1
1
LM = f 0 Z(Z 0 Z)1 Z 0 f = f 0 PZ f
2
2
with f = g i, i is a vector of n ones.
Basic Concepts
The Trilogy
Note
i0 g =
PZ i = i
2 = n.
e2i /
(Why?)
Then
LM
= 1/2 f 0 PZ f
= 1/2 (g i)0 PZ (g i)

= 1/2 g 0 PZ g i0 PZ g g 0 PZ i + i0 PZ i

= 1/2 g 0 PZ g n

= 1/2 g 0 PZ g (i0 g)2 /n
= 1/2 ESSg,z
Basic Concepts
The Trilogy
On the small sample performance
Key result: under all the assumptions and

H0 : LMbp 2 (p 1), asymptotically.
This result is used to determine the acceptance region, for an
exogenously set level .
How reliable is this this result?
A key assumption is normality.
Basic Concepts
The Trilogy
Evans (1992): a Monte Carlo exploration

Setup:
yi = x0i + ui
xi is generated using a log-normal and a uniform distribution
V (ui ) = 2 (1 + zi ).
2000 replications for alternative disributions for ui .
n = 64, = 0.05.
Tests: Breusch-Pagan (BPTrue), Modified BP (BPasym),
Koenker (BPmod), White.
Basic Concepts
The Trilogy
The size is severly affected by non-normalities.

Mle Testing

Diunggah oleh

Informasi Dokumen

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Mle Testing

Diunggah oleh

Hak Cipta:

Format Tersedia

Basic Concepts

ML Based Hypothesis Testing

April 21, 2009

ML Based Hypothesis Testing

ML Based Hypothesis Testing

A test statistic is a random variable that has one particular

ML Based Hypothesis Testing

The probability that the test rejects H0 when it is true (type-I

ML Based Hypothesis Testing

and hi () denotes the ith element of h(), i = 1, . . . , r. We will

ML Based Hypothesis Testing

Example: = (1 , 2 , 3 )0 . Consider H0 : h() = 0, with

ML Based Hypothesis Testing

The nested testing problem implies two estimators.

and under suitable regularity conditions, the FOCs are:

ML Based Hypothesis Testing

Similarly, the restricted MLE, R is defined as

The lagrangean function for the restricted maximization problem is

ML Based Hypothesis Testing

Score (or Lagrange Multiplier):

ML Based Hypothesis Testing

Result: Under all the regularity assumptions, if Zi f (z; 0 ),

The three tests are asymptotically equivalent.

ML Based Hypothesis Testing

Intuition: is always consistent for 0 . R is consistent for 0 only

ML Based Hypothesis Testing

Consider the following simplification

ML Based Hypothesis Testing

Asymptotic Distributions under H0

LR and LM depend on R . We only know it is consistent.

ML Based Hypothesis Testing

ML Based Hypothesis Testing

Then, taking the normed quadratic form:

ML Based Hypothesis Testing

AN of the restrited MLE

To prove the other two cases, we need two additional results:

ML Based Hypothesis Testing

Proof: from the FOCs

ML Based Hypothesis Testing

Take a mean value expansion around 0 for both equations:

ML Based Hypothesis Testing

Solving the system (do it as homework):

ML Based Hypothesis Testing

Note that by our asymptotic normality result:

which is the desired result.

We leave as homework to prove the similar result for .

ML Based Hypothesis Testing

ML Based Hypothesis Testing

Consider a second order mean value expansion of (R ) around ,

ML Based Hypothesis Testing

Replacing and dividing and multiplying by n:

Substracting both sides:

ML Based Hypothesis Testing

ML Based Hypothesis Testing

Consistency of LR, LM and W

Note that under HA , plim s(R ; z)/n 6= 0 (Why?). Then:

ML Based Hypothesis Testing

We can use a similar argument with LR

(Why: recall the consistency proof...)

ML Based Hypothesis Testing

ML Based Hypothesis Testing

An application: A test for heteroskedasticity

Estimate by OLS, and save squared residuals e2i .

This is an LM based test. The goal is to derive it from basic