Lecture 4
Maximum likelihood and generalized least squares
estimators
Irene Mammi
irene.mammi@unive.it
1 / 19
outline
I References:
I Johnston, J. and J. DiNardo (1997), Econometrics Methods, 4th
Edition, McGraw-Hill, New York, Chapter 5.
2 / 19
MLEs in a nutshell
I let the joint density be written f (y; θ): this density may either
indicate, for given θ, the probability of a set of sample outcomes, or it
may be interpreted as a function of θ, conditional on a set of sample
outcomes
I in the latter interpretation it is referred as likelihood function:
3 / 19
MLEs in a nutshell (cont.)
4 / 19
ML estimation of the linear model
y = X β + u with u ∼ N (0, ff 2 I )
1 2 0
f (u ) = e −(1/2σ )(u u )
(2πσ2 )n/2
I so that the multivariate density for y conditional on X is
∂u
f (y |X ) = f (u )
∂y
5 / 19
ML estimation of the linear model (cont.)
I the log-likelihood function is
n n 1
` = ln f (y |X ) = ln f (u ) = − ln 2π − ln σ2 − 2 u 0 u
2 2 2σ
n n 1
= − ln 2π − ln σ − 2 (y − X β)0 (y − X β)
2
2 2 2σ
I The vector of unknown parameters, θ, has k + 1 elements, namely,
θ0 = β0 , σ2
∂` 1
= − 2 (−X 0 y + X 0 X β)
∂β σ
∂` n 1
2
= − 2 + 4 (y − X β ) 0 (y − X β )
∂σ 2σ 2σ
6 / 19
ML estimation of the linear model (cont.)
β̂ = (X 0 X )−1 X 0 y
and
σ̂2 = (y − X β̂)0 (y − X β̂)/n
I the MLE, β̂ is the OLS estimator, b, and σ̂2 is e 0 e/n with e being
OLS residuals
I the max of the likelihood function is L( β̂, σ̂2 ) = constant · (e 0 e )−n/2
7 / 19
ML-based tests
H0 : R β = r
8 / 19
ML-based tests (cont.)
L( β̃, σ̃2 )
λ=
L( β̂, σ̂2 )
I a generally applicable large-sample test is
a
LR = −2 ln λ = 2[ln L( β̂, σ̂2 ) − ln L( β̃, σ̃2 )] ∼ χ2 (q )
LR = n(ln e 0∗ e ∗ − ln e 0 e )
9 / 19
ML-based tests (cont.)
10 / 19
ML-based tests (cont.)
(R β̂ − r )0 [R (X 0 X )−1 R 0 ]−1 (R β̂ − r ) a 2
W = ∼ χ (q )
σ̂2
which can also be expressed as
n(e 0∗ e ∗ − e 0 e ) a 2
W = ∼ χ (q )
e0e
11 / 19
ML-based tests (cont.)
Lagrange multiplier (LM) test
I the LM test, also known as the score test, is based on the score vector
∂ ln L ∂`
s (θ) = =
∂θ ∂θ
I the unrestricted estimator, θ̂, is found by solving s (θ̂) = 0; the score
vector will in general not be zero when evaluated at θ̃, the restricted
estimator
I if the restrictions are valid, the restricted maximum, `(θ̃), should be
close to the unrestricted maximum, `(θ̂), and so the gradient of the
former should be close to zero
I under the null hypothesis,
a
LM = s 0 (θ̃)I −1 (θ̃)s (θ̃) ∼ χ2 (q )
12 / 19
ML-based tests (cont.)
LM = nR 2
13 / 19
ML-based tests (cont.)
14 / 19
ML estimation with nonspherical disturbances
y = X β + u with u ∼ N (0, σ2 Ω)
15 / 19
ML estimation with nonspherical disturbances (cont.)
I the multivariate normal density for u is
1
f (u ) = (2π )−n/2 |σ2 Ω|−1/2 exp − u 0 (σ2 Ω)−1 u
2
which, noting that |σ2 Ω| = σ2n |Ω|, can be rewritten as
and
1
σ̂2 = (y − X β̂)0 Ω−1 (y − X β̂)
n
16 / 19
generalized least squares
Ω −1 = P 0 P
y ∗ = X ∗ β + u∗
17 / 19
generalized least squares (cont.)
I since Ω = P −1 (P 0 )−1 , we have
var(u ∗ ) = E(Puu 0 P 0 )
= σ2 PΩP 0
= σ2 PP −1 (P 0 )−1 P 0
= σ2 I
b GLS = (X 0∗ X ∗ )−1 X 0∗ y ∗
= (X 0 Ω −1 X ) −1 X 0 Ω −1 y
H0 : R β = r
can be based on
19 / 19