Anda di halaman 1dari 15

1

Heteroscedasticity

Assumption of CLRM:-

The error term are homoscedastic [ n i E
i
,..., 2 , 1 ; ) (
2 2
= = o ]
the variance of each
i
c is constant

For example: Given
i i o i
X Y c | | + + =
1
, as income (x) increase mean value of savings
(Y) increase but variance of savings remains constant.

Remember: Heteroscedasticity more commonly found in cross-sectional rather than time
series data, because cross sectional data usually deals with members of population at a
given point in time (small, medium @ large firms) scale effect in cross-sectional data.

When assumption 3 holds,
i.e. the errors u
i
in the regression equation have common variance (ie
constant or scalar variance)
then we have homoscedasticity.
or a scalar error covariance matrix

When assumption 3 breaks down, we have what is known as heteroscedasticity.
or a non-scalar error covariance matrix

Homoskedasticity => variance of error term constant for each observation












Each one of the residuals has a sampling distribution, each of which
should have the same variance -- homoscedasticity




scalar a is e wher
0 0
0 0
0 0

) var( ) , cov( ) , cov(
) , cov( ) var( ) , cov(
) , cov( ) , cov( ) var(
) ,.... , cov(
2
2
2
2
2 1
2 2 1 2
1 2 1 1
2 1
o
o
o
o
(
(
(
(
(

=
(
(
(
(

n n n
n
n
n
u u u u u
u u u u u
u u u u u
u u u
2
Generalized Least Squared Estimation

For general linear statistical model

e X Y + = |

where
E[e]=0

Heteroscedasticity exists when diagonal element of + are not all identical

+ =
(
(
(
(
(

= u = '
2
2
2
2
2
1
0 0
0 0
0 0
] [ o
o
o
o
T
e e E




GLS for | (BLUE)

y X X X
1 1 1
' ) ' (


u u = |


where + = u
2
o , with
2
o unknown and + known

These 2 estimators are same because

y X X X
y X X X
y X X X
1 1 1
2
1
1
2
1
1 1 1
' ) ' (
' '
' ) ' (


=
+
(
(

+
= u u
o o



If
1
'

u = P P , then

* *' *) *' (
' ' ) ' ' (

1
1
y X X X
Py P X PX P X

=
= |


where
PX X = *
Py y = *
The tth observation for whole model can be written as

t
t
t
t
t
t
e X y
o
|
o o
+ =
'


3
The variance of transform disturbance
t t t
e e o /
*
= constant

1 ] [
1
] [
2
2
2
2
2
2
*
= = =
(
(

|
|
.
|

\
|
=
t
t
t
t t
t
t
e E
e
E e E
o
o
o o


heteroscedasticity error model the GLS estimator obtained by
(Weighted Least Squares):-

(a) Divide each observation (dependent & independent) by standard deviation of eror
term for that observations
(b) Apply usual LS procedures to transformed observations


Recall:

GLS estimator is | that minimizes
) ( )' (
1
2
1
| |
o
X y X y
e
T
t
t
t
u =
|
|
.
|

\
|

=

|
|
.
|

\
|
=
T
t
t t t
T
t
t t t
y x x x
1
2
1
1
' 2

o o |

Covariance matrix for |


















1
1
' 2 1 1 1
) ' ( *) *' (


|
|
.
|

\
|
= u =
|
o
T
t
t t t
x x X X X X
4
(a) Causes (Why variance of
i
vary?)

(i) Omitted Variables

Suppose the true model of y is:
y
i
= a + b
1
x
i
+ b
2
z
i
+ u
i

but the model we estimate fails to include z:
y
i
= a + b
1
x
i
+ v
i

then the error term in the model estimated by SPSS (v
i
) will be capturing the effect of the
omitted variable, and so it will be correlated with z:
v
i
= c

z
i
+ u
i

and so the variance of v
i
will be non-scalar



(ii) Non-constant coefficient

Suppose that the slope coefficient varies across i:
y
i
= a + b
i
x
i
+ u
i

suppose that it varies randomly around some fixed value b:
b
i
= b + e
i

then the regression actually estimated by SPSS will be:
y
i
= a + (b + e
i
)

x
i
+ u
i


= a + b x
i
+ (e
i
x
i
+ u
i
)
where (e
i
x + u
i
) is the error term in the SPSS regression. The error term will thus vary
with x.


(iii) Non-linearities

If the true relationship is non-linear:
y
i
= a + b

x
i
2
+ u
i

but the regression we attempt to estimate is linear:
y
i
= a + b

x
i
+ v
i

then the residual in this estimated regression will capture the non-linearity and its
variance will be affected accordingly:
v
i
= f(x
i
2
, u
i
)


(iv) Aggregation

Sometimes we aggregate our data across groups:
e.g. quarterly time series data on income = average income of a group of
households in a given quarter
if this is so, and the size of groups used to calculate the averages varies,
variation of the mean will vary
larger groups will have a smaller standard error of the mean.
5
the measurement errors of each value of our variable will be correlated
with the sample size of the groups used.
Since measurement errors will be captured by the regression residual
regression residual will vary the sample size of the underlying groups
on which the data is based.


Overall:

Mis-specification error
- wrong functional form
- Non-linearities
- Non-constant coefficient
- incorrect data transformation
- omitted variables
- Aggregation
Outlier
Improvement in data collecting technique
Errors of behavior become smaller over time


(b) Consequences:

OLS Estimators still linear and remain unbiased
The property of minimum variance no longer holds (not efficient)
Heteroskedasticity does, however, bias the OLS estimated standard errors for the
estimated coefficients:
which means that the t tests will not be reliable:
t = b
hat
/SE(b
hat
).
F-tests are also no longer reliable













6
Unbiased and Consistent Estimator



Biased but Consistent Estimator





Asymptotic Distribution of OLS Estimate
hat
The Estimate is Unbiased and Consistent since as the sample size increases, the mean of the
distribution tends towards the population value of the slope coefficient
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
-
4
-
3
.
7
-
3
.
3
-
3
-
2
.
6
-
2
.
3
-
1
.
9
-
1
.
6
-
1
.
2
-
0
.
9
-
0
.
5
-
0
.
2
0
.
2
0
.
5
5
0
.
9
1
.
2
5
1
.
6
1
.
9
5
2
.
3
2
.
6
5 3
3
.
3
5
3
.
7
4
.
0
5
4
.
4
4
.
7
5
5
.
1
5
.
4
5
5
.
8
6
.
1
5
6
.
5
6
.
8
5
7
.
2
7
.
5
5
7
.
9

|
hat
n = 1,000
n = 500
n = 300
n = 200
n = 150
Asymptotic Distribution of OLS Estimate
hat
The Estimate is Biased but Consistent since as the sample size increases, the mean of the
distribution tends towards the population value of the slope coefficient
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
-
4
-
3
.
7
-
3
.
3
-
3
-
2
.
6
-
2
.
3
-
1
.
9
-
1
.
6
-
1
.
2
-
0
.
9
-
0
.
5
-
0
.
2
0
.
2
0
.
5
5
0
.
9
1
.
2
5
1
.
6
1
.
9
5
2
.
3
2
.
6
5 3
3
.
3
5
3
.
7
4
.
0
5
4
.
4
4
.
7
5
5
.
1
5
.
4
5
5
.
8
6
.
1
5
6
.
5
6
.
8
5
7
.
2
7
.
5
5
7
.
9

|
hat
n = 1,000
n = 500
n = 300
n = 200
n = 150
7
(c) Detecting:

1. Graphical Examination of residuals (Informal Tests)

-plot residual square against y

-plot residual square against x


If we plot the residual against Rooms, we can see that its variance
increases with No. rooms:





























Number of rooms
14 12 10 8 6 4 2 0
U
n
s
t
a
n
d
a
r
d
i
z
e
d

R
e
s
i
d
u
a
l
300000
200000
100000
0
-100000
-200000
8
Formal Test
2. Whites General Hetero Test

Given the model:

i i i i
u X X Y + + + =
3 3 2 2 1
| | |

White Test Prosedure (basic idea):

(1) Estimate the model and obtain the residuals
]

[

3 3 2 2 1 i i i i i i i
u X X Y Y Y e + + + = = | | |

(2) Estimate the auxiliary regression

i i i i i i i i
v X X X X X X u + + + + + + =
3 2 6
2
3 5
2
2 4 3 3 2 2 1
2
o o o o o o

squared terms of all the Xs & cross products

(3) Obtain the R-squared from the auxiliary regression then can be used to
compute the test statistics

2
1
2
~

k
R n _

test-stat
2 2
R n = _
critical value f d.
2
_

k-1=degree of freedom

(4) H
0
= no heteroscedasticity (variance constant)
H
0
= heteroscedasticity
@
2 2
0
: o o =
i
H for all i.
0 1
: H Not H

If test-stat > critical value, then reject H
0
Hetero


If test-stat < critical value, then fail to reject H
0
No

Hetero







9

Or

The correct covariance matrix for the LS estimator is
1 1 2
] ][ [ ] [ ] | [

' O ' ' = X X X X X X X b Var
i
o and
1 2
] [

' = X X s V . Is there is no
heteroscedasticity, then V will give a consistent estimator of ] | [ X b Var .




3. Spearmans rank correlation test

Spearmans rank correlation

(
(

=

) 1 (
6 1
2
2
n n
d
r
i
s


where
d
i
= difference in ranks assigned to 2 different characteristics of phenomenon
n = number of phenomena ranked

Steps:
1) Fit the regression to data on Y and X and obtain residuals

2) Ignore sign of residuals, take & rank
i
u &
i
X (or
i
Y

) & compute
s
r


3) Assume population rank correlation coefficient = 0 & n>8 . Use
2
1
2
s
s
r
n r
t

= , df = n-2

If computed t value < critical t value, fail to reject homoscedasticity












10
4. Park test (strictly exploratory method)

The Park test is a test of the hypothesis:

H
0
= 0
1
= o [which is constant]
H
1
= 0
1
= o [hetero]

2 stage procedure:

(1) run OLS regression
i i
u X Y + + = ln

1 0
o o disregarding heteroscedasticity
question

(2) run
i i i
v X u + + = ln ln
2
| o to test particular which independent variable
causing hetero.


Park suggests

vi
i i
e X
|
o o
2 2
= or
i i i
v X + + = ln ln ln
2 2
| o o



Unknown, using
2

i
u as proxy

i i i
v X u + + = ln ln ln
2 2
| o
i i
v X + + = ln | o

If | insignificant, fail to reject assumption of homoscedasticity.
















11
5. Glejser test

-Similar in spirit to Park test

Step:
After obtain residuals from OLS regression, regress absolute values of residuals
on regressor variable that close associated with
2
i
o .

6. Goldfeld-Quandt test

-critic the error term may not satisfy OLS assumption (hetero) in Park test

-assumes the heteroscedastic variance positive related to one of regressors in
Model
2 2 2
i i
X o o = ------hetero if
2
i
o large when
i
X larger

-need to depend on number of central observation to be omitted & identify
correct regressor variable to order observation (on of the limitation of this test)

The Goldfeld-Quandt test is a test of the hypothesis:

H
0
= 0 ...
3 2
= = = =
m
o o o [
1
2
o o =
i
, which is constant]

H
1
=
2 2
2
1
2
...
T
o o o s s s

Steps:
1) rank observation according to the values of
i
X

2) omit c (specified prior) central observations, divide remaining (n-c)
into 2 groups each 2 / ) ( c n observations

3) fit separate OLS regressions to first and last observation, obtain
respective residual sums of squares RSS
1
and RSS
2



k
c n

2
) (
or df
k c n
2
) 2 (



4) compute

df RSS
df RSS
/
/
1
2
=

-If computed ) ( f = < critical F, fail to reject hypothesis of
homoscedasticity
RSS from regression
corresponding to smaller
i
X values
12
7. Breusch-Pagan-Godfrey test

The limitation of Goldfeld-Quandt test can be avoided by using BPG test.

If it known that a set of variables influence the error variance such as Z
1
, Z
2
,, Zm , we
can write:

mi m i i
Z Z o o o o + + + = ...
2 2 1
2


The Breusch-Pagan-Godfrey test is a test of the hypothesis:

H
0
= 0 ...
3 2
= = = =
m
o o o [
1
2
o o =
i
, which is constant]

Steps:
1) Estimate linear regression model by OLS, obtain residuals
2) Get

= n u
i i
/
~ 2 2
o
3) Construct variables
2 2 ~
/ o
i i
u p =
4) Regress
i mi m i i
v Z Z p + + + + = o o o ...
2 2 1

5) Obtain ESS from steps 4, define ] [
2
1
ESS = O
If computed O< critical value [ 1
2
m _ ], fail to reject hetero.























13
(d) Remedial Measures:

Weighted Least Square (WLS)

The GLS estimator is

y X X X
1 1 1
) (


O ' O ' = |

Consider
i i i
X Var e o o c
2 2
] | [ = = .
1
O is a diagonal matrix whose ith diagonal
element is
i
e
1
. The GLS estimator is obtained by regressing

(
(
(
(
(
(
(
(

=
n
n
y
y
y
Py
e
e
e

2
2
1
1
on
(
(
(
(
(
(
(
(

'
'
'
=
n
n
X
X
X
Px
e
e
e

2
2
1
1



Applying OLS to transformed model, we obtained the WLS estimator

], [ ] [

1
1
1

' =
=

=
n
i
i i i
n
i
i i i
y X w X X w | where
i
i
e
e
1
= .

For simplify version

- When
2
i
o
known

Given model:
i i
X Y c | | + + =
1 0


Assume the true error variance
2
i
o is know, then we can transform the model
(divide both side by error variance for this case):


i
i
i
i
i i
i
X Y
o
c
o
|
o
|
o
+
|
|
.
|

\
|
+
|
|
.
|

\
|
=
1 0
1


14
Is the
i
i
i
v =
o
c
homescedastic?
As we known
i
i
i
v
o
c
=
2
2
2
i
i
i
v
o
c
=
1
1
) ( ) (
2
2
2
2
2
=

|
|
.
|

\
|
=
=
i
i
i
i
i
E v E
o
o
o
c


Since
2 2 2
) ( ) var( o c = =
i i
E v (homoscedastic)





























15
- When
2
i
o
unknown

If the variance is proportional to X
2
, divide all variables by
i
X
2

i
i
i
i
i i
i
X X
X
X X
Y c
| | +
|
|
.
|

\
|
+
|
|
.
|

\
|
=
1 0
1


Is the
i
i
i
v
X
=
c
homescedastic?
As we known
i
i
i
X
v
c
=
i
i
i
X
v
2
2
c
=
2
2
2
2
) (
) (
o
o
c
=

|
|
.
|

\
|
=
=
i
i
i
i
i
i
X
X
X
E
v E


Since
2 2 2
) ( ) var( o c = =
i i
E v (homoscedastic)

If the variance is proportional to X
2
2
, divide all variables by X
2



Re-specification model

-Note: Hetero problem may be reduced as log transformation compresses the scale

Whites Heteroscedasticity Corrected Standard Error

-Take into consideration of hetero without changing the value of estimated
coefficient.