Chapter 12

Analysis of Covariance

Any scientific experiment is performed to know something that is unknown about a group of treatments
and to test certain hypothesis about the corresponding treatment effect.

When variability of experimental units is small relative to the treatment differences and the experimenter
do not wishes to use experimental design, then just take large number of observations on each
treatment effect and compute its mean. The variation around mean can be made as small as desired by
taking more observations.

When there is considerable variation among observations on the same treatment and it is not possible to
take an unlimited number of observations, the techniques used for reducing the variation are
(i) use of proper experimental design and
(ii) use of concomitant variables.

The use of concomitant variables is accomplished through the technique of analysis of covariance. If both
the techniques fail to control the experimental variability then the number of replications of different
treatments (in other words, the number of experimental units) are needed to be increased to a point where
adequate control of variability is attained.

Introduction to analysis of covariance model

In the linear model
Y= X 11 + X 2 2 + ... + X p p + ,

if the explanatory variables are quantitative variables as well as indicator variables, i.e., some of them
are qualitative and some are quantitative, then the linear model is termed as analysis of covariance
(ANCOVA) model.

Note that the indicator variables do not provide as much information as the quantitative variables. For
example, the quantitative observations on age can be converted into indicator variable. Let an indictor
variable be

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

1
 years
1 if age17
D=
0 if age<17 years.
Now the following quantitative values of age can be changed into indicator variables.

Ages (years) Ages

14 0
15 0
16 0
17 1
20 1
21 1
22 1

In many real application, some variables may be quantitative and others may be qualitative. In such
cases, ANCOVA provides a way out.

It helps is reducing the sum of squares due to error which in turn reflects the better model adequacy
diagnostics.

See how does this work:

In one way model : Yij = + i + ij , we have TSS1 = SSA1 + SSE1
In two way model : Yij = + i + j + ij , we have TSS 2 = SSA2 + SSB2 + SSE2
In three way model : Yij = + i + j + k + ik , we have TSS3 = SSA3 + SSB3 + SS 3 + SSE3

If we have a given data set, then ideally

=
TSS1 =
TSS 2 TSS3
=
SSA1 =
SSA2 SSA3 ;
SSB2 = SSB3

So SSE1 SSE2 SSE3 .

SS (effects ) / df
Note that in the construction of F -statistics, we use .
SSE / df
So F -statistic essentially depends on the SSEs .
Smaller SSE large F more chance of rejection.

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

2
Since SSA, SSB etc. here are based on dummy variables, so obviously if SSA, SSB , etc. are based on
quantitative variables, they will provide more information. Such ideas are used in ANCOVA models and
we construct the model by incorporating the quantitative explanatory variables in ANOVA models.

In another example, suppose our interest is to compare several different kinds of feed for their ability to
put weight on animals. If we use ANOVA, then we use the final weights at the end of experiment.
However, final weights of the animals depend upon the initial weight of the animals at the beginning of
the experiment as well as upon the difference in feeds.

Use of ANCOVA models enables us to adjust or correct these initial differences.

ANCOVA is useful for improving the precision of an experiment. Suppose response Y is linearly related
to covariate X (or concomitant variable). Suppose experimenter cannot control X but can observe it.
ANCOVA involves adjusting Y for the effect of X . If such an adjustment is not made, then the X can
inflate the error mean square and makes the true differences is Y due to treatment harder to detect.

If, for a given experimental material, the use of proper experimental design cannot control the
experimental variation, the use of concomitant variables (which are related to experimental material)
may be effective in reducing the variability.
Consider the one way classification model as
E (Yij i
= = =
i 1,..., p, j 1,..., N i ,
Var (Yij ) = 2 .

If usual analysis of variance for testing the hypothesis of equality of treatment effects shows a highly
significant difference in the treatment effects due to some factors affecting the experiment, then consider
the model which takes into account this effect
i + tij
E (Yij ) = i=
1,..., p, j =
1,..., N i ,
Var (Yij ) = 2

where tij are the observations on concomitant variables (which are related to X ij ) and is the

regression coefficient associated with tij . With this model, the variability of treatment effects can be

considerably reduced.

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

3
For example, in any agricultural experimental, if the experimental units are plots of land then, tij can be

measure of fertility characteristic of the j th plot receiving i th treatment and X ij can be yield.

In another example, if experimental units are animals and suppose the objective is to compare the growth
rates of groups of animals receiving different diets. Note that the observed differences in growth rates can
be attributed to diet only if all the animals are similar in some observable characteristics like weight, age
etc. which influence the growth rates.

In the absence of similarity, use tij which is the weight or age of j th animal receiving i th treatment.

If we consider the quadratic regression in tij then in

E (Yij ) =i + i tij + 2tij2 , i =1,..., p, j =1,..., ni ,

Var (Yij ) = 2 .

ANCOVA in this case is the same as ANCOVA with two concomitant variables tij and tij2 .

In two way classification with one observation per cell,

E (Yij ) = + i + j + tij , i =1,..., I , j =1,..., J
or
E (Yij ) = + i + j + i tij + 2 wij
i 0,=
with=
i
j 0, j

then ( yij , tij ) or ( yij , tij , wij ) are the observations in (i, j )th cell and tij , wij are the concomitment variables.

The concomitant variables can be fixed on random.

We consider the case of fixed concomitant variables only.

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

4
One way classification
Let = =
Yij ( j 1... ni , i 1... p ) be a random sample of size ni from i th normal populations with mean

=
ij ) i + tij
E (Yij=
Var (Yij ) = 2

where i , and 2 are the unknown parameters, tij are known constants which are the observations on

a concomitant variable.

The null hypothesis is

H 0 : 1= ...= p .

Let
1 1 1
=yio =
ni j
yij ; yoj =
p i
yij , yoo yij
n i j
1 1 1
=tio =
ni j
tij ; toj =
p i
tij , too tij
n i j
n = ni .
i

Under the whole parametric space ( ) , use likelihood ratio test for which we obtain the i ' s and
using the least squares principle or maximum likelihood estimation as follows:
=
Minimize S ( y
i j
ij ij ) 2

= ( y
i j
ij i tij ) 2

S
= 0 for fixed
i
i = yio tio

S
Put i in S and minimize the function by = 0,

y
2
i.e., minimize ij yio (tij tio ) with respect to gives
i j

( y y )(t tij io ij io )
= i j
.
(t t ) i j
ij io
2

Thus =
y t
i io io

= + t .

ij i ij

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

5
Since yij ij = yij i tij
= yij yio (tij tio ),

we have

( yij yio )(tij tio )
) 2 ( yij yio ) 2
( yij ij= .
i j

i j (tij tio ) 2

i j

y
2
Under H 0 : 1= ...= p= (say), consider
= Sw ij tij and minimize S w under sample
i j

space ( w ) ,

S w
= 0,

S w
=0

= yoo too
( y y )(t t ij oo ij oo )
= i j

(t t ) i j
ij oo
2

ij= + tij .

Hence
2

( yij yoo )(tij too )
) 2 ( yij yoo ) 2
( yij ij=
i j

i j i j (tij too ) 2

i j

and
2
(
i j
ij ij ) 2
= ( y y
i j
i oo ) + (tij tio ) (tij too ) .

The likelihood ratio test statistic in this case is given by

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

6
 max L( , , 2 )
= w

max L( , , 2 )

( ij ij ) 2
=
i j
.
( y
i j
ij ij )
2

Now we use the following theorems:

Theorem 1: Let Y = (Y1 , Y2 ,..., Yn ) follow a multivariate normal distribution N ( , ) with mean

vector and positive definite covariance matrix . Then Y AY follows a noncentral chi-square

distribution with p degrees of freedom and noncentrality parameter A , i.e., 2 ( p, A ) if and only
if A is an idempotent matrix of rank p.
Theorem 2: Let Y = (Y1 , Y2 ,..., Yn ) follows a multivariate normal distribution N ( , ) with mean

vector and positive definite covariance matrix . Let Y AY

1 follows 2 ( p1 , A1 )

and Y A2Y follows 2 ( p2 , A2 ) . Then Y AY

1 and Y A2Y are independently distributed if A1A2 =
0.

Theorem 3: Let Y = (Y1 , Y2 ,..., Yn ) follows a multivariate normal distribution N ( , 2 I ) , then the
maximum likelihood (or least squares) estimator L of estimable linear parametric function is

n 2
independently distributed of 2 ; L follow N L , L( X X ) 1 L and follows 2 (n p ) where
2

rank ( X ) = p.

Using these theorems on the independence of quadratic forms and dividing the numerator and
denominator by respective degrees of freedom, we have

n p 1
( ij ij ) 2
F= i j
~ F ( p 1, n p ) under H 0
p 1 ( y ij ij ) 2

So reject H 0 whenever F F1 ( p 1, n p ) at level of significance.

The terms involved in can be simplified for computational convenience follows:

We can write

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

7
 ( y
i j
ij ij ) 2

2
y t
=
i
ij
j
ij

2
=
i j
( yij yoo ) (tij too )

2
= ( y
i j
ij yoo ) (tij too ) + (tij tio ) (tij tio )

( y
2
= ij yio ) (tij tio )
i j
2
= ( y
i j
ij yoo ) + (tij tio ) (tij too )

= ( y
i j
ij ij ) 2 + ( ij ij ).
i j

For computational convenience

Tyt2 E yt2
( ij ij )
2
Tyy E yy
Ttt Ett

=
i j
( yij ij ) 2
E yy
E yt2

E yy
i j

where
=Tyy ( y
i j
ij yoo ) 2

=Ttt (t
i j
ij too ) 2

Tyt= ( y
i j
ij yoo )(tij too )

=E yy ( y i j
ij yio ) 2

=Ett (t
i j
ij tio ) 2

E yt= ( y i j
ij yio )(tij tio ).

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

8
Analysis of covariance table for one way classification is as follows:
Source of Degrees Sum of products Adjusted sum of squares F
variation of yy yt tt
Degress Sum of squares
freedom
of feedom
Population p 1 Pyy=( Tyy E yy ) Pyt=( Tyt E yt ) Ptt=( Ttt Ett ) p 1 q=
1 q0 q2 n p 1 q1
p 1 q2
E yt2
n p 1 =
q2 E yy
E yy E yt Ett E yy
Error
n p
Tyt2
Total n 1 Tyy Tyt Ttt n2 q=
0 Tyy
Ttt

If H 0 is rejected, employ multiple comprises methods to determine which of the contrasts in i are
responsible for this.

For any estimable linear parametric contrast

p p
=
=i 1 =i 1

= Ci i with Ci 0,
p p p
=
= Ci i
=i 1 =i 1 =i 1
Ci yi Cii ti

2
Var ( ) =
(t
i j
ij ti ) 2

Ci2 Ci ti
2
)
Var (= + i .
i ni (tij ti ) 2
i j

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

9
Two way classification (with one observations per cell)
Consider the case of two way classification with one observation per cell.
Let yij ~ N ( ij , 2 ) be independently distributed with

E ( yij ) = + i + j + tij , i =1...I , j =1...J

V ( yij ) = 2

where
: Grand mean
I
1 : Effect of i th level of A satisfying i
i =0

J
1 : Effect of j th level of B satisfying
i
j =0

tij : observation (known) on concomitant variable.

The null hypothesis under consideration are

H 0 : 1= 2= ...= I= 0
H 0 : 1= 2= ...= J= 0

Dimension of whole parametric space ( ) : I + J

Dimension of sample space ( w ) : J + 1 under H 0

Dimension of sample space ( w ) : I + 1 under H 0

with respective alternative hypotheses as

H1 : At least one pair of ' s is not equal

H1 : At least one pair of ' s is not equal.

Consider the estimation of parameters under the whole parametric space ( ) .

Find minimum value of ( y

i j
ij ij ) 2 under .

To do this, minimize

( y
i j
ij i j tij ) 2 .

For fixed , which gives on solving the least squares estimates (or the maximum likelihood estimates) of
the respective parameters as

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

10
 yoo to
=
i =yi yoo ( tio too ) (1)
j = yoj yoo ( toj too ).

Under these values of , i and j , the sum of squares ( y

i j
ij i j tij ) 2 reduces to

y
2
ij yio yoj + yoo + (tij tio toj + too ) . (2)
i j

Now minimization of (2) with respect to gives

I J

( y
=i 1 =j 1
ij yio yoj + yoo )(tij tio toj + too )
= I J
.
(t
=i 1 =j 1
ij tio toj + too ) 2

Using , we get from (1)

yoo too
=
i = ( yio yoo ) ( tio too )
j = ( yoj yoo ) ( toj too ).
Hence

( y
i j
ij ij ) 2

( yij yio yoj + yoo )(tij tio toj + too )
= ( yij yio yoj + yoo ) 2
i j

i j (tij tio toj + too ) 2

i j

E yt2
= E yy
Ett
where
=
E yy ( y
i j
ij yio yoj + yoo ) 2

=
E yt ( y y y + y )(t ij io oj oo ij tio toj + too )

(t t t + t ) .
2
=
Ett ij io oj oo
i j

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

11
Case (i) : Test of H 0
Minimize ( y
i j
ij j tij ) 2 with respect to , j and gives the least squares estimates) (or the

maximum likelihood estimates) of respective parameters as

= yoo too

j = yoj yoo ( toj too )
( y y )(t t ij oj ij oj )
= i j
(3)
(t t ) i j
ij oj
2

= + j + tij .
Substituting these estimates in (3) we get

( yij yoj )(tij toj )
( yij = ( yij y j ) 2
2 i j
ij )
i j i j (tij toj ) 2

i j

E yt + Ayt
= E yy + Ayy
Ett + Att

where
=Ayy J(y i
io yoo ) 2

=Att J (t
i
io too ) 2

Ayt = J(y i
io yoo )( tio too ) 2

=
E yy ( y i j
ij yio yoj + yoo ) 2

=
Ett (t i j
ij tio toj + too ) 2

=
E yt ( y i j
ij yio yoj + yoo )(tij tio toj + too ).

Thus the likelihood ratio test statistic for testing H 0 is

( y ij ij ) 2 ( yij ij ) 2
1 = i j i j
.
( y
i j
ij ij ) 2

Adjusting with degrees of freedom and using the earlier result for the independence of two quadratic
forms and their distribution
Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur
12
 ( yij ij ) 2 ( yij ij ) 2
( IJ I J ) i j
F1 =
i j
~ F ( I 1, IJ I J ) under H o .
( I 1) ( yij ij ) 2

i j
So the decision rule is to reject H o whenever F1 > F1 ( I 1, IJ I J ).

Case b: Test of H 0
Minimize ( y
i j
ij i tij ) 2 with respect to , i and gives the least squares estimates (or

maximum likelihood estimates) of respective parameters as

yoo too
=
j = yio yoo ( tio too )
( y y )(t t ij io ij io )
= i j
(4)
(t t ) i j
ij io
2

ij = + i + ij .

From (4), we get

( yij yio )(tij toj )
) 2 ( yij yio ) 2
( yij ij=
i j

i j i j ij io
(t t ) 2

i j
2
E yt + Byt
= E yy + Byy
Btt

=Byy I(y j
oj yoo ) 2

=
where Btt I (t
j
oj too ) 2

Byt = I(y j
io yoo )( toj too ) 2 .

Thus the likelihood ratio test statistic for testing H 0 is

( yij ij ) 2 ( yij ij ) 2
( IJ I J ) i j
~ F ( J 1, IJ I J ) under H o .
i j
F2
( J 1)

i j
( yij ij ) 2

So the decision rule is to reject H 0 whenever F2 F1 ( J 1, IJ I J ).

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

13
If H o is rejected, use multiple comparison methods to determine which of the contrasts i are

responsible for this rejection. The same is true for H o .

The analysis of covariance table for two way classification is as follows:

Degrees Sum of products F

Source of of yy yt tt
variation freedom
Between I 1 Ayy Ayt Att I 1 q0 =
q 3 q2 IJ I J q0
F1 =
evels of A I 1 q2

Between
levels of J 1 Byy Byt Btt J 1 q= q4 q2
IJ I J q1
1

B F2 =
J 1 q2

Error ( I 1)( J 1) E yy E yt Ett E yt2

IJ I J q2= E yy
Ett

Total IJ 1 Tyy Tyt Ttt IJ 2

( Ayt + E yt ) 2
Error + IJ J q3 = ( Ayy + E yy )
levels Att + Ett
of A

( Byt + E yt ) 2
Error + q4 = ( Byy + E yt )
IJ I Btt + Ett
levels
of B

Analysis of Variance | Chapter 12 | Analysis of Covariance | Shalabh, IIT Kanpur

14