RELIABILITY

The RELIABILITY procedure employs one of two different computing methods,

depending upon the MODEL specification and options and statistics requested.
Method 1 does not involve computing a covariance matrix. It is faster than
method 2 and, for large problems, requires much less workspace. However, it can
compute coefficients only for ALPHA and SPLIT models, and it does not allow
computation of a number of optional statistics, nor does it allow matrix input or
output. Method 1 is used only when alpha or split models are requested and only
FRIEDMAN, COCHRAN, DESCRIPTIVES, SCALE, and/or ANOVA are
specified on the STATISTICS subcommand and/or TOTAL is specified on the
SUMMARY subcommand.
Method 2 requires computing a covariance matrix of the variables. It is slower
than method 1 and requires more space. However, it can process all models,
statistics, and options.
The two methods differ in one other important respect. Method 1 will continue
processing a scale containing variables with zero variance and leave them in the
scale. Method 2 will delete variables with zero variance and continue processing if
at least two variables remain in the scale. If item deletion is required, method 2 can
be selected by requesting the covariance method.

Notation
There are N persons taking a test that consists of k items. A score X ji is given to
the jth person on the ith item.

i items
1
1

X 11

X 12

X 1k

X ji

Pj

M
N

P1

X N1

X N2

T1

T2

Ti

X Nk

PN

Tk

2 RELIABILITY

If the model is SPLIT, k1 items are in part 1 and k 2 = k k1 are in part 2. If the
number of items in each part is not specified and k is even, the program sets
k1 = k 2 = k 2 . If k is odd, k1 = k + 1 2 . It is assumed that the first k1 items are
in part 1.

1 6

wj

W=

Sum of the weights, where w j is the weight for case j

j =1

Pj =

ji

The total score of the jth person

i =1

Pj = Pj k

Mean of the observations for the jth person

Ti =

X ji w j

The total score for the ith item

j =1
k

G=

X ji w j

Grand sum of the scores

i =1 j =1

G =

G
Wk

Grand mean of the observations

Scale and Item StatisticsMethod 1

Item Means and Standard Deviations
Mean for the ith Item

Ti = Ti W

RELIABILITY 3

Standard Deviation for the ith Item

N

w j X 2ji WTi 2
j =1

Si =

W 1

Scale Mean and Scale Variance

Scale Mean

M =G W
For the split model:
Mean Part 1
k1

M1 =

i =1

Mean Part 2
k

M2 =

Ti

i = k1 +1

Scale Variance

S 2p

6 !

1
=
W 1

j =1

 
W T 
 
k

Pj2 w j

i =1

"#
##
$

4 RELIABILITY

For the split model:

Variance Part 1

S 2p1

1
=
W 1


w
 X
!
k1

j =1

  
 W  T 
  
k1

ji

i =1

i =1

"#
##
$

Variance Part 2

1
S 2p2 =
W 1

 N  k  2  k  2 "#
w j  X ji  W  Ti  ##
! j =1  i = k +1   i = k +1  $
1

Item-Total Statistics
Scale Mean if the ith Item is Deleted

~
M i = M Ti

Scale Variance if the ith Item is Deleted

~
Si2 = S 2p + Si2 2 cov X i , P

where the covariance between item i and the case score is

cov X i , P =

1
W 1


 P X

N

j =1



k

ji w j

Tl Ti

l =1

RELIABILITY 5

Alpha if the ith Item Deleted


 S


k 1
Ai =
1
k 2

2
l

~
Si2

l =1
l i





Correlation between the ith Item and Sum of Others

Ri =

cov X i , P Si2
~
Si Si

The ANOVA Table (Winer, 1971)

Source of
variation

Sum of Squares

df

Between
people

Pj2 w j

W 1

j =1
k

Within
people

k G 2 Wk

w j X 2ji

i =1 j =1

Pj2 w j

1 6

W k 1

j =1

Between
measures

i =1 j =1

Total

W G 2 Wk

k 1

i =1
k

Residual

w j X 2ji

Pj2 w j

j =1

Ti2 W G 2 Wk 1W 161k 16
i =1

w j X 2ji G 2 Wk
i =1 j =1

Wk 1

6 RELIABILITY

Each of the mean squares is obtained by dividing the sum of squares by the
corresponding degrees of freedom. The F ratio for between measures is

F=

MS between measures
,
MS residual

61 67

df = k 1, W 1 k 1

Friedman Test or Cochran Test

2 =

SS between measures
,
MS within people

df = k 1

Note: Data must be ranks for the Friedman test and a dichotomy for the Cochran test.

Kendalls Coefficient of Concordance

W=

SS between measures
SS total

(Will not be printed if Cochran is also specified.)

Tukeys Test for Nonadditivity

The residual sums of squares are further subdivided to

SS nonadd = M 2 D ,

df = 1

RELIABILITY 7

where

D = SS bet. meas SS bet. people

8 1Wk 6

 

"
 = 3T G 8 "# w 3 P G 8 ##
#$ !
#$
 !
k

i =1

j =1

T P X w G P w


 = w 3 P G 8 X 3T G 8"##

!
$
M=

i =1

ji

2
j

j =1

k G SS bet. meas

j =1

ji

j =1

i =1

SS bal = SS res SS nonadd ,

61 6

df = W 1 k 1 1

The test for nonadditivity is

F=

MS nonadd
MS balance

2 1

61 6 7

df = 1, W 1 k 1 1

The regression coefficient for the nonadditivity term is

B$ = M D ,
and the power to transform to additivity is

$
p$ = 1 BG

8 RELIABILITY

Scale Statistics
Reliability coefficient alpha (Cronbach 1951)


 S
k 
A=
1 S
k 1

k

2
i

i =1

2
p






If the model is split, separate alphas are computed:

A1 =



 SS 

k1
1
k1 1

k1

2
i
2
p1

i =1




k

A2 =

2
Si2
k2
1
k2 1
S2
i = k +1 p 2
1





For Split Model Only

Correlation Between the Two Parts of the Test

1 2
S p S 2p1 S 2p 2
2
R=
S p1S p 2

Equal Length Spearman-Brown Coefficient

Y=

2R
1+ R

RELIABILITY 9

Guttman Split Half

G=

2 S 2p S 2p1 S 2p2

S 2p

Unequal Length Spearman-Brown

ULY =

R 2 + R 4 + 4 R 2 1 R 2 k1k 2 k 2

2 1 R 2 k1k 2 k 2

Basic ComputationsMethod 2
Items with zero variance are deleted from the scale and from k , k1 , and k 2 . The
inverses of matrices, when needed, are computed using the sweep operator
described by Dempster (1969). If V < 10 30 , a warning is printed and statistics
that require V 1 are skipped.

Covariance Matrix V and Correlation Matrix R

%K 1 
 W 1  X X w
K
=&
KKr S S
K'




li

vij

lj

WTi Tj , i , j = 1, K , k

l =1

vij
Si Si

, where Si2 =

if raw data input

if correlation matrix and SD input

ij i j

rij =




vij
W 1

10 RELIABILITY

Scale Variance
k

S 2p =

Si2 + 2

i =1

vij
i< j

If the model is split,

S p21 =

k1

Si2 + 2

i =1

k1

ij

i< j

S p22 =

k1

Si2 + 2

i = k1 +1

ij

i = k1 +1 j > i

where the first k1 items are in part 1.

Scale StatisticsMethod 2
Alpha Model
Estimated Reliability

 S
k 
1
k 1 
 S
k

i =1

2
i

2
p

Standardized Item Alpha

k Corr

1 6

1 + k 1 Corr





RELIABILITY 11

where
k

rij
1 6 i <
j

2
Corr =
k k 1

Split Model
Correlation between Forms
k1

ij
i =1 j = k1 +1

S p1S p2

Guttman Split-Half
k1

v
G=

ij
i =1 j = k1 +1
S 2p

Alpha and Spearman-Brown equal and unequal length are computed as in method 1.

12 RELIABILITY

Guttman Model (Guttman 1945)

k

2
i

L1 = 1

i =1

S p2
2k
k 1

L2 = L1 +

L3 =

2
ij

i< j
2
Sp

k
L1
k 1
k

4
L4 =

ij

i< j
S p2

2 max
i

L5 = L1 +

2
ij

j i

S p2
k

L6 = 1

2
i

3 8

S p2 ; where i2 = V 1

i =1

Parallel Model (Kristof 1963)

Common Variance

CV = var =

1
k

S
i =1

2
i

1
ii

RELIABILITY 13

True Variance

TV = cov =

vij
1 6 i<
j

2
k k 1

Error Variance
EV = var cov

Common Inter-Item Correlation

R$ = cov var

Reliability of the Scale


 S
k 
A=
1 S
k 1

k

2
i

i =1

2
p






Unbiased Estimate of the Reliability

1 6
1 6

2+ W 3 A
A$ =
W 1
where A is defined above.

14 RELIABILITY

Test for Goodness of Fit





k 1 k + 16 12 k 36
= 1W 16 1
 121k 16 k 1k + 16 2 1W 16  log L
 2  

2

where

L=

3var cov8 3var + 0k + 15cov8

k 0k + 15
df =
2
k 1

Log of the Determinant of the Unconstrained Matrix

log UC = log V

Log of the Determinant of the Constrained Matrix

 4


log C = log var cov

9k 14var + 1k 16cov9

Strict Parallel (Kristof 1963)

Common Variance

CV = var +

1
k

3T G 8
i

i =1

RELIABILITY 15

Error Variance

EV = MS within people
All mean squares are calculated as described later in the analysis of variance table.
True Variance

TV = var +

1
k

3T G 8
i

EV

i =1

Common Inter-Item Correlation

R$ =

1 6 3T G 8
1
var + 3T G 8
k

1
cov
k 1 k

i =1

i =1

Reliability of the Scale

Re l =

kR$
1 + k 1 R$

1 6

Unbiased Estimate of the Reliability

Re l =

3 + W 3 Re l
W

16 RELIABILITY

Test for Goodness of Fit


6


2 = W 1 1

1 6 12k 36  log L
121 k 162 k 1 k + 36 2 371W 16 
k k +1

where

L=

k

 k 1
2
1
4var + 1k 16cov9 var cov + k 3Ti G 8 


i =1
df = k 1 k + 36 2 3

Log of the Determinant of the Unconstrained Matrix

log UC = log V

Log of the Determinant of the Constrained Matrix



1
log C = log4 var + 1 k 16cov9 var cov +
T

G

3
8

k 1

k

i =1

k 1

RELIABILITY 17

Additional StatisticsMethod 2
Descriptive and scale statistics and Tukeys test are calculated as in method 1.
Multiple R 2 if an item is deleted is calculated as

2
~
Ri2 = 1 i2
Si

i2 =

4V 9ii
1

Analysis of Variance Table

Source of
variation

Sum of Squares

df

1 
1W 16 k  S
! 
k

Between
people

Within
people

i =1

k 1

 "# 2
v  # + 1k 16 v
 #$
ij

i< j


W T


  
 T  
 
1W 161k 16  S 2
k 1v
k
!
i

i =1

i =1

Between SS + Within SS
2

T 2 = WY B 1Y

k 1

2
i

Hotellings T (Winer, 1971)

1 6

W k 1

i =1

Total

W 1

i< j

Residual

ij

"
1W 161k 16  k Si2 2
vij # + 1W 16SS bet. people

k
! i =1 k 1 i < j ##$
k

Between
measures

2
i

ij

i< j

"#
##$

1W 161k 16
Wk 1

18 RELIABILITY

where

 T T "#
T T #
Y=
##
M
!T T #$
B = CVC
1

k 1

where C is an identity matrix of rank k 1 augmented with a column of 1 on the right.

bij = vij vik v jk + S k2

The test will not be done if W < k or B < 10 30 .
The significance level of T 2 is based on

F=

W k +1

1W 161k 16 T

Item Mean Summaries

k

Mean =

i =1

 
T  T 
Variance =
1k 16
k

i =1

i =1

Maximum = max Ti
i

Minimum = min Ti
i

Range = Maximum Minimum

Ratio =

Maximum
Minimum

, with df = k 1, W k + 1

RELIABILITY 19

Item Variance Summaries

Same as for item means excepts that Si2 is substituted for Ti in all calculations.

Inter-Item Covariance Summaries

v
Mean =
k 1 k 16

ij

i< j

1 6
!

1
Variance =
k k 1 1

vij2

i< j




1 6  v 

k k 1

ij

i< j

"#
##
$

Maximum = max vij

i, j

Minimum = min vij

i, j

Range = Maximum Minimum

Ratio =

Maximum
Minimum

Inter-Item Correlations
Same as for inter-item covariances, with vij being replaced by rij .
If the model is split, statistics are also calculated separately for each scale.

Intraclass Correlation Coefficients

Intraclass correlation coefficients are always discussed in a random/mixed effects
model setting. McGraw and Wong (1996) is the key reference for this document.
See also Shrout and Fleiss (1979).
In this document, two measures of correlation are given for each type under each
model: single measure and average measure. Single measure applies to single
measurements, for example, the ratings of judges, individual item scores, or the
body weights of individuals, whereas average measure applies to average

20 RELIABILITY

measurements, for example, the average rating for k judges, or the average score
for a k-item test.

One-Way Random Effects Model: People Effect Random

Let Xji be the response to the i-th measure given by the j-th person, i = 1, , k, j =
1, , W. Suppose that Xji can be expressed as Xji = + pj + wji, where pj is the
between-people effect which is normal distributed with zero mean and a variance
of 2p , and wji is the within-people effect which is also normal distributed with zero
mean and a variance of w2 .
Let MS BP and MSWP be the respective between-people Mean Squares and withinpeople Mean Squares. Formulas for these two quantities can be found on page 479
of SPSS 7.5 Statistical Algorithms by dividing the corresponding Sum of Squares
with its degrees of freedom.
Single Measure Intraclass Correlation
The single measure intraclass correlation is defined as

(1) =

2p
2p + 2w

Estimate

The single measure intraclass correlation coefficient is estimated by

ICC (1) =

MS BP - MSWP
.
MS BP + ( k - 1) MSWP

In general,

-1
< ICC(1) 1.
k -1
Confidence Interval

RELIABILITY 21

For 0 < < 1, a (1- )100% confidence interval for (1) is given by

Fp / w - F / 2 ,W -1,W ( k -1)
Fp / w + ( k - 1) F / 2 ,W -1,W ( k -1)

< (1) <

Fp / w - F1- / 2 ,W -1,W ( k -1)

Fp / w + ( k - 1) F1- / 2 ,W -1,W ( k -1)

where

MS BP
MSWP

Fp / w =

and F ,v1 ,v2 is the upper point of a F-distribution with degrees of freedom v1

and v2.
Hypothesis Testing

The test statistic F (1) for H0 : (1) = 0 , where 1 > 0 0 is the hypothesized
value, is

F (1) = Fp / w

1- 0
.
1 + ( k - 1) 0

Under the null hypothesis, the test statistic has an F-distribution with
W - 1, W ( k - 1) degrees of freedom.
Average Measure Intraclass Correlation
The average measure intraclass correlation is defined as

(k ) =

2p
2p + 2w / k

Estimate

The average measure intraclass correlation coefficient is estimated by

22 RELIABILITY

ICC ( k ) =

MS BP - MSWP
.
MS BP

Confidence Interval

A (1- )100% confidence interval for ( k ) is given by

Fp / w - F / 2 ,W -1,W ( k -1)
Fp / w

< (k) <

Fp / w - F1- / 2 ,W -1,W ( k -1)

Fp / w

Hypothesis Testing

The test statistic F ( k ) for H0 : ( k ) = 0 , where 1 > 0 0 is the hypothesized

value, is

F ( k ) = Fp / w 1 - 0 .
Under the null hypothesis, the test statistic has an F-distribution with
W - 1, W ( k - 1) degrees of freedom.

Two-Way Random Effects Model: People and Measures Effects Random

Let Xji be the response to the i-th measure given by the j-th person, i = 1, , k, j =
1, , W. Suppose that Xji can be expressed as Xji = + pj + mi + pmji + eji, where pj
is the people effect which is normal distributed with zero mean and a variance of
2p , mi is the measures effect which is normal distributed with zero mean and a
variance of m2 , pmji is the interaction effect which is normal distributed with zero
mean and a variance of 2pm , and eji is the error effect which is again normal
distributed with zero mean and a variance of e2 .
Let MS BP , MS BM and MS Res be the respective between-people Mean Squares,
between-measures Mean Squares and Residual Mean Squares. Formulas for these
quantities can be found on page 479 of SPSS 7.5 Statistical Algorithms by dividing
the corresponding Sum of Squares with its degrees of freedom.

RELIABILITY 23

Type A Single Measure Intraclass Correlation

The type A single measure intraclass correlation is defined as

( A,1,r )

%K
K
=&
KK
K'

2p
2
p

if interaction effect pm ji is present

+ 2m + 2pm + 2e

2p
2
p

if interaction effect pm ji is absent

+ 2m + 2e

Estimate

The type A single measure intraclass correlation coefficient is estimated by

ICC ( A,1, r ) =

MS BP

MS BP - MS Res
.
+ ( k - 1) MS Res + k ( MS BM - MS Res ) / W

Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval

A (1- )100% confidence interval is given by

W ( MS BP - F / 2,W -1,v MS Res )

< ( A,1,r )
F / 2 ,W -1,v [ k MS BM + ( kW - k - W ) MS Res ] + W MS BP
<

W ( MS BP - F1- / 2 ,W -1,v MS Res )

F1- / 2 ,W -1,v [ k MS BM + ( kW - k - W ) MS Res ] + W MS BP

where

(aMS BM + bMS Res ) 2

 (aMS )
! k -1
BM

(bMS Res ) 2
(W - 1)( k - 1)

"#
#$

24 RELIABILITY

a=

k ICC ( A,1, r )
W (1 - ICC ( A,1, r ))

and

b = 1+

k ICC ( A,1, r ) (W - 1)
.
W (1 - ICC ( A,1, r ))

Hypothesis Testing

The test statistic F ( A,1,r ) for H0 : ( A,1,r ) = 0 , where 1 > 0 0 is the

hypothesized value, is

F ( A,1,r ) =

MS BP
a0 MS BM + b0 MS Res

where

a0 =

k 0
W (1 - 0 )

and

b0 = 1 +

k 0 (W - 1)
.
W (1 - 0 )

Under the null hypothesis, the test statistic has an F-distribution with W - 1, (01)
degrees of freedom.

(01) =

(a0 MS BM + b0 MS Res ) 2

 (a MS
! k -1
0

BM )

(b MS Res ) 2
+ 0
(W - 1)( k - 1)

"# .
#$

RELIABILITY 25

Type A Average Measure Intraclass Correlation

The type A average measure intraclass correlation is defined as

( A,k ,r )

%K
K
=&
KK
K'

2p
2
p

2
+ ( m
+ 2pm + 2e ) / k

2p
2
p

2
+ ( m
+ 2e ) / k

if interaction effect pm ji is present

.

if interaction effect pm ji is absent

Estimate

The type A average measure intraclass correlation coefficient is estimated by

ICC ( A, k , r ) =

MS BP

MS BP - MS Res
.
+ ( MS BM - MS Res ) / W

Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval

A (1- )100% confidence interval is given by

W ( MS BP - F / 2 ,W -1,v MS Res )
< ( A,k ,r )
F / 2 ,W -1,v ( MS BM - MS Res ) + W MS BP
<

W ( MS BP - F1- / 2,W -1,v MS Res )

F1- / 2,W -1,v ( MS BM - MS Res ) + W MS BP

where is defined as in the Type A single measure confidence interval, with

ICC ( A,1, r ) replaced by ICC ( A, k , r ) .
Hypothesis Testing

The test statistic for H0 : ( A,k ,r ) = 0 , where 1 > 0 0 is the hypothesized value,
is

26 RELIABILITY

F ( A, k ,r ) =

MS BP
c0 MS BM + d 0 MS Res

where

c0 =

0
W (1 - 0 )

and

d0 = 1 +

0 (W - 1)
.
W (1 - 0 )

Under the null hypothesis, the test statistic has an F-distribution with W - 1, (0k )
degrees of freedom.

(0k ) =

(c0 MS BM + d 0 MS Res ) 2

 (c MS
! k -1

BM )

(d MS Res ) 2
+ 0
(W - 1)( k - 1)

"# .
#$

Type C Single Measure Intraclass Correlation

The type C single measure intraclass correlation is defined as

( C ,1,r )

%K
K +
=&
KK
K' +

2
p
2
pm

2
p

2
p

2
p

2
e

+ 2e

if interaction effect pm ji is present

.

if interaction effect pm ji is absent

Estimate

The type C single measure intraclass correlation coefficient is estimated by

RELIABILITY 27

ICC (C ,1, r ) =

MS BP - MS Res
.
MS BP + ( k - 1) MS Res

Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval

A (1- )100% confidence interval is given by

Fp / r - F / 2 ,W -1,(W -1)( k -1)

Fp / r - F1- / 2 ,W -1,(W -1)( k -1)
< ( C ,1,r ) <
Fp / r + ( k - 1) F / 2,W -1,(W -1)( k -1)
Fp / r + ( k - 1) F1- / 2,W -1,(W -1)( k -1)
where

Fp / r =

MS BP
.
MS Res

Hypothesis Testing

The test statistic for H0 : ( C ,1,r ) = 0 , where 1 > 0 0 is the hypothesized value,
is

F ( C ,1,r ) = Fp / r

1- 0
.
1 + ( k - 1) 0

Under the null hypothesis, the test statistic has an F-distribution with
W - 1,(W - 1)( k - 1) degrees of freedom.
Type C Average Measure Intraclass Correlation
The type C average measure intraclass correlation is defined as

28 RELIABILITY

( C , k ,r )

%K
K
=&
KK
K'

2p
2
p

+ ( 2pm + 2e ) / k

2p
2
p

+ 2e / k

if interaction effect pm ji is present

.

if interaction effect pm ji is absent

Estimate

The type C average measure intraclass correlation coefficient is estimated by

ICC (C , k , r ) =

MS BP - MS Res
.
MS BP

Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval

A (1- )100% confidence interval is given by

Fp / r - F / 2,W -1,(W -1)( k -1)

Fp / r - F1- / 2 ,W -1,(W -1)( k -1)
.
< ( C , k ,r ) <
Fp / r
Fp / r

Hypothesis Testing

The test statistic for H0 : ( C ,k ,r ) = 0 , where 1 > 0 0 is the hypothesized value,

is

F ( C ,k ,r ) = Fp / r (1 - 0 ) .
Under the null hypothesis, the test statistic has an F-distribution with
W - 1,(W - 1)( k - 1) degrees of freedom.

RELIABILITY 29

Two-Way Mixed Effects Model: People Effects Random, Measures Effects Fixed
Let Xji be the response to the i-th measure given by the j-th person, i = 1, , k, j =
1, , W. Suppose that Xji can be expressed as Xji = + pj + mi + pmji + eji, where pj
is the people effect which is normal distributed with zero mean and a variance of
2p , mi is considered as a fixed effect, pmji is the interaction effect which is normal
distributed with zero mean and a variance of 2pm , and eji is the error effect which
is again normal distributed with zero mean and a variance of e2 . Denote m2 as
the expected measure square of between measures effect mi.
Let MS BP and MS Res be the respective between-people Mean Squares and
Residual Mean Squares. Formulas for these quantities can be found on page 479 of
SPSS 7.5 Statistical Algorithms by dividing the corresponding Sum of Squares with
its degrees of freedom.
Type A Single Measure Intraclass correlation
The type A single measure Intraclass correlation is defined as

%K - / (k - 1)
K + + ( +
=&
KK
K' + +
2
p

( A,1,m)

2
p
2
p

2
pm

2
m
2
p
2
m

2
pm

2
e

2
e)

if interaction effect pm ji is present

.

if interaction effect pm ji is absent

Estimate

The type A single measure intraclass correlation is estimated by

ICC ( A,1, m) =

MS BP

MS BP - MS Res
.
+ ( k - 1) MS Res + k ( MS BM - MS Res ) / W

Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval

30 RELIABILITY

A (1- )100% confidence interval for ( A,1,m) is the same as that for ( A,1,r ) , with
ICC ( A,1, r ) replaced by ICC( A,1, m) .
Hypothesis Testing

The test statistic for H0 : ( A,1,m) = 0 , where 1 > 0 0 is the hypothesized value,
is the same as that for ( A,1,r ) , with the same distribution under the null hypothesis.
Type A Average Measure Intraclass Correlation
The type A average measure intraclass correlation is defined as

( A,k ,m)

%K
K
=&
KK
K'

2p - 2pm / ( k - 1)
2
p

if interaction effect pm ji is present

+ ( 2m + 2pm + 2e ) / k

2p
2
p

if interaction effect pm ji is absent

+ ( 2m + 2e ) / k

Estimate

The type A single measure intraclass correlation is estimated by

%KNot estimable
MS - MS
ICC ( A, k , m) = &
K' MS + ( MS - MS
BP

BP

BM

if interaction effect pm ji is present

Res
Res ) / W

if interaction effect pm ji is absent

Confidence Interval

A (1- )100% confidence interval for ( A,k ,m) is the same as that for ( A,k ,r ) ,
with ICC ( A, k , r ) replaced by ICC ( A, k , m) . Notice that the hypothesis test is not
available when the interaction effect pmji is present.
Hypothesis Testing

The test statistic for H0 : ( A,k ,m) = 0 , where 1 > 0 0 is the hypothesized value,
is the same as that for ( A,k ,r ) , with the same distribution under the null

RELIABILITY 31

hypothesis. Notice that the hypothesis test is not available when the interaction
effect pmji is present.
Type C Single Measure Intraclass Correlation
The type C single measure intraclass correlation is defined as

( C ,1,m)

%K -
K + (
=&
KK
K' +
2
p
2
p

2
pm / ( k - 1)
2
2
pm + e )

2
p

2
p

2
e

if interaction effect pm ji is present

.

if interaction effect pm ji is absent

Estimate

The type C single measure intraclass correlation is estimated by

ICC (C ,1, m) =

MS Between people - MS Residual

MS Between people + ( k - 1) MS Residual

Notice that the same estimator is used whether or not the interaction effect pmji is
present.
Confidence Interval

A (1- )100% confidence interval is given by

Fp / r - F / 2 ,W -1,(W -1)( k -1)

Fp / r + ( k - 1) F / 2,W -1,(W -1)( k -1)
where

Fp / r =

Hypothesis Testing

MS BP
.
MS Res

< ( C ,1,m) <

Fp / r - F1- / 2 ,W -1,(W -1)( k -1)

Fp / r + ( k - 1) F1- / 2,W -1,(W -1)( k -1)

32 RELIABILITY

The test statistic for H0 : ( C ,1,m) = 0 , where 1 > 0 0 is the hypothesized value,
is

1- 0
.
1 + ( k - 1) 0

F ( C ,1,m) = Fp / r

Under the null hypothesis, the test statistic has an F-distribution with
W - 1,(W - 1)( k - 1) degrees of freedom.
Type C Average Measure Intraclass Correlation
The type C average measure intraclass correlation is defined as

%K - / ( k - 1)
K + ( + ) / k
=&
KK
K' + / k
2
p

( C ,k ,m)

2
pm
2
pm

2
p

2
e

2
p

2
p

2
e

if interaction effect pm ji is present

.

if interaction effect pm ji is absent

Estimate

The type C average measure intraclass correlation is estimated by

%K
&K
'

Not estimable
ICC (C, k , m) = MS BP - MS Res
MS BP

if interaction effect pm ji is present

if interaction effect pm ji is absent

Confidence Interval

A (1- )100% confidence interval is given by

Fp / r - F / 2,W -1,(W -1)( k -1)

Fp / r - F1- / 2 ,W -1,(W -1)( k -1)
.
< ( C ,k ,m) <
Fp / r
Fp / r

RELIABILITY 33

Notice that the confidence interval is not available when the interaction effect pmji
is present.
Hypothesis Testing

The test statistic for H0 : ( C ,k ,m) = 0 , where 1 > 0 0 is the hypothesized value,
is

F ( C ,1,m) = Fp / r 1 - 0 .
Under the null hypothesis, the test statistic has an F-distribution with
W - 1,(W - 1)( k - 1) degrees of freedom. Notice that the F-test is not available
when the interaction effect pmji is present.

References
Cronbach, L. J. 1951. Coefficient alpha and internal structure of tests.
Psychometrika, 16: 297334.
Dempster, A. P. 1969. Elements of Continuous Multivariate Analysis. Reading,
Mass.: Addison-Wesley.
Guttman, L. 1945. A basis for analysing test-retest reliability. Psychometrika,
10(4): 255282.
Kristof, W. 1963. The statistical theory of stepped-up reliability coefficients when
the test has been divided into several equivalent parts. Psychometrika, 28: 221
238.
Kristof, W. 1969. Estimation of true score and error variance under various
equivalent assumptions. Psychometrika, 34: 489507.
McGraw, K. O., and Wong, S. P. (1996). Forming Inferences About Some
Intraclass Correlation Coefficients. Psychological Methods, 1, p30-46.
Novick, M. R., and Lewis, C. 1967. Coefficient alpha and the reliability of
composite measurements. Psychometrika, 32: 113.
Shrout, P. E., and Fleiss, J. L. (1979). Intraclass Correlations: Uses in Assessing
Rater Reliability. Psychological Bulletin, 86, p420-428.

34 RELIABILITY

Winer, B. J. 1971. Statistical principles in experimental design, 2nd ed. New York:
McGraw-Hill.