Introduction to hypothesis testing

Determine characteristics of a population

from a sample
Sample does not match population
Differences due to chance
Differences reflect real effects
Research treatment
Treatment population mean different from original
mean?
ogic o hypothesis testing: State
hypotheses
H
0
: the null hypothesis
No effect of treatment
Any differences due to sampling error
H
1
: the alternative or scientific hypothesis
Treatment had an effect
Real differences
Hypotheses contradictory and mutually exclusive
H
0
states a specific value for population parameter
E.g.,
treatment
=
population
How likely are the results if H
o
is true?
f unlikely H
0
is probably not true
ogic o hypothesis testing
Set criteria for decision
How unlikely?
Collect sample
Observe sample statistic
Evaluate sample statistic consistency with the
population parameter given by the H
0
!robability that the H
0
is true
'ery unlikely reject the H
0
Reasonable chance H
0
truefail to reject H
0
Assume H
0
true
Argument by contradiction
;aluating the

Alpha d or significance
level
Traditional levels
.05 or .01
5 value 5(H
0
true)
Critical regioncritical
values
Choice of .05 or .01
how important to be
certain
-5 -4 -3 -2 -1 0 1 2 3 4 5
X
0.0
0.1
0.2
0.3
0.4
Y
Reject H0
Reject H0
ample using coin tossing
H
0
: Coin is fair and the !(head) = .5
H
1
: Coin is not fair and !(head) = .5
Sample 10 heads in a row
Set d at .05
!(10 heads) = 1/2 X 1/2 X 1/2 X...X =
1/2
10
= .000976
Based on assumption H
0
true
rrors
Actual Situation
o effect
H
0
is true
Effect Exists
H
0
is false
Reject H
0

Type error
(!robability = d)
Correct decision
(!robability = 1 )
!ower Decision
Retain H
0

Correct decision (!robability =
1 d)
Type error
(!robability = )

Type or d errors consist of rejecting the H
0
when the H
0
is true
5 (type error) = d
Type or errors consist of failing to reject
the H
0
when the H
0
is false
5 (type error) =
test
= $1,000, o = $500
= $3,000, n = 20
H
0
: = $1,000
H
1
: = $1,000
d = .05
111.803
4.472
500
20
500
= = = =
3

o
o
)
17.89
803 . 111
) 000 , 1 000 , 3 (
=

600 800 1000 1200 1400

X
0.000
0.001
0.002
0.003
0.004
Y
f = 1,050
Area in tail for (0.45) = .3264, 5 = 2*.3264
= 0.6528
! .05, fail to reject H
0
)
0.45
803 . 111
) 000 , 1 050 , 1 (
=

Steps
State H
0
and H
1
Set d
Determine critical value and region
Determine
Evaluate H
0
, exceed critical value
)

o

=
nother eample
= 4, o = .45
= 3.85, n = 25
H
0
: = 4, H
1
: = 4
- = .01
Critical = 2.57583
= 1.667, p.01
Retain the H0
1.667
09 .
15 .
25
45 .
0 . 4 85 . 3
=

asic lements
Hypothesized population parameter H
0
Sample statistic
Estimate of error/chance, standard error
Alpha level -
- = .05 the test statistic critical value will be around 2.00
- = .01 the test statistic critical value will be around 2.50
- = .001 the test statistic critical value will be around 3.00
)
o

and between error standard

mean population - mean sample

=
.an.e by expe.ted diIIeren.e
diIIeren.e obtained
statisti. test =
Reporting
Significant test statistic = value, 5 < al5a
= 2.50, 5 < .05
Not significant test statistic = value, 5 > al5a or n.s.
= 1.22, 5 >.05
!recise 5 from computer
Assumptions
Random sample
ndependent observations
o unchanged by the treatment, constant added to scores
irectional ;ersus nondirectional tests
One-tailed and two-tailed tests
Two tailed, difference regardless of direction
H
0
: = 100, H
1
: = 100
One tailed, specific direction
H
0
: ^ 1,000, H
1
: 1,000
d = .05
z two tailed 1.96
z one tailed 1.65
-5 -4 -3 -2 -1 0 1 2 3 4 5
X
0.0
0.1
0.2
0.3
0.4
Y
oncerns
Criticisms
All or none 1.95 versus 1.97
H
0
artificial
gnores magnitude of effect
= 3.9. The o = .45.
H
0
: = 4, H
1
: = 4
- = .01, = 2.58
n = 25
= 1.11, 5 .01
1.11
09 .
1 .
25
45 .
0 . 4 9 . 3
=

n = 900
z = 6.67, 5 < .01
Statistical versus real world significance
Effect size
67 . 6
015 .
1 .
900
45 .
0 . 4 9 . 3
=

o

= =

/
deviation standard
diIIeren.e mean
s Coen'
22 .
45 .
1 .
45 .
0 . 4 9 . 3
= =

2 3 4 5 6

0.0
0.3
0.6
0.9
Y
2 3 4 5 6

0.0
0.3
0.6
0.9
Y
2 3 4 5 6

0.0
0.3
0.6
0.9
Y
ean = 3.5
Small 0 < / < 0.2 (.25)
edium 0.2 < / < 0.8 (.5)
arge effect / 0.8 (1.25)
11 . 1
45 .
5 .
45 .
0 . 4 5 . 3
s Coen' = =

= /
2 3 4 5 6

0.0
0.3
0.6
0.9
Y
2 3 4 5 6

0.0
0.3
0.6
0.9
Y
Statistical Power
Ability to reject false H
0
!ower depends on:
agnitude of treatment effect
Alpha level
Sample size
One tailed versus two tailed test
agnitude of treatment effect
Small .25 o
edium .75 o
arge 1.25 o
-5 -3 -1 1 3 5
X
0.0
0.1
0.2
0.3
0.4
Y
Small Effect .25
-5 -3 -1 1 3 5
X
0.0
0.1
0.2
0.3
0.4
Y
-5 -3 -1 1 3 5
X
0.0
0.1
0.2
0.3
0.4
Y
edium Effect .75
-5 -3 -1 1 3 5
X
0.0
0.1
0.2
0.3
0.4
Y
-5 -3 -1 1 3 5
X
0.0
0.1
0.2
0.3
0.4
Y
arge Effect 1.25
-5 -3 -1 1 3 5
X
0.0
0.1
0.2
0.3
0.4
Y
ect Magnitude and Power
= 100, o

= 10, Real = 110 (actual

population ), d = .05
Critical value assuming H
0
is true

.1685 above 119.6

6 . 119 10 * 96 . 1 100 * 96 . 1 = + = +

o
60 90 120 150
X
0.0
0.01
0.02
0.03
0.04
Y
60 90 120 150
X
0.0
0.01
0.02
0.03
0.04
Y
96 .
10
6 . 9
10
110 6 . 119
= =

= 100, o

= 10, real = 135, d = .05

Critical value
.
.9382 above 119.6
6 . 119 10 * 96 . 1 100 * 96 . 1 = + = +

o
54 . 1
10
4 . 15
10
135 6 . 119
=

60 80 100 120 140 160 180 200

X
0.0
0.01
0.02
0.03
0.04
Y
60 80 100 120 140 160 180 200
X
0.0
0.01
0.02
0.03
0.04
Y
Sample Size and Power
= 100, o = 10, real = 110, n = 4, o

5, d
= .05
Critical value
.5160
above 109.8
8 . 109 5 * 96 . 1 100 * 96 . 1 = + = +

o
04 .
5
2 .
2
10
2 .
4
10
110 8 . 109
=

80 90 100 110 120 130

X
0.00
0.02
0.04
0.06
0.08
Y
80 90 100 110 120 130
X
0.00
0.02
0.04
0.06
0.08
Y
Sample Size and Power
= 100, o = 10, real = 110, n = 25, o

2,
d = .05
Critical value
.9987
above 103.92
92 . 103 2 * 96 . 1 100 * 96 . 1 = + = +

o
04 . 3
2
08 . 6
5
10
08 . 6
25
10
110 92 . 103
=

60 90 120 150
X
0.0
0.05
0.10
0.15
0.20
Y
60 90 120 150
X
0.0
0.05
0.10
0.15
0.20
Y
lpha, eta, and Power
= 100, o = 5, d = .05, = 1.96, = 110
Critical value = 109.8
= 100, o = 5, d = .01, = 2.575
Critical value = 114.9
ore stringent d (.01)
ower 5 (Type error)
Higher 5 (Type error)
ower power
80 100 120 140
X
0.0
0.02
0.04
0.06
0.08
Y
80 100 120 140
X
0.0
0.02
0.04
0.06
0.08
Y
One ;ersus 1wo 1ailed
Two tailed, = 100, o = 5, d = .05, = 1.96,
Real = 110
Critical value = 109.8
One tailed, = 100, o = 5, d = .05, = 1.65,
Real = 110
Critical value = 108.25
80 100 120 140
X
0.0
0.02
0.04
0.06
0.08
Y
80 100 120 140
X
0.0
0.02
0.04
0.06
0.08
Y
ect Size, Power, and 3
ne sample two tailed test sample sizessmall eIIe.t.25o
Power
alpha 0.6 0.7 0.8 0.9
0.1 64 83 110 151
0.05 87 109 139 185
0.01 143 170 207 262
0.001 224 258 303 369
ne sample two tailed test sample sizesmedium eIIe.t.75o
power
Alpha 0.6 0.7 0.8 0.9
0.1 8 10 13 17
0.05 11 13 16 21
0.01 18 21 25 30
0.001 28 32 36 43
ne sample two tailed test sample sizeslarge eIIe.t1.25o
power
Alpha 0.6 0.7 0.8 0.9
0.1 4 5 6 7
0.05 6 6 7 9
0.01 9 10 11 13
0.001 14 15 17 19

Inter;al stimation
Critical 'alue of = 1.96
= 50, o

=4
Hypothesized < 42.16 or 57.84 reject
H
0

o
84 . 57 16 . 42 84 . 7 50 4 * 96 . 1 50 = = =