Probability and Statistics

Grinshpan

The most powerful test for the mean of a normal distribution

Let X1 , . . . , Xn , be a random sample from a normal distribution with unknown mean µ and
known variance σ 2 . Suggested are two simple hypotheses: µ = µ0 and µ = µ1 .
Given 0 < α < 1, what would the likelihood ratio test at significance level α be?
The answer turns out to be directly related to the sample mean X.

Let us write Xi ≈ xi to indicate the event that Xi takes a value in a small interval about xi ,
1 2 2
so that P (Xi ≈ xi ) = √ e−(xi −µ) /2σ dxi . By independence, we then have
2πσ
n
Y
P (X1 ≈ x1 , . . . , Xn ≈ xn |µ = µ0 ) = P (Xi ≈ xi |µ = µ0 )
i=1
n
Y 1 2 2
= √e−(xi −µ0 ) /2σ dxi
i=1
2πσ
n
( )
√ 1 X
= ( 2πσ)−n exp − 2 (xi − µ0 )2 dx1 . . . dxn .
2σ
i=1
Similarly,
n
( )
√ −n 1 X
P (X1 ≈ x1 , . . . , Xn ≈ xn |µ = µ1 ) = ( 2πσ) exp − 2 (xi − µ1 )2 dx1 . . . dxn .
2σ
i=1
So the likelihood ratio is
P (X1 ≈ x1 , . . . , Xn ≈ xn |µ = µ0 )
l(x1 , . . . , xn ) =
P (X1 ≈ x1 , . . . , Xn ≈ xn |µ = µ1 )
n
( )
1 X 2 2
= exp (xi − µ1 ) − (xi − µ0 )
2σ 2
i=1
  
n(µ0 − µ1 ) µ0 + µ1
= exp x− .
σ2 2
Observe that the condition l(x1 , . . . , xn ) < c is equivalent to
µ0 + µ1 σ 2 ln c
x > c̃ = + ,
2 n(µ0 − µ1 )
if µ0 < µ1 , and to x < c̃, if µ0 > µ1 . Therefore, any likelihood ratio criterion of the form
l(X1 , . . . , Xn ) < c reduces to the a simple bound on the sample mean, X > c̃ or X < c̃.
One says that X is the statistic of the criterion. The critical threshold c̃ is determined by α.

Assume that µ0 < µ1 .

Under the hypothesis µ = µ0 , we have X ∼ N (µ0 , σ 2 /n), and so

 
c̃ − µ0
α = P (l(X1 , . . . , Xn ) < c|µ = µ0 ) = P (X > c̃|µ = µ0 ) = P Z > √ .
σ/ n
√
Hence c̃ = µ0 + zα σ/ n, where zα is the upper α–quantile for the standard Gaussian.
Note that c̃ does not depend on µ1 .
Let us also determine the probability of a type II error.

Under the hypothesis µ = µ1 , we have X ∼ N (µ1 , σ 2 /n), and therefore

 
c̃ − µ1
β = P (l(X1 , . . . , Xn ) ≥ c|µ = µ1 ) = P (X ≤ c̃|µ = µ1 ) = P Z ≤ √ .
σ/ n
 √ 
n
Hence β = P Z ≤ zα +(µ0 − µ1 ) . Note that β is monotone decreasing as a function of n.
√ σ
n
Furthermore, zα +(µ0 − µ1 ) = −zβ or
σ
 2
zα + z β
n= σ ,
µ1 − µ0
which is the minimum sample size needed for the type I and type II errors not to exceed the
respective values of α and β. In fact, dne should be taken.

EXAMPLE Let n = 25, µ0 = 0, µ1 = 1, σ = 1, and α = 0.05. Determine the critical

threshold c̃, the probability of a type II error β, and the power of the test 1 − β.
√
Solution. We have c̃ = µ0 + z0.05 σ/ 25 = 0 + 1.645/5 = 0.329. So the hypothesis µ = 0 is to
be rejected whenever X > 0.329. The chance of a type II error is
β = P (Z ≤ 1.645 + (0 − 1) · 5) = P (Z ≤ −3.355) = 0.0004.
The power is 1 − β = 0.9996.

EXAMPLE Let µ0 = 0, µ1 = 1, σ = 1, and α = 0.05. How many observations are needed to

be made for the probability of a type II error not to exceed 0.1?

Solution. Since z0.05 = 1.645 and z0.1 = 1.282,

 2
1.645 + 1.282
n= · 1 ≈ 8.6.
1−0
Hence 9 observations are needed to be made.