Hypothesis Testing

Ingredients of a Statist. Test

•  Null hypothesis H0
  a statement about the data, which one hopes
to reject / falsify
  e.g. “Winter temperatures have not gradually
changed over the 20th century.”

•  One or several alternative hypotheses HA

  statements about the alternative possibilities contrasting H0
  often just the inverse of H0
  e.g. “Winter temperatures have a non-zero linear trend.”

•  Test statistic T
  a scalar function of the data
  suitable to distinguish HA from H0
Example: SON Temp Zürich
•  Test for mean
  H0: Expected value of SON Temp is 9 degC (µ = µ0)
  HA: … is different from 9 deg C (µ ≠ µ0)
  Test statistic:
n
x − µ0 1 1
T= x = 1 ∑ xi sx = sx = ⋅ 1 ∑ ( x − x )2
sx n n n n−1
i i
x = 8.7, s x = 0.1
•  Null-Distribution
€   If H0 is true, T is distributed like Tn-1 (T-dist with n-1 degrees
of freedom) … €
  … if temperatures are iid (independent, identically distributed),
normally-distr., stationary (no trend).
Testing
T(data) is close to expected value
of T under H0. Data is compatible with
H0. No reason to reject H0.

T(data) is far from expected value

of T under H0. It is unlikely to obtain
this T if H0 is true. Reject H0!

Significance level α: the area of the

null distribution pdf in the rejection
region. Typically α=0.05.
rejection no rejection rejection

q0.025 q0.975
 •  Test
  Value of T (= –3.02) is in
rejection region
T=-3.02   Expected value of Temps is
different from 9 degrees at
the 5% significance level
  Test has p-value of 0.003

•  p-value
  The probability under the
null distribution outside the
rejection rejection
value of +/– T.
no rejection
  The maximum significance
level (smallest α) for which
H0 would be rejected.
q0.025 q0.975
Errors in Testing
•  Type I Errors: •  Type II Errors:
  H0 is rejected although it is   H0 is not rejected although it is
true. false.
  Probability of error is α.   Probability (β) depends on the
separation/overlap of the
distributions for H0 and HA
  Can be large. Never accept H0
when it is not rejected!
β

α
Power of a Test
•  Power 1-β
  The probability to reject H0 when HA is true
  Depends on signal magnitude (Alternative) and sampling
uncertainty of test statistic
signal

sampling
uncertainty
1-β
Power of a Test
•  Power of a test can be very small if …
  … signal is small
  … sample size is small

•  Then testing is not informative.

•  Non-rejection is not „acceptance“ of Null-

Hypothesis.

•  It may be useful to quantify the power of a test.

  E.g. by stochastic simulation for given alternatives
One-Sided or Two-Sided Tests
•  One-sided
  If all considered HAs are on one side of the null distribution

•  Two-sided
  If HAs could be either side of the null distribution
  Tests on mean and trends usually two-sided
Categories of Tests
•  One-sample tests
  Is expected value of a sample equal to a predefined value?
  E.g. measurements to infer the universal gas constant

•  Independent two-sample tests

  Is expected value the same for two independent samples?
  E.g. the occurrence of hail in seeded/unseeded thunderstorms

•  Paired two-sample tests

  Is expected value the same between two paired samples?
  E.g. is there climatological preference for solar energy
between two regions.
Two-Sample T-Test parametric test

•  Null hypothesis
  Expected values of two independent samples {x}, {y} are equal

•  Test statistic
x−y
Z= sx, sy : the standard deviations
2 s 2
sx y of the two samples
+
nx ny
•  Null distribution
  Z is approx. distributed like Student’s T with min(nx,ny)-1
degrees
€ of freedom if:
  (a) the data within and between the samples are independent
  (b1) the samples are normally distributed or
  (b2) nx, ny are large so that means are normally distrib.. (CLT)