Analysis of Variance (ANOVA)

Engineering Experimental Design

Valerie L. Young
 Example Problem
• As a chicken nutritionist, you
need reliable measures of
lasalocid sodium levels in
Laboratory A Laboratory B Laboratory C chicken feed, so you decide to
87 88 85 test the analytical labs.
88 93 84
84 88 79
• You send a sample of feed
84 89 86 containing 85 mg/kg lasalocid
87 85 81 sodium to each of three
81 87 86 independent labs, and get
86 86 88
84 89 83
back the results at left.
88 88 83 • Do the analyses agree with
86 93 83 each other?
• Do the analyses agree with the
true value?
 Graph it First

95
Lasalocid Sodium Content, mg/kg

90

85

80

75

70
Laboratory A Laboratory B Laboratory C
 Interpret the Graph

• Laboratory B results appear to be higher

• All labs’ results overlap with each other
• All labs’ results overlap with the true value
(85 mg/kg)
• Based on the graph alone, I cannot tell
whether any of the labs differ significantly
from each other or from the true value.
 ANOVA – What does it tell me?

• ANOVA = Analysis of Variance

• ANOVA will tell me whether I have
sufficient evidence to say that
measurements from at least one lab differ
significantly from at least one other.
– It will not tell me which ones differ, or how
many differ.
 ANOVA vs. t-test
• ANOVA is like a t-test among multiple data
sets simultaneously
– t-tests can only be done between two data sets,
or between one set and a “true” value
• ANOVA uses the F distribution instead of
the t-distribution
• ANOVA assumes that all of the data sets
have equal variances
– Use caution on close decisions if they don’t
• Consult a professional
 ANOVA – a Hypothesis Test
• H0: There is no significant difference
among the results provided by these three
laboratories.
• H1: At least one of these laboratories
provides results significantly different from
at least one other.
 Excel and ANOVA
• Tools > Data Analysis >
– ANOVA: Single Factor
– ANOVA: Two-Factor with Replication
– ANOVA: Two-Factor without Replication
• So how many factors do we have here?
– Factor = Independent Variable
– The I.V. here is Laboratory
– We have a SINGLE factor with THREE levels
(A,B,C)
 ANOVA Results

Anova: Single Factor

Using a significance level of 0.05.
SUMMARY
Groups Count Sum Average Variance
Laboratory A 10 855 85.5 4.944444
Laboratory B 10 886 88.6 6.933333
Laboratory C 10 838 83.8 6.844444

ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 118.4667 2 59.23333 9.491395 0.000756 3.354131
Within Groups 168.5 27 6.240741

Total 286.9667 29
 Focus on ANOVA Table
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 118.4667 2 59.23333 9.491395 0.000756 3.354131
Within Groups 168.5 27 6.240741
Treatments F = MSTr / MSE
MSTr = 59.233 / 6.2407
Error MSE

• F = ratio of variability (between groups) due

to treatment to variability (within groups)
due to random error
• P = probability of getting an F value at least
this large if these were 3 sets of 10
measurements from the same population
 Decision Based on ANOVA
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 118.4667 2 59.23333 9.491395 0.000756 3.354131
Within Groups 168.5 27 6.240741
Treatments F = MSTr / MSE
MSTr = 59.233 / 6.2407
Error MSE
• F > F critical
• Reject H0
• P < 0.05 (chosen significance level)
• Reject H0
• If H0 were true, the probability of getting
3 sets of data like this is less than 0.1 %
Where Does the Difference Lie?
• ANOVA only shows that a difference exists
• To find the difference, consider
– Graphical representation
• Mean with confidence limits
• Effects plot
– Analysis of Means (ANOM)
• For one factor at multiple levels, ANOM is a better
technique than ANOVA
• For multiple factors, ANOVA is required. We are
showing ANOVA for one factor so you can better
understand it when it is properly applied for multiple
factors.
 Descriptive (Summary) Statistics
Laboratory A Laboratory B Laboratory C

Mean 85.5 Mean 88.6 Mean 83.8

Standard Error 0.703167 Standard Error 0.832666 Standard Error 0.827312
Median 86 Median 88 Median 83.5
Mode 84 Mode 88 Mode 83
Standard Deviation 2.223611 Standard Deviation 2.633122 Standard Deviation 2.616189
Sample Variance 4.944444 Sample Variance 6.933333 Sample Variance 6.844444
Kurtosis 0.246073 Kurtosis 0.065974 Kurtosis 0.103614
Skewness -0.79585 Skewness 0.772335 Skewness -0.28668
Range 7 Range 8 Range 9
Minimum 81 Minimum 85 Minimum 79
Maximum 88 Maximum 93 Maximum 88
Sum 855 Sum 886 Sum 838
Count 10 Count 10 Count 10
Confidence Level(95.0%) 1.590676 Confidence Level(95.0%) 1.883624 Confidence Level(95.0%) 1.87151

Note: I have not used the pooled variance (MSe) to calculate the
confidence limits. Since the analysis was performed by different
labs, I decided to allow for them to have different uncertainties.
 Focus on Descriptive Statistics
Mean of 10 replicate measurements
Laboratory A
by laboratory A. This sample mean
Mean 85.5 is an estimate of the “true”
Standard Error 0.703167
concentration.
Median 86
Mode 84
Standard Deviation 2.223611
Standard deviation of 10 replicate
Sample Variance 4.944444 measurements by laboratory A. This
Kurtosis 0.246073 sample std dev is an estimate of the
Skewness -0.79585
Range 7
“true” std dev. for lab A
Minimum 81
Maximum 88
Sum 855
Count 10 95 % confidence interval on the
Confidence Level(95.0%) 1.590676
mean.
 Graphical Representation

91

90 95 % CI
An "effects plot" is just this plot with the
on mean
Lasalocid Sodium Content, mg/kg

89 dots connected and no 95% CI error

bars.
Mean
88

87

86

85

84
Target Value
83

82
Laboratory A Laboratory B Laboratory C
 Interpretation of Graph

• Results from Lab B differ significantly from

Lab C and from the known value (85 mg/kg)
• Results from Lab A and Lab C agree with one
another and with the known value
• Lab B analysis is unacceptable
– This is the appropriate conclusion even though
analyses by Labs A and B are apparently not
significantly different
 Example – Lead Contamination
• Lead was banned as a gasoline additive in 1978.
Soil lead levels were monitored at 80 randomly
chosen locations in the United States to
determine whether the lead ban resulted in
significant reduction of environmental lead
levels over time. The following results were
obtained.

Year # samples Mean Std Dev

1976 80 34.47 4.3
1982 80 32.88 4.4
1987 80 31.78 4.5
 ANOVA – Lead Contamination
Source df SS MS F F-crit

Year (between groups 2 292.0 146.0 7.53 2.996

Error (within groups 237 4589.9 19.4

Total 239 ?

• You must have all of the original data to calculate

the values for SSyear and SStotal .
• F-crit can be found on Table B.7, last row (), 3rd
column (2).
 CRD = Completely Randomized
Design
• An experiment with multiple independent
groups of data is a “Completely Randomized
Design” (CRD)
– ANOVA is like a t-test among multiple independent
groups of data
• If specific data points between groups are linked
(like in a paired t-test) then it is not single-
factor ANOVA
– Linking data between groups with some other factor
is called “blocking”