5801 ANOVA

Learning objectives
1.! 2.! 3.! 4.! Using SSs to determine group differences Interpretation of SSs in ANOVA Overall vs. specific hypotheses Attitudes towards post hoc comparisons

Copyright 2013 by E. Kwan

5801 ANOVA

From regression to ANOVA

! ! ! As we noted regression involves studying the relationship between two continuous variables. There are situations where the predictor (independent variable) may be discrete. Examples ! How does the university attended by a CEO relate to that CEO's salary? ! How do Ontario universities differ with regards to GPA? ! How do different makes of cars differ in terms of owner satisfaction? ! ! ! In these situations, the relationship involves a discrete predictor and a continuous response. Such models are commonly known as ANOVA (analysis of variance) models. The underlying calculations and statistical theory of ANOVA models are identical to that of regression models.

Copyright 2013 by E. Kwan

5801 ANOVA

Just to clarify, "ANOVA" or "ANOVA models" refer to a statistical model where ! The response is a continuous variable. ! The predictor is a discrete variable (with 2 or more levels). A fundamental goal ! In regression: How the independent variable relates to the response. ! In ANOVA: How do the groups (i.e., levels of the discrete predictor) differ in terms of the response. Why not just call these regression models? ! Technically, there is nothing wrong with doing so. ! But ANOVA models were developed separately.

Copyright 2013 by E. Kwan

5801 ANOVA

Let us consider two situations

Situation 1 Situation 2

! ! ! !

In each situation we have 3 groups, where each group only has 2 observations. In Situation 1 the groups have very different means. Whereas in Situation 2 the groups all have near identical means. Line in the middle is the mean for all six observations (irrespective of group); call this the "grand" mean.

Copyright 2013 by E. Kwan

5801 ANOVA

Look at Situation 1 more closely

Situation 1

! !

We consider the group mean for each group, and add that to the plot. Next we will consider deviations of the observations.

Copyright 2013 by E. Kwan

5801 ANOVA

Deviations of an observation
Situation 1

A deviation from the grand mean can be split into two parts

A deviation of the observation from its group mean A deviation of the group mean from the grand mean

Copyright 2013 by E. Kwan

5801 ANOVA

Consider another observation

Situation 1

Deviation from group mean

A deviation of the group mean from the grand mean

Copyright 2013 by E. Kwan

5801 ANOVA

Summing up (the square of) such deviations

SSTO: Sum of squares total

SSE: Sum of squares error

SST: Sum of squares treatment

"treatment" because ANOVA originated in experiments (e.g., a treatment is a particular experimental manipulation)

Copyright 2013 by E. Kwan

5801 ANOVA

Let's compare Situation 1 and 2 again

SSTO: Sum of squares total

SSE: Sum of squares error

SST: Sum of squares treatment

Copyright 2013 by E. Kwan

5801 ANOVA

Copyright 2013 by E. Kwan

10

5801 ANOVA

Sums of squares
Situation 1 Situation 2

! ! !

Calculate SSTO, SSE, and SST in each situation. The comparison of these various SSs in each situation will tell us if the group means are different or not. They can also reflect how much do the group means differ (remember this is the fundamental goal of ANOVA).

Copyright 2013 by E. Kwan

11

5801 ANOVA

How to interpret sum of squares in ANOVA

Since deviations are closely related to variance

SSTO: Sum of squares total

SSE: Sum of squares error

SST: Sum of squares treatment

Reflects the total variance in the data.

Reflects the variance of the data around each group mean;

Reflects the variance of the group means around the grand mean;

Copyright 2013 by E. Kwan

12

5801 ANOVA

The usefulness of SSs in ANOVA

! ! ! With SSs no matter how many groups we have, we can summarize all the differences between group means by just three numbers: SSTO, SSE, and SST. The relative sizes of SST and SSE compared to SSTO will tell us to what extent the group means are different. Traditionally speaking, ANOVA refers to the situation we have seen here, where the SSs are calculated to determine how much several groups differ in terms of their means. Of course, if interest lies in comparison of means between specific groups, that can be done as well. In fact, there are many specialized techniques available for such comparisons. But almost always such specific comparisons are complemented by an assessment of overall differences via SSs.

! ! !

Copyright 2013 by E. Kwan

13

5801 ANOVA

Hypotheses in ANOVA
The overall hypothesis ! This hypothesis evaluates the overall differences between s; also called the omnibus hypothesis. ! Here H1 does not say that all means must be different, only that they're not all the same.

Specific hypotheses ! We may also have hypotheses that declare a particular pattern of difference between the groups, or between a certain subset of the groups; for example,

Copyright 2013 by E. Kwan

14

5801 ANOVA

Example: CEO salaries

Copyright 2013 by E. Kwan

15

5801 ANOVA

The need for specific comparisons/hypotheses

! ! ! ! While the omnibus hypothesis is important, clearly we also need to make specific comparisons between groups. Example: Establishing all Ontario universities have quite different average GPAs is useful, but we should also care about how certain universities fare relative to others. Evaluating the omnibus/overall hypothesis is only one aspect of an ANOVA. When research goes no further than demonstrate an overall significant F test, then very little has been learned. Specific comparisons provide better description of the means; such comparisons are often called post hoc comparisons, meaning "after-the-fact". When a researcher actually formulates post hoc comparisons, he/she is inclined to think more about the underlying theory. Post hoc comparisons could entail more precise hypotheses than the overall H1, thus greater demand is placed on the legitimacy of the theory.

! !

Copyright 2013 by E. Kwan

16

5801 ANOVA

The fear over too many post hoc comparisons

Example ! Suppose we conduct an ANOVA to compare the degree of driver satisfaction between 4 makes of cars (Honda, Toyota, Ford, Chrysler). ! Beyond just determining that the group means (on satisfaction) are different, we could compare H ! T F C

Or, we formulate more precise post hoc hypotheses H0: (H + T)/2 ! (F + C)/2 H1: (H + T)/2 < (F + C)/2

So what is wrong with doing all these comparisons (i.e., significance tests)? ! Every time we carry out one significance test, there is a chance for a Type I error. ! So when we carry out many significance tests, this chance gets bigger and bigger. ! We need a way to keep this error rate from getting too big.
Copyright 2013 by E. Kwan 17

5801 ANOVA

Attitudes toward post hoc comparisons

Traditional attitude ! Many believe that such comparisons should only be conducted if the omnibus null hypothesis has been rejected (i.e., unless we have demonstrated that group means do differ, we are not justified to carry on any more comparisons). ! Thus, if H0 has not been rejected, any subsequent comparisons might reflect "spurious" differences. More sensible attitude ! Post hoc comparisons should not be dictated by the results of the overall F test. ! Such comparisons, when they are of interest to us, should always be carried out (supplemented by CIs and effect sizes), even if the overall F test is statistically nonsignificant. ! Don't worry so much about Type I errors. ! The protection against spurious findings is not necessary.

Copyright 2013 by E. Kwan

18

5801 ANOVA

Special procedures for post hoc comparisons

! ! There are many such procedures (e.g., Bonferroni adjustment, Tukey's HSD, Scheffe's LSD). They all serve the same purpose: Protection against "spurious" findings (i.e., finding a mean difference when one does not exist, which is essentially a Type I error).

The Bonferroni adjustment is the easiest to understand ! Suppose we have deemed that a 0.05 Type I error rate is acceptable. ! Then, if we need to carry out 10 post hoc comparisons (and conduct a significance test for each one), then do so at a pre-set Type I error rate of 0.05/10 for each test. ! So the Type I error rate for all 10 tests won't be bigger

Copyright 2013 by E. Kwan

19