Experimental Research

12 1

LEARNING OUTCOMES After studying this chapter, you should be able to

1. 2. 3. 4. 5. 6. 7. Identify the independent variable, dependent variable, and construct a valid simple experiment to assess a cause and effect relationship Understand and minimize experimental error Know ways of minimizing experimental demand characteristics Avoid unethical experimental practices Understand the advantages of a between-subjects experimental design Weigh the trade-off between internal and external validity Use manipulations to implement a completely randomized experimental design, a randomized-block design, and a factorial experimental design

122

 Subjects
The sampling units for an experiment, usually human respondents who provide measures based on the experimental manipulation.

Independent Variables
Experimental conditions One of the possible levels of an experimental (independent) variable manipulation. Blocking variables Variables included in the statistical analysis as a way of controlling or accounting for variance due to that variable: Categorical variables

123

EXHIBIT 12.1

Experimental Conditions in Self-Efficacy Experiment

124

 Main Effect
The experimental difference in dependent variable means between the different levels of any single experimental variable.

Interaction Effect
Differences in dependant variable means due to a specific combination of independent variables.

125

EXHIBIT 12.2

Job Satisfaction Means in Self-Efficacy Experiment

126

EXHIBIT 12.3

Experimental Graph Showing Results within Each Condition

127

 Manipulation of the Independent Variable

Experimental treatment: the way an experimental variable is manipulated. Categorical variables: described by class or quality Continuous variables: described by quantity (level) Experimental Group A group of subjects to whom an experimental treatment is administered. Control Group A group of subjects to whom no experimental treatment is administered.

128

 More than One Independent Variable

Cell: a specific treatment combination associated with an experimental group.

Computation of the number of cells in an experiment:

K = (T1)(T2)..(Tm)
Repeated Measures
Experiments in which an individual subject is exposed to more than one level of an experimental treatment.

129

 Selection and Measurement of the Dependent Variable

Selecting dependent variables that are relevant and truly represent an outcome of interest is crucial. Choosing the right dependent variable is part of the problem definition process. Thorough problem definition will help the researcher select the most important dependent variable(s).

12 10

 Selection and Assignment of Test Units

Test units: the subjects or entities whose responses to treatment are measured or observed.

Sample Selection And Random Sampling Errors

Systematic or nonsampling error Subject selection, experimental design, and unrecognized extraneous variables Overcoming sampling errors Randomization Matching Control over extraneous variables

12 11

 Sample Selection And Random Sampling Errors

Experimental Confound When there is an alternative explanation beyond the experimental variables for any observed differences in the dependent variable. Once a potential confound is identified, the validity of the experiment is severely questioned. Extraneous variables Variables that naturally exist in the environment that may have some systematic effect on the dependent variable.

12 12

 Demand Characteristic
An experimental design element or procedure that unintentionally provides subjects with hints about the research hypothesis.

Demand Effect
Occurs when demand characteristics actually affect the dependent variable.

Hawthorne Effect
People will perform differently from normal when they know they are experimental subjects.

12 13

12 14

 Constancy of Conditions
Subjects in all experimental groups are exposed to identical conditions except for the differing experimental treatments.

Counterbalancing
Attempts to eliminate the confounding effects of order of presentation by varying the order of presentation (exposure) of treatments to subject groups.

12 15

 Debriefing experimental subjects

Communicating the purpose of the experiment Explaining the researchers hypotheses

Attempts to interfere with a competitors test-marketing efforts

Such acts as changing prices or increasing advertising to influence (confound) competitors test-marketing results are ethically questionable.

12 16

 Basic versus Factorial Experimental Designs

Basic experimental designs a single independent variable and a single dependent variable. Factorial experimental design allows for an investigation of the interaction to two or more independent variables.

Laboratory Experiment
A situation in which the researcher has more complete control over the research setting and extraneous variables.

Field Experiments
Research projects involving experimental manipulations that are implemented in a natural environment.
12 17

EXHIBIT 12.5

The Artificiality of Laboratory versus Field Experiments

12 18

 Within-Subjects Design
Involves repeated measures because with each treatment the same subject is measured.

Between-Subjects Design
Each subject receives only one treatment combination. Usually advantageous although they are usually more costly. Validity is usually higher.

12 19

EXHIBIT 12.6

Within- and BetweenSubjects Designs

12 20

 Internal Validity
The extent that an experimental variable is truly responsible for any variance in the dependent variable. Does the experimental manipulation truly cause changes in the specific outcome of interest?

Manipulation Checks
A validity test of an experimental manipulation to make sure that the manipulation does produce differences in the independent variable.

12 21

History

Maturation

Mortality

Internal Validity

Testing

Selection

Instrumentation

12 22

 History Effect
Occurs when some change other than the experimental treatment occurs during the course of an experiment that affects the dependent variable. Cohort Effect A change in the dependent variable that occurs because members of one experimental group experienced different historical situations than members of other experimental groups.

12 23

 Maturation Effects
Effects that are a function of time and the naturally occurring events that coincide with growth and experience.

Testing effects
A nuisance effect occurring when the initial measurement or test alerts or primes subjects in a way that affects their response to the experimental treatments.

12 24

 Instrumentation Effect
A change in the wording of questions, a change in interviewers, or a change in other procedures causes a change in the dependent variable.

Selection
The selection effect is a sample bias that results from differential selection of respondents for the comparison groups, or a sample selection error.

Mortality Effect (Sample Attrition)

Occurs when some subjects withdraw from the experiment before it is completed.

12 25

Extraneous Variable
History Uncontrollable events occurring in the environment between before and after measurements
Maturation Changes in subjects during the course of the experiment Testing A before measure that alerts or sensitizes subject to the nature of experiment or second measure.

Example
A major employer closes its plant in test market area.

Subjects become tired during the experiment.

A questionnaire about the traditional role of women triggers enhanced awareness of females in an experiment.
12 26

Extraneous Variable
Instrument Changes in instrument result in response bias

Example
New questions about women are interpreted differently from earlier questions.

Selection Sample selection error because of differential selection comparison groups Mortality Sample attrition; some subjects withdraw from experiment

Control group and experimental group is self-selected group based on preference for soft drinks

Subjects in one group of a hair dying study marry rich widows and move to Florida

12 27

 External Validity
The accuracy with which experimental results can be generalized beyond the experimental subjects. Student surrogates: Atypical?

Trade-Offs Between Internal and External Validity

Artificial laboratory experiments usually are high in internal validity, while naturalistic field experiments generally have less internal validity, but greater external validity.

12 28

Laboratory Experiment

Field Experiment

Artificial: Low Realism

Few Extraneous Variables High control Low Cost Short Duration Subjects Aware of Participation

Natural: High Realism

Many Extraneous Variables Low control High Cost Long Duration Subjects Unaware of Participation
12 29

 Basic Experimental Design

An experimental design in which only one variable is manipulated.

Diagramming Experimental Designs: Symbols

12 30

 Quasi-experimental Designs
Experimental designs that do not involve random allocation of subjects to treatment combinations.

One Shot Design (After Only): X O1 One Group Pretest Posttest: O1 X O2 Static Group Design: Experimental X O1 Control O2
12 31

 PretestPosttest Control Group Design (BeforeAfter with Control)

Experimental R O1 X O2 Control R O3 X O4

Posttest Only Control Group (After-Only with Control)

Experimental R X O1 Control R O2

12 32

EXHIBIT 12.7

Product Preference Measure in an Experiment

12 33

EXHIBIT 12.8

Selected Time Series Outcomes

12 34

Completely Randomized Design

Randomized Block Design

Factorial

12 35

 Completely Randomized Design

An experimental design that uses a random process to assign subjects (test units) to treatment levels to investigate the effects of an experimental variable.

12 36

 Randomized Block Design

An extension of the completely randomized design in which a single categorical extraneous variable that might affect test units responses to the treatment is identified and the effects of this variable are isolated by being blocked out.

Blocking Variable
A categorical variable that is expected to be associated with different values of a dependent variable for each group. It effectively controls for an extraneous cause in experimental analysis.

12 37

EXHIBIT 12.9

Randomized Block Design

12 38

 Factorial Design
An experiment that investigates the interaction of two or more independent variables on a single dependent variable.

12 39

EXHIBIT 12.10

Factorial DesignSalary and Vacation

12 40

 Main effect
The influence of a single independent variable on a dependent variable.

Interaction effect
The influence on a dependent variable by combinations of two or more independent variables. Interaction occurs if the effect of one treatment differs at various levels of the other treatment.

12 41

EXHIBIT 12.11

A 2 2 Factorial Design That Illustrates the Effects of Sex and Ad Content on Believability

12 42

EXHIBIT 12.12

Graphic Illustration of Interaction between Gender and Advertising Copy

12 43