2.0 INTRODUCTION
Psychologists make decisions about hypothesised relationships between independent
and dependent variables based upon observations of behaviour. One way to
organise the observational process is to employ an experimental design.
This unit tries to acquaint you with control group design and two factor design
(factorial design) which are used as true experimental design in psychological
researches. It begins with the nature and basic elements of experimental design
and focuses on the terminology of experimental design. Further, you will find the
description of control group design and three types of control group one as post
test only, one as control group design, and one as pretest posttest control group
design and one as Solomon four group design with relevant examples.
Moreover this unit continues with the discussion of the factorial design in which
you will study the nature of factorial design. This will provide answers to the
questions like how can you sketch a layout of factorial design, how basic
terminology of experimental design are worked out, how can you study the main
effect and interaction effect of the different variables and how can you interpret
your result with the help of graphical presentation. Finally, advantage and
14 disadvantage of factorial design are described.
Experimental Design
2.1 OBJECTIVES (Control Group Design and
Two Factor Design)
After reading this unit, you will be able to:
Explain experimental design;
Describe types of control group design;
Distinguish posttest only, control group design with pretest posttest control
group design;
Interpret main effect and interaction effect theoretically and graphically;
List advantage and disadvantage of factorial design; and
Draw the layout plan of various types of control group design and factorial
design.
R1 X O
R2 O
In the above design are two groups. One group R1 is given treatment (X), usually
called the experimental group, and the other group R2 is not given any treatment
and R2 is called the control group. Both groups are formed on the basis of
random assignment of the subjects and hence, they are equivalent. Not only that,
subjects of both groups is initially randomly drawn from the population (R). This
fact controls for selection and experimental mortality. Besides these, in this design
no pretest is needed for either group, which saves time and money. As both
groups are tested after the experimental group has received the treatment, the
most appropriate statistical tests would be those tests which make a comparison
between the mean of O1 and O2. Thus either t- test or ANOVA is used as the
appropriate statistical test.
Let us take an example. Suppose the experimenter, with the help of the table of
random numbers, selects 50 students out of a total of 500 students. Subsequently,
these 50 students are randomly assigned to two groups. The experimenter is
interested in evaluating the effect of punishment over retention of verbal task. The
hypothesis is that punishment enhances the retention score. One group is given
punishment (X) while learning a task and another group receives no such punishment
while learning a task. Subsequently, both groups are given the test of retention.
A simple comparison of mean retention scores of the two groups is carried out
through the t- test which provides the basis for refuting or accepting the hypothesis.
The only problem with the post-test is that there is no direct indication of what
actual change is found in the treatment group. This is corrected by measuring them
before and after the treatment. The control group is still useful as additional
factors may have had an effect, particularly if the treatment occurs over a long
time or in a unique context. 17
Research Designs 2.4.2.2 Pretest- Posttest Control Group Design
This is also called the classic controlled experimental design, and the randomized
pre-test/post-test design because it
1) Controls the assignment of subjects to experimental (treatment) and control
groups through the use of a table of random numbers.
This procedure guarantees that all subjects have the same change of being
in the experimental or control group. Because of strict random assignment
of subjects, it is assumed that the two groups are equivalent on all important
dimensions and that there are no systematic differences between the two
groups. Researchers may substitute matching for random assignment. Subjects
in the two groups are matched on a list of characteristics that might affect
the outcome of the research (e.g., sex, race, income). This may be cheaper
but matching on more than three or four characteristics is very difficult. And
if the researcher does not know which characteristics to match on, this
compromises internal validity.
2) Controls the timing of the independent variable (treatment) and which group
is exposed to it.
Both groups experience the same conditions, with the exception of the
experimental group, which receives the influence of the independent variable
(treatment) in addition to the shared conditions of the two groups.
3) Controls all other conditions under which the experiment takes place.
Nothing but the intervention of the independent (treatment) variable is assumed to
produce the observed changes in the values of the dependent variable.
The steps in the classic controlled experiment are:
1) Randomly assign subjects to treatment or control groups;
2) Administer the pre-test to all subjects in both groups;
3) Ensure that both groups experience the same conditions except that in addition
the experimental group experiences the treatment;
4) Administer the post-test to all subjects in both groups;
5) Assess the amount of change on the value of the dependent variable from
the pre-test to the post-test for each group separately.
These steps are diagramed as follows:
R O1 X O2
R O1 O2
18
Table: Diagram representing experimental design Experimental Design
(Control Group Design and
Scientific 1st observation Exposure to the 2nd observation Two Factor Design)
The difference in the control group’s score from the pre-test to the post-test
indicates the change in the value of the dependent variable that could be expected
to occur without exposure to the treatment (independent) variable X
Control group Post test scores – control group Pre test scores = control group
difference
Experimental group pre-test score – Experimental group post-test scores = the
difference obtained as a result of independent variable.
The difference in the experimental group’s score from the pre-test to the post-test
indicates the change in the value of the dependent variable that could be expected
to occur with exposure to the treatment (independent) variable X.
(Experimental group – (Experimental group = (Experimental group difference
pre-test score) post-test score) on the dependent variable)
The difference between the change in the experimental group and the change in
the control group is the amount of change in the value of the dependent variable
that can be attributed solely to the influence of the independent (treatment) variable
X.
Control group difference – experimental group difference = difference attributable
to X (the manipulation of the independent variable)
This design follows all the same steps as the classic pre-test/post-test design
except that it omits the pre-test. There are many situations where a pre-test is
impossible because the participants have already been exposed to the treatment,
or it would be too expensive or too time-consuming. For large groups, this
design can control most of the threats to internal and external validity as does
the classic controlled experimental design. For example, it eliminates the threat
to internal validity of pre-testing by eliminating the pre-test. It may also decrease
the problem of experimental mortality by shortening the length of the study (no
pre-test). For small groups, however, a pre-test is necessary. Also, a pre-test is
necessary if the researcher wants to determine the exact amount of change
19
attributable to the independent variable alone.
Research Designs For many true experimental designs, pretest-posttest designs are the preferred
method to compare participant groups and measure the degree of change occurring
as a result of treatments or interventions.
Pretest-posttest designs grew from the simpler posttest only designs, and address
some of the issues arising with assignment bias and the allocation of participants
to groups.
One example is education, where researchers want to monitor the effect of a new
teaching method upon groups of children. Other areas include evaluating the
effects of counseling, testing medical treatments, and measuring psychological
constructs. The only stipulation is that the subjects must be randomly assigned
to groups, in a true experimental design, to properly isolate and nullify any nuisance
or confounding variables.
Problems with Pretest-Posttest Designs
The main problem with this design is that it improves internal validity but sacrifices
external validity. There is no way of judging whether the process of pre-testing
actually influenced the results because there is no baseline measurement against
groups that remained completely untreated. For example, children given an
educational pretest may be inspired to try a little harder in their lessons, and both
groups would outperform children not given a pretest, so it becomes difficult to
generalise the results to encompass all children.
The other major problem, which afflicts many sociological and educational research
programs, is that it is impossible and unethical to isolate all of the participants
completely. If two groups of children attend the same school, it is reasonable to
assume that they mix outside of classrooms and share ideas, potentially
contaminating the results. On the other hand, if the children are drawn from
different schools to prevent this, the chance of selection bias arises, because
randomization is not possible.
The two-group control group design is an exceptionally useful research method,
as long as its limitations are fully understood. For extensive and particularly
important research, many researchers use the Solomon four group methods, a
design that is more costly, but avoids many weaknesses of the simple pretest-
posttest designs.
Factor B A1 A2
(Task complexity) (high) (low)
B1 (simple) A1 B1 A2 B1
B2 (complex) A1 B2 A2 B2
21
Research Designs Let us take an example to illustrate the meaning of factorial design.
Suppose the experimenter wants to know about how do the level of arousal and
task difficulty affect the task performance. Obviously, there are two independent
variables or factors. One is ‘arousal level’ and the other is ‘task complexity’. You
can denote the first independent variable as A and second independent variable
as B.
The dependent variable is task performance. Further, suppose the arousal level
is manipulated in two ways: high arousal and low arousal - these are two levels
of A. Let the high arousal be A1 and low arousal A2 . Similarly the task complexity
is manipulated in two ways: Simple and Complex Task. Let the simple task be
called B1 and the complex task be called B2. Now you see that both independent
variables have been manipulated in two ways. The resulting factorial design is
2*2.
In the simplest form, with two levels of each independent variable there would be
four groups of randomly assigned participants. They would receive treatments that
represent all possible combinations of the two levels of arousal high and low and
the two levels of task complexity simple and complex. This design is known as
a 2*2 factorial experiments with 2*2=4 groups in all. In the notation system we
have been using all along, the design for this factorial experiment could be
diagrammed as follows.
G1 R XA1B1 O1
G2 R XA1B2 O2
G3 R XA2B1 O3
G4 R XA2B2 O4
The hypothesised outcome of this study would be O2 > O1 and O3 > O4. That
is, the prediction would be that performance on the simple task would be better
under conditions of high arousal, while performance on the complex task would
be worse under conditions of higher arousal – an interaction or interactive effect
of the two independent variables.
Factorial experiments can also be represented in another way. Our hypothetical
experiment, with two independent variables each represented by two levels, could
be diagrammed as follows:
Task Complexity A
SimpleA1 ComplexA2
The four cells in this 2*2 table represent the four possible combinations of
treatments in 2*2 factorial experiments.
You can see that data in two left cells are performance of condition A1 that is,
the two simple task. Data in two cells on the right side are the performance of
condition of A2 that is, the two complex task. Therefore, we can average the data
in the two left cells and find average performance to simple task is 7.6, as shown
below the left column. Similarly, averaging the data in the two right cells gives the
average performance to the complex task, 13.1. Because 7.6 and 13.1 were
obtained by averaging the columns of the table, they are called the column
means. Looking at the column mean, we find that the complex task appear to
have taken more time than simple task. Therefore, it can be concluded that
complex task takes more time.
The effect of arousal level on performance in terms of time taken can be seen
by looking at the rows of the table. The two upper cells shows the performance
of condition B1 and the two lower cells show the performance of condition B2.
Averaging across the rows, you find that average performance in the low arousal
condition was 10.4 and the average for the high arousal was 10.3. Because 10.3
and 10.4 are obtained by averaging across the rows of the table they are called
the rows means. From the rows means it can be concluded that arousal difference
makes no difference at all to performance but when arousal combines with the
task complexity it is clear that arousal makes a big difference. However, the
nature of that difference becomes clear only when both independent variables are
taken into account.
It is now obvious that the factorial designs tell us the effect of each independent
variable individually. It indicates effect of arousal on performance and the effect
of task complexity on performance. These are the effects we would observe in
separate univariate experiments. These effects are known in statistical terms as the
main effects of independent variables. A main effect is the effect of one independent
variable averaged across the levels of the other independent variable.
Factorial design also tells us about the joint effect of two independent variables
or interaction between the two variables. These variables interact if effect of one
variable depends on the level of the other. You can see the interaction in the above
cited example here because effect of arousal depends on the task complexity.
(refer to graphs given below)
26
3) The experimental results of a factorial experiment are more comprehensive Experimental Design
(Control Group Design and
and can be generalised to a wider range due to the manipulation of several Two Factor Design)
independent variables in one experiment. From this point of view the single
IV experiments suffer a major setback.
4) In factorial experiments there is an additional gain occurring due to the
hidden replication arising from the factorial arrangement itself.
Disadvantages
Despite these advantages, factorial design has some disadvantages which are as
follows:
1) Sometimes the experimental setup and the resulting statistical analysis become
so complex that the experimenter may wish to drop this design and return
to a single IV experiment. This is especially true when more than three
independent variables each with three or more levels are to be manipulated
together.
2) In factorial experiments when the number of treatment combinations or
treatments becomes large, it becomes difficult for the experimenter to select
a homogeneous experimental unit (or subject).
3) Sometimes, it happens that some treatment combinations arising out of the
simultaneous manipulation of several independent variables becomes
meaningless. Then, the resources spent in those combinations are simply
wasted.
28