Anda di halaman 1dari 51

Siska Nopa Tambunan_1031100_National Chiayi University

Instructional Objectives
Describe characteristics of experimental research
Define levels of the independent variable
Explain the concept of control and its importance in experimental research
Distinguish between true control groups and comparison groups in
experimental research
Define internal and external validity
Identify threats to internal validity and strategies for avoiding or
minimizing them
Identify internal validity problems in research proposals and reports
Describe the difference between random selection and random assignment
and show how they are related to internal and external validity

Siska Nopa Tambunan_1031100_National Chiayi University


Experimental Research
An experiment is a scientific investigation in which the researcher
manipulates one or more independent variables, controls any other
relevant variables, and observes the effect of the manipulations on the
dependent variables.

Experiment has 3 characteristics:


1. An independent variable is manipulated
2. All other variables that might affect the dependent variable
are held constant
3. The effect of the manipulation of the independent variable on
the dependent variable is observed

Siska Nopa Tambunan_1031100_National Chiayi University


Characteristics of Experimental
Research

Control Over Extraneous Variable


Manipulation
Observation and Measurement

Siska Nopa Tambunan_1031100_National Chiayi University


Control Over Extraneous Variable

The purpose of control in an experiment is to arrange a


situation in which the effect of a manipulated variable
on a dependent variable can be investigated
An extraneous variable is not related to the purpose of
the study but may affect the dependent variable

Siska Nopa Tambunan_1031100_National Chiayi University


Manipulation

Manipulation of an independent variable


is a deliberate operation performed by the
experimenter

Levels of the independent variable is the


different treatment conditions administered
to the subjects in the experiment

Siska Nopa Tambunan_1031100_National Chiayi University


Observation & Measurement
After applying the experimental treatment,
the researcher observes to determine if the
hypothesized change has occured.
For the variable that cannot observed
directly, then measurement is used.

Siska Nopa Tambunan_1031100_National Chiayi University


Experimental Comparison
A group comparison is the process of a
researcher obtaining scores for individuals or
groups on the dependent variable and
comparing the means and variance both within
the group and between the groups.

Siska Nopa Tambunan_1031100_National Chiayi University


Experimental Design
The experimental design sets up the
conditions required for demonstrating
cause-effect relationships.
Experimental Design differ in number of
independent variable, method of assigning
subjects and differ in how often dependent
variable measured.

Siska Nopa Tambunan_1031100_National Chiayi University


Validity of Research Designs
There are four types of validity:
- Internal Validity
The validity of inferences about whether the effect of variable A (the
treatment) on variable B (the outcome) reflects a causal relationship
- External Validity
The validity of the inference about whether the cause-effect relationship
holds up with other subjects, settings, and measurement
- Construct Validity
The validity of the inferences about psychological constructs involve in
the subject, settings, treatment and outcome
- Statistical Conclusion Validity
The validity of the inferences about the covariation between treatment
and outcome

Siska Nopa Tambunan_1031100_National Chiayi University


Threats to Internal Validity
History
Maturation
Testing
Instrumentation
Statistical Regression
Selection Bias
Experimental mortality (attrition)
Selection-maturation interaction
Experimenter Effect
Subject Effect
Diffusion

Siska Nopa Tambunan_1031100_National Chiayi University


Threats to Internal Validity

Siska Nopa Tambunan_1031100_National Chiayi University


Dealing With Threats to Internal
Validity
Random Assignment
Randomized Matching
Homogeneous Selection
Building Variables into the Design
Statistical Control
Using Subjects as their Own Control
Controlling Situational Differences

Siska Nopa Tambunan_1031100_National Chiayi University


Dealing With Threats to Internal
Validity
Random assignment is the process of assigning individuals
at random to groups or to different groups in an experiment.
Randomized matching is the process of identifying one or
more personal characteristics that influence the outcome and
assigning individuals with that characteristic equally to the
experimental and control group.
Homogeneous selection, makes groups reasonably
comparable on an extraneous variable by selecting sample
that are as homogeneous as possible on that variable.
Analyis of covariance (ANCOVA) is a statistical technique
used to control for the effect of an extraneous variable known
to be correlated with the dependent variable.

Siska Nopa Tambunan_1031100_National Chiayi University


Threats to External Validity
Selection-treatment interaction
(nonrepresentativeness)
Setting-treatment interaction (artificiality)
Pretest-treatment interaction
Subject effects
Experimenter effects

Siska Nopa Tambunan_1031100_National Chiayi University


Threats to External Validity

Siska Nopa Tambunan_1031100_National Chiayi University


Dealing With Threats to External
Validity
Randomly sample the target population to select subjects for the
study and then randomly assign them to treatment groups
Identify the relevant characteristics of individuals in the target
population, and determine the impact of these characteristics by
incorporating them into the research study
Control problem from pretest-treatment by choosing a design
that not use pretest
Control the reactive effects by arranging for a second control
group to experience an interesting interaction with the
researchers like experimental group
Replicate the research study in new setting

Siska Nopa Tambunan_1031100_National Chiayi University


Instructional Objectives
Explain the function of a research design
Explain the relationship between research design and internal validity
Define preexperimantal design and discuss its limitations
Describe the most common randomized experimental designs and
state the advantages of each
Define factorial design and state its advantages
Distinguish between-groups and within groups experimental designs
Define quasi experimental and discuss its limitations
Describe single-participant experimental research and its uses

Siska Nopa Tambunan_1031100_National Chiayi University


Experimental Designs Meaning
The general plan for carrying out a study with
an active independent variable (Ary et al,
2014)

The traditional approach to conducting


quantitative research where you test an idea
(or practice or procedure) to determine
whether it influences an outcome or dependent
variable (Creswell, 2012)

Siska Nopa Tambunan_1031100_National Chiayi University


Classifying Experimental Designs
Preexperimental Designs
True Experimental Designs
Factorial Designs
Repeated-measures Design
Quasi-Experimental Designs
Time Series Designs
Single Subject

Siska Nopa Tambunan_1031100_National Chiayi University


Preexperimental Designs
Provide little or no control of extraneous variables.

There are 2 types:


Design 1: One-Group Pretest-Posttest Design
Design 2 :Static Group Comparison

Siska Nopa Tambunan_1031100_National Chiayi University


Design 1: One-Group Pretest-Posttest
Design
Conducted in three steps:
1. Administering a pretest to measure the dependent
variable;
2. Applying experimental treatment X to the subjects; and
3. Administering a posttest to measure dependent variable
No control of extraneous variable; history and maturation
No way to assess the effect of the pretest
Design 1: One-Group Pretest-Posttest Design
Pretest Independent Posttest
Y1 X Y2

Siska Nopa Tambunan_1031100_National Chiayi University


Design 2: Static Group Comparison
The static group comparison uses two or more preexisting
or intact (static) groups, only one of which is exposed to the
experimental treatment
No randomized or matching
No pretest
Design 2: Static Group Comparison
Group Independent Posttest
E X Y2
C - Y2

Siska Nopa Tambunan_1031100_National Chiayi University


True Experimental Designs
Subjects are randomly assigned to an experimental group and at
least one control group

There are 5 types :


Design 3: Randomized Subjects, Posttest-Only Control Group
Design
Design 4: Randomized Matched Subjects, Posttest-Only Control
Group Design
Design 5: Randomized Subjects, Pretest-Posttest Control Group
Design
Design 6: Solomon Three-Group Design
Design 7: Solomon Four-Group Design

Siska Nopa Tambunan_1031100_National Chiayi University


Design 3 : Randomized Subjects, Posttest-Only
Control Group Design

Design 3 is one of the simplest yet most powerful of all experimental


designs
Randomized and have a control group
Control main effects of history, maturation, regression, and presenting
No pretest
Design 3: Randomized Subjects, Posttest-Only Control Group
Design

Group Independent Posttest


E X Y2
C - Y2

Siska Nopa Tambunan_1031100_National Chiayi University


Design 4 : Randomized Matched Subjects,
Posttest-Only Control Group Design
Similar to Design 3, except it uses a matching technique to form
comparison groups
Matching : Select pairs of individuals with identical or almost
identical characteristics and randomly assign one member of the
matched pair to treatment A and the other to treatment B

Design 4 : Randomized Matched Subjects, Posttest-Only Control Group Design

Group Independent Posttest


E X Y2
C - Y2

Siska Nopa Tambunan_1031100_National Chiayi University


Design 5: Randomized Subjects, Pretest-
Posttest Control Group Design
Randomly assigns subject to the experimental and control groups
and administers a pretest measuring the dependent variable Y.
To determine the effectiveness of the treatment, compare the two
groups mean scores on the posttest. If no significant result,
compare the average between posttest and pretest (Y2-Y1) scores for
each group.
Design 5: Randomized Subjects, Pretest-Posttest Control Group Design

Group Independent Posttest


(R) E X Y2
(R) C - Y2

Siska Nopa Tambunan_1031100_National Chiayi University


Design 5: Randomized Subjects, Pretest-
Posttest Control Group Design

Using analysis of covariance (ANCOVA)with


posttest as the dependent variable and pretest as
covariate to control any initial differences on the
pretest.
Threats external validity : pretest-treatment
interaction

Siska Nopa Tambunan_1031100_National Chiayi University


Design 6: Solomon Three-Group Design

Identical with design 5, but the advantage is of employing a


second control group labeled C2 that is not pretested but is
exposed to the treatment X
The interactive effect of pretesting and the experimental
treatment
Design 6: Solomon Three-Group Design
Group Pretest Independent Posttest
(R) E Y1 X Y2
(R) C1 Y1 - Y2
(R) C2 - X Y2

Siska Nopa Tambunan_1031100_National Chiayi University


Design 7 : Solomon Four-Group Design
Extending Design 6 to include one more control group that receives neither
pretest nor treatment
Design 7: Solomon Four-Group Design
Group Pretest Independent Posttest
(R) E Y1 X Y2
(R) C1 Y1 - Y2
(R) C2 - X Y2
(R) C3 - - Y2

Combination of the randomized subjects pretest-posttest control group design


and randomized subjects posttest-only control group design.
The disadvantages are more time and effort are required to conduct two
experiments simultaneously, problem about number of subjects, difficulty of
statistical analysis

Siska Nopa Tambunan_1031100_National Chiayi University


Factorial Designs
Researcher manipulates two or more
variables simultaneously in order to study
the independent effect of each variable on
the dependent variable, as well as the
effects caused by interactions among the
several variables.

Siska Nopa Tambunan_1031100_National Chiayi University


Design 8: Simple Factorial Design
The simplest factorial design is the 2 x 2 read as
2 by 2. this design has two variables (factors)
and each factor has two levels
Variable 1 (X1)
Variable 2 (X2)
Treatment A Treatment B
Level 1 Cell 1 Cell 3
Level 2 Cell 2 Cell 4

Siska Nopa Tambunan_1031100_National Chiayi University


Design 8: Simple Factorial Design
The effect of one independent variable on the
dependent variable irrespective of the influence of
the other independent variable is referred to as
main effect
Main effect does not mean the most important effect
but rather the effect of one independent variable
(factor) ignoring the other factor
Interaction effect between two independent
variables that is the different effects of one of them
at different levels of the other.

Siska Nopa Tambunan_1031100_National Chiayi University


Repeated-measures Design
A design in which a researcher observes each
individual in all of the different treatment is called a
within-subjects design.
It is also called repeated-measures design because
the research repeats measurement of the same
indiduals under different treatment conditions

Siska Nopa Tambunan_1031100_National Chiayi University


Quasi-Experimental Designs
Quasi-Experimental Designs are similar to
randomized experimental designs that they involve
manipulation of an independent variable but differ in
that subjects are not randomly assigned to treatment
groups.

There are 2 types:


Design 9: Nonrandomized control group, pretest-
posttest design
Design 10: Counterbalanced Design

Siska Nopa Tambunan_1031100_National Chiayi University


Design 9: Nonrandomized Control Group,
Pretest-posttest Design

Group Pretest Independent Posttest


E Y1 X Y2
C Y1 - Y2

One of the most widely used in educational research.


This design does not permit random assignment of
subjects to the experimental and control group.

Siska Nopa Tambunan_1031100_National Chiayi University


Design 9: Nonrandomized Control
Group, Pretest-posttest Design

Internal Validity Threats


Interaction of selection and maturation
Interaction of selection and regression
Interaction of selection and instrumentation

Siska Nopa Tambunan_1031100_National Chiayi University


Design 10: Counterbalanced Design
Used with intact class groups, rotates the groups at
interval during the experimentation
Design 10: Counterbalanced Design
Experimental Treatments
Replication X1 X2
1 Group 1 Group 2
2 Group 2 Group 2

Siska Nopa Tambunan_1031100_National Chiayi University


Time Series Designs
There are 2 types :
Design 11: One-Group Time-Series Design
Design 12: Control Group Time-Series Design

Siska Nopa Tambunan_1031100_National Chiayi University


Design 11: One-Group Time-Series Design
The one-group time series is a quasi-experimental design that
involves periodic measurement on one group and the
introduction of an experimental treatment into this time series
of measurements.
Design 11: One-Group Time-Series Design
Y1 Y2 Y3 Y4 X Y5 Y6 Y7 Y8

The major weakness of design 11 is its failure to control


history: that is, you cannot rule out the possibility that it is not
X but rather some simultaneous event that procedures the
observed change

Siska Nopa Tambunan_1031100_National Chiayi University


Design 12: Control Group Time-Series Design

Control group time-series design is an extension of Design


11 to include a control group.
This design overcomes the weakness of Design 11 that is
failure to control history as a source of extraneous variance.

Design 12: Control Group Time-Series Design

E Y1 Y2 Y3 Y4 X Y5 Y6 Y7 Y8
C Y1 Y2 Y3 Y4 - Y5 Y6 Y7 Y8

Siska Nopa Tambunan_1031100_National Chiayi University


Single-Subject Experimental Designs

Single-subject experimental designs share a unique


feature: the sample size is just one, or is composed of a few
participants who are treated as one unit.
Researcher measure the dependent variable repeatedly
during at least two different points in time, when the
treatment is not present and again when a treatment is
present.
Treatment periods: the periods when the treatment is given
Baseline periods: the treatment is not present.

Siska Nopa Tambunan_1031100_National Chiayi University


Single-Subject Experimental Designs
ABAB Designs
Consist of a period of no treatment, or baseline (A), during which the
behavior of interest is repeatedly measured.

Multiple-Baseline Designs
In this design, observations are made on several participants, different
target behaviors of one or more subjects, or different situations.
There are 3 basic designs: multiple-baseline across-participants
design, multiple-baseline across-behaviors design, and multiple-
baseline across-settings design.

Siska Nopa Tambunan_1031100_National Chiayi University


Comparison of Single-subject and
Group Designs

The goal of both designs is to establish as unequivocally as


possible the connection between the manipulation of the
independent variable (treatment) and its effect on the
dependent variable (behavior)
Random assignment of subject eliminates many rival
explanation of differences
Single-participant design controls amount of time which
baseline and treatment.
Single-subject threats to external validity generalizability
of experimental findings

Siska Nopa Tambunan_1031100_National Chiayi University


Example of Article
True Experimental Designs

Independent Variable: Multiple Intelligences Approach


Dependent Variable: Students achievement and attitude toward science
Experimental Group: Multiple Intelligences Approach
Control Group: Traditional Teaching
Research Design: Pretest-Posttest Control Group (Design 5)
Sample 60 students randomly selected from 183 students
Randomly assigned to the experimental and control groups

Siska Nopa Tambunan_1031100_National Chiayi University


Example of Article
Quasi-Experimental Designs

Independent Variable: Collaborative Writing Activity using Google Docs


Dependent Variable: Students Writing Abilities
Experimental Group: Collaborative Writing Activity using Google Docs
Control Group: Group face to face
Research Design: Quasi-experimental Design (Design 9)
Sample 80 students (2 sections) cluster sampling from 5,625 students
(123 sections)
No Randomly assigned to the experimental and control groups
Content Validity by three experts
Pretest for both groups

Siska Nopa Tambunan_1031100_National Chiayi University


Continue

Quasi-Experimental Designs
Threats to internal Validity (analysis result)
Subject Effects (Hawthorne effect):
The students know that their teacher can check who
works less or more throughout the learning process. As
such, student in experimental group were more serious
about collaborating and willing to follow the group
convention and practices

Siska Nopa Tambunan_1031100_National Chiayi University


References
Ary, D., Jacobs, L. C., Sorensen, C. K., & Walker, D. A. (2014). Introduction to
Research in Education 9th Edition. Wadsworth, USA: Cengage Learning.
Creswell, J. W. (2012). Educational Research 4th Edition. Boston: Pearson
Education.
Suwantarathip, O., & Wichadee, S. (2014). The effects of collaborative
writing activity using google docs on students writing abilities. The
Turkish Online Journal of Educational Technology, 13(2), 148-156
Kaya, O. N., Dogan, A., Gokcek, N., Kilic, Z., & Kilic, E. (2007). Comparing
multiple intelligences approach with traditional teaching on eight grade
students achievement in and attitudes toward science. Presented at The
Annual Meeting of The American Educational Research Association Chicago.

Siska Nopa Tambunan_1031100_National Chiayi University


Siska Nopa Tambunan_1031100_National Chiayi University