QUANTITATIVE RESEARCH

Ferry Efendi, S.Kep., Ns., M.Sc., PhD

OVERVIEW

 For quantitative studies, no aspect of a study’s methods has a bigger

impact on the validity and accuracy of the results than the research
design—particularly if the inquiry is cause probing
 The research design of a study spells out the basic strategies that
researchers adopt to answer their questions and test their hypotheses
KEY RESEARCH
DESIGN FEATURES
CAUSALITY

 Many research questions and questions for evidence-based practice (EBP) are
about causes and effects
 Although it might be easy to grasp what researchers mean when they talk about a
cause, what exactly is an effect? A good way to grasp the meaning of an effect is
by conceptualizing a counterfactual (Shadish et al., 2002)
 A counterfactual is what would have happened to the same people exposed to a
causal factor if they simultaneously were not exposed to the causal factor
CRITERIA FOR ESTABLISHING CAUSE-AND-
EFFECT RELATIONSHIPS
 The first criterion is temporal: a cause must precede an effect in time
 Second, there must be an empirical relationship between the presumed cause
and the presumed effect
 The third criterion for inferring a causal relationship is that the relationship
cannot be explained as being caused by a third variable
 Coherence involves similar evidence from multiple sources, and the criterion of
consistency involves having similar levels of statistical relationship in several
studies
 Biologic plausibility, that is evidence from basic physiologic studies that a causal
pathway is credible
EXPERIMENTAL, QUASI-EXPERIMENTAL,
AND NONEXPERIMENTAL DESIGNS
EXPERIMENTAL DESIGN

Characteristics of True Experiments a true experimental design or randomized

controlled trial (RCT) is characterized by the following properties:
 Manipulation: the experimenter does something to some subjects that is, there is
some type of intervention.
 Control: the experimenter introduces controls into the study, including devising a
good approximation of a counterfactual usually a control group that does not
receive the intervention.
 Randomization: the experimenter assigns subjects to a control or experimental
condition on a random basis.
BASIC DESIGNS

 The most basic experimental design involves randomizing subjects to

different groups and then measuring the dependent variable. This design is
sometimes called a posttest-only (or after-only) design.
FACTORIAL DESIGN RESEARCHERS

 Sometimes manipulate two or more

independent variables simultaneously.
Suppose we were interested in
comparing two therapeutic strategies
for premature infants: tactile
stimulation versus auditory
stimulation.
 In factorial experiments, subjects are
assigned at random to a combination
of treatments
CROSSOVER DESIGN

 Designs is called between-subjects designs because the comparisons are

between different people.
 Involves exposing participants to more than one treatment.
 A crossover design has the advantage of ensuring the highest possible
equivalence among the subjects exposed to different conditions.
EXPERIMENTAL AND CONTROL
CONDITIONS
Among the possibilities for the counterfactual are the following:
1. No intervention—the control group gets no treatment at all
2. An alternative treatment (e.g., auditory versus tactile stimulation)
3. A placebo or pseudo-intervention presumed to have no therapeutic value
4. “Usual care”—standard or normal procedures used to treat patients
5. An attention control condition—the control group gets researchers’ attention
but not the active ingredients of the intervention
6. A lower dose or intensity of treatment, or only parts of the treatment
7. Delayed treatment (i.e., control group members are wait-listed and exposed to
the experimental treatment at a later point).
ADVANTAGES OF EXPERIMENTS

 Experimental designs are considered the “gold standard” for intervention studies
because they yield the highest quality evidence regarding the effects of an
intervention
 Experiments offer greater corroboration than any other research approach that, if
the independent variable is manipulated, then certain consequences in the
dependent variable may be expected to ensue
 Through the controls imposed by manipulation, comparison, and—especially—
randomization, alternative explanations to a causal interpretation can often be
ruled out or discredited
DISADVANTAGES OF EXPERIMENTS

 A number of interesting variables simply are not amenable to manipulation

 There are many variables that could technically—but not ethically—be
manipulated.
 Another potential problem is the Hawthorne effect
QUASI-EXPERIMENTS

 Quasi-experiments (called controlled trials without randomization in the

medical literature), also involve an intervention; however, quasi-
experimental designs lack randomization, the signature of a true
experiment
 Some quasi-experiments even lack a control group.
QUASI-EXPERIMENTAL DESIGNS

1. Nonequivalent Control Group Design

 The most frequently used quasi-experimental design
 Involves two or more groups of subjects observed before and after the
implementation of an intervention
 Identical to the before–after experimental design discussed in the previous
section, except subjects were not randomly assigned to the groups
 The quasi-experimental design is weaker because, without randomization, it
cannot be assumed that the experimental and comparison groups are
equivalent at the outset
 The term comparison group is sometimes used in lieu of control group to refer
to the group against which outcomes in the treatment group are evaluated.
QUASI-EXPERIMENTAL DESIGNS

2. Time–Series Design
 Involves collecting data over an extended time period, and introducing the
treatment during that period.
 Although the time–series design does not eliminate all the problems of
interpreting changes in turnover rate, the extended time perspective
strengthens the ability to attribute change to the intervention.
 The time–series design rules out the possibility that changes in resignations
represent a random fluctuation of turnover measured at only two points
QUASI-EXPERIMENTS
Advantages
 Introduce some research control
when full experimental rigor is not
possible.
 That people are not always willing
to be randomized in clinical trials.
Disadvantages
 That causal inferences cannot be
made as easily as with experiments.
 With quasi experiments, there is
almost always at least one plausible
rival explanation.
NON-EXPERIMENTAL STUDIES

 There are various reasons for doing a non-experimental study, including situations
in which the independent variable inherently cannot be manipulated or in which it
would be unethical to manipulate the independent variable.
 There are also research questions for which an experimental design is not
appropriate, such as studies whose purpose is description
CORRELATIONAL DESIGNS

 A correlation is an interrelationship or association between two variables, that is,

a tendency for variation in one variable to be related to variation in another (e.g.,
people’s height and weight).
 Investigators do not control the independent variable, which often has already
occurred
 Correlational studies with a retrospective design are ones in which a phenomenon
observed in the present is linked to phenomena occurring in the past.
 Correlational studies divided by case-control design and prospective design
CORRELATIONAL DESIGNS

 In designing a case-control study, researchers try to identify controls without the

disease who are as similar as possible to the cases with regard to key confounding
variables
 Correlational studies with a prospective design (called a cohort design by medical
researchers) start with a presumed cause and then go forward to the presumed
effect
 Prospective studies are more costly, but much stronger, than retrospective
studies. For one thing, any ambiguity about the temporal sequence of
phenomena is resolved in prospective research
DESCRIPTIVE RESEARCH

 The purpose of descriptive studies is to observe, describe, and document

aspects of a situation
 Researchers seek to describe relationships among variables, without
attempting to infer causal connections
NON-EXPERIMENTAL RESEARCH
Advantages
 Correlational research is often an
efficient and effective means of
collecting a large amount of data
about a problem
Disadvantages
 Its inability to illuminate causal
relationships with assurance
 Correlational studies are often
undertaken with an underlying desire
to discover causes
 Correlational results should be
interpreted cautiously, especially if
the research has no theoretical basis.
QUANTITATIVE DESIGNS AND RESEARCH
EVIDENCE
 Evidence for nursing practice depends on descriptive, correlational, and
experimental research
 These studies can provide hints about how to structure an intervention, who can
most profit from it, and when it can best be instituted.
 Thus, it is appropriate to think of alternative evidence hierarchies. For questions
about the effects of an intervention, experimental designs are the “gold standard”
and meta-analyses of multiple RCTs are at the pinnacle of the hierarchy. For
questions about prognosis or etiology/harm; however, strong prospective (cohort)
studies are usually the best design available for studying causal relationships.
THE TIME DIMENSION IN RESEARCH
DESIGN
 The research design incorporates decisions about when and how often data will
be collected in a study
 In many nursing studies, data are collected just once but other studies involve
data collection on multiple occasions
 The major distinction is between cross-sectional and longitudinal designs.
CROSS-SECTIONAL DESIGNS

 Involve the collection of data at one point in time (or multiple times in a short time
period, such as 2 hours and 4 hours postoperatively)
 Crosssectional designs are especially appropriate for describing the status of
phenomena or relationships among phenomena at a fixed point
 Retrospective studies are usually crosssectional: data on the independent and
dependent variables are collected concurrently but the independent variable
usually captures events or behaviors occurring in the past.
 Sometimes used to study time-related phenomena, but the designs are less
persuasive than longitudinal ones.
 The main advantage of cross-sectional designs is that they are economical and
easy to manage
LONGITUDINAL DESIGNS

 Researchers who collect data at more than one point in time over an extended
period
 Useful for studying changes over time and for ascertaining the temporal
sequencing of phenomena, which is an essential criterion for establishing
causality
 Longitudinal studies involve collecting data from different people in a population
to examine trends over time
 In a more typical longitudinal study, the same people provide data at two or more
points in time.
LONGITUDINAL DESIGNS

 Longitudinal studies of general (nonclinical) populations are sometimes called

panel studies
 Panel studies typically yield more information than trend studies because
researchers can examine correlates of change. That is, researchers can identify
individuals who did and did not change
 Panel studies are appealing as a method of studying change but are difficult and
expensive to manage
 The term longitudinal implies multiple data collection points over an extended
period of time.
 The most serious challenge in longitudinal studies is the loss of participants
(attrition) over time
TECHNIQUES OF RESEARCH CONTROL

1. Controlling the Study Context

Various external factors, such as the research environment, can affect study
outcomes. In carefully controlled quantitative research, steps are taken to minimize
situational contaminants (i.e., to achieve constancy of conditions for the collection
of data) so that researchers can be confident that outcomes reflect the influence of
the independent variable and not the study context.
TECHNIQUES OF RESEARCH CONTROL

2. Controlling Intrinsic Factors

- Randomization: is to secure a close approximation to an ideal counterfactual,
that is, to have groups that are equal with respect to confounding variables
- Homogeneity: which only subjects who are homogeneous with respect to
confounding variables are included in the study
- Matching: which involves using information about subject characteristics to
form comparable groups
- Statistical Control: researchers are increasingly using powerful statistical
techniques to control extraneous variables
- Evaluation of Control Methods
CHARACTERISTICS OF GOOD DESIGN

1. What is the strength of the evidence that a relationship exists between two
variables?
2. If a relationship exists, what is the strength of the evidence that the
independent variable of interest (e.g., an intervention), rather than other
factors, caused the outcome?
3. What is the strength of evidence that observed relationships are generalizable
across people, settings, and time?
4. What are the theoretical constructs underlying the related variables and are
those constructs adequately captured?
FOUR ASPECTS OF A STUDY’S VALIDITY

(1) Statistical conclusion validity;

(2) Internal validity;
(3) External validity; and
(4) Construct validity
STATISTICAL CONCLUSION VALIDITY

 Statistical tests are used to support inferences about whether or not such a
relationship exists
 Design decisions can influence whether statistical tests will actually detect true
relationships, and so researchers need to make decisions that protect against
reaching false statistical conclusions
INTERNAL VALIDITY

 Internal validity refers to the extent to which it is possible to make an inference that the
independent variable is truly causing or influencing the dependent variable
 Experiments tend to have a high degree of internal validity because randomization to
different groups enables researchers to rule out competing explanations
 Threats to Internal Validity
- Temporal Ambiguity
- Selection
- History
- Maturation
- Mortality/Attrition
EXTERNAL VALIDITY

 Concerns inferences about the extent to which relationships observed in a study

hold true for different people, conditions, and settings
 External validity has emerged as a major concern in an EBP world in which it is
important to generalize evidence from controlled research settings to real-world
practice settings.
 An important concept relevant to external validity is that of replication
CONSTRUCT VALIDITY

 Construct validity involves inferences from the particulars of the study to the
higher-order constructs they are intended to represent
 Construct validity is important because constructs are the means for linking the
operations used in a study to a relevant conceptualization and to mechanisms for
translating the resulting evidence into practice.
CRITIQUING QUANTITATIVE RESEARCH
DESIGNS
 If the research purpose is descriptive or exploratory, an experimental design is not
appropriate.
 If the researcher is searching to understand the full nature of a phenomenon
about which little is known, a highly structured design that allows little flexibility
might block insights
 Research control as a mechanism for reducing bias, but in certain situations, too
much control can introduce bias