Anda di halaman 1dari 54

Sources of Knowledge

The Scientific Method

• 4 Steps
• Observe a phenomenon
• Formulate hypotheses – tentative statements
of cause & effect
• Further observation / experimentation to rule
out alternative explanations
• Refining & retesting of explanations
• Provides a check on the validity of its
Hypothesis Formation
Library Research Based on Lib. Rsh., propose
Learning “what is known” some “new knowledge” Research Design
about the target behavior Determine how to
obtain the data to test
the RH:
The “Research Loop”
Data Collection
• Novel RH:
Choose sample,
Draw Conclusions • Replication measures, method
Decide how your “new • Convergence
knowledge” changes
“what is known” about
the target behavior Data Analysis
statistical analysis
Hypothesis Testing
Based on design properties
and statistical results
Scientific Explanations
• Accepted because they have the
following properties:
• Empirical: based on objective &
systematic observation with the senses
• Rational: follows rules of logic,
consistent with known facts
• Testable: able to be verified, disproved
• Parsimonious: uses fewest assumptions
• General: apply to broad circumstances
• Tentative: admittedly imperfect
• Rigorously evaluated
Scientific Explanations
• Mechanistic
• Describes the physical components and the
chain of cause and effect through which the
behavior is produced
• Describes how it works (but not why)
• Functional
• Describes something in terms of what is it
does, but not how it happens
• Need both for a full understanding of the
Research Hypotheses
• The whole research process revolves around
hypotheses -- literature reviews to form them,
designs to generate data to be analyzed to test
them, replication and convergence of them, etc.

• A research hypothesis is a tentative explanation

or “guess” about the target behavior
• What you will find when you complete your
research and data analysis
Basic vs. Applied
• Basic research
• Conducted in order to test theories
or empirical positions – basic goal
is to obtain general information
about a phenomenon – not really
interested in applying finding to
real-world situations
• Applied research
• Purpose is to investigate a
problem that occurs in the real
Sources of Research
• Research starts with an interesting
• Experience
• Unsystematic Observation
• You read/see/hear/experience something, and
ask WHY?
• Systematic Observation
• Planned observation of real world behavior
• Previous observations of others (published
• Own research
• Internet research
Scientific Theories
• Theory - a partially verified
statement of a scientific
relationship that cannot be
directly observed (Martin, 1985)
Sources of Research
• Theory
• A developed set of assumptions and
rules about the causes of behavior
• Research can involve applying a theory
to a novel situation
• Testing competing theories about
Classifying Theories
• Can be classified along 3 dimensions
• Quantitative vs. qualitative
• Level of description
• Domain
• Quantitative Theory
• Theory expressed in mathematical terms
• Formulas used to explain behavior
• Qualitative Theory
• States relationship between variables in
verbal rather than mathematical terms
Classifying Theories
Level of description
• Descriptive Theory
• Merely describes the relationships
among variables – it does not explain
the relationship

• Analogical Theory
• Explains the relationships among
variables through analogies to something
that is well-understood
• E.g., Memory as a computer
Classifying Theories
• Fundamental Theory
• Proposes a new structure to
explain the relationships among
• Not only describes behavior, it
explains behavior – not through
analogies but through a new
• Highest level of theory
• E.g., Cognitive dissonance theory
Classifying Theories

The theory’s domain (scope)

• The range of situations to which

the theory may be applied
• E.g., Cognitive dissonance –
applicable to many situations
A Good Theory
• Ability to Account for Data
• Account for existing data and well-
established facts within its domain
• Explanatory Relevance
• Offer good grounds for believing that
the phenomenon would occur under
specified conditions
• Theory must define logical links
between variables
• Testability
• It must be capable of failing some
empirical test
A Good Theory

• Prediction of Novel Events

• Should predict phenomena that the
theory was not specifically
designed to account for but that
are within its domain
• Parsimony
• Should explain phenomena within
its domain with the fewest
possible assumptions and the
simplest terms possible
Testing Theories

• Confirmational Strategy
• Look for evidence to confirm
predictions from a theory
• Important part of theory testing,
but has limits
• Confirmation does not prove a
theory is correct
• Confirmation may occur when
predictions are too loosely defined
Testing Theories
• Disconfirmational Strategy
• Using a positive research result to
disconfirm a theory’s predictions
* Should use both strategies
• Begin with confirmational
• Can it explain the phenomena?
• Move to disconfirmational
• Do unexpected outcomes happen?
Strong Inference
• Theory is tested and modified based on
outcome of research and then tested
• Cycle of testing and modification
continues until theory adequately
accounts for behavior
• Several alternative explanations can
be tested with an experiment
• Some ruled out
• New experiment tests remaining
• Continue until only one alternative remains
Good Questions
• Not too broad
• Operational definitions – describe
how a variable will be measured
− Limits generalizability
Good Questions
• Ask IMPORTANT questions
• Clarify theoretical or empirical issues
• Support one hypothesis/theory over
• Address important practical issues
• Probably unimportant if:
• Answer already firmly established
• Small effects of no theoretical interest
• No reason to believe that 2 variables are
Purposes of Literature
• To fully describe the results from
prior research
• What is the “state of the knowledge”?
• To clearly state the purposes of the
• Purpose is to address some need
• To clearly state the hypotheses,
which should follow logically from
the literature review
Sources of Research
• Primary vs. Secondary Sources
• A primary source includes a full
report of a research study, including
methodological details
* Primary sources are preferred because
secondary sources may be biased or
author may not have discussed
something you may think is important.
• A secondary source summarizes
information from a primary source
(e.g. review article)
• Exception – meta-analysis
• Books
• General textbooks or specialized
professional publications
• Anthologies - assemble papers that
an editor feels are important in a
• Possibility for editor’s bias
• Most useful in early stages of
literature search
• Should be used with caution - may
not undergo rigorous review; info
Sources Cont.
• Scholarly Journals
• Current research and theoretical
• Refereed vs. nonrefereed journal
* Prefer refereed sources
• You can evaluate the quality of a
journal by
• Consulting Journals in Psychology
• Consulting the Social Science
Citations Index
• Using the method of authority
Sources Cont.
• Conventions and Professional
• Gatherings of researchers to
present findings
• Provide the most up-to-date info
• Advantages of attending
• Information is at the frontiers of
• Meet others in your field and
exchange ideas
Evaluating a Research
Article: The Introduction
• Has relevant research been adequately
• Are assertions supported with the
appropriate citations?
• Are the purposes of the study clearly
• Are the hypotheses clearly stated, and
do they flow logically from the info in
the introduction?
Evaluating a Research Article:
The Method Section
• Was the nature of the subject sample
• Does the design of the study allow an
adequate test of the hypotheses?
• Are there any methodological flaws
that might affect the validity of the
• Is sufficient detail presented to allow
one to replicate the study?
Evaluating a Research Article:
The Results Section
• Did the statistically significant effects
support or refute the hypotheses?
• Are the differences reported large or
• Were the appropriate statistics used?
• Do the tables, figures, and text match?
Evaluating a Research Article:
The Discussion Section
• Do the conclusions presented match the
results reported?
• If the author speculates about
implications of results, does he or she
stray too far from the results reported?
• How well do the results mesh with
existing theory and empirical data?
• Does the author point the way to
directions for future research?
Factors Affecting the Quality
of Research Information
• Statistical Significance
• Journals typically do not publish
findings that do not meet the
minimum .05 level of statistical

• File drawe r p henomenon: Findings

that don’t reach significance at .05
end up in the file drawer, perhaps
masking true effects
• Causal relationships
• One variable influences another
• Unidirectional vs. bidirectional
• Correlational relationships
• Changes in one variable associated with
changes in another variable (variables
• Can’t say one change causes the other
* Our ability to distinguish between the 2 types of
relationships depends on the level of control in
the study
Correlational Research

• Form of nonexperimental
• Goals
• Determine whether two variables
• Establish direction, magnitude,
forms of the relationship
• No variables are manipulated
• Instead just observed
Correlational Research
Descriptive Purposes
• Determine whether a relationship
exists between 2 variables

Predictive Purposes
• Knowledge of value of one variable
can help us predict value of related
• Predic to r v aria ble used to predict
the value of a Criterion variable
Correlation & Causality
• Correlational research ca nno t be
used to establish caus al
re lat ion ship s among variables –
correlation does not equal causation
• Causality: 3 conditions to say that A
causes B
• A precedes B
• A is related to B
• All alternative explanations have been
ruled out
Obstacles to Causal
• Third variable problem
• An unmeasured variable may account
for changes in both variables
• Both observed variables may vary together
although they are not directly related
• Examples
• Aggressive video games and aggression –
aggressive personality
• Ice cream sales and crime – hot weather
• There are ways to statistically
account for the third variable problem
Obstacles to Causal
• The directionality problem
• When a causal relationship does exist,
it is often hard to know the direction of
the effect
• Does A cause B, B cause A, or both?
• Examples
• Playing aggressive video games and
• Weight and frequency of exercise
When to Use Correlational
• Gathering data in the early stages
of research
• Identify possible causal relationships
which can then be tested
• Inability to manipulate variables
• Manipulating independent variable
may be impossible or unethical
• Relating naturally occurring
Experimental Research
• Involves high degree of control over
• Allows us to establish causal

• An Independent Variable is
• A variable whose values are chosen
and set by the experimenter
• Participants must be exposed to at
least two levels of this variable
Experimental Research
• A Dependent Variable is measured
• The variable whose value you observe (the
• The value of the dependent variable depends
on the participants’ behavior

• The most basic experiment consists of

an experimental and a control group
• The exper im ental gr oup receives the
• The cont rol group does not receive the
• Assignment to these groups/conditions must
be completely random
Experimental Research
• Control over extraneous variables
• Extraneous variables = other variables
besides the IV that may affect the DV
• May mask any influence of IV on DV or
may produce chance differences in DV
that are unrelated to IV
• To control these possible effects:
• Hold extraneous variables constant
• Random assignment of participants to
Research Settings
• Depends on: costs, convenience,
ethical considerations, research

• The laboratory setting

• Anywhere other than where the
behavior actually occurs – a formal
lab, a classroom, etc.
• Affords greatest control over
extraneous variables
Research Settings
• Simulations
• Attempt to recreate the real world in a lab
• When attempting to control extraneous
variables and interested in generalizing
results to real-world settings
• Realism is an issue
• Mundane real ism : How well does a
simulation mimic the real world event being
• Experi mental real ism : How engaging
(psychologically) is the simulation for
* Experimental realism is more important than
mundane realism
Research Settings
• The field setting
• Study conducted in a real world
environment – where the behavior
actually occurs
• Field experiment: Manipulate variables
in the field
• Could be nonexperimental, or contain
all the characteristics of a true
• Easily generalizable to the real world
• Little control over confounding
Internal Validity
• Internal validity
• The degree to which your design tests
what it was intended to test
• The degree to which inferences about
whether variations in the independent
variable cause variations in the
dependent variable are warranted
• The extent to which the research
design adequately tests your
Threats to Internal
• History
• Events may occur between multiple
• E.g., High profile news story
• Maturation
• Participants may become older or
• Repeated Testing
• Taking a pretest can affect results of a
later test
Threats to Internal
• Instrumentation
• Changes in instrument calibration or
observers may change results
• Statistical Regression
• Subjects may be selected based on
extreme scores
• Biased Subject Selection
• Subjects may be chosen in a biased
Threats to Internal
• Experimental Mortality
• Differential loss of subjects from
groups in a study may occur

• These threats make it impossible

to tell if results (changes in DV) are
due to changes in the IV
External Validity
• External validity - the degree to which
results can be accurately generalized
beyond your sample and research setting
• Other participants, tasks, stimuli, settings &

• Population: Will the results generalize to

other persons or animals ?
• Will a study of college students generalize to
your target population of “consumers”?
• Will a study of chronically depressed patients
transfer to those who are acutely depressed?
Components of External
• Setting: Will the findings apply to
other settings ?
• Will a laboratory study generalize to what
happens in the classroom?
• Will a study in a psychiatric hospital
generalize to a out-patient clinic?
• Societal/Temporal: Will the findings
continue to apply?
• Will a study conducted in 1965 generalize
to today?
• Will a study conducted today still be
useful 10 years from now? … 5 years
from now?
Components of External
• Task/Stimuli: Will the results
generalize to other tasks or stimuli ?
• Usually the participant is “doing
something” that directly or indirectly
generates the behavior that is being
• What do I learn about “consumer
decision making” from a study that asks
participants to select the best “widgit”?
• Will research using visual illusions
inform us about the perception of
everyday objects ?
Threats to External
• Degree of control
• Data obtained in highly controlled
lab settings may not generalize to
real-world situations

• Sample used
• Results may apply only to subjects
representing a unique group (i.e.
college students)
Threats to External
• Reactive effects of
experimental arrangements
• Participants’ knowledge that they
are research subjects may affect
results (demand characteristics)

• Multiple treatment interference

• Exposure to early treatments may
affect responses to later
Internal vs. External
• Trade-off characterization
• Impossible to promote both internal
and external validity within a single
• Steps taken to increase internal
validity may decrease external validity
and vice versa
• Must choose which will be emphasized
• Internal validity (control)
• External validity
Internal vs. External
• Internal validity may be more
important in basic research,
external validity  in applied
• Which is more important depends on
the reasons why the research is being

• Internal validity an important

• Without causal interpretability (from