1

Chapter 3: Alternatives to
Experimentation: Non
Experimental Designs
Learning Objectives
Understand the pros and cons of different
research methodologies
Introduce the concepts of internal and external
validity
Explore some of the issues related to
constructing a valid survey
Identify different types of measurement
techniques used in surveys

2
Experimental vs. Non-
Experimental
Experimental (manipulation, random assignment,
control over variables)
Cause effect relationships
High in internal validity
Internal validity -- when a changes in behavior
can genuinely be attributed to the manipulation
Non Experimental (observation, no manipulation)
3
Advantages of Non-Experimental
Designs
rare instances
Quasi-Independent Variable -- a characteristic of an
individual that is used to group him/her for comparison
Not manipulated
used to create independent groups
E.g., High versus low; gender; year in school
High in external validity: external validity is the
ability to generalize the results to situations
outside the research setting
Experiments lower on external validity than most non
experimental designs

4
Describing Research Activities
two major dimensions
1) Control over the antecedent conditions
(independent variables)
2) Control over the subjects responses
(dependent variables) or control over the
imposition of units
5
Manipulation of Antecedent
Conditions
experiments are high on manipulation of
antecedent conditions
set up the situation for the participants
Or, set up quasi independent variables for
groups
6
Degree of Imposition of Units
Imposition of units: the degree to which the
researcher constrains or limits the responses
a subject can make
E.g., high -- using surveys with few choices;
low -- free response; low -- observing behavior
in real world settings
How much control is there over the subjects
responses?

7
Internal Validity and Confounds
Experiments tend to be high on internal validity
Experiments attempt to reduce plausible rival
hypotheses by controlling the environment and
randomly assigning people to conditions
Goal is to reduce confounds: confound is a
random variable that co-occurs with the
antecedent condition (IV) and acts as a plausible
interpretation for your results
Varies systematically with the IV
E.g., time of day and manipulation
8
Types of Research in Psychology
Descriptive
Simply describes behavior without trying to control or
manipulate the setting or the people involved;
objective and systematic Advantage: in-depth
Correlational
Research that investigates relationships between
variables Advantage: allows prediction
Experimental
Determines cause-effect explanations by manipulating
behavior in a controlled setting. Advantage: can
determine cause
9
Programs of Research
Multimethod approach: investigate research
problem using multiple methods
One methods limitation is offset by another
methods strength
10
11
Descriptive Research
preferred for its external validity
case studies
field studies
naturalistic observation
participant observer
survey research
12
Case Studies
descriptive record
one or just a few individuals (e.g., rare
disorder)
goal is to obtain general principles about
people by studying one person in depth
13
Elements of a Case Study
systematically record
low on manipulation
low on imposition of units
usually no hypotheses
describe does not explain (provides what not
why)
typical in clinical psychology, early brain research
Examples: Sybil, Phineas Gage, Intersex
individuals
14
Five Major Purpose of Case
Studies
source of hypotheses (serendipity)
develop therapeutic techniques
allow study of rare phenomenon
provide exceptions to the rule
persuasive value
15
Limitations of Case Studies
people may not be representative
missing data
retrospective data
social desirability
16
Field Studies
gather data in real life situations
outside lab
no manipulation
with hypothesis and systematic recording
17
Naturalistic Observation
type of field study
recording and observing predetermined by
hypotheses
different from just people watching
low in manipulation, low on control
coding strategy
reduce reactivity -- people act differently when the
know they are being watched (e.g., Hawthorne)
Naturalistic Observation in Lab (a blend); e.g.,
infants with parents and one-way mirror
18
Participant
Observer/Ethnography
Participant Observer: research method in which
researcher becomes an inside part of the group
not told that they are being studied
Ethnography: research method in which the
objective is to document the customs habits
actions of people culture or subculture
In-depth interviews, careful recording of conversations
and behavior
sensemaking and meaning
Importance of context
Limitations
friendships and objectivity; privacy; rejection

19
Advantages of Field Research
behavior in context
observer fleeting events
behavior as it occurs
compare observations to lab
20
Survey Research
common and easy
gather lots of information efficiently
questionnaires, interviews
self report method
self report occurs when participants
describes their own behavior, emotion, or
thought
21
Constructing a Questionnaire
representative content
content valid = survey has a
representative sample of the
behavior/attitude being assessed
E.g., exams and content validity
22
Measures in Survey Research
imposition of units
open ended
less structured
closed ended
more structured
23
Advantages/Disadvantages of
Types of Items
Open Ended +/-
researcher does not make assumptions
time consuming
Rambling
Low on imposition of units
Closed Ended +/-
easy to code
people forced to have an opinion
High on imposition of units
24
Scaling
most common measurement tools
use different numbers to represent response
Likert Scale
Resnis Likert
numbers associated with responses
1 = disagree; 7 = agree
higher numbers more agreement
25
Likert Scales
Please indicate your response by circling the response that best represents how you feel.
Remember there are no right or wrong answers, just your opinions are what matter.




1 2 3 4 5 6 7 I am afraid of losing my mind.
Disagree Agree




1 2 3 4 5 6 7 8 9 I often believe I am more nervous than other
people.
Disagree Agree

Middle Point
Middle Point
26
Forced Choice Scales
disadvantage of Likert people can choose 4
forced choice make people choose one
direction or the other
e.g., honest or intelligent
use even number on each scale
1 2 3 4 5 6 7 8 9 10 I often cross the street to avoid
meeting people.
Disagree Agree
No middle point
27
Semantic Differential
have participants evaluate a concept
assign meaning
evaluation (positive, negative)
potency (strong, weak)
activity (fast, strong)
28
Semantic Differential
Please rate the baby on the following scales:
1 2 3 4 5 6 7
Weak Strong
1 2 3 4 5 6 7
Inattentive Alert
1 2 3 4 5 6 7
Beautiful Plain
1 2 3 4 5 6 7
Hardy Delicate
29
Rating Errors in Surveys
we assume that people are objective
we assume people are truthful
we assume that people are interested
We assume that people have introspective
access
Psychological state in which people can
accurately reflect on their behaviors, emotions,
and thought processes
30
Errors in Person Perception
Halo Effect
positive impression of person interferes with objective
judgment
initially categorize people as positive, we interpret all
subsequent information in a positive manner
Pitchfork Effect
negative impression contributes to a subsequent negative
evaluation
Leniency Bias
rate people we know extremely positively
Logical Error in Rating
Rating error in which a person rates variables similarity
because they appear to be logically related to one another
31
Measurement Error: Central
Tendency Bias
no extreme ratings
no evaluation
1 2 3 4 5 6 7 I am afraid of losing my mind.
Disagree Agree
1 2 3 4 5 6 7 Sometimes I feel like I am being watched
Disagree Agree
1 2 3 4 5 6 7 People hate me.
Disagree Agree
1 2 3 4 5 6 7 I think euthanasia is a good idea for humans as
well as animals
Disagree Agree

32
Measurement Error: Floor Effects
all people rate the object on the low end
(e.g., resume)
reduces variability
decreases chance of observing difference
1 2 3 4 5 6 7 I am afraid of losing my mind.
Disagree Agree
1 2 3 4 5 6 7 Sometimes I feel like I am being watched
Disagree Agree
1 2 3 4 5 6 7 People hate me.
Disagree Agree
1 2 3 4 5 6 7 I think euthanasia is a good idea for humans as
well as animals
Disagree Agree

33
Ceiling Effects
all people rate the high end
occurs because there is little possible
variability in response; reduced variability
decreases chance of observing differences
1 2 3 4 5 6 7 Education is important.
Disagree Agree
1 2 3 4 5 6 7 I like chocolate
Disagree Agree
1 2 3 4 5 6 7 I am a creative person
Disagree Agree
1 2 3 4 5 6 7 My family is weird sometimes.
Disagree Agree
34
Acquiescent Response Set
similar to ceiling effects
yea sayers
people understand what the survey is assessing,
then keep responding the same way
reverse scoring is used to recode responses on a
questionnaire so that the sum and average are
meaningful
higher numbers indicate more endorsement of
conservatism
35
Acquiescent Response Set
1 2 3 4 5 6 7 Education is important.
Disagree Agree
1 2 3 4 5 6 7 I like chocolate
Disagree Agree
1 2 3 4 5 6 7 I am a creative person
Disagree Agree
1 2 3 4 5 6 7 My family is weird sometimes.
Disagree Agree
All responses are on the ceiling
36
Social Desirability
people give the socially accepted response
instead of the one that reflects their
behaviors
people tend to put themselves in a positive
light (self serving biases)
37
Questions
What are the three main types of methods?
Describe an advantage and disadvantage of each.
What is the difference between internal validity
and external validity?
If a survey simply requests participants to give
their impressions of product X, is the degree of
imposition of units most likely high or low?
Identify one of errors in person perception and
describe how it might be a problem when
administering a survey.