Constructing questions for

interviews & questionnaires

Theory and practice in social research
William Foddy

Preface
These slides include one theoretical
framework for constructing questions and a
small set of good and bad practices for
constructing questions
This is not only or complete set, but gives
hints about what should you think when
constructing questions.

Errors in gathering data through

survey procedures are
Responders failure to understand questions an intended
A lack of effort or interest on the part of responders
Respondents unwillingness to admit certain attitudes or
behaviours
The failure of respondents memory or comprehension
process in the stressed condition of the interview
Interviewers failures of various kinds (e.g. the tendency to
change wording, failures in presentation procedures and
the adoption of faulty recording process

Examples that illustrate the inadequate of

many of the questions
Factual questions sometimes elicit invalid answer: As simple as age
Relationship between what respondents say and what they actually do in not always
very strong: E.g. racism: Questions: Do you let a black person inside a bar? Do they
really?
Responders attitudes, beliefs, opinions, habits interests often seem to be
extraordinary unstable
Small changes in wording sometimes produce major changes in the distribution of
responses
Respondents commonly misinterpret questions
Answer to earlier question can affect to later question
Order of options changes the respondents: e.g. middle option
Format of question: Open vs. close
Respondents answer even if they don't know about the topic
Cultural context affects

The principal assumptions that have defined the

general orientation adopted by survey researcher in
the past

The researcher has clearly defined the topic about which information is
required
Respondents have the information that researcher requires
Respondents are able to access the required information under the conditions
of the research situation
Respondents can understand every question as intended
Respondents are willing (or at least, can be motivated) to give the information
Responses are more valid if they know why that is asked
Answers are more valid if they are not suggested
The research situation does not affect to the results
The process does not affect the respondent
All responses are meaningfully comparable

The traditional survey model

A carefully standardised physical
stimulus (i.e. question)

Researcher/
Interviewer

Respondent
A response (i.e. answer)
expressed in term of a
standardised format provided by
the researcher

The key issue: the comparability

of aswer
The reseacher must be clear about the nature of the information
required and encode a request for this format
The respondent must decode this request in the way the researcher
intends it to be decoded
The respondent must encode an answer that contains the information
researcher has requested
The reseacher must decode an answer as the respondent intendedd it to
be encoded

Symbolic interactioinist theory

Human beings interpret and define each other's actions. Not only
simple stimulus-response
Human beings can be objects of their own attention. Argue themselves,
take pride themselves
Conscious social behavior is intentional behavior. Situation affects.
Interpreting, planning and acting are ongoing process which begin
anew at every stage of a social process
In situation there is a role and person wants to hold on it.
Always exists
In brief: social situation is constantly negotiating a shared definition of
the situation; taking one another's viewpoint into account; and
interpreting another's behavior

A model of symbolic interactionist view of question

answer behaviour
Interviewer

Responder

Encodes question, taking into

account own purposes and
presumptions/knowledge about
the respondent, and perceptions
of the respondent's
presumptions/knowledge about
self

Decodes question, taking into

account own purposes and
presumptions/knowledge about
the interviewer, and perceptions
of the interviewer's
presumptions/knowledge about
self

Encodes question, taking into

account own purposes and
presumptions/knowledge about
the respondent, and perceptions
of the respondent's
presumptions/knowledge about
self

Encodes question, taking into

account own purposes and
presumptions/knowledge about
the interviewer, and perceptions
of the interviewer's
presumptions/knowledge about
self

The implication of the symbolic interaction

theory for social research
Roles
Respondents try to find a mutually shared
definition of the situation
Interpretation of researcher acts
Clues of what kind of information
researcher wants

Other
Multidimensionality. Nothing is onedimensional
Level of generality in the statements
Level of generality in the topic
Utterance frame: descriptive, explanatory or
evaluative

Defining topic properly

Interviewer

Responder

Encodes question.
The need to clearly specify the
information that is required. The
assumption that the respondent
have the required information.
The assumption that the
respondents can access the
information

Decodes answer

Decodes question

Encodes answer

Researcher has clearly defined

the required information
Defined topic
Global vs. local (29)
Dimensions (economic, technical...)
Is PL good thing: Marketing vs. Technical

E.g. better social secure in cost of taxes

Researcher has clear idea what kind of

information is required

Responder have required

information
Hypothetical question => hypothetical Answer
Willingness to answer vs. information to answer
Opinions vs. facts
Grounded opinions
Serious opinions

Responds are capable of verbalising information.

Not hurry, give time to answer

Formulating intelligible requests

for information
Interviewer

Responder

Encodes question.
Formulating an intelligible
request for information.

Decodes question

Defining key terms for

respondents and virtues of
simplicity and breity

Decodes answer

Encodes answer

The meaning of individual words

Context specific meanings
Ambiguities words
Different meanings
Cultural decencies

Relative difficulty of words

foreign words

The operation of unintended nuances associated

with apparently similar words
Forbid vs. allow

The meaning of concepts

Lack of empirical references
How should be
Difficul, unclear, wide concepts

Context decencies (often, usually etc.)

Concrete: Not: often, quite often, seldom.
Use 1,2,3,4,5

Structural complexities
Number of words
Less is better, but not complex ones
Grammatical
Asking too much in one question
divide & conquer
not what you like this and that!

Not too much negatives: Double is bad

Good practices
Descriptive introduction: I'd like to
describe
Explain before question
The addition of clauses, phrases and
instructions

Anecdotes
Education of the respondent affects

Contextual influences on respondents'

interpretation of questions
Interviewer

Responder

Decodes question
Encodes question.

Contextual clues that influence

respondents interpretations

Decodes answer

Encodes answer

Clues afforded by either the

Question itself or its components
Clues associated with the question
Some existing idea always
Leading questions

Clues in components
Additional phrases
Note: Vegetables e.g. spinach not associated vegetarian
in general as intended

Clues in response options.

Approve or disapprove - not only approve in question
Scale
Pre-set response options as memory clues

May not remember other options

Information range covered by the options.
Who counts own averages?
Abstract: no knowledge of normative levels

Response biased with number of options

Respondent favor first when reads, last when hears - vary the order
Let responder tell, if no-one match

Impact of preceding questions

Influence associated with prior Q. Issues:
Does one have any knowledge?
How deep is knowledge or how thorough
How one interprets

Psychological need for consistent

Need to stay in one opinion

Even-handedness
Fair for all sides

Impact of preceding questions

(contd)
The impact of the previous answers
The impact of the overall situation
Semantics of human mind.
Willingness to give information

Contextual influences on respondents'

interpretation of questions
Interviewer

Encodes question.

Responder

Decodes question

Encodes answer
Decodes answer

The kind of answer given is defined by the

dimension of the response framework that
is employed

Descriptive accounts

How to describe the situation

Perspectives

Level of social generality of responses

You? Singular or plural?

Explanations
Respondent can frame an explanation in many
different ways
Why did you do X

Causal antecedent - what caused to do X

Goal antecedent - purpose for X
Enablement factors - how possible
Causal consequences - what happened after X
The researcher's expectations

Evaluations
Always relative
Standards necessary does not exists
Evaluative standard that are external to the
question
Pleasant/easy flight - pilot vs. passenger

Evaluative standard that are built in to the question

Agree -disagree

Limitations of human memory

Limitations of human memory

Long term memory problems
Nobody remembers all
Ask from present to past, not past to present.

Short term memory problems

Simple questions

Filters

Filters
Establishing the relevance of questions to
respondent
Respondents tend to answer all the questions
I don't know enough

Don't know vs. don't have opinion

Position of filters
Middle and dont know are problematic
Middle category at all

subjective viewpoint to strongly (dis)agree

How strongly?

Reducing question threat

Reducing question threat

Always some threat
Biased
Refuse

How?

Casual, Do you happen to have?

Imputation of deviance, You know everyone does.
Anonymous
Lessen psychological immediacy of the Q (e.g. other instead self.
Numeric coding of alternatives)
Decrease specificity of the information called (e.g. broad response
categories)
Adopt knowing so that respondents have to confirm rather than volunteer
'Kinsey' straight at eyes and ask
Adopt indirect so that respond give answer without knowing
Place threatening Q at the end of series

How? (contd)
Door in the face: Ask direct, if does not
answer ask indirect/about e.g. salary.
Ask long, dont hesitate top repeat, give
time and encourage to use time
The definition of Q threat

Threat causes bias

Topics that are desirable and over-reported
Be good citizen
Be well informed and cultural person
Have fulfilled of moral and social responsibilities

Topics that are socially undesirable and under-reported

Illness and disabilities
Illegal and contra-normative behavior
Financial status

Feeling of guilt or personal fears

Threat associated with the nature of the relationship between interviewer and
respond
Social equity
Fear of political or economical sanctions

Open vs. closed question debate

Open vs. closed question debate

Coding responses to open questions
Formulating response option for closed
question

Evaluation of the use of open questions

Open Q doesnt suggest answer -- or does it?
Respondents use of "probes"

Indicate level of knowledge

Assumptions
will answer open Q if knows
don't try to answer open Q if don't know
wil answer closed question if don't know

Answers indicate the salience of the topic in the responder's mind

Is there evidence?

Avoid format effect

Not in the middle!
Is it in the closed?

Evaluation of the use of open questions

Allows complex motivation
Indicates more how respondent has interpreted it
Indicate motivation that have influenced respondent's orientation to the
topic
Indicates the frame of reference

Problems associated with probing inadequate answer

Turns to close

Problems associated with coding response!

Evaluation assumptions
associated with closed questions
Answers the question in the same way
-meaningfully comparable?
Easier to answer
More easily analyzed
Interpretations

Problems associated with recording responses to

closed questions
Start with general instructions
Explain why to answer

Measuring attitudes

Measuring attitudes
List of respond alternatives p. 153
Define topic clearly
Applicability of the topic to respondents has to be
established
Dont know

Respondents has to know what sort of answers they should

know
Level of generality

Specs of standards
"Strongly agree"

Measuring attitudes
Stimulus centered effects
Number of categories
7 +/- 2

Anchoring effects of he category labels

The word in the positive/negative end?

Problems in the batteries of rating scales

Meaning can alter
Some category
long list. Don't know last when answering the first

Ambiguity

Checks to ensure that questions

work as intended

Checks to ensure that questions

work as intended
Editing rules (list p. 184)
Piloting Question
Observation hard, but must be done in the beginning to
ensure that questions work e.g. had to repeat?

Question testing
Rephrase the question in responder's own words
Double interview
Come in the beginning

Allow aloud thinking

Tieto vai mielipide

Halu vastata vai
Vihjaa ett pitisi olla? vastaus olemassa?

Mik on tyypillinen
tuote?

Esim KKK

Koponentti parametrinen vs
komponetin mukanaolo parametrista
Mik on komponentti?

Kuinka monta parametroitua komponenttia on

tyypillisess toimitettavassa tuotteessa (0=ei kytet)?
Miten paljon parametreja on tyypillisess
parametroidussa komponentissa?
Parametrien mrn vaihteluvli?
Millaisia parametreja komponenteissa on?
Millaisia arvoalueita parametreilla on (kokonaisluku,
arvoalue, yksi kokonaisluku joukosta , yksi annetuista
vaihtoehdoista, reaalilukuja, joukkoja)? Mit muita?

Jos ymmrsi aluksi vrin, halu

konsistenttiin ilmaisuun

Strukturaalinen kompleksisuus, liian

monimutkainen lause:
Kun toimitatte tuotteen, tuotteessa om
komponentteja. Kompoenenteilla voi
olla toimituskohtaisia parametreja.
Onko olemassa tllaisia
komponentteja, joissa on parametreja?
Montako tllist kompoenttia, jossa
on parametreja, tuotteessa on?