Chapter 14

Questionnaire
Development

Winston Jackson and Norine Verberg

Methods: Doing Social Research, 4e
Questionnaire Design
 Questionnaires have many applications and
are widely used
 The textbook covers:
 General guidelines
 Types of questions illustrated
 Steps and rules for questionnaire design
 Rules for ordering, formatting, and presenting

© 2007 Pearson Education Canada 14-2

General Guidelines
 Consult the respondent
 Able to express opinion freely, opinions valued
 Keep it short
 Ask only pertinent questions
 Achieve precise measurement
 Collect data in its rawest form (income in
dollars, precise occupation, age to the year,
not age category)
 highest level of measurement possible

© 2007 Pearson Education Canada 14-3

Types of Questions Illustrated
1. Pre-coded, single-choice questions
2. Open-ended questions
3. Presence-absence questions
4. Rank-ordering questions
5. Likert-type questions
6. Index development

© 2007 Pearson Education Canada 14-4

Types: Pre-Coded, Single-Choice
 Pre-coded, single-choice questions ask
respondents to indicate which one category
applies
 Answers are pre-coded for easy data entry
 If not all options can be listed, include a
category entitled “Other” with a space to
indicate what the “Other” category implies
 “Please specify” ______________
 Categories should be mutually-exclusive
 i.e., no categories overlap with one another

© 2007 Pearson Education Canada 14-5

Pre-Coded, Single-Choice: Example

4. What year are you in?

Freshman 1
Sophomore 2
Junior 3
Senior 4

See Box 14.1 (p. 383) for more examples

© 2007 Pearson Education Canada 14-6

Types: Open-Ended Questions
 Open-ended questions do not have pre-set
answers. Excellent way to explore new areas
 Use open-ended questions when:
 Too many response categories (year of birth)
 You don’t wish to impose categories on
respondents
 “Really” consulting respondents
 Qualitative – source of quotations
 Determining appropriate categorization
 You want a change in pace for respondents
© 2007 Pearson Education Canada 14-7
Open-Ended: Example
21. What is (or was) your father’s occupation (e.g.,
supervisor, railway machine shop … supervises work of
about 25 people)?
Job ________________________________________
Brief Job Description __________________________
___________________________________________

23. What is the one thing you would like to see changed
at the university?
____________________________________________
____________________________________________

More examples in Box 14.3 on page 385

© 2007 Pearson Education Canada 14-8
Open-Ended: Caution
 Minimize the number of open-ended questions
because they:
 Are time-consuming to code
 May generate responses that are inconsistent
 Are more likely to be left blank
 May decrease response rate
 Takes more time to complete the questionnaire
 Use opinion-seeking questions sparingly
 Place open-ended questions at 2/3 mark

© 2007 Pearson Education Canada 14-9

Types: Presence-Absence Questions
 Presence-absence questions ask
respondents to check off which items in a list
do or do not apply to them
 Less commonly used than other types of
questions

© 2007 Pearson Education Canada 14-10

Presence-Absence: Example
23. Have you ever had contact with a physically
handicapped person in any of these groups? (Circle
to indicate “yes” or “no” for each group)
Yes No
Community ----------- 1 0
Family ----------------- 1 0
Relatives -------------- 1 0
Elementary school -- 1 0
High school ----------- 1 0
University class ------ 1 0
As a co-worker ------- 1 0

© 2007 Pearson Education Canada 14-11

Types: Rank-Ordering Questions
 Rank-order questions ask respondents to
indicate an ordering of response items,
usually from most preferred to least preferred
 Must be done with great care
 Ask for only three most important items
 Must make instructions explicit
 These types of questions should be avoided
or minimized because they take time

© 2007 Pearson Education Canada 14-12

Rank-Ordering Questions: Example
31. Rank-order the three most important characteristics you
want in the job you make your life’s work. (Place a 1
beside the most important one; a 2 beside the second
most important one; and a 3 beside the next most
important one.)

High salary.…………. ____

Security.…………….. ____
Continued interest….. ____
Power……………….. ____
Prestige……………… ____
Excitement………….. ____

© 2007 Pearson Education Canada 14-13

Types: Likert-Type Questions
 Likert-type questions ask respondents to
indicate how much they agree or disagree
with a statement
 Response options originally included: strongly
disagree, disagree, are undecided or neutral,
agree, strongly agree
 Today often used with numbered response
options (see example, next slide)
 Used in the development of indexes (see
Chapter 13)

© 2007 Pearson Education Canada 14-14

Likert-Type Questions: Example
In the following, circle a number to indicate the extent to
which you agree or disagree with each statement:
52. I believe capital punishment represents the most effective
deterrent to murder.
Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree

53. I believe a murderer can be rehabilitated to become a

resonsible, functioning member of society.
Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree

54. I believe a life sentence is a satisfactory penalty for

murder.
Strongly disagree 1 2 3 4 5 6 7 8 9 Strongly agree
© 2007 Pearson Education Canada 14-15
Tips for Likert-Style Questions
1. Avoid the word “and”
 Makes the questions multidimensional

2. Always place “strongly agree” on right side,

with 9 indicating strong agreement
 Response set, a situation in which the respondent
answers similarly to all answers, is best avoided
by wording some statements positively, others
negatively
3. Avoid negatives that can confuse
respondents
 Use direct negative statements
© 2007 Pearson Education Canada 14-16
Tips for Likert-Style Questions
(cont’d)
4. Vary “strength of wording” to produce
variation in response
1. The nursing care I received was good.
2. The nursing care I received was perfect in
every instance
5. Before the first Likert-type item, provide a
brief explanation of how respondents are to
to indicate their answers

© 2007 Pearson Education Canada 14-17

Steps in Questionnaire Development
1. Make a list of variables. Usually includes:
1. Background variables
2. Dependent variable(s)
3. Independent variables
4. Others: intervening, antecedent, spurious

2. Anticipate how data will be analyzed

 Procedures depend on level of measurement

3. Write the proposed questions on index cards

 Facilitates editing and re-arranging order

4. Double check: do you have all the

variables?
© 2007 Pearson Education Canada 14-18
Steps (cont’d)
5. Review the conceptual definitions of
variables
 Done in anticipation of step 6 - developing
the wording
 Example: how to measure socioeconomic
status?
 As reflecting various levels of respect and
prestige: use Pineo and Porter’s scale of
occupational prestige to measure SES
 As reflecting variation in access to scarce
resources: use total income to measure SES
© 2007 Pearson Education Canada 14-19
Steps (cont’d)
6. Develop wordings for questions
 After considering the conceptual definition,
also:
 Use simple words: Grade 7 or 8 level
 Avoid “and”: make sure only one question
 Vary wording to produce variation in replies
 Avoid complexity – make it simple
 Use existing wording if comparative study
 Take the edge off sensitive questions
 Be precise, highly specific when choosing
wordings

© 2007 Pearson Education Canada 14-20

 Steps (cont’d)
7. Pre-test the questionnaire
 Start by filling out the survey yourself, then ask
individuals to do so
 Ask for feedback on the questionnaire
 Is it too long? Any problem with wording?

8. Pilot study
 Send questionnaire to a small sample of
respondents
 Use data to determine which items will be used for
indexes, modify unclear questions, create pre-
coded, single-choice questions based on responses
to open-ended questions
© 2007 Pearson Education Canada 14-21
Ordering the Questions
 Use a brief statement to introduce the survey
to the respondent – re: legitimacy of study
 Say who is conducting/sponsoring the study
 Assure confidentiality and anonymity
 Ease them into it
 Place easy to answer/salient questions at the
beginning
 Place key and repeated questions at 1/3 point
 Group questions by type
 E.g., put Likert-type questions together

© 2007 Pearson Education Canada 14-22

Formatting the Questions
 Begin conditioning respondents
 E.g., have consistent formatting so respondents can get
used to how to answer (format types shown in next two
slides)
 Anticipate computer data entry
 Pre-code questions before data collection
 Vary placement of response categories
 Response options are appealing and easy to understand

 Clearly indicate branching (shown on slide 14-26)

 Clear respondents away from irrelevant questions
 Make it easy to fill out; will increase response rate
© 2007 Pearson Education Canada 14-23
Four Format Styles Illustrated
1. Babbie Format Style for Fixed-Choice Responses:
23. Have you ever smoked marijuana?
[ ] Yes
[ ] No

2. Dillman Format Style for Fixed-Choice Responses:

Q-22 Your Sex (Circle number of your answer)
1 MALE
2 FEMALE

Sources:
Earl Babbie (1992) The Practice of Social Research, 6th ed. Belmont, CA: Wadsworth Publishing
Company. p. 155.
Don A. Dillman (1978). Mail and Telephone Surveys: the Total Design Method. New York: John Wiley
and Sons. p. 134.
© 2007 Pearson Education Canada 14-24
Four Format Styles Illustrated
3. Krahn Format Style for Fixed-Choice Responses:
54. In the past year, has any member of your
immediate family (not counting yourself) been
unemployed (out of work and not looking for work)?
No 1
Yes 2

4. Jackson Format Style for Fixed-Choice Responses:

4. What year are you in?
Freshman ------------- 1
Sophomore ----------- 2
Junior ------------------ 3
Senior ----------------- 4
Sources:
Harvey Krahn (1991). “Sociological Methods of Research.” In Lorne Tepperman and R. Jack
Richardson, eds., The Social World: An Introduction to Sociology. Toronto: McGraw-Hill
Ryerson. p. 51.
Winston Jackson (1995). Methods: Doing Social Research. Toronto: Prentice Hall. p. 388.
© 2007 Pearson Education Canada 14-25
Branching Question Illustrated

© 2007 Pearson Education Canada 14-26

Presentation of Questionnaire
 Have a distinctive look
 E.g., coloured paper, graphics for questions is
appealing
 Can increase the appeal of doing the
questionnaire
 Don’t squeeze too much onto one page
 Crowding questions on a page can make the
questionnaire appear too long

© 2007 Pearson Education Canada 14-27