Questionnaire Design

A Bose
BIMTECH

February 2010

A Bose (BIMTECH)

Questionnaire Design

February 2010

1 / 19

Table of Contents

What is a Questionnaire Questionnaire Design Process 1. Specify the information needed 2. Type of Interviewing Method 3. Individual Question Content 4. Overcoming Inability to Answer 5. Overcoming Unwillingness to Answer 6. Choosing Question Structure 7. Choosing Question Wording 8. Determining the Order of Questions 9. Reproduction of the Questionnaire 10. Pretesting of the Questionnaire

A Bose (BIMTECH)

Questionnaire Design

February 2010

2 / 19

Material for this set of slides is from Marketing Research - An Applied Orientation by Naresh Malhotra Chapter 10

A Bose (BIMTECH)

Questionnaire Design

February 2010

3 / 19

What is a Questionnaire

A questionnaire is a formalized set of questions for obtaining information from respondents. Can be called a schedule, interview form, measuring instrument. Might accompany other things, e.g. eldwork procedures, reward / gift, communication aids. Specic Objectives Must translate the information needed into a set of specic questions that the respondents can and will answer Must uplift, motivateand encourage the respondent to become involved in the interview, to cooperate and to complete the interview Should minimize response error

A Bose (BIMTECH)

Questionnaire Design

February 2010

4 / 19

Questionnaire Design Process

1 2 3 4

Specify the information needed Specify the type of interviewing method Determine the content of individual questions Design the questions to overcome the respondents inability and unwillingness to answer Decide on the question structure Determine the question wording Arrange the questions in proper order Identify the form and layout Reproduce the questionnaire Eliminate bugs by pretesting

5 6 7 8 9 10

A Bose (BIMTECH)

Questionnaire Design

February 2010

5 / 19

Questionnaire Design Process

1. Specify the information needed

Information needed becomes more and more clearly dened as the research progresses. Review components of the problem, the approach, the research questions, hypotheses and information needed. Prepare a set of dummy tables (blank tables used to catalog data) describing how the analysis will be structured once the data is collected. Have a clear idea of the target population. Characteristics of the respondent group have a great inuence on questionnaire design. Also clear understanding is needed to prevent high incidence of uncertain or no-response responses.

A Bose (BIMTECH)

Questionnaire Design

February 2010

6 / 19

Questionnaire Design Process

2. Type of Interviewing Method

2. Type of Interviewing Method

Type of interviewing will inuence questionnaire design.
1

Personal Interviews - respondents see the questionnaire and interact face-to-face with interviewer. So lengthy, complex and varied questions can be asked. Telephone interviews - Respondents interact with the uinterviewer but do not see the questionnaire. Limits type of questions that can be asked Mail questionnaires - Self administered. Questions should be simple and detailed instructions must be provided. Computer assisted interviewing (CAPI, CATI) - Complex skip patterns and randomization of questions to eliminate order bias can be accommodated Internet questionnaires - Share some characteristics of CAPI, but e-mail questionnaires have to be simpler

In mail and electronic surveys respondent records his response, in personal interview, the interviewer records the responses

A Bose (BIMTECH)

Questionnaire Design

February 2010

7 / 19

Questionnaire Design Process

3. Individual Question Content

3. Individual Question Content

1 2

Is the Question necessary Are several questions needed instead of one Wrong: Do you think Coca-Cola is a tasty and refreshing soft drink Right: Do you think Coca-Cola is a tasty soft drink Do you think Coca-Cola is a refreshing soft drink Do not have in one question, multiple, ambiguous, questions, or questions liable to be interpreted in many ways question. Wrong: Why do you shop at Shopprix Mall?

A Bose (BIMTECH)

Questionnaire Design

February 2010

8 / 19

Questionnaire Design Process

4. Overcoming Inability to Answer

4. Overcoming Inability to Answer

1

Is the Respondent Informed? In situations where all respondents may not be informed about the topic of interest, lter questions that measure familiarity, product use and past experience should be asked before questions about the topics themselves. Filter questions enable the researcher to lter out respondents who arre not adequately informed. A do-not-know option is provided to reduce uninformed responses without reducing the overall response rate. Can the Respondent Remember? Do not ask quantitative questions - soft drink consumed last month. Rather ask How often do you consume soft drinks in a typical week. Follow by alternative responses. Can the Respondent Articulate? If asked to describe the atmosphere of the department store they would like to patronize, most respondents may be unable to phrase their answers. On the other hand, if they are provided with alternative descriptions, they will be able to indicate the best choice.
A Bose (BIMTECH) Questionnaire Design February 2010 9 / 19

Questionnaire Design Process

5. Overcoming Unwillingness to Answer

5. Overcoming Unwillingness to Answer

1

Eort required of the respondents Minimize eort needed to be put in by the respondents Context Some questions may not be appropriate in all some contexts. Use discretion Legitimate Purpose Respondents may be unwilling to divulge information that they do not see as serving a legitimate purpose. Why should a rm marketing cereals want to know their age, income or occupation? Explanations could legitimize questions and make the respondents more willing to answer them Sensitive Information Respondents could feel that that their prestige or image is threatened. Questions about money, family life, political and religious beliefs, involvement in accidents or crimes should be avoided as far as possible. Increasing the Willingness of Respondents
1 2

Place sensitive topics at the end of the questionnaire Preface the question with a statement that the behaviour of interest is common. E.g. before requesting information on credit card debt say, Recent studies show that most Indians are in debt. This itechnique is called counterbiasing
Questionnaire Design February 2010 10 / 19

A Bose (BIMTECH)

Questionnaire Design Process

6. Choosing Question Structure

6. Choosing Question Structure

1

Unstructured Questions Open ended questions that respondents have to answer in their own words. Principal disadvantages are that the interviewer bias is high, coding of responses is costly and time consuming. Implicit greater weight to respondnets who are more articulate Structured Questions Structured questions specify the set of response alternatives and the response format. Following types:
1

Multiple-choice Questions Alternatives should include all possible choices. Include an alternative labelled Other (please specify). Response alternatives should be mutually exclusive. Generally, respondents should be able to identify one, and only one alternative Order or position bias To control for order bias, several forms of the questionnaire should be preparedwith the order in which the alternatives are listed varied from form to form. Continued on next slide

A Bose (BIMTECH)

Questionnaire Design

February 2010

11 / 19

Questionnaire Design Process

6. Choosing Question Structure

6. Choosing Question Structure

1

Structured Questions Structured questions specify the set of response alternatives and the response format. Following types:
1 2 3

Multiple-choice Questions Order or position bias Dichotomous Questions Only two possible responses, sometimes includes a neutral alternative, e.g. yes, no, do not know. Easiest to code and analyze. But response can be inuenced by the wording of the question. Split Ballot technique (next slide) Scales Discussed in Measurement Scales Lecture. Questions are formed using scales such as Likert Scale, e.g. Do you intend to buy a new car within the next six months? Denitely not Probably will Probably Denitely will not buy not buy Undecided will buy will buy 1 2 3 4 5

A Bose (BIMTECH)

Questionnaire Design

February 2010

12 / 19

Questionnaire Design Process

6. Choosing Question Structure

Split Ballot Technique

Response can be inuenced by the wording of the question. Individuals are more to blame than social conditions for crime and lawlessness in this country produced agreement from 59.6% of respondents However on a matched sample that responded to the opposite statement: Social conditions are more to blame than individuals for crime and lawlessness in this country produced 43.2% agreement - not (100-59.6=) 40.4% To overcome this problem, the question should be framed in one way on one half of the questionnaires and in the opposite way on the other half.

A Bose (BIMTECH)

Questionnaire Design

February 2010

13 / 19

Questionnaire Design Process

7. Choosing Question Wording

7. Choosing Question Wording

1

Dene the Issue A question should clearly dene the issue being addressed. Wrong: Which brand of shampoo do you use Right: Which brand/brands of shampoo have you personally used at home during the last month? In case of more than one brand, please list all brands that apply. Use Ordinary Words Should match the vocabulary level of the respondents. Wrong: Do you think the distribution of soft drinks is adequate? Right: Do you think soft drinks are readily available when you want to buy them? Use Unambiguous Words. For a question: In a typical month, how often do you shop in department stores? Wrong Right Never Less than once Occasionally 1 or 2 times Sometimes 3 or 4 times Often More than 4 times Regularly
A Bose (BIMTECH) Questionnaire Design February 2010 14 / 19

Questionnaire Design Process 1

7. Choosing Question Wording

Avoid Leading or Biasing Questions A leading question is one that clues the respondent to what anser is desired or leads the respondent to answer in a certain way Avoid Implicit Alternatives Wrong: Do you like to y when travelling short distances? Right: Do you like to y when travelling short distances, or would you rather drive? Avoid Implicit Assumptions Wrong: Are you in favour of a balanced budget? Right: Are you in favour of a balanced budget if it would result in an ncrease in th epersonal income tax? Avoid Generalizations and Estimates Wrong: What is the annual per capita expenditure on groceries in your household? Right: What is the monthly (or weekly) expenditure on groceries in your household? and How many members are there in your household? Dual Statements: Positive & Negative Balance positively and negatively worded questions
A Bose (BIMTECH) Questionnaire Design February 2010 15 / 19

Questionnaire Design Process

8. Determining the Order of Questions

8. Determining the Order of Questions

1

Opening Questions Crucial in gaining condence and cooperation of respondents. Should be interesting, simple and non-threatening. Questions that ask respondents for their opinions can be good opening questions Type of Information As a general guideline follow order as below
1 2 3

basic information classication information identication information

Dicult Questions Senitive, embarrassing, complex or dull should be placed late in the sequence; only after rapport has been established Eect on Subsequent Questions General questions should precede specic ones. This prevents specic questions from biasing responses to general questions General to specic - funnel approach Logical Order Questions should be asked in a logical order. Branching questions should be designed carefully. Use owchart to see that all paths are covered.
A Bose (BIMTECH) Questionnaire Design February 2010 16 / 19

Questionnaire Design Process

9. Reproduction of the Questionnaire

9. Reproduction of the Questionnaire

How a questionnaire is reproduced for administration can inuence the results.
1

Questionnaire should be reproduced on good quality paper and should have a professional appearance Multi page questionnaires should be in the form of booklets (not sheets stapled together) A questionm should not be split across pages. This applies also to the response categories for the question. Do not overcrowd the questionnaire Instructions for individual questions should be as close to the questions as possible Above all, the questionnaire should be easy to read and answer

4 5

A Bose (BIMTECH)

Questionnaire Design

February 2010

17 / 19

Questionnaire Design Process

10. Pretesting of the Questionnaire

10. Pretesting of the Questionnaire

Refers to the testing of the questionnaire on a small sample of respondents to identify and eliminate potential problems. Pretests are best done by personal interviews, even if the actual survey is to be conducted by mail, telephone or electronic means, because the interviewer can observe respondents reactions and attitudes. A variety of interviewers should be used for pretests. This will give a good feel of the potential problems and the nature of expected data. Protocol analysis and debrieng are two commonly used procedures in pretesting. In protocol analysis the respondent is asked to think alod while answering the questionnaire. The remarks are taped and analyzed.

A Bose (BIMTECH)

Questionnaire Design

February 2010

18 / 19

Questionnaire Design Process

10. Pretesting of the Questionnaire

Debrieng occurs after the questionnaire has been completed. Respondents are told that the questionnaire they just completed was a pretest and the objectives of pretesting aredescribed to them. They are then asked to describe the meaning of each question, to explain their answers and to state any problems they encountered while answering the questionnaire. Finally, the responses from pretest are coded and analyzed. Thisd serves as a check of adequacy of the problem denition and data and analysis required to obtain the necessary information.

A Bose (BIMTECH)

Questionnaire Design

February 2010

19 / 19