From Paper-and-Pencil to Screen-and-Keyboard

Studies on the Effectiveness of

Internet-Based Marketing Research

Elisabeth C. Brüggen-Deutskens

1
2
q 50% European researchers say they have replaced face-to-
face studies with online research over the last year
(Greenfield/Ciao Online Research Barometer 2005)

q 75% of research decision makers in the US tried online

research and expect to use it again in the future (Acorn / IIR)

q Companies currently doing research online spent, on

average, 28 percent of their total research budgets on
online methodologies in 2004

q By 2010, almost one-third of all survey research

expenditures will be directed towards online methods

q Increase from €3 mill. in 1996 to 500 mill. in 2002

(Hogg 2002)

q Value of the online research market is set to triple in three

years, reaching $4 billion by 2008 (Cambiar / GMI 2005)

q Online research is surging, and within the next few years is

likely to become the most common form of all marketing
research fieldwork conducted
(Marketing Research Association Executive Director 2006)

q Online research is the largest paradigm shift in the industry

since the late 1950's

3
 Online Surveys

Benefits
q Fast responses

q Electronic capture of data

Efficiency of
q Lower costs
online surveys
q Large scale

q Flexibility

Effectiveness of Online Surveys

Effectiveness = weight1 * Quality - weight2 * Costs

Where
- Quality: Data quality (response rate and quality)
- Costs: Time, money

- Weights: magnitude of risk, constraints

4
 Central Research Question

What are the critical factors

that influence the effectiveness
of online marketing research?

Overview

HOW Study 1: Design of Online Surveys

Study 2: Equivalence of Online & Mail Surveys

WHAT
Study 3: Generalizability of Online & Mail Surveys

WHO Study 4: Quality of Online Panels

5
Study 1: Design of Online Surveys

q Length: Short vs. long

q Type of incentives: Vouchers, lotteries, donations
q Presentation: Pictures vs. text
q Timing of follow-ups: 1 vs. 2 weeks

Field experiment with 730 consumers

Effectiveness: Response rate and quality

6
 Conclusions Study 1

q Length: Long survey good response

q Incentives: Lotteries most effective
q Visual Presentation: Lower response rate but
higher response quality
q Timing of follow-up: no significant influence

Study 1 provides important design guidelines to

academics and marketers alike

7
 Study 2&3:
Equivalence and Generalizability

q Core concern: Data quality of online surveys

q Online survey results:
§ Reliable?
§ Comparable to mail surveys?
q Trade-off costs and quality?

Study 2&3:
Equivalence and Generalizability

q Study 2:
§ Effectiveness: Equivalence
è accuracy, completeness, measurement invariance

§ Field experiment in US
§ Respondents: mail 694 (16.58%)
online 255 (28.47%)

8
 Study 2&3:
Equivalence and Generalizability

q Study 3:
§ Effectiveness: Generalizability theory
§ Field experiment US & UK

US UK
§ Respondents: Mail 1000 439
(16.58%) (16.74%)
Online 377 141
(28.48%) (26.60%)

Conclusions Study 2&3

q Equivalence in response quality

q Comparable level of generalizability

q Online surveys
§ More cost efficient
§ Higher response rate
§ More answers to open questions

Online surveys can be equally effective

9
 Conclusions Study 2&3

Practical Implications
q Support multi-mode
surveys

www.oce.com

Because we want to make it as easy as possible for you

to give us your feedback, we have made the survey
available on our Internet side as well. To fill it in, simply
go to www.oce.com and use your customer number
(printed at the top of this letter) to log on.

10
Alternatively, please fill in the questionnaire enclosed and
return it to Océ using the postage-paid return envelope.

Study 4: Quality of Online Panels

q Fastest growing segment due to lack of

representative online sampling lists
q Online panel: People who agreed to participate
regularly in surveys

q What motivates members of online access

panels to join and participate regularly?

Effectiveness: response rate and quality

11
Survey Participation Inventory

Survey Participation Inventory

Intrinsic Motives Extrinsic Motives

Interest
Self Enjoyment Incentives
Focus Curiosity

Other Helping Obligation

Focus Give opinion Need for recognition

Response Rate
Response Quality

12
 Analysis Study 4

q 13 different surveys with wide range of topics

q Response quality: Style
Tendency to dis/agree, provide extreme or neutral responses
q Response rate: Participation last year
q Link to SPI results
q 3815 respondents

Conclusions Study 4

Respondents Size Motives Response

1. Voicing assistants 27% Opinion Neutral response style

Helping

2. Reward seekers 25% Incentives Lowest response rate

Lowest self-reported levels
Good response style
Nay-sayers

3. Intrinsics 48% All intrinsic Highest response rate

motives Worst response style
Yea-sayers

13
 Conclusions Study 4

q Practical implications:
Design of E-mail invitation
Triggers in subject
line or text:

q Your opinion counts

q We need your help
q Participate and win

5 Key Recommendations

1. Use lotteries with several small prizes

2. Long surveys are possible
3. Equivalenceè make use of multi-mode surveys
4. Optimize measurement design (costs vs. level of
generalizability)
5. Examine link between response motives and
styles in online panels

14
 Overall Conclusion

From Paper-and-Pencil to Screen-and-Keyboard

Studies on the Effectiveness of

Internet-Based Marketing Research

Elisabeth C. Brüggen-Deutskens

15