Pilot testing

Why and how

Ken Black

Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008

Key reasons to pilot test

Questions able to be administered in a real world environment by interviewers or selfreported, depending on method used, and can then assess:
Respondent understanding of question meaning Appropriateness of response categories Question clarity

Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008

Key reasons to pilot test (cont.)

Correct sequencing of respondents if needed Adequacy of training procedures Adequacy of instructions Effectiveness of data capture Edits field (if electronic collection) and post field

Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008

Key reasons to pilot test (cont.)

Interviewer field notes of problems/issues arising Interviewer debriefing opportunities preferably group face-to-face Form receipt and follow-up procedures Processing instructions and procedures Mock-ups of data output for user liaison
Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008

Desired results of pilot testing

Need for change identified in question design, wording or instructions Effectiveness of operational procedures tested Do the questions asked do what was intended, and are they likely to provide the output data required?

Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008

Desired results of pilot testing

(cont.)

In the end a better standard product for the final enumeration, and assurance of more meaningful and accurate data More likely to meet data users needs!

Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008

Essential elements
Adequate sample size selected to test all facets of the disability questions and processes against a broad spectrum of the population, including coverage of people with a disability both urban and rural
minimum 1,000 households

Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008

Essential elements (cont.)

Adequate resources available for:
development training enumeration data capture editing and processing analysis final report

Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008

Your needs
Each of you will have different strengths and weaknesses I need your input here to identify what advice you need, and where you see the most difficulty in conducting an effective pilot test as outlined

Regional workshop on promoting disability data collection through the 2010 population and housing censuses Bangkok, Thailand 8-10 April 2008