Questionnaire development: an

examination of the Nordic Musculoskeletal

Questionnaire
C E Dickinson, K Campion, A F Foster, S J Newman, A M T O'Rourke and P G Thomas
Technology and Health Sciences Division, Health and Safety Executive, Magdalen House,
Trinity Road, Bootie, L20 3QZ, UK
This paper describes the o u t c o m e of user trials of the Nordic Musculoskeletal
Questionnaire which e n c o m p a s s e d the views of the following groups: data entry
clerks, technical staff, administrative clerks and 481 subjects e m p l o y e d in 10
supermarkets. A significant n u m b e r of i m p r o v e m e n t s was identified, especially
concerning its wording, layout and administration. This has led to a standardized
version being produced for use in studying the prevalence of r e p o r t e d s y m p t o m s in
many types of occupational groups.

Keywords: Musculoskeletal, questionnaire design

Introduction
Questionnaire development is a surprisingly complex
and onerous task. As a rule of thumb, a good
questionnaire may be thought of as a tool that is
acceptable to all parties handling the form. The most
important group to consider is the subjects - the
questionnaire must be understandable to them and the
layout such that they answer all of the questions. These
needs of the subjects can compromise other requirements which would permit straightforward computer
data entry, and the gaining of all the relevant information
for a successful analysis, interpretation and report,
unless care is taken. One approach to developing a
questionnaire is to travel through the flow-diagram
shown in Figure 1. This recommends that time is given
to defining what needs to be learnt from the questionnaire, the staff and financial resources available (this
has an effect on such issues as how the questionnaire
will be administered and whether there are funds for it to
be type-set) and how the questionnaire is to be utilized
- which occupational groups and to whom in that group
(sampling frame). Once the definition stage is completed
it is timely to consider if a well recognized questionnaire
exists that matches these objectives but, if not, to begin
constructing a questionnaire and making arrangements
for future pilot studies.
A project group in the Nordic Council of Ministers
tackled this area when producing the Nordic Musculoskeletal Questionnaire (NMQ) 1. When questioning the
prevalence of such disorders, this distanced itself from
using diagnostic labelling, and simply asked for 'ache,
pain, discomfort' arising in nine body areas. It was an
Vol 23 No 3 June 1992

approach attractive to the Health and Safety Executive

(HSE) who needed a standard methodology for use by
many individuals for identifying the number of workers
experiencing musculoskeletal symptoms. It was recognized that the NMQ was suitable for application in a
wide diversity of workplaces and could accommodate
very large numbers of workers in a study very quickly
and cheaply, whilst the extent of its application stands
as testimony to its acceptability to the workforce.

Planning the questionnaire

Define objectives, resources, terminology

[ S a m p l i n g [
Define population
Sample size
Sampling frame

Prologue

Questions

Classification

Questionnaire construction
Question sequence
Language
Layout

Epilogue

Question specificity

Data hand ling

I
Piloting
Distribute to friends
Distribute to I0 subjects
Distribute to subjects
Interview = Time them, question order, word changes
Action : Check analysis routine, data handling,
estimate repeatability, response rate,
meaningless questions.

I
I

Undertake survey

Figure 1 Questionnaire development strategy

0003-6870/92/03 0197-05 $03.00 ~) 1992 Butterworth-Heinemann Ltd

197

Questionnaire development: an examination of the Nordic Musculoskeletal Questionnaire

However, the acceptability of the questionnaire to the
research team has not been extensively reported.
A questionnaire typically contains four sections: a
prologue, core questions of the study, a series of
classification questions and an epilogue 2. It is suggested
that the prologue might include a letter describing the
purpose of the survey, and instructions on how to
complete the questionnaire. The core questions are
obviously prepared to meet the survey's objectives and
are to be arranged in an order which will seem natural
to those faced with completing the questionnaire. The
classification questions are those such as sex or age or
handedness; that is, those which provide for subgrouping for the data analysis stage. Finally, the
epilogue thanks the subjects for completing the
questionnaire and often instructs them what next to do
with it.

stores, the questionnaires were issued to staff who

took them away and were asked to return them to
centrally positioned box or person, sealed in an
envelope. In each of the three approaches a record
was taken in each store of the number of cashiers
potentially available to complete the questionnaire
versus the numbers that actually did so as an
indication of response rate.
One week after completing the questionnaire, 44
of the cashiers were asked to complete the questionnaire again in exactly the same manner to produce
the repeatability exercise. A further 10% of the
cashiers were interviewed using a checklist to
ascertain suggestions for improved layout and
phrasing of words or questions.
Improvements were introduced along the way by
virtue of a multi-disciplinary working group that met
periodically to review progress and agree changes.

The purpose of this paper is to describe our experience

and the development of the Nordic Musculoskeletal
Questionnaire.

Pilot study results

Methods

Question changes

A questionnaire was constructed with a prologue

citing HSE's logo, provision for identification numbers
(for both subject and occupational group) and a title to
establish the topic of interest. A letter was included to
describe the purpose of the study and invite subject
participation. Assurances of confidentiality were made,
and a section included to explain how to complete the
questionnaire.
This was followed by two groups of classification type
questions, firstly to identify the subject's age and
handedness and, secondly, five questions about the
individual's job as a record of exposure. This was
followed by a picture of the human form with nine body
areas shaded and defined (neck, shoulders, upper back,
lower back, elbows, wrist/hands, thighs, knees and
ankles). This was relevant to a table that requested a
'yes' or 'no' response for each body area to three
questions concerning annual prevalence, any disability
during the last year (annual disability) and weekly
prevalence. Finally, a statement of thanks constituted
the epilogue.
The questionnaire was distributed to a succession of
specific groups as part of a piloting exercise.
1. Six HSE professional staff (statisticians, doctors and
nurses) for discussions on the technical content.
2. Data entry clerks for their view on improving the
layout for ease of input.
3. Ten HSE administrative clerks to gain an idea of
time needed for its completion and to highlight any
questions not readily understood.
4. Check-out staff at 10 supermarkets (total number =
481) were administered the questionnaire in three
different ways. Firstly, by HSE staff seeing small
groups of cashiers in a designated room at the store
and inviting them to complete the questionnaire.
Secondly, at three stores, the retailer's training staff
invited their check-out staff to complete the
questionnaire in a designated room. Finally, at three
198

Several questions were modified from those in the

NMQ. For instance, year of birth was amended to date
of birth and today's date inserted to enable age to be
calculated more precisely. Provision was made so that
weight and height could be input in either imperial or
SI units. In fact, of the 481 cashiers, all entered their
weight in stones and pounds and height in feet and
inches.

Repeatability exercise
Of the 44 cashiers who completed the questionnaire
on two occasions, the number of non-identical answers
varied from 7% to 26% for annual prevalence, 0% to
8% for annual disability and 6% to 19% for weekly
prevalence.
All variables were judged to be repeatable though
precise duplication was not always evident, even in
factual type questions. Whilst all cashiers reproduced
their original answer for their sex, 7% of the population
were undecided about whether they were right- or lefthanded. Date of birth was also found to vary for just
over 2% of the cashiers.

Meaningless answers
1% of subjects reported trouble in a body area in the
last week but not so in the last year.

Distribution of missing values

The majority of missing values were located on the
questions about annual disability and weekly prevalence for the ankles and knees - essentially those
questions in the bottom right-hand corner of the page.
Observation of people completing the questionnaire
suggested that this was due to the subjects working
their way through the three columns of questions in a
vertical direction. This was to have been expected given
that bold vertical lines separated the three questions.
When bold horizontal lines were drawn in and tried', at
three of the supermarkets this problem of missing
values was eliminated, as all individuals were unconsciously steered to work horizontally from left to
right across the page (Figure 2). Further, the cashiers
Applied Ergonomics

C E DICKINSON, K CAMPION, A F FOSTER, S J NEWMAN, A M T O'ROURKE AND P G THOMAS

Please answer by using the tick boxes

-

[2]

one tick for each question

Please note that this part of the questionnaire should be answered, even
if you have never had trouble in any )arts of your body.
Have you at any time during the last 12
months had trouble (such as ache,
:)ain, discomfort, numbness) in:

Neck

Have you had trouble during the last 7

days:

Neck

No

Yes

No

,[-1

21-]

,1-1

4 Shoulders
No
1D

1D

in the left shoulcler

in bothshoulders

4~

in both shoulders

8
Yes
in the right elbow

1[-~

1~1

2D

in the right elbow

3D

in the left elbow

4~

tn bothelbows

4 [""] in both elbows

11 Wsirth/andSNo

2 ["'~ in the right wrisVnand

,I-I

,D
,I--1

Yes

No

'D

,I-]

'D

No

Yes

,I-I

,l-1

19 One or both hips/thighs/buttocks

No

Yes

,I--]

21-1

22 One or both knees

No

,I-1

Yes

,D

25 One or both ankles/feet

No

tD

Yes

,D

No
in the right wrist/hand

2D

'D

Yes

,D

in both wrists/hands

14 Upper back

No

16 Lower back (small of the back)

,1-1

Yes

in the left wrist/hand

in bothwrists/hancls

13 Upper back

Yes

,D

12 Wrists/hands(both/either)

Yes

3 [ ~ ] in the left wrist/hand

4~

t[-]

No

Yes

3 [ ~ ] in the left elbow

Yes

,D

19 Elbows (both/either)

Elbows
No

Yes

Shoulders (both/either)
No

3D

4D

1r]

6
Yes

in the left shoulder

3 []

10 Wrists/hands
No

,l-1

2D
in the right shoulcler

in the right shoulaer

2r-]

No

2D

2D

17 Elbows
No

Neck
Yes

Shoulders
No

Yes

During the last 12 months have you

been prevented from carrying out
normal activities (eg. job, housework,
hobbies) because of this trouble:

No

Yes

21-1

,I-I

,I-I

17 Lower back
No

,I-1
,D

No

Yes

,I-I

,I-I

,I-]

,~

121 Hips/thighs/buttocks

Yes

No

21-1

,1-1

Yes

,I-1

24 Knees

23 Knees
No

18 Lower back

Yes

20 Hips/thighs/buttocks
No

15 Upperback

Yes

Yes

,[-]

26 Ankles/feet
No

Yes

t[]

,I-1

No

Yes

27 Anklee/feet
No

Yes

Figure 2 Musculoskeletal questionnaire

Vol 23 No 3 June 1992

199

Questionnaire development: an examination of the Nordic Musculoskeletal Questionnaire

indicated that it was more natural to consider one body
area and complete the three questions before moving
on to considering their next body area.

The time taken to complete the questionnaire was

approximately 4 min though it varied from about 2 to 9
min.

Layout interviews
Suggestions were made by the cashiers to include an
option for being ambidextrous on the question of
handedness. 2% of subjects were found to be ambidextrous.
The wording was amended for annual disability
following suggestions from the administrative clerks
and cashiers from:

The cashiers claimed to understand all wording being

used. Efforts had been made to avoid officious and
unfriendly words and included 'you' or 'we' rather than
the subject or employee in the introductory letter. All
acronyms, abbreviations and technical terms were
absent from the draft.

Have you at any time during the last 12 months

been prevented from doing your day's work (at
home or outside the home) because of the
trouble?
Replaced with:
During the last 12 months have you been prevented from carrying out normal activities (eg,
job, housework, hobbies) because of this trouble?
The definition of trouble began with "ache, pain or
discomfort", this was expanded to "ache, pain, discomfort or numbness" given the outcome of the
interviews with some of the cashiers who had severe
and prolonged numbness but failed to associate this as
being discomfort despite explaining it so to the interviewer.
The data entry clerks were generally happy with the
layout of questions but particularly emphasized their
preference that questions be located in a column to the
left, and the answers be positioned in a column over to
the right. This meant that they did not have to search
across the page for their next input and so were able to
enter the data at speed. This was resolved for the
classification questions by arranging the questions in a
left-hand column and the answer boxes aligned below
each other but positioned over to the right leaving a
blank column in between. For the questions shown in
Figure 2, the order for entering the data was changed so
that the clerk worked vertically down each of the three
columns. Notating the questionnaires with small-sized
input coding (eg, a tick in the 'yes' box was entered as
2), and indications of which computer column numbers
this question was assigned, eased both data input and
checking.
Very often the preferred order of questions is that
judged by subjects as being the most natural order.
Subsequently, whereas the Nordic Musculoskeletal
Questionnaire asked for Annual Prevalence - Annual
Disability - Weekly Prevalence, this was amended to
Annual Prevalence - Weekly Prevalence - Annual
Disability (as shown in Figure 2).
The use of right justified text in the Nordic Musculoskeletal Questionnaire was removed given the known
difficulties this posed to poor readers unable to handle
the uneven amount of spacing between successive
words. The use of bold key words was included in
places, though care was taken to balance such instances
so that several bold words did not cluster together.
200

Administrative type
The results were very markedly affected by the
response rate arising from the different methods of
administering the questionnaire. Figure 3 illustrates
this effect for the weekly prevalence of neck trouble in
seven of the ten stores visited.
In stores where HSE administered the questionnaire
(stores coded 1), all were returned including those on
sick leave or annual leave. In stores where the retail
staff issued the questionnaires (stores coded 2) the
response rate fell slightly. The response rate fell further
for those stores where the individuals returned their own
questionnaires (stores coded 3) and in the latter, the
weekly prevalence of neck trouble was much higher.
This suggested that the individuals who returned their
questionnaires were, in the main, the one with troubles
and a self-interest in doing so.

Discussion
The results demonstrate the value in gaining experience of a questionnaire prior to its inclusion in any
major survey. The repeatability exercise was consistent
with findings of non-identical answers varying from 0 to
23% reported by Kuorinka et al 1. In their reliability
studies, comparisons were made between the NMQ
data and information gleaned by a physiotherapist
taking a clinical history. When leaving two to three
weeks between questioning sessions they concluded
that the method of administration had an effect on the
repeatability but they did not describe the effect on the
prevalence outcome. In this study, the method of
administration was demonstrated to be of considerable
significance and underlines the need for response rate

I00

8060-

~- 4 o -

,,- /

//

y/

/./

/ /

/ /

/ /

/ /

/ /

.//

/ /

/ /

/ /

/ /

/ /

/ /

/ /

/ /

/ /

/ /

/ /

20- : , - . ,
/

W /

L//

--

/ /

/ /

/ /

i i

/ /t
//i

/ A
/

mid

2
2
2
3
Administration s t y l e
Prevalence
~
% R e s p o n s e rote

I HSE

2 Store 3 Freely administered

1 %

Figure 3 Weekly prevalence of neck trouble in supermarket

cashiers
Applied Ergonomics

C E DICKINSON, K CAMPION, A F FOSTER, S J NEWMAN, A M T O ' R O U R K E AND P G THOMAS

figures to be reported in studies of this kind. That the

data are vulnerable to such distortion also casts serious
limitations on utilizing the reference manuals and data
gathered from the Swedish system and reported by
Ydreborg and Kraftling3.
There are benefits attributed to subjects not having
to provide their name but this requires detailed
arrangements being made in workplaces. Arising from
the response rate:prevalence relationship (Figure 3) is
the recommendation that future studies necessitate a
return exceeding about 80%. Subsequent HSE studies
indicate this to be realistic when care is taken.
Whilst this work has been valuable in producing an
improved English version of the NMQ, further improvements should be considered in the future. Of
particular relevance is the inclusion of questions to
indicate which reports of symptoms are believed to be
work-related.
The HSE's eventual questionnaire is to become its
standardized questionnaire tool for identifying the size
of a musculoskeletal problem in workplaces. Large
workplaces can be studied very quickly, cheaply and
validly by having the questionnaire and computer
analysis programs immediately available. Further benefits will accrue from an expanding databank of occupational populations to facilitate comparisons to be made.
Its simplicity permits a sizeable number of specialists to
use the questionnaire as a means to identifying those
workplaces and subjects necessitating a more in-depth

Vol 23 No 3 June 1992

examination. Precise standardization of the method of

administration in some workplaces is not always
practical, though recommendations are provided to
HSE's specialists with the caveat that surveys are not
analyzed unless a response rate exceeding 80% is
achieved.
References
1. Kuorinka, I, Jonsson, B, Kilbom, A, Vinterberg, H,
Biering-Sorenson, F, Anderson, G and Jorgensen, K
'Standardized Nordic Questionnaires for the analysis of
musculoskeletal symptoms' Appl Ergonomics Vol 18 No 3
(1987) pp 233-237
2. Wilson, J R, and Corlett, E N (eds) Evaluation of human
work. A practical ergonomics methodology Taylor and
Francis (1990) pp 71-85
3. Ydreborg, B and Kraftling, A Referensdata Till Formularen FHV 001 D, FHV 002 D, FHV 003 D, FHV 004 D
och FHV 007 D. Rapport 6 (1987)
Bibliography
1. Abramson J H Survey methods in community medicine.
An introduction to epidemiological and evaluative studies
Churchill Livingstone (1984)
2. Andersson, K, Karlehagen, S and Jonnsson, B 'The
importance of variations in questionnaire administration'
Appl Ergonomics Vol 18 No 3 (1987) pp 229-232
3. Cutts, M and Maher, C Writing plain English: A guide for
writers and designers of official forms, leaflets, letters,
labels and agreements Plain English Campaign (1980)

201