Gait & Posture 29 (2009) 360–369

Contents lists available at ScienceDirect

Gait & Posture

journal homepage: www.elsevier.com/locate/gaitpost

Review

The reliability of three-dimensional kinematic gait measurements:

A systematic review
Jennifer L. McGinley a,d,*, Richard Baker a,b, Rory Wolfe a,c, Meg E. Morris a,d
a
Centre for Clinical Research Excellence in Clinical Gait Analysis and Gait Rehabilitation, Murdoch Childrens Research Institute, Royal Children’s Hospital, Melbourne, Australia
b
Hugh Williamson Gait Analysis Service, Royal Children’s Hospital, Melbourne, Australia
c
Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Australia
d
School of Physiotherapy, The University of Melbourne, Melbourne, Australia

A R T I C L E I N F O A B S T R A C T

Article history: Background/Aim: Three-dimensional kinematic measures of gait are routinely used in clinical gait analysis
Received 5 March 2008 and provide a key outcome measure for gait research and clinical practice. This systematic review
Received in revised form 5 September 2008 identifies and evaluates current evidence for the inter-session and inter-assessor reliability of three-
Accepted 5 September 2008
dimensional kinematic gait analysis (3DGA) data.
Method: A targeted search strategy identified reports that fulfilled the search criteria. The quality of full-
Keywords:
text reports were tabulated and evaluated for quality using a customised critical appraisal tool.
Gait
Results: Fifteen full manuscripts and eight abstracts were included. Studies addressed both within-
Gait analysis
Reliability
assessor and between-assessor reliability, with most examining healthy adults. Four full-text reports
Reproducibility evaluated reliability in people with gait pathologies. The highest reliability indices occurred in the hip and
Measurement error knee in the sagittal plane, with lowest errors in pelvic rotation and obliquity and hip abduction. Lowest
reliability and highest error frequently occurred in the hip and knee transverse plane. Methodological
quality varied, with key limitations in sample descriptions and strategies for statistical analysis. Reported
reliability indices and error magnitudes varied across gait variables and studies. Most studies providing
estimates of data error reported values (S.D. or S.E.) of less than 58, with the exception of hip and knee
rotation.
Conclusion: This review provides evidence that clinically acceptable errors are possible in gait analysis.
Variability between studies, however, suggests that they are not always achieved.
ß 2008 Elsevier B.V. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
2. Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
2.1. Study identification and selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
2.2. Data extraction and quality appraisal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
3.1. Sample selection, composition and description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
3.2. Study procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
3.3. Statistical analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
3.4. Reliability findings: overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 364
4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
4.1. Methodological considerations: participant and assessor samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
4.2. Methodological considerations: study design and procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
4.3. Methodological considerations: statistical analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367

* Corresponding author at: Gait CCRE, Murdoch Children’s Research Institute, Hugh Williamson Gait Laboratory, Royal Children’s Hospital, Flemington Rd Parkville, Victoria
3052, Australia. Tel.: +61 3 9345 5354; fax: +61 3 9345 5447.
E-mail address: jennifer.mcginley@mcri.edu.au (J.L. McGinley).

0966-6362/$ – see front matter ß 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.gaitpost.2008.09.003
 J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369
5. Considerations and recommendations for future research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Acknowledgements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1. Introduction
Three-dimensional kinematic gait measurements are used
widely in clinical gait analysis services and clinical research.
Despite the increasing number of gait laboratories, there is limited
cohesive information regarding the reliability of kinematic gait
measurements. Two recent reports [1,2] highlighting between-
laboratory differences in 3DGA measures have raised concerns
from the wider orthopaedic community [3,4]. This paper presents a
systematic review and qualitative appraisal of the evidence
describing the reliability of three-dimensional kinematic gait data
(3DGA).
The reliability and validity of gait measurements should be
known in order to be used appropriately [5]. As repeated gait
measurements typically show some differences, these can be
assumed to contain a proportion of error. This review addresses
reliability, which is the extent to which gait measurements are
consistent or free from variation. The term ‘error’ in this paper is
used within the context of reliability and refers to the variation
found across repeated measures. Knowledge and understanding of
typical measurement variation is helpful to guide the use and
interpretation of data.
Clinical gait analysis typically seeks to discriminate between
normal and abnormal gait and to assess change in walking over
time [6]. Repeated gait measurements can be used to evaluate
the response to therapeutic interventions such as surgery,
physiotherapy, medications and orthotics. Variability between
‘before’ and ‘after’ measurements may be due to treatment effects
or measurement variation, or a combination of both. Knowledge of
the error magnitude can enable clinical teams to minimise the risk
of over-interpreting small differences as meaningful, [7] and to
have greater confidence that the treatment effect exceeds the
measurement error. Additionally, the use of measurements with
low reliability in clinical research may lead to underestimation or
failure to detect significant effect sizes; with too much noise (error)
drowning out real effects [8].
The reliability or consistency of 3DGA can be examined in
various ways. Typically multiple walking trials are collected
within a single session. Variability between these trials can be
regarded as ‘intrinsic variation’, and reflects the inherent variation
within unimpaired individuals or those with pathology [7]. These
intrinsic variations cannot be reduced, yet provide a baseline
indication of variation independent of other error sources. Other
measurement variation arises from extrinsic factors such as
procedural errors [7]. Reliability of data obtained from different
testing sessions conducted by the same assessor (inter-session or
within-assessor) and by different assessors (inter-assessor) is
susceptible to these extrinsic errors. Inconsistent marker place-
ment is generally regarded as a key factor, although other factors
such as inconsistent anthropometric measurements, variation in
walking speed, data processing or measurement equipment errors
may also contribute to data variation [9]. Reliability across
sessions is of immediate relevance to clinical gait analysis
practices; as observations are routinely and regularly repeated
to measure patient performance over time, and different assessors
may conduct tests for an individual patient. This aim of this review
was to identify and critically evaluate the evidence describing the
reliability of lower body kinematic gait data across repeated
sessions.
361
368
368
368
2. Method
2.1. Study identification and selection
The search strategy for this review began with retrieval of published reports
indexed on health or biomechanics related electronic databases from MEDLINE
(1970 to July 2007), EMBASE (1980 to July 2007), CINAHL (1982 to July 2007), RECAL
Bibliographic Database (pre-1990 to July 2007) and Inspec (1970 to July 2007). The
search was limited to literature reporting studies of human subjects with abstracts
written in English. The search terms were customised to each database and
included the following keywords; gait, gait disorders, gait analysis, observer
variation, reproducibility of results, and reliability. Bibliographies of identified
papers and relevant conference proceedings were hand searched.
The review was conducted to be of primary relevance to gait laboratories
collecting typical multi-joint lower body gait kinematic data. The titles and
abstracts identified by the initial search strategy were screened by the first named
author (JM) to identify potentially eligible reports and retrieve full-text reports.
When the title or abstract did not clearly indicate whether an article should be
included then the complete article was obtained and reviewed. Full-text reports
were then evaluated by two authors (JM and RB) for the following inclusion criteria:
(1) reports of the inter-session or inter-assessor reliability of three-dimensional
kinematic gait or running measures of human participants; (2) including at least
three joints of the lower body (pelvis, hips, knees, ankles); (3) reporting numerical
findings from repeated kinematic data capture from more than one measurement
occasion (with markers replaced each occasion); (4) full papers or abstracts (not
later published as full papers); (5) published with an English abstract.
2.2. Data extraction and quality appraisal
Reports were retained as either full-text reports or published abstracts. A
standardised data extraction and appraisal form was constructed to identify and
detail key features of each study. Two reviewers (JM and RB) initially independently
piloted the form with a small subset of representative studies to confirm the
content and to assess the reliability. The extracted study details focused on
participant characteristics and recruitment, study procedures and biomechanical
models, and the statistical analysis techniques.
The quality of study design and conduct are key elements in evaluating scientific
evidence, with contemporary systematic reviews providing study quality
appraisals in addition to quantitative reviews. Although a large body of literature
exists to provide guidelines for the systematic evaluation of research methodology
[10,11], the majority are focussed primarily upon studies of healthcare interven-
tions, in particular randomized controlled trials. As no standardised or established
guidelines were located for reviews of reliability, a customised quality appraisal
form was developed. The appraisal component was developed to integrate relevant
examples of methodological quality criteria from other systematic reviews of
reliability [12–14], and gait classification [15]. Relevant quality themes and
principles were also adapted from quality criteria proposed for the measurement
properties of health status questionnaires [16], and the QUADAS tool used to
appraise studies of diagnostic accuracy [17]. Additionally, an initial expert panel
was formed to consider and define the data extraction and appraisal criteria for the
study. Quality appraisal indicators were developed into a standardised form to
ensure a structured approach to evaluation of key quality elements and to ensure
equal appraisal of all papers. Appraisal items were not scored as the validity of such
scoring systems is currently unproven [18]. The appraisal criteria included themes
related to external validity such as sampling methods and description, standardisa-
tion and description of procedures, and selection of statistical analysis techniques.
Appraisal criteria were not applied to the abstract-only reports because their
brevity limited the provision of methodological detail.
The data extraction and appraisal form were used independently by two
reviewers (JM and RB) to extract key details from each report and to evaluate the
quality of each full-text paper. Any rating disagreements on quality criteria were
checked against the original article to ascertain the correct scoring according to a
pre-defined procedure, in accordance with established and recommended protocols
[15,18].
3. Results
The electronic searches and hand-search of references and
selected conference proceedings yielded a total of 510 articles.
Following the application of the inclusion/exclusion criteria, 23
362 J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369
studies were identified for inclusion in the systematic review; 15
full papers and 8 abstracts.
Details of the 23 identified studies are provided in Table 1.
Within-assessor reliability was reported in 15 of the studies, and
between-assessor in 10. Four of the studies were described as test–
retest, as either the number of assessors was not stated, or it was
uncertain whether the same assessors applied the markers in
repeated sessions [19–22]. Gait participant sample sizes ranged
from 1 to 50 (median of 10), with approximately 70% of studies
examining healthy subjects. Three studies included groups of both
healthy and disabled participants [20,22,23]. Thirteen studies
included adults, eight included children and five did not report the
age of participants. The number of assessors ranged from 1 to 24,
and included physiotherapists and technicians. Commercially
available biomechanical models were most frequently used in
the studies. Measurement sessions commonly numbered two or
three and occurred over time intervals ranging from 2 h to 20
weeks.
3.1. Sample selection, composition and description
The quality indicators related to sampling methods and
description varied widely across the 15 full-text reports. Key
criteria are reported in Table 2. The sampling method for
recruitment of gait participants was frequently not reported. In
the studies with unimpaired people, it is likely that convenience
sampling was employed. In studies with clinical participants, the
sampling method was also predominantly convenience samples
from local known clinical populations [2,21,23]. Yavuzer et al. [24]
sought consecutive patients with stroke who fulfilled the study
criteria.
The study inclusion/exclusion criteria for gait participants were
stated in around half of the full-text reports and varied greatly in
detail. Of the studies including ‘healthy’, ‘unimpaired’ or ‘normal’
subjects, seven specified inclusion/exclusion criteria such as the
absence of previous musculo-skeletal, neurological or other
conditions that may affect gait. A single study chose to recruit
only male healthy participants to minimise any potential influence
of gender variation on the findings [25]. Another study selected
participants older than 16 in order to minimise potential variation
due to adolescent growth and variable walking velocity [9]. Of the
three full-text studies including participants with CP, common
inclusion criteria included age, type of CP, ability to walk
independently, and adequate cognition to cooperate with gait
analysis. Required gait ability varied from the ability to walk
without an orthosis, [2] to the ability to walk continuously for
15 min without walking aids or orthoses [23]. Noonan et al. [2]
offered study participation to a wide range of subjects, including
subjects from mild to severe disability, both pre- and post-
operatively, with and without bracing and with varied distribution
of spasticity. This sample seems likely to be representative of a
typical gait laboratory population, although it is uncertain whether
the patients with prior surgery had stabilised prior to study
inclusion. Children with prior therapeutic intervention (therapy/
casts/surgery) were excluded in the study by Steinwender et al.
[23].
The quality of the descriptions of the gait participants also
varied markedly across the full-text reports. Gait participants were
fully described in terms of age, gender, health status and
anthropometric characteristics in only seven studies. Relevant
pathology-specific detail was provided in the descriptions of the CP
and stroke participants, although only one of the three studies
including subjects with CP reported the Gross Motor Function
Classification System (GMFCS) [26] to detail gait ability. Adequate
clinical participant descriptors are particularly important as they
reflect the heterogeneity of the sample, and allow insight into the
generalisability of the findings to other populations. The number of
gait participants varied widely across full-text studies from 1 [27]
to 40 [23,28], with 10 reports including more than 10 participants.
Justification for the sample size of gait participants was not
provided in any study.
The sampling method used to recruit assessors or the related
inclusion and exclusion criteria were not reported in any of the
full-text reports. Descriptions of the assessors were generally
poor, with only two studies reporting the desired complete
details including the number of assessors, professional back-
ground and experience or training [29,30]. Physiotherapists were
most often reported as the group, with six of the reports also
describing their assessors as either experienced or highly trained
[9,24,28–31]. The number of assessors was frequently small,
between one and five.
3.2. Study procedures
The majority of full-text reports appeared to use standardised
measurement protocols on repeated occasions. Of the full-text
reports, twelve studies reported capturing data at self-selected
or normal speed, with one study selecting a fixed speed (of
running) [32], and one study not reporting speed [27].
Metronome paced or beep test controlled speed protocols were
also reported in abstracts [33,37]. Associated spatio-temporal
(ST) gait measures or measures of between-session ST variation
were provided in 11 of the 15 reports. Data capture systems
were generally adequately described with most full-text reports
providing adequate overall descriptions of the biomechanical
models used, or providing appropriate reference to available
descriptions. Desirable model-specific details regarding within-
model options such as Knee Angle Device (KAD) utilisation,
post-testing adjustment of thigh rotation angles and specifica-
tion of hip joint centre location techniques were inconsistently
provided.
The duration between measurement sessions varied and ranged
from 2 h [24] up to 20 weeks [2], with several studies failing to
state their time interval (e.g. [7,34]). Within-study standardisation
of the measurement interval did not occur uniformly, with some
studies reporting varied time intervals or wide within-study
ranges of 6–20 weeks [2].
3.3. Statistical analysis
The coefficient of multiple correlation (CMC), or coefficients
of multiple determination (CMD) were used in eight of the 23
studies. These techniques examine consistency across the entire
gait cycle and are expressed as an index of agreement between 0
and 1. Intra-class correlations (ICCs) were reported in six
studies. Various forms of ICCs are available for different study
designs and different methods of estimation exist with
corresponding differences in the underlying assumptions and
generalisability [35].
Absolute measures of measurement variation in degrees were
also included in numerous reports, including standard deviations
(S.D.), standard error (S.E.), range, mean absolute difference and
Bland and Altman limits of agreement. The majority of studies used
techniques to examine the reliability of the entire kinematic curve,
with others choosing to examine selected key kinematic peaks,
amplitudes or events [9,24,32,34,36,37]. Many authors also
presented within-session reliability data. Schwartz et al. and
Murphy et al. [7,38] reported sources of variance related to within-
session (inter-trial), inter-session (within-assessor) and inter-
therapist (between-assessor).
Table 1
Characteristics of the identified studies of the reliability of 3DGA data.
Study Biomechanical model
Besier et al. [31] Custom models; Anatomical
landmark model, Functional
joint model
Charlton et al. [27] VCM, OLGA
Cowman et al. [37] (A) Bilateral CODA mpx30
Eve et al. [40] (A) PiG
Ferber et al. [32] Unilateral model MOVE3D
Gok et al. [25] VCM
Gorton et al. [19] (A) VCM
Gorton et al. [1] (A) Vicon and Motion Analysis
Corporation software
Gorton et al. [33] (A) Vicon and Motion Analysis
Corporation software
Growney et al. [34] 21 marker model similar to
‘Kabada’ model with ANALYZE
software.
Kadaba et al. [28] Conventional gait model
Leardini et al. [29]# Custom model: Anatomically
based protocol
Mackey et al. [21] Cleveland clinic marker set, EvA
and Orthotrak (Motion Analysis
Software)
Maynard et al. [36] CODA mpx30 model
Miller et al. [20] (A) Modified Helen Hayes
Monaghan et al. [9] Unilateral CODA
Murphy et al. [38] (A) Conventional biomechanical
model
Noonan et al. [2] VCM, CCM/Orthotrak
Quigley et al. [22] (A) NS
Steinwender et al. [23] Conventional gait model
Schwartz et al. [7] VCM
Tsushima et al. [30] VCM
Yavuzer et al. [24] VCM
(A), Abstract only; F, female; M, male; VCM, Vicon Clinical Manager; GC, gait cycle; S.D., standard deviation; SEM, standard error of measurement; W-Ass, Within-assessor; ANOVA, analysis of variance; CODA, Cartesian
Optoelectric Dynamic Anthropometer; OLGA, optimised lower-limb gait analysis; LOA, limits of agreement; CCM, Cleveland Clinic Model; NS, not stated; PiG, Plug-in-Gait; PT, Physiotherapist; CMC, coefficient of multiple
correlation; CMD, coefficient of multiple determination; CV%, coefficient of variation; ICC, intra-class correlation; CI, confidence interval; NS, not stated; # data refers to Leardini et al. [29] Study 2 (inter-examiner).
Participant characteristics
(n, age (years), type, gender)
n = 10, Age: NS, Able-bodied, 6
M;4 F
n = 1, Age: NS, Healthy
n = 2, Age: 9, 21 years, Normal
n = 10, Age: 31.4, Healthy
n = 20, Age: 21.4 (mean),
Healthy, 7 M; 13 F
n = 11, Age: 32 (mean), Healthy,
11 M
n = 50, Age: 5–16, Normal
n = 1, Age: NS
n = 1, Age: NS
n = 5, Age: NS, Normal, 3 F; 2 M
n = 40, Age: 18–40, Normal
n = 1 Age: 7, Healthy, F
n = 10, CP; Age: 6 M; 9+/ 4, 4 F;
12+/ 3
For inter-session: n = 10, Age:
39.2 (mean) 5 M 5 F, For
inter-assessor: n = 19, Age: 34.4
(mean) 4 M, 15 F
n = 10, 5 = CP, 5 = non-disabled,
Age 5–16
n = 10, Age: 28.5 (mean), 7 F, 3
M, Healthy
n = 3, Age: 50 (mean, S.D. = 8),
Stroke
n = 11, Age: 5–17, CP, 6 M; 5 F
n = 10, CP (n = 5), Typically
developing (n = 5), Age: 9.6
(mean)
n = 40, Healthy (n = 20), CP
(n = 20), Age: 7–15
n = 2, Age: 40, 36, Healthy
n = 6, Age: 34.8 (mean),
Unimpaired, 3 M; 3 F
n = 20, Age: 54.2 (mean), 7 F; 13
M, Stroke
Assessor characteristics Type of reliability, session number and interval Statistical analysis
(n, profession)
n = 5, Discipline: NS Inter-session (W-Ass), 2  sessions, Analysis across GC curve: CMD, Average
Inter-assessor, Interval: 4 h systematic error
n = 3, Discipline: PT Inter-session (W-Ass), 3  sessions, Analysis across GC curve: S.D. of averaged joint
Inter-assessor, Interval: NS angle
n = 2, Discipline: PT Inter-session (W-Ass), 3  sessions, Kinematic key points selected for analysis:
Inter-assessor Error indices (% of 95% confidence ranges)
n = 1, Discipline: Inter-session (W-Ass), 2  sessions, Analysis across GC curve: S.E.
therapist Interval: Within a week
n = 1, Discipline: NS Inter-session (W-Ass), 2  sessions, Kinematic key points (stance phase) selected
Interval: 1 week for analysis: ICC, Mean (S.E.M.)
n = 1 or 2, Discipline: Inter-session (W-Ass), 2  sessions, Kinematic key points selected for analysis, ICC,
Physician & technician Interval: 3 days Wilcoxon signed ranks test
n = 2, Discipline: NS Inter-session (test-retest), Interval: >1 week Analysis across GC curve: CMC
n = 24 (in 12 labs), Inter-assessor, Interval: within a 3 month Linear mixed model, S.D. and range
Discipline: clinicians period
J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369
n = 24 (in 12 labs), Inter-assessor, Interval: within 1 month Linear mixed model, S.D. and range
Discipline: clinicians
n = 1, Discipline: NS Inter-session (W-Ass), 3  sessions, Analysis across GC curve: CMC, S.D.
Interval: NS; 3 separate days
n = 1, Discipline = NS Inter-session (W-Ass), 3  sessions >1 week Analysis across GC curve: CMC with mean
subtracted
n = 5, Discipline: PT Inter-assessor, Interval: NS Analysis across GC curve: Averaged S.D.
n = NS, Discipline = NS Inter-session (test-retest), 2  sessions, Analysis across GC curve: CMC (S.D.), Mean
Interval: 1 week absolute difference (S.D.)
For inter-session: NS, Inter-session (W-Ass), 3  sessions, Kinematic key points selected for analysis:
For inter-assessor: Inter-assessor, Interval: within day, & 1 week Bland and Altman LOA, ICC
n = 3,
Discipline = NS
n = NS, Discipline: NS Inter-session (test-retest), 5  sessions Analysis across GC curve: ICC
n = 1, Discipline: NS Inter-session (W-Ass), 2  sessions, Kinematic key points selected for analysis,
Interval: 1 week Bland and Altman LOA, ICC
n = 3, Discipline: Inter-session (W-Ass), 2  sessions, Analysis across GC curve: Multi-level,
Clinician Interval: NS random-effects linear regression model
n = 4 laboratories, Inter-assessor, Interval: 6–20 weeks Analysis across GC curve, Discordance index,
Discipline: NS Absolute variability
n = NS, Discipline: NS Inter-session (test-retest), 5  sessions; Kinematic key points selected for analysis, CV
Interval: Each session >2 days apart (S.D.)
n = 1, Discipline: NS Inter-session (W-Ass), 3  sessions, Analysis across GC curve, CMC (S.D.)
Interval: 3 days within a week
n = 4, Discipline: PT Inter-session (W-Ass), 3  sessions, Analysis across GC curve, Variance components
Inter-assessor, Interval: NS estimation (S.D.)
n = 2, Discipline: PT Inter-session (W-Ass), 2  sessions, Analysis across GC curve, CMC (S.D.)
Inter-assessor, Interval: within 2 weeks
n = 1, Discipline: Inter-session (W-Ass), 2  sessions, Analysis across GC curve and at selected
Technician Interval: 2 h kinematic key points, CV%, CMC, ICC
363
364 J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369

Table 2
Methodological quality of the reviewed full-text studies.

Gait participants Assessor Protocol Model Data Statistical

participant standardisation description description analysis
Sampling Inclusion and Description description and description
method exclusion criteria

Besier et al. [31] Not stated Not stated Partial Partial Adequate Adequate Adequate Adequate
Charlton et al. [27] Not stated Not stated Inadequate Partial Limited Adequate Limited Adequate
Ferber et al. [32] Not stated Not stated Adequate Inadequate Adequate Limited Limited Adequate
Gok et al. [25] Convenience Stated Partial Partial Adequate Adequate Limited Limited
Growney et al. [34] Not stated Not stated Partial Inadequate Limited Adequate Adequate Adequate
Kadaba et al. [28] Not stated Limited Partial Partial Limited Adequate Adequate Limited
Leardini et al. [29] Not stated Not stated Adequate Adequate Limited Adequate Adequate Adequate
Mackey et al. [21] Convenience Stated Adequate Inadequate Limited Adequate Adequate Adequate
Maynard et al. [36] Not stated Stated Partial Inadequate Adequate Adequate Adequate Adequate
Monaghan et al. [9] Not stated Stated Adequate Partial Adequate Adequate Adequate Adequate
Noonan et al. [2] Convenience Stated Partial Partial Limited Adequate Adequate Adequate
Schwartz et al. [7] Not stated Not stated Adequate Partial Limited Adequate Adequate Adequate
Steinwender et al. [23] Convenience Stated Partial Inadequate Adequate Limited Adequate Limited
Tsushima et al. [30] Not stated Stated Adequate Adequate Adequate Adequate Adequate Limited
Yavuzer et al. [24] Case consecutive Stated Adequate Partial Limited Adequate Adequate Limited

3.4. Reliability findings: overview
The diversity in the reported studies precludes a simple
synthesis of results. Meta-analysis of the results was not
considered to be appropriate given the diversity among a fairly
small number of studies, the varied participant ages and pathology,
the marked variability in the quality, methods and selected
statistical analysis and the heterogeneity of results. Under these
circumstances, the review comprised a qualitative analysis of the
research available, a ‘‘best evidence synthesis’’ [39].
Limited comparisons are possible across the seven studies
reporting within-assessor reliability using the CMC or CMD (see
Table 3). Reliability varied widely across the studies and gait
variables. Excluding pelvic tilt, very high values were typically
reported for the sagittal plane data, with the transverse plane
generally showing the lowest reliability (median < .72). The
lowest obtained reliability indices (<.6) were reported for pelvic
tilt, [23,28,30], knee varus [23], and hip, knee and foot (transverse
plane) [23,28,31,34].
Evaluation of the reliability indices (either CMC or ICC) across
all studies confirms that sagittal plane reliability was typically
higher than .8, excluding pelvic tilt. For the coronal plane, most
studies reported reliability indices of >.7. The majority of studies
reported indices <.7 for the transverse plane (excluding the pelvis).
Results from gait studies reported as either S.D. or S.E. provide
the magnitude of error across different gait variables and are
reported in Fig. 1. The diversity of study types, participants and
analyses limits between-study comparisons; however grouping
data in this manner is useful to look broadly at patterns. In general,
sagittal plane errors were typically <48, and coronal plane around
28. Highest errors were seen in hip and knee rotation, and the
lowest errors commonly at the pelvis in the transverse and coronal
plane, and hip abduction. The pattern of these findings broadly
concurs with other reports in terms of range [1,33] or ‘absolute
variability’ [2,29], with reported values of hip rotation ranging
from 168 [29] to 348 [33], in contrast to lower estimates of pelvic
obliquity of less than 68 [1,2,29,33].
Reports of the distribution and relative error sizes across gait
variables did not always coincide with the findings of studies using
CMCs to assess reliability. Both statistical methods suggested that
hip and knee rotation measures showed most variation, with
higher errors reported and generally low CMC values. For some gait
variables, however, the error magnitudes did not reflect the
reported CMC indices. For example, pelvic rotation was frequently
reported with relatively low error (<28), yet showed only moderate
reliability with CMC values ranging from .67 to 89 (median of .72).
Similarly, knee flexion showed uniformly high CMC values, yet
showed relatively larger error magnitudes, with some studies
showing errors in excess of 48.
Of the six studies reporting both within-assessor and between-
assessor error, three found single assessors to be more repeatable
than multiple assessors [7,27,30], one found similar repeatability
[31] and a single study reported between-assessor reliability to be
better than within-assessor [36]. Of the three reports comparing

Table 3
Summary of studies reporting within-assessor reliability of 3DGA, data as coefficient of multiple correlation (CMC) (within-assessor).

Besier et al. Besier et al. Gorton Growney Kadaba Steinwender Steinwender Tsushima, Yavuzer Median
[31] (AL)a [31] (FUN)a et al. [19] et al. [34] et al. [28] et al. [23] et al. [23] et al. [30] et al. [24]
Healthy Healthy Healthy Healthy Healthy Healthy Children Healthy Adults with
adults adults children adults children with CP adult stroke

Sagittal Pelvic Tilt – – .79 .64 .24 .32 .56 .38 .95 .56
Hip flexion .97 .98 .99 .96 .98 .96 .96 .99 .89 .96
Knee flexion .96 .96 .99 .99 .99 .96 .96 .99 .85 .96
Ankle d’flexion .92 .93 .96 .98 .93 .87 .83 .98 .85 .93
Coronal Pel obliquity – – .85 .89 .75 .73 .98 – .85
Hip abduction .93 .92 – .90 .89 .85 .76 .97 – .89
Knee varus/val .80 .82 – .74 .61 .49 .58 .79 – .74
Transverse Pel rotation – – – .88 .72 .67 .71 .89 – .72
Hip rotation .62 .63 .91 .74 .41 .59 .57 .82 – .62
Knee rotation .83 .87 .54 .49 .34 .41 .81 – .54
Foot progression – – – .55 .58 .37 .49 .82 – .55
a
Besier et al. CMCs derived from coefficient of multiple determination data, L side.
 J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369 365

Fig. 1. Summary of gait studies reporting 3DGA reliability as S.D. or S.E.

Study details: Besier [31]; average systematic error between and within-assessors, Charlton [27]a*; SD PIG inter-assessor, Charlton [27]b*; SD OLGA inter-assessor, Eve [40];
SE within-assessor, Gorton [1]; SD inter-assessor, Gorton [33]; SD inter-assessor, Growney [34]; SD within-assessor (right side), Leardini [29]; SD inter-assessor, Maynard
[36]; SD diff. inter-assessor (averaged across events), Monoghan [9]; SD diff. within-assessor (averaged across events), Murphy [38]*; SD within and inter-assessor, Schwartz
[7]; SD inter-assessor. * data estimated from Figure provided.

the repeatability of healthy children and those with CP, no clear
findings emerged. The only full-text report described the repeat-
ability of children with CP as lower than healthy children, although
the provided data show broadly comparable CMC values with
higher values obtained for the CP group pelvic tilt and foot rotation
[23]. In the abstract reports from Quigley et al. and Miller et al.
[20,22], the normal children appeared to be slightly less consistent
than those with CP, although varying across different gait
variables.
4. Discussion
The diversity of study participants, methods, biomechanical
modelling techniques, statistical analyses and results precludes a
366 J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369
simple conclusion about the reliability of 3DGA. Data from studies
reporting reliability indices do however suggest that the majority
of studies reported moderate to good reliability for sagittal and
coronal plane variables, with the exception of pelvic tilt and knee
varus/valgus in some reports. Likewise, estimates of error (S.D. or
S.E.; Fig. 1) suggest most studies reported error of less than 58 for all
gait variables, excluding hip and knee rotation.
Whether 3DGA data is ‘reliable enough’ remains a question that
can be answered only in the context of proposed use, with the
degree of acceptable measurement variation relating directly to
the intended application. It is clearly beyond the scope of this
review to specify ‘acceptable limits’ of reliability for 3DGA data. We
do however believe that in most common clinical situations that
error of 28 or less is highly likely to be widely considered
acceptable, as such errors are probably too small to require explicit
consideration during data interpretation. Errors of between 28 and
58 are also likely to be regarded as reasonable but may require
consideration in data interpretation. We suggest that errors in
excess of 58 should raise concern and may be large enough to
mislead clinical interpretation. Data from the studies reporting
error reveals that the majority of studies and gait variables show
errors that fall between 28 and 58. Hip rotation clearly shows the
highest error, although it is noteworthy that some studies report
lower error of <58 for this variable, suggesting that lower error is
currently achievable [7,27,38,40,41]. This compares well with
clinical measurements of similar variables. For example, both
Fosang et al. [42] and McDowell et al. [43] report variability of
between 58 and 108 in clinical assessment of sagittal plane range of
movement of the major joints of the lower extremity.
4.1. Methodological considerations: participant and assessor samples
The widespread use of 3DGA as part of clinical services in
clinical populations warrants careful consideration of best-quality
study methodology. Appropriate sample composition and inclu-
sion/exclusion criteria should ensure that the range of character-
istics of interest in a clinical population is most likely to be present
in a sample, and that the findings can be generalised. Of the four
full-text reports including clinical participants [2,21,23,24], three
chose convenience samples. Such samples may be susceptible to
sampling bias, such as selective inclusion of the most cooperative
or compliant participants. Ideally, subjects participating in a study
of a measurement should consist of individuals who would be
likely to undergo the test in clinical practice, and reflect a
continuum of severity from mild to severe [44]. If the target
population is intended to be typical clinical gait analysis service
patients then recruitment strategies could consider use of a
prospective cohort design with consecutive clinical subjects, such
as the case consecutive sampling described by Yavuzer et al. [24].
Such designs are recognized as the best method in studies of
diagnostic tests to ensure a representative sample and avoid
selection bias [45]. Alternate strategies could include stratified
random sampling of typical gait disorder populations.
The prevalence of healthy participants in the majority of the
studies is noteworthy and contrasts with the widespread clinical
and research application of 3DGA to evaluate gait disorders. Gait
analysis services typically include those with gait conditions such
as CP [46], spinal cord injury [47,48], spina bifida and acquired
brain injury [49]. Furthermore, research studies have used 3DGA to
characterise gait disorders and examine intervention efficacy in
diseases such as Parkinson’s disease [50], myelomenigocele [51],
and CP [52]. Generalisability of the error associated with repeated
measures of healthy adults to children or those with gait pathology
should be viewed with some caution. Adult gait data is generally
found to be less variable than children’s [53], and younger children
also more variable than older children [19]. Children with CP were
more variable for some kinematic gait variables than healthy
children [23]. Measures of gait data reliability are intrinsically
related to the variability within the studied group [54], with
measurements widely considered to be population-specific [55].
Whether estimates of error can be reasonably generalised across
clinical populations should be carefully considered, in the context
of the characteristics of the specific pathology, and the associated
impairment and gait dysfunction characteristics. Furthermore,
although different gait disorders may be associated with variable
levels of intrinsic gait repeatability, it is not clear whether the
nature of the gait disorder has any direct effect on procedural
sources of error such as marker placement. It is likely that such
errors may be related to patient-specific factors such as cognition,
compliance and cooperation which may or may not be related to
the gait disorder.
The potential influence of the assessor characteristics on the
reliability of 3DGA data received very limited focus within the
studies in this review, with generally poor detailing of assessor
recruitment and descriptions. Kinematic 3D gait measurement
using landmark-specific models requires specialised staff skills,
including accurate and consistent placement of markers, and
expert knowledge of the underlying biomechanical model.
Training of clinical staff in standardised protocols is widely
considered to be important [1,29]. The consistency of the measures
may therefore be influenced by assessor experience, expertise,
professional background and additional training [56], with
experience of the clinical team potentially contributing to random
error in gait data [57]. Inclusion criteria or sampling methods for
assessors were not reported in any study, and it seems probable
that assessors were convenience samples of staff working within
the authors’ laboratories. Whether the samples were influenced by
any biasing factors, such as recruitment of only the most
experienced or ‘best’ assessors is uncertain. If experience or
discipline-specific training is a determinant of 3DGA measurement
reliability, then it is uncertain whether the results of ‘best’
assessors can be applied to other inexperienced assessors, or those
from different professional backgrounds. Similarly, if the findings
are from novice assessors, then the error sizes reported may be
larger than those typically achieved by experienced assessors with
greater expertise.
4.2. Methodological considerations: study design and procedures
Although the majority of studies described the use of
standardised protocols, wide variation was apparent in the
duration between measurement sessions. Justification of the time
interval duration is recognized as a desirable attribute of study
quality [16], but was absent in the majority of reports. Selection of
an optimal interval in repeated 3DGA measures requires con-
sideration of both practical and theoretical issues. In principle,
intervals should be far apart to minimise fatigue or memory bias
effects, but short enough to avoid genuine change in the
measurements [16,55]. Artificially short intervals within a day
are often most feasible to achieve, yet may leave visible signs of
marker placement on skin to ‘unblind’ a repeat assessment or
subsequent assessor, or increase the possibility that assessors may
remember aspects of anthropometric measures or landmark
identification. Fatigue may also cause true variations in the gait
patterns of clinical subjects when measured repeatedly within a
day by multiple assessors. In contrast, longer time periods of
months increase the possibility that real change has occurred
within the measurement interval, potentially introducing disease
progression bias [17]. In clinical populations such as CP,
deterioration in gait has been documented over periods of 1–2
 J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369
years [58,59], and may potentially occur over shorter periods.
Further, the exact level of intrinsic stability of able-bodied human
gait patterns over hours, days, weeks, months or years has not been
well detailed.
Blinding of assessors to prior measurements is typical practice
within repeatability studies of measurement tools other than
3DGA (e.g. [60]). Although the potential for assessor bias is less
apparent with instrumented measures, it remains a possible factor
in some studies which good research design may minimise. It is
particularly relevant to within-laboratory or within-assessor
studies using biomechanical models that rely on landmark-specific
marker placement and anthropometric measures. When measures
are repeated over short duration intervals, assessors may recall
anthropometric measures and/or bony landmark identification.
Bias may also be introduced in data processing, by unblinded
selective trial inclusion or post hoc data adjustment. Three authors
reported efforts aiming to minimise assessor bias. Tsushima et al.
[30] ensured that any traces of marker placement were absent
prior to repeated marker placement, and both Maynard et al. [36]
and Noonan et al. [2] reported that assessors/study sites were blind
to previous measurements. We suggest that future studies reflect
upon potential sources of bias within study design and when
relevant consider blinding assessors to previous measurements.
Provision of concurrent ST data are a potentially useful
additional indicator of the ‘true’ level of between-session gait
stability. Kinematic gait patterns are known to vary with changes
in walking speed [61]. Significant changes in speed or step size
across sessions are therefore more likely to be associated with
‘true’ change in kinematic variability, rather than error related to
inconsistent marker placement. Inspection of the reported ST data
shows marked across-study differences in ST variation. Healthy
adults varied little in mean walking speed (0.03 m/s; 3%
Coefficient of Variation (CV)) across four measurement sessions
within 2 weeks [30]. Higher variation was evident in the study of
children with CP by Noonan et al. [2], which included four visits to
separate laboratories over 6–20 weeks, reporting a mean absolute
variability of 0.3 m/s, with a maximum absolute variability of
0.6 m/s. These wide variations suggest that the resultant dis-
cordance index may include kinematic changes due to differences
in walking speed in some individuals across measurement
sessions.
The data selected for the evaluation of reliability may
potentially influence between-session data variation and differed
markedly across studies. Measurement sessions ranged from two
to five (Table 1) and trial numbers varied from a single or ‘typical’
or ‘representative’ trial [2,36], up to 10 trials (e.g. [1,9,33]). Some
evidence suggests that the number of analysed trials may influence
the reliability of gait measurements. Monoghan et al. [9] examined
the inter-session reliability of two, four, six, eight and 10 trials,
finding that reliability improved with higher trial numbers,
subsequently advocating that 10 trials be used in analysis.
Similarly, in a study of inter-session reliability of the kinematics
of able-bodied running, Diss [62] found higher reliability indices
from inclusion of five trials, in contrast to the lower values
obtained from a single trial. Wide variations in methodology
prevent a detailed examination of the influence of trial number
within this review. It is notable however, that the two studies
including only single trials reported generally lower values of
reliability [36] and larger data variability [2].
The majority of studies either stated or are presumed to have
captured data in barefoot conditions, inferred from the description
of skin-mounted foot markers. No study examined gait with an
orthosis. It is common for children with CP or adults after stroke to
wear lower limb Ankle-Foot Orthoses (AFOs). Typical clinical gait
analysis for these people includes measures of gait in both barefoot
367
and AFO conditions, requiring marker replacement for the AFO
condition. The data from both gait conditions are commonly used
for clinical interpretation and evaluation of AFO prescription and
efficacy. Further studies are needed to examine the reliability of
3DGA data from gait conditions including orthoses.
Comparisons of the repeatability of alternate biomechanical
models are likely to influence and guide the development and
adoption of more reliable models. Adequate model description is
therefore necessary to allow ready identification of the models
used. Two studies evaluated the repeatability of alternate models
with concurrent data capture. Charlton et al. [27] compared the
within- and between-assessor repeatability of a new model using
optimisation techniques (OLGA) with a conventional gait model
(VCM), finding lower error with OLGA. In contrast, Besier et al. [31]
found few differences in either within- and between-assessor
repeatability of a conventional anatomical landmark model
compared to a newer model with functional (motion) calibration.
No conclusions can be made from this review as to whether
particular models are more repeatable than others, due to the
diverse methodology used, the varied statistical analysis and
variable study quality.
4.3. Methodological considerations: statistical analysis
A key question in the reliability of 3DGA data is whether the
measures are reliable enough for clinical decision-making.
Although indices such as the CMC and ICC were commonly
reported, it is now well-recognized that, in isolation, correlation
indices do not tell us whether the measures are ‘reliable enough’,
with even high values potentially hiding measurement errors
judged to be of clinical importance [63]. Furthermore, expressing
data variability as a coefficient results in units that are difficult to
interpret clinically [29]. To be most useful, variability should be
expressed in a manner that can be directly related to the
measurement itself, in the same measurement units (e.g. degrees)
[64]. This is a significant limitation to much of the existing
literature, with only around half of the papers reporting error in
absolute terms. Interpretation of reliability indices according to
reference ranges of arbitrary ‘acceptable or unacceptable’ values
also occurred in some of the studies. This is now generally regarded
as unreasonable with preference that the adequacy of reliability
outcomes should be reported in the context of the intended
research or clinical utilisation.
The prevalence of reports using the CMC warrants particular
attention, as the calculation method of the CMC is markedly
influenced by the joint range of motion (ROM). As noted by
previous authors [23,34], joints with large ROM typically record
high CMCs, and conversely, joints with low ROM typically show
poorer reliability. Furthermore, lower limb joints vary greatly in
ROM across patient groups and individual patients, and subjects
with gait pathology may show either increased (e.g. pelvic tilt) or
reduced (e.g. knee flexion) ROM. Inspection of reported data
confirms this pattern, with generally low and variable CMC values
achieved for pelvic tilt (for example see [23,28,30]) and only
relatively high values of >.85 reported for sagittal knee motion
(Table 3). The marked variation in joint range across the lower limb
seriously limits the utility of this measure, and caution is
advocated in interpreting such results. We recommend that this
technique should not be used in isolation in future reliability
studies.
It is recommended that future studies reporting reliability of
3DGA data include absolute measures of measurement error such
as the S.D., S.E.M. or alternate forms. Consideration should also be
given to the investigation and development of minimum levels of
detectable change (MDC), or minimal clinically important differ-
368 J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369

Table 4 As an alternative to research with clinical participants, small

Factors to consider when planning or reporting a 3DGA gait reliability study.
Descriptor
Methods
Participants (gait) Eligibility criteria. Recruitment strategy.
Participants (assessors) Eligibility criteria. Recruitment strategy.
Protocol and model Description of setting, measurement
protocol, data capture systems and
biomechanical models (in sufficient
detail to allow study to be repeated).
Study design Single or multiple assessors and/or labs.
Number and timing of sessions and
trials within session. Standardisation of
assessment intervals. Variables to be
investigated.
Steps to reduce bias Has blinding of assessors occurred if
appropriate?
Sample size How has sample size been determined?
Statistical methods Description of statistical measurements.
Do these provide outcomes with the same
units as the measured variables to ensure
clinical applicability of results?
Results
Participants (gait) Description of participant characteristics.
Participants (assessors) Description of participant characteristics
with specific emphasis on professional
background and experience.
Data Report of basic temporal data parameters
along with more complex gait data.
Consider reporting estimates of variance
of various sources: i.e. inter-trial,
within-assessor, between-assessor etc
ences (MCID) [65]. Further evidence may also be sought for the
responsiveness of 3DGA measures. Whether the error magnitudes
are sufficiently low will be relative to the magnitude of expected
intervention effect size and specific population context. Further
studies are necessary in typical clinical populations to provide high
quality evidence indicating whether 3DGA measures are suffi-
ciently reliable to detect clinically important change.
5. Considerations and recommendations for future research
studies using low numbers of healthy participants may also be
appropriate, to more easily enable between-laboratory compar-
isons of specific techniques or biomechanical models. Further
refinement and adoption of a ‘standard test protocol’ using
methods such as those outlined by Schwartz et al. [7] may be
useful. Such a protocol could specify an agreed number of trials and
sessions, incorporate methods to minimise assessor bias, and
adopt a specified time interval such as 1 week. This may provide a
useful and more feasible approach to investigating model or
technique-specific questions, prior to definitive studies in clinical
populations when necessary.
This review concludes that although most errors in gait analysis
are probably acceptable, they are generally not small enough to be
ignored during clinical data interpretation. A goal of any clinical
measurement technology must be to provide measurements that
are free from any measurement error that might affect inter-
pretation. There is thus still a need for modifying measurement
techniques to reduce levels of error. Many current techniques rely
heavily on the skill of assessors in accurately placing markers, and
inaccurate marker placement is almost certainly the principal
source of error. New techniques are now emerging based on
functional calibration techniques which are, in principle, less
dependent on the accuracy of marker placement (for example, see
[66,67]). It is hoped that these may further reduce measurement
error in clinical gait analysis. The definition of what measurement
error is acceptable is, of course, dependent on the particular clinical
application.
This review provides evidence that clinically acceptable errors
are possible in gait analysis. Variability between studies, however,
suggests that they are not always achieved and that particular care
is required to achieve acceptable results.
Acknowledgements
This project was funded by a National Health and Medical
Research Council Grant (ID 264597) to the Centre for Clinical
Research Excellence in Gait Analysis and Gait Rehabilitation,
Murdoch Childrens Research Institute, Melbourne, Australia.

A number of limitations should be considered when interpret-

ing the findings of this review. All papers were retained for
inclusion regardless of study quality, in order to provide a
comprehensive overview of available data. Statistical synthesis
of the data was not performed. The findings of this review are
limited to the published papers identified by the search strategies.
Potential publication bias was not assessed and may have resulted
in an over-estimation of reliability. Study quality was only
reviewed by the criterion tool developed for the study purpose.
Future studies of the reliability of 3DGA require careful
consideration of optimal design to enhance the generalisability
of the findings. If the intention is to apply the reliability estimates
to clinical populations, then careful attention is necessary to
recruit and describe samples which are representative of the
clinical populations of interest. Assessor recruitment and char-
acterization warrants comparable attention. Protocols should
carefully consider what standardised measurement interval is
most appropriate and minimise predictable sources of assessor
bias. Appropriate statistical strategies should include reliability
estimates in units of degrees to enhance interpretation. Future
studies should also consider evaluation of the reliability of kinetics
and consider study designs that allow evaluation of the respon-
siveness of 3DGA. Table 4 proposes a list of factors that should be
considered when designing or reporting a study of the reliability of
3DGA.
Conflict of interest
Author RB has received research support funding from VICON.
The other authors state there were no conflicts of interest.
References
[1] Gorton G, Hebert D, Goode B. Assessment of the kinematic variability between
12 Shriners motion analysis laboratories. Gait & Posture 2001;13:247.
[2] Noonan K, Halliday S, Browne R, O’Brien S, Kayes K, Feinberg J. Inter-observer
variability of gait analysis in patients with cerebral palsy. Journal of Pediatric
Orthopaedics 2003;23:279–87.
[3] Wright JG. Pro: interobserver variability of gait analysis. Journal of Pediatric
Orthopaedics 2003;23:288–9.
[4] Gage JR. Con: interobserver variability of gait analysis. Journal of Pediatric
Orthopaedics 2003;23:290–1.
[5] Rothstein JM, Echternach JL. Primer on measurement: an introductory guide to
measurement issues. Alexandria, VA: American Physical Therapy Association
(APTA); 1993.
[6] Baker R. Gait analysis methods in rehabilitation. Journal of NeuroEngineering
and Rehabilitation 2006;3:4.
[7] Schwartz MH, Trost JP, Wervey RA. Measurement and management of errors in
quantitative gait data. Gait & Posture 2004;20:196–203.
[8] Kallen M. Understanding reliability when using measurement instruments in
the VA population. METRIC Newsletter (Measurement Excellence and Training
Resources Information Center); 2005 [Fall].
[9] Monaghan K, Delahunt E, Caulfield B. Increasing the number of gait trial
recordings maximises intra-rater reliability of the CODA motion analysis
system. Gait & Posture 2007;25:303–15.
 J.L. McGinley et al. / Gait & Posture 29 (2009) 360–369
[10] National Health and Medical Research Council. How to review the evidence:
systematic identification and review of the scientific literature. Canberra,
Australia: Biotext; 1999.
[11] Mulrow C, Cook DJ, Davidoff F. Systematic reviews: critical links to the great
chain of evidence. Annals of Internal Medicine 1997;126:389–91.
[12] Hestbaek L, LeBoeuf-Yde C. Are chiropractic tests for the lumbo-pelvic spine
reliable and valid? A systematic critical literature review. Journal of Manip-
ulative and Physiological Therapeutics 2000;23:258–75.
[13] Jordan K. Assessment of published reliability studies for cervical spine range-
of-motion measurement tools. Journal of Manipulative and Physiological
Therapeutics 2000;23:180–95.
[14] van der Wurff P, Hagmeijer RHM, Meyne W. Clinical tests of the sacroiliac
joint. A systematic review. Part 1: reliability. Manual Therapy 2000;5:
30–6.
[15] Dobson F, Morris ME, Baker R, Graham HK. Gait classification in children with
cerebral palsy: a systematic review. Gait & Posture 2007;25:140–52.
[16] Terwee C, Bot S, de Boer M, van der Windt D, Knol D, Dekker J, et al. Quality
criteria were proposed for measurement properties of health status ques-
tionnaires. Journal of Clinical Epidemiology 2007;60:34–42.
[17] Whiting P, Rutjes A, Dinnes J, Reitsma J, Bossuyt P, Kleijnen J. Development and
validation of methods for assessing the quality of diagnostic accuracy studies.
Health Technology Assessment 2004;8.
[18] Higgins J, Green S, editors. Cochrane handbook for systematic reviews of
interventions 426 [updated September 2006] The Cochrane Library, vol. issue
4. Chichester, UK: John Wiley & Sons, Ltd.; 2006.
[19] Gorton G, Stevens C, Masso P, Vannah W. Repeatability of the walking patterns
of normal children. Gait & Posture 1997;5:155.
[20] Miller F, Castagno P, Richards J, Lennon N, Quigley E, Njiler T. Reliability of
kinematics during clinical gait analysis: a comparison between normal and
children with cerebral palsy. Gait & Posture 1996;4:169–70.
[21] Mackey AH, Walt SE, Lobb GA, Stott NS. Reliability of upper and lower limb
three-dimensional kinematics in children with hemiplegia. Gait & Posture
2005;22:1–9.
[22] Quigley E, Miller F, Castagno P, Richards J, Lennon N. Variability of gait
measurements for typically developing children and children with cerebral
palsy. Gait & Posture 1999;10.
[23] Steinwender G, Saraph V, Scheiber S, Zwick EB, Uitz C, Hackl K. Intrasubject
repeatability of gait analysis data in normal and spastic children. Clinical
Biomechanics 2000;15:134–9.
[24] Yavuzer G, Oken O, Elhan A, Stam HJ. Repeatability of lower limb three-
dimensional kinematics in patients with stroke. Gait & Posture 2008;27:
31–5.
[25] Gok H, Ergin S, Yavuzer G. Reliability of gait measurement in normal subjects.
Journal Rheumatic Medicine and Rehabilitation 2002;13:76–80.
[26] Palisano R, Rosenbaum P, Walter S, Russel LD, Wood E, Galuppi B. Develop-
ment and reliability of a system to classify gross motor function in children
with cerebral palsy. Developmental Medicine and Child Neurology 1997;39:
214–23.
[27] Charlton IW, Tate P, Smyth P, Roren L. Repeatability of an optimised lower
body model. Gait & Posture 2004;20:213–21.
[28] Kadaba MP, Ramakrishnan HK, Wootten ME, Gainey J, Gorton G, Cochran GV.
Repeatability of kinematic, kinetic, and electromyographic data in normal
adult gait. Journal of Orthopaedic Research 1989;7:849–60.
[29] Leardini A, Sawacha Z, Paolini G, Ingrosso S, Nativo R, Benedetti MG. A new
anatomically based protocol for gait analysis in children. Gait & Posture 2007
Oct;26:560–71.
[30] Tsushima H, Morris ME, McGinley J. Test-retest reliability and inter-tester
reliability of kinematic data from a three-dimensional gait analysis system.
Journal of the Japanese Physical Therapy Association 2003;6:9–17.
[31] Besier TF, Sturnieks DL, Alderson JA, Lloyd DG. Repeatability of gait data using
a functional hip joint centre and a mean helical knee axis. Journal of Biome-
chanics 2003;36:1159–68.
[32] Ferber R, McClay Davis I, Williams D, Laughton C. A comparison of within- and
between-day reliability of discrete 3D lower extremity variables in runners.
Journal of Orthopaedic Research 2002;20:1139–45.
[33] Gorton G, Hebert D, Goode B. Assessment of the kinematic variability between
twelve Shriners motion analysis laboratories Part 2: short-term follow up. Gait
& Posture 2002;16:S65–66.
[34] Growney E, Meglan D, Johnson M, Cahalan T, An K-N. Repeated measures of
adult normal walking using a video tracking system. Gait & Posture 1997;6:
147–62.
[35] Shrout PE, Fleiss JL. Intra-class correlations: uses in assessing rater reliability.
Psychology Bulletin 1979;86:420–8.
[36] Maynard V, Bakheit AMO, Oldham J, Freeman J. Intra-rater and inter-rater
reliability of gait measurements with CODA mpx30 motion analysis system.
Gait & Posture 2003;17:59–67.
[37] Cowman J, Jenkinson A, O’Connell P, O’Brien T. A model for establishing
reliability and quantifying error associated with routine gait analysis. Gait
& Posture 1998;8:79.
369
[38] Murphy A, McGinley J, Tirosh O. Reliability of kinematic gait measurements in
adult hemiplegic stroke. In: Proceedings of the 12th annual gait and clinical
movement analysis society; 2007.
[39] Deville WL, Buntinx F, Bouter LM, Montori VM, de Vet HCW, van der Windt
DAWM, et al. Conducting systematic reviews of diagnostic studies: didactic
guidelines. BMC Medical Research Methodology 2002.
[40] Eve L, McNee A, Shortland A. Extrinsic and intrinsic variation in kinematic data
from the gait of healthy adult subjects. Gait & Posture 2006;24:S56–7.
[41] Schwartz MH, Viehweger E, Stout J, Novacheck TF, Gage JR. Comprehensive
treatment of ambulatory children with cerebral palsy: an outcome assess-
ment. Journal of Pediatric Orthopedics 2004;24:45–53.
[42] Fosang AL, Galea MP, McCoy AT, Reddihough DS, Story I. Measures of muscle
and joint performance in the lower limb of children with cerebral palsy.
Developmental Medicine & Child Neurology 2003;45:664–70.
[43] McDowell B, Hewitt V, Nurse A, Weston T, Baker R. The variability of gonio-
metric measurements in ambulatory children with spastic cerebral palsy. Gait
& Posture 2000;12:114–21.
[44] Lijmer JG, Willem B, Heisterkamp S, Bonsel GJ, Prins MH, van der Meulen JHP,
et al. Empirical evidence of design-related bias in studies of diagnostic tests.
Journal of the American Medical Association 1999;282:1061–6.
[45] Fritz J, Wainner R. Examining diagnostic tests: an evidence-based perspective.
Physical Therapy 2001;81:1546–64.
[46] Gage JR, Koop SE. Clinical gait analysis: application to management of cerebral
palsy. In: Allard P, Stokes IAF, Blanchi J-P, editors. Three-dimensional analysis
of human movement. Champaign, IL: Human Kinetics; 1995. p. 349–62.
[47] Patrick J. Case for gait analysis as part of management of incomplete spinal
cord injury. Spinal Cord 2003;41:497–582.
[48] Smith PA, Hassani S, Reiners K, Vogel LC, Harris GF. Gait analysis in children
and adolescents with spinal cord injuries. Journal of Spinal Cord Medicine
2004;27:S44–9.
[49] Perry J. The use of gait analysis for surgical recommendations in traumatic
brain injury. Journal of Head Trauma Rehabilitation 1999;14:116–35.
[50] Morris M, Iansek R, McGinley J, Matyas T, Huxham F. 3-Dimensional gait
biomechanics in Parkinson’s disease: evidence for a centrally mediated ampli-
tude regulation disorder. Movement Disorders 2005;20:40–50.
[51] Gutierrez E, Bartonek A, Haglund-Akerling Y, Saraste H. Kinetics of compen-
satory gait in persons with myelomeningocele. Gait & Posture 2005;21:12–23.
[52] Gage JR, DeLuca PA, Renshaw TS. Gait analysis: principle and applications with
emphasis on its use in cerebral palsy. Instructional Course Lectures
1996;45:491–507.
[53] Stolze H, Kuhtz-Buschbeck J, Mondwurf C, Johnk K, Friege L. Retest reliability
of spatiotemporal gait parameters in children and adults. Gait & Posture 1998;
7:125–30.
[54] Bruton A, Conway JH, Holgate ST. Reliability: what is it, and how is it
measured? Physiotherapy 2000;86:94–9.
[55] Portney LG, Watkins MP. Foundations of clinical research. Applications to
practice. New Jersey: Prentice Hall Health; 2000.
[56] de Vet HCW, Terwee CB, Bouter LM. Current challenges in clinimetrics. Journal
of Clinical Epidemiology 2003;56:1137–41.
[57] Davis R, Davids J, Gorton G, Aiona M, Scarborough N, Oeffinger D, Tylkowski
CAB. A minimum standardized gait analysis protocol: development and
implementation by the Shriners Motion Analysis Laboratory Network (SMAL-
net). In: Harris GF, Smith PA (Eds.). Pediatric gait: a new millennium in clinical
care and motion analysis technology. IEEE; 2000.
[58] Bell KJ, Ounpuu S, DeLuca PA, Romness MJ. Natural progression of gait in children
with cerebral palsy. Journal of Pediatric Orthopaedics 2002;22:677–82.
[59] Gough M, Eve LC, Robinson RO, Shortland AP. Short-term outcome of multi-
level surgical intervention in spastic diplegic cerebral palsy compared with the
natural history. Developmental Medicine & Child Neurology 2004;46:91–7.
[60] Watkins M, Riddle D, Lamb R, Personius W. Reliability of goniometric mea-
surements and visual estimates of knee range of motion obtained in a clinical
setting. Physical Therapy 1991;71:90–6.
[61] van der Linden ML, Kerr AM, Hazlewood ME, Hillman SJM, Robb JE. Kinematic
and kinetic gait characteristics of normal children walking at a range of
clinically relevant speeds. Journal of Pediatric Orthopaedics 2002;22:800–6.
[62] Diss CE. The reliability of kinetic and kinematic variables used to analyse
normal running gait. Gait & Posture 2001;14:98–103.
[63] Luiz RR, Szklo M. More than one statistical strategy to assess agreement of
quantitative measurements may usefully be reported (commentary). Journal
of Clinical Epidemiology 2005;58:215–6.
[64] Keating J, Matyas T. Unreliable inferences from reliable measurements. Aus-
tralian Journal of Physiotherapy 1998;44:5–10.
[65] Haley SM, Fragala-Pinkham MA. Interpreting change scores of tests and
measures used in physical therapy. Physical Therapy 2006;86:735–43.
[66] Schwartz MH, Rozumalski A. A new method for estimating joint parameters
from motion data. Journal of Biomechanics 2005;38:107–16.
[67] Reinbolt JA, Schutte JF, Fregly BJ, Koh BI, Haftka R, George A, Mitchell K.
Determination of patient-specific multi-joint kinematic models through two-
level optimization. Journal of Biomechanics 2005;38:621–6.