British Journal of Orthodontics

ISSN: 0301-228X (Print) (Online) Journal homepage: http://www.tandfonline.com/loi/yjor19

Angle's Classification of Malocclusion: an

Assessment of Reliability

J. F. Gravely M.Ch.D., D.Orth. R.C.S., L.D.S. & D. B. Johnson B.D.S., F.D.S.,

D.Orth. R.C.S.

To cite this article: J. F. Gravely M.Ch.D., D.Orth. R.C.S., L.D.S. & D. B. Johnson B.D.S., F.D.S.,
D.Orth. R.C.S. (1974) Angle's Classification of Malocclusion: an Assessment of Reliability, British
Journal of Orthodontics, 1:3, 79-86, DOI: 10.1179/bjo.1.3.79

To link to this article: http://dx.doi.org/10.1179/bjo.1.3.79

Published online: 21 Jun 2016.

Submit your article to this journal

Article views: 1

View related articles

Citing articles: 18 View citing articles

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=yjor19

Download by: [Tufts University] Date: 19 October 2016, At: 12:56

British Journal of Orthodontics/Yo! 1/No 3/79-86 Printed in Great Britain
Angle's Classification of
Malocclusion: an Assessment
of Reliability
Abstract
The reliability of Angle's system of classification of malocclu-
sion has been examined. Between-examiner errors were found
to be high and there were also high within-examiner error levels
in categorizing Angle Class II division 2 malocclusions. It is
concluded that comparisons of the distribution of malocclu-
sions in different communities, classified according to Angle's
system, should not be made unless observations are made in
each community by the same examiner. The usefulness of
Angle's classification to both clinician and epidemiologist is
questioned.
Introduction
Several methods of classifying malocclusions have
been described: the one which has gained most
widespread use is that of Angle (1898). Angle
defined three classes of malocclusion based on the
antero-posterior relationship of the upper and
lower buccal segments (Fig. 1): Class I in which
there is a normal antero-posterior relationship;
Class II in which the mandibular buccal segments
are distal to those of the maxilla and Class III in
which they are mesial. Class II malocclusions are
further divided into two categories: division 1 in
which there is an excessive overjet; and division
J. F. Gravely, M.Ch.D., D.Orth. R.C.S., L.D.S.
University of Leeds, The Dental School, Blundell
Street, Leeds LS1 3EU
D. B. Johnson, B.D.S., F.D.S., D.Orth. R.C.S.
Orthodontic Department, St. James's Hospital,
Beckett Street, Leeds 9
2 in which the upper central incisors are retro-
clined, the overbite is greater than normal and the
overjet normal.
In addition to its clinical use, Angle's classifica-
tion has frequently been used in assessing the
prevalence of malocclusions in communities.
Epidemiological studies published prior to 1956
were reviewed by Brash eta!. (1956): more recent
studies include those of Goose eta!. (1957), Walther
(1960), Miller and Hobson (1961), Ast eta!. (1962;
1965), Heifer and Lovius (1963), Moss and Picton
(1968) and Murray (1968).
Considerable variations in the prevalence of
malocclusions have been reported in different com-
munities. For example, the studies quoted by Brash
eta!. (1956) give the prevalence of normal occlusion
as ranging from 8·6 per cent to 77·6 per cent; for
Angle Class I from 26·2 per cent to 65·9 per cent;
for Class II division I from 4·8 per cent to 21 per
cent; for Class II division 2 from 2·6 per cent to
13·3 per cent and for Class III from 0·9 per cent to
12·2 per cent. The authors pointed out, however,
79
J. F. Gravely and D. B. Johnson
that such contrasting results probably reflect differ-
ences in diagnostic standards rather than differences
between communities. That wide differences in
diagnostic standards can occur has been demon-
strated by Ast et a/. (1962) who reported that two
orthodontists disagreed on the classification of
30 per cent of 302 sets of study models indepen-
dently classified according to Angle. The authors
did not report the nature of these differences but
stated that they were readily resolved by discussion.
On the other hand Murray (1968), a dental epide-
miologist, was able to classify the occlusion of 50
children on two separate occasions without dis-
agreement. The paucity of investigations into the
reliability of Angle's classification is surprising in
view of its widespread use both by clinician and
epidemiologist.
Method and Material
The material consisted of 102 sets of pre-treatment
study models and 80 school children. The models
were those of patients who had received treatment
at the Leeds Dental School. All were of the mixed
or permanent dentitions: the cases were unselected
except for the exclusion of damaged models. The
children, aged eleven to fifteen years, were selected
at a routine school dental inspection as being in
need of orthodontic treatment. The selection was
carried out by the Principal School Dental Officer
who did not participate in the subsequent examina-
tions.
Classification of malocclusions from study models
was undertaken by three experienced orthodontists
who are referred to as Examiners A, B and C. Each
assessed the malocclusions independently: after a
fortnight had elapsed (to eliminate memory bias)
a fresh assessment was made. Two of the examiners
(Examiners A and B) then proceeded to a clinical
examination of children. Again assessments were
made independently and were repeated two weeks
later. Of the 80 children examined only 73 were
available for the second examination.
To overcome difficulties which would arise if an
occlusion could not be classified, either because the
examiner considered it normal or because he could
not fit the malocclusion into any of the categories
of Angle's Classification, three additional groups
were added: "Angle Class II division uncertain",
"Malocclusion unclassifiable" and "Normal
occlusion".
Up to this stage of the investigation no discus-
sions took place on diagnostic criteria. Differences
in diagnostic standards became apparent when the
data were analysed. Discussions between examiners
80
followed and agreement was reached on the criteria
to be used, after which the three examiners again
classified the study models applying standardized
criteria.
Results
1. Within-Examiner Constituency
(i) Study of Models (Tables Ia, b, c and II)
Of the 102 sets of study models examined, the
malocclusions were classified consistently by
Examiner A on 89 occasions, by Examiner B on 90
occasions and by Examiner C on 8 I occasions.
Inconsistency was not confined to any class but
occurred in each group. The level of consistency
of Examiners A and B is almost identical. Examiner
C used the "Class II division uncertain" more than
did either of the other examiners and this probably
accounts for the lower number of Class II division
1 cases recorded. The "Class II division uncer-
tain" is also the category classified least consistently
by Examiner C and this probably accounts for his
lower over-all level of reliability.
In order to make comparisons between consistency
of examiners or between reliability of the different
categories, values referred to as the "reliability
ratio" and "level of agreement per cent" will be used
hereafter. The reliability ratio is calculated by
dividing the number of cases consistently placed in a
particular category by the total number placed in
that category. For example, in diagnosing Class I
malocclusion Examiner A recorded 24 cases on his
first examination and 25 on the second (Table Ia).
Of these only 21 were consistently classified on both
occasions. Three cases, originally classified as Class
I were allocated to different categories on the
second occasion, while four cases classified as
Class I in the second assessment had previously
been placed in other groups. Thus the reliability
ratio in this example is:
21
21 +4+3=21 :28,
and the level of agreement is 75 per cent.
The reliability ratio and the level of agreement
percent for the classification of malocclusions made
by our three examiners from study models is given
in Table II. Where the level of agreement per cent
has been calculated from a ratio in which there is a
low denominator the values are unreliable estimates
of observer consistency: where the denominator is
less than ten the values are given in brackets
(e.g. the "unclassifiable" group in Table II).
Angle Classes III and II division 1 were the most
reliable categories with agreement per cent values
Angle's Classification of Malocclusion

TABLE f(a)
Within-Examiner Consistency in Classifying 102 Malocclusions From Study Models
Examiner A
Second Assessment

Cl>
:0
c: "'
c;::
"iii
"iii
c:
0
t::
Cl>
(.)
0"'"'c:
c: ::>
"iii
::> .... c-1 ::> c:
0(.) .:: >
:c .:=
-c
.!:!
-c
- - --- "'::>
--
0
itS - 0(.) .J
<{
§ "'"' "' "'"' "'"' "'"' 0 1-
0
z u"' u"'"' u"' u"' u"' itS
::2:
0
1-
-- - - - - - - - -- - - -
Normal occlusion 0 0
.....
c: Class I 21 2 1 24
Cl>
Class II div. 1 42 2 44
E
"'"'Cl> Class II div. 2 2 2 8 12
Class II div. uncertain 2 0 2
"'
<"'
.....
Class Ill 1 18
0
19
1
Malocclusion unclassifiable 1
~
u: TOTAL 0 25 46 13 0 18 0 102

TABLE f(b)
Within-Examiner Consistency In Classifying 102 Malocclusions From Study Models
Examiner B
Second Assessment

Cl>
:0
c: "'
c;::
"iii
"iii
c:
t::
Cl>
"'
0"'
0 (.) c:
"iii c: ::>
::> ,... c-1 ::> c:
> :c> ~
0(.) 0
"iii
:c -c
-=
0
"iii - -- -- - ::>
0(.) .J
<{
§
0
"'"'
..!!!
"'"'
..!!!
"'"'
..!!! "'"'
..!!!
"'
"'
..!!!
0
"iii
1-
0
z u u u u u ::2: 1-
-- -- -- - - - - - - 1 - --
Normal occlusion 0 2 2
.....c: Class I 27 2 1 1 31
Ql
Class II dlv. 1 2 46 2 50
E
Class II dlv. 2 1 8 9
"'"'Ql Class II div. uncertain 0 0
<"'"' Class Ill 1 9 10
..... Malocclusion unclassifiable 0 0
....
u:"' TOTAL 0 33 48 11 0 9 1 102

81
J. F. Gravely and D. B. Johnson

TABLE /(c)
Within-Examiner Consistency in Classifying 102 Malocclusions From Study Models
Examiner C
Second Assessment

Gl
:0
Ill
c;:::
c 'iii
'iii
c
0
t:
Gl
0
0"'"'c
'iii
:l ...:> 1:11
c
:l
:>
c
:l

0
0
:> 0
'iii
'C 'C 'C
0
'iii - -- = --"' -- 0
:l

0
...J
<(
"'"'
Ul

0
E
~
Ul
Ul
"' ~ 'iii ~"' ~
"'"'
~
0 1-
0
z u u u u u ::!: 1-
- - - - - -1 - - - -- - - - -
-
c
Gl
E
Ul
Normal occlusion
Class I
Class II div. 1
Class II div. 2
0
1
1
23
1 28
1 8
2
1
4
3
1
1
29
31
13
Ul
Gl
Class II div. uncertain 3 1 1 15 20
"'
1/)
<( Class Ill 1 6 7
...
iii
i.i:
Malocclusion unclassifiable
TOTAL 30 9 22 6
1
5
1
102
1 29

ranging from 86-95 per cent for Class III and 85-88
per cent for Class II division 1. The levels of
agreement for Class I were lower, ranging from
66-75 per cent; that of Class II division 2 lower
still with a range 47-67 per cent. The numbers in
the remaining groups were too small for conclusions
to be drawn except for Class II division uncertain
which in the case ofExaminerCwasusedconsistently
on 15 out of 27 occasions: thus, the level of agree-
ment was 56 per cent, which is almost identical to
the level of reliability of Class II division 2 classifi-
cation by the same examiner.
(ii) Study of Children (Table liD
The clinical examination was carried out by
Examiners A and B only: fewer children than
models were examined. The categories Class I
and Class II division I have a reliability level similar
to that found in the model study: the level of
agreement for Class I ranged from 75-78 per cent;
for Class II division 1 from 76-90 per cent. Examiner
A classified Class II division 2 and Class III
malocclusions with a similar level of agreement as
he did in the model study: Class III was classified
most consistently with a reliability ratio of 83 per

TABLE II
Within-Examiner Consistency in Classifying 102 Malocclusions From Study Models

Examiner A Examiner 8 Examiner C

Reliability Agreement Reliability Agreement Reliability Agreement

ratio % ratio % ratio %
Normal 0:2 (0) 0:2 (O)
Class I 21 :28 75 27:37 73 23:35 66
Class II division 1 42:48 88 46:52 88 28:33 85
Class II division 2 8:17 47 8: 12 67 8: 14 57
Class II division uncertain 0:2 (0) 15 : 27 56
Class Ill 18: 19 95 9: 10 90 6:7 (86)
Malocclusion unclassifiable 0: 1 (D) 0i1 (0) 1 :5 (20)

82
Angle's Classification of Malocclusion
cent and Class II division 2 least consistently with a
43 per cent level of agreement. Examiner B was
much more consistent in his classification of Class
II division 2 and much less consistent in classifying
Class III: the agreement levels were 75 per cent and
57 per cent respectively. The numbers in these
groups are, however, too small for valid judgements.
From these observations it is concluded that
classification of malocclusions can be carried out
by clinical examination with a very similar level of
reliability to that of classifying study models. The
group most consistently classified in both studies
was Class II division I.
2. Between-Examiner Consistency (Table IV)
Consistency in classifying malocclusions was
analysed, between Examiners A and B for the
clinical study, and between all three examiners in
the model study.
Study of Children
Between-examiner differences of assessment were
greater than within-examiner differences. For
example, whereas in the diagnosis of Class I
within-examiner level of agreement for Examiner A
was 75 per cent and for Examiner B it was 78 per
cent, the between-examiner level of agreement was
only 57 per cent. Similarly the diagnosis of Class II
division I within-examiner levels of agreement
were 76 per cent and 90 per cent respectively as
against 69 per cent for the between-examiner level
of agreement.
Study of Models
Wide differences between the between- and within-
examiner levels of agreement are equally apparent
when assessments were made from models. For
example, when either Examiners A orB diagnosed
TABLE Ill
Within-Examiner Consistency in Classifying 80 Malocclusions From Examination of Children
Examiner A
Reliability
ratio
Normal
Class I 30:40
Class II division 1 19:25
Class II division 2 3:7
Class II division uncertain
Class Ill 10: 12
Malocclusion unclassiflable
Class I, they agree in only 41 per cent of instances.
The within-examiner levels of agreement were 75
per cent and 73 per cent respectively. They more
frequently agreed in the diagnosis of Class II
division I (77 per cent) but for Class II division 2
and Class III levels of agreement were as low as
40 per cent and 53 per cent respectively. This order
of difference was much the same when analyses
were made between Examiners C and B, and B and
A; and there can be no doubt that the examiners
in the present study were applying different
diagnostic criteria.
3. Between-Examiner Consistency following Standardization
of Diagnostic Criteria (Table V)
At this stage of the study discussions took place
and agreement was reached on the following
diagnostic criteria:
(i) The "Unclassifiable" group was eliminated.
Any malocclusion which was not clearly
classifiable as Class II or III would be classified
as Class I.
(ii) Class II malocclusions were defined as being
one-half a "unit" or more post-normal, a "unit"
being defined as the mesio-distal width of a
premolar or canine tooth.
(iii) The Class II "uncertain" group was eliminated.
If doubt existed on whether a Class II case
was to be classified as division 1 or division
2, it would be classified as Class II division
1.
(iv) A malocclusion would only be categorized as
Class III if all, or all but one, of the maxillary
incisors occluded lingual to the lower incisors
or met their incisal edges in centric occlusion.
By eliminating unnecessary and ill-defined cate-
gories, and by providing a standard procedure for
classifying border-line cases, an improvement in
Examiner B
Agreement Reliability Agreement
% ratio %
75 28:36 78
76 27:30 90
(43) 6:8 {75)
83 4:7 {57)
83
J. F. Gravely and D. B. Johnson

TABLE IV
Between-Examiner Consistency in Classifying Malocclusions From Examination of 73 Children and 102 Sets
of Study Models
Children Study models
Examiners A and B Examiners Examiners C and B Examiners
compared A and C compared compared B and A compared
Reliability Agreement Reliability Agreement Reliability Agreement Reliability Agreement
ratio % ratio % ratio % ratio %
Normal 0: 1 (0) 0:3 (0) 0:2 (0)
Class I 26:46 57 17: 36 47 18:42 43 16: 39 41
Class II division 1 24:35 69 30: 45 67 30:51 59 41 :53 77
Class II division 2 3:8 (38) 8: 17 47 6: 16 38 6: 15 40
Class II division
uncertain 0:22 0 0:20 0 0:2 (0)
Class Ill 6: 12 50 7: 19 37 6: 11 55 10 : 19 53
Malocclusion
unclasslfiable 0: 2 (O) 0: 1 (0) 0: 1 (0)

TABLE V
Between-Examiner Consistency in Classifying 102 Malocclusions From Study Models Using Standardised
Criteria
Examiners Examiners Examiners
A and C compared C and B compared B and A compared
Reliability Agreement Reliability Agreement Reliability Agreement
ratjo % ratio % ratio %
Class I 19:42 45 30:46 65 23:46 50
Class II 27:46 59 28:47 60 33:53 62
Class II 2 12:30 40 7:26 27 6: 17 35
Class Ill 11 : 17 65 9 : 11 82 9: 17 53

the between-examiner level of agreement would be
anticipated. But with the exception of Angle Class
III malocclusion in which the level of agreement
increased there was no clear over-all improvement.
In as many cases there was less agreement after
standardization than there was before. For example,
before standardization agreement levels for Class
II division 2 between A and C, between C and B
and between B and A were 47 per cent, 38 per cent
and 40 per cent respectively: after standardization
the comparable figures were 40 per cent, 27 per cent
and 35 per cent.
Discussion
Angle's classification is often taught to under-
graduate dental students as a precise classification
upon which orthodontic treatment is planned. It is
also used by orthodontists as a convenient method
of description when writing reports on cases referred
by dental practitioners for advice or treatment. The
evidence of this investigation questions the validity
of Angle's system of classification and therefore
its usefulness must also be questioned.
There are several reasons why Angle's system is
unreliable. It is based on a concept of normal
occlusion between maxillary and mandibular
dentitions: marked antero-posterior disharmonies
of occlusion are clearly defined. Difficulty in
classification arises, however, where disharmonies
are only slight especially if there is also an under-
lying asymmetry between left and right sides or
where tooth movements have occurred because of
such factors as crowding and premature loss of
deciduous teeth. Thus there may be differences
between the antero-posterior relationship of left
and right sides or between incisor and buccal
segments (see Fig. 2). In Class II malocclusions the
incisor classification may be division I on one side
and division 2 on the other (see Fig. 3). In such
cases judgements must be made as to the nature of
the underlying malocclusion: it is in making these
judgements that inconsistencies between- and
within-examiners occur.
One might expect that the separate classification
of buccal and incisor occlusion would improve
reliability. In a study similar to the present investi-
gation, Gravely (I 972) compared the reliability of
Angle's Classification of malocclusion with that of
classifying incisor occlusion into Class I, II (lh
II12l and III as described by Ballard and Wayman
(1964). In that study it was found that in some
instances classification of incisor occlusion was more
reliable than Angle's classification, but for the
Class II 12 > group, the least reliable category, the

84
Angle's Classification of Malocclusion
Class I
Class II
Division 1
Division 2
Class Ill
Fig. 1. Angle's classification.
level was almost identical to that of Angle's
system. The chief source of error in both systems is
in the classification of border-line cases, particularly
some Class II(2J cases which are not clearly defined
from Class I, or in a few instances Class II UJ·
So far, the reliability of Angle's Classification has
been considered on an individual basis. When used
in epidemiology the classification is applied to
communities and random errors will tend to cancel
out. How reliable is the classification when used in
this way? If the same examiner examined two
identical communities (in our study the same
community has been examined twice), how con-
sistent would be his classification of the malocclu-
sions? What would be the effect of employing a
different observer for each community? One
method of comparing samples from two com-
munities is to apply a Chi-square test: if differences
between the samples are small they may be attri-
butable to sample variation and the value of Chi-
square would be low: the greater the value of
Chi-square the greater the probability of a true
difference between the communities. Thus the Chi-
square test may be used as a measure of the reliabi-
lity of an assessment such as Angle's classification
by applying the test to two separate assessments
made of the same sample. If observer error is low,
the value of Chi-square would be low: if high,
the value of Chi-square would be correspondingly
high. The data for the present study are given in
Table VI.
Fig. 2. Models in which there is an Angle Class Ill
buccal relationship but no reverse overjet.
Fig. 3. Models classified inconsistently as Class II
division 1 and division 2.
On the first occasion Examiner A recorded
prevalence values of 37, 24 and 12 for Angle Class
I, II and III respectively: on the second occasion
the corresponding values were 33, 30 and I 0.
When these data were tested for statistical signifi-
cance using the Chi-square analysis the differences
were not significant at even a low level of probability.
Clearly the errors were random and their effect
had been neutralized. Thus community differences
with respect to these parameters could be assessed
by Examiner A, provided he made the assessments
in each community. The same conclusions apply
to Examiner B. The value of Chi-square is much
greater, however, when the classification by
Examiner A is compared with that of Examiner B.
This indicates that the comparison of the communi-
ties with respect to these parameters will not
provide reliable data if communities are examined
by different orthodontists.
85
J. F. Gravely, D. B. Johnson

TABLE VI
Chl 2 Analysis to Test the Reliability of Angle's Classification In an Epidemiological
Investigation
Number of children in each class Probability
Examiner Examination Class I Class II Class Ill Chi 2 at D.F.=2
A 1st 37 24 12 1·08 c60%
2nd 33 30 10

B 1st 31 36 6 0·11 c90%

2nd 33 35 5

A 39 29 12 4·56 c 10%
B 33 41 6

Conclusions Ast, D. B., Carlos, J.P. and Cons, N.C. (1965)

The findings of the present investigation show the
classification of malocclusions according to Angle's
system cannot be carried out with a high level of
reliability. The value of Angle's classification to the
clinician and the emphasis laid on it by teachers of
orthodontics must, therefore, be questioned. In
group studies, observer errors are randomly distri-
buted and tend to cancel each other out. Angle's
classification is, therefore, a reliable method of
classifying malocclusion in community studies,
provided that observations in different communities
are made by the same observer. If, however, the
classification is of doubtful diagnostic value to the
clinician, its usefulness to the epidemiologist must
also be questioned.
Th~ pre':'alence and characteristics of malocclusion among
semor hrgh school students in upstate New York,
American Journal of Ortlrodontics,
51, 437.
Ballard, C. F. and Wayman, J. B. {1964)
A report on a survey of the orthodontic requirements of 310
army apprentices,
Transactions of the British Society for the Study of
Orthodontics,
81.
Brash, J. C., McKeag, H. T. A. and Scott, J. H. (1956)
The aetiology of irregularity and malocclusion of the teeth
2nd edition: the Dentai:Board of the United Kingdom, '
London.
Goose, D. H., Thompson, D. G. and Winter, F. C. (1957)
Malocclusion in school children of the West Midlands
British Dental Journal, '
102., 174.

Gravely, J. F. (1972.)
Orthodontic diagnosis: an analysis of reliability,
Acknowledgements M.Ch.D. Thesis, University of Leeds.
The authors wish to thank Professor D. Jackson, Hefl'er, P. T. and Lovlus, B. B. (1963)
University of Leeds Dental School, for encouragement and An orthodontic survey of the islanders of Tristan da Cunha
supervision; Mr P. M. Benzies, Consultant Orthodontist, Dental Practitioner,
Leeds Dental Hospital, for participating in the study; Mr 14, 129.
G. Turner, Principal School Dental Officer, City of York,
Miller, J. and Hobson, P. (1961)
the Headmaster, staff and pupils of Manor School, York The relationship between malocclusion, oral cleanliness,
for their co-operation; Messrs. L. Jepson and D. Chapman, gingival conditions and dental caries in school children,
Photographic Department, Leeds Dental Hospital, for the British Dental Journal,
photographs; and Miss D. Limon, Mrs E. Johnson and 111, 43.
Mrs J. Taylor for clerical assistance.
Moss, J. P, and Picton, D. c. A. (1968)
The problems of dental development among the children on a
Greek island,
References Dental Practitioner,
18, 442.
Angle, E. (1898)
Treatment of malocclusion of the teeth and fractures of the Murray, J. (1968)
maxillae, The dental condition of school children in fluoride and
The S.S. White Dental Manufacturing Company, Philadelphia. non-fluoride areas,
Ast, D. B., Allaway, N. and Draker, H. L. (1962.) M.Ch.D. Thesis, University of Leeds.
The prevalence of malocclusion, related to dental caries and
lost first permanent molars, in a tluoridatcd city and a Walther, D. P. (1960)
fluoride-deficient city, Some of the causes and effects of malocclusion,
American Journal of Orthodontics, Dental Practitioner,
48, 106. 10, 139.

86