Assessment of Singing
Katherine Eberle Fink
S1IPITMBFhR/OCTOBER 2006 35
KatherineEberle Fink
Kreirnan, Vanlancker-Sidtis, and Gerratt suggest that in Teaching Kids to Sing' offers his own version of an
when assessing voices, one must not only judge the per- evaluation form that includes the following criteria: pos-
formance but also "assess the interactions between lis- ture, breathing, lower and upper adjustment and coor-
teners and signals, rather than treating quality solely as dination, resonance, vowel uniformity, vocal coordina-
a function of the voice signals themselves." They go on tion, open throat, pronunciation, consonant production,
to say that "[o] nce the relationship between a signal and phrasing, dynamic/tempo variation, and agility/
a percept is understood, it may be possible to determine range. He uses numerical rankings of 1 = minimal or no
which physiological parameters create perceptually achievement; 2 below average; 3 = average; 4 = above
meaningful changes in phonation."3 Until researchers average; and 5 superior achievement. He goes on to
such as Kreiman et al. identify the best method for per- list specific criteria for each of these numerical assign-
ceptual acoustic measures, singing teachers will have to ments regarding respiration, resonant tone production,
depend on unanimity in the profession to establish a re- diction, and expression.
liable assessment method. James McKinney's evaluation form lists criteria includ-
Perhaps professional singing teachers should develop ing posture, breathing, facial expression, phonation, sup-
evaluation methods based on standards used in speech port, quality, quantity (volume), vibrato, intonation, artic-
pathology. Some scholars might argue that singing per- ulation, vowel accuracy, basic musicianship, range, and
formances historically have been effectively evaluated voice classification.6 He includes a footnote that commu-
without the application of science. Though the variety of nication, interpretation, phrasing, sensitivity to the text,
literature defining the many individual approaches to platform technique, agility, dynamic control, flexibility,
teaching voice is serious and scholarly, when it comes and tone color are to be added as desired. Rather than a
to perceptual assessments most professional singing numerical analysis, he uses abbreviations such as G =
teachers use an informal method of evaluation not far good; VG = very good; NB = not bad; F fair; Br = breathy;
removed from the judging found in American Idol. Eval- T = tight; N = nasal; W = weak; and L lacking.
uators decide if they like the singer or not, based on per- Richard Davis's evaluation form includes respira-
sonal, subjective observations. A critical comment such tion, phonation, resonation, articulation, interpreta-
as "That singer's voice is pitchy" probably makes com- tion, and presentation as his categories for a jury ex-
plete sense to the judge. However, the term "pitchy" amination. He then offers subcriteria for each of these
leaves some listeners wondering whether (and why) this categories; for example, under respiration he includes
is to be considered a negative. Would a more scientific "breath inadequate at:' and then he allows room for
approach lend more credibility to singing adjudication? written comments.
The Consensus Auditory Perceptual Evaluation of Voice The test format typically used in competitions, ac-
(CAPENV) form instituted by the American Speech and cording to J. David Boyle and Rudolf E. Radocy, is the
Hearing Association is an example of a form that is be- norm-referenced test (where a relative standard is em-
ing implemented throughout this country in the evalua- ployed and judges discriminate among individuals).'
tion of the speaking voice. Over fifty different instruments Boyle and Radocy go on to say,
(test forms) for voice assessment were combined into one
The measurement of musical performance is inherently sub-
standardized form that attempts to "identify a minimal
jective. While many listeners may agree ... regarding "right"
set of meaningful perceptual voice parameters with ex- notes, decisions involving correct tempi, phrasing nuances, ex-
pedient result capabilities which can be applicable to all ecution of ornamentation and tone quality are the decisions of
clients while demonstrating optimal reliability."4 If singing individuals functioning as subjects, hence the decisions are sub-
teachers could agree on terminology, agree on a similar for- jective. While one may structure a performance measurement
mat of evaluation, and use anchors or models of recorded process to enhance objectivity and minimize nonmusical as-
performances to establish category boundaries, they could pects, it never can be as objective as a multiple-choice test. 9
establish a starting point for standardization. However, published measurements exist for wind instru-
To develop a standardized singing profile, the author ments and snare drum and for orchestral strings;" thus,
looked for other educators' approaches. Kenneth Phillips the classical singing voice deserves similar consideration.
36 JOURNAL OF SINGING
PerceptualAcoustic Assessment of Singing]
What should be measured when scoring competi- the scoring is for judging purposes alone. Singers should re-
ceive only written comments on their adjudication sheets. A
tions? Detailed evaluative criteria like those advocated
plan that has proven very satisfactory and practical is to have
above by Phillips, McKinney, and Davis can help judges
adjudicators rate the performer with a score between 70 and
rank performances. Boyle and Radocy have derived from 100. No student should receive a grade below 70. Their sug-
the study of psychology several techniques for the meas- gested scoring is: 90- 100 excellent, 85-89 very good, 80-84
urement of musical performance: Likert scales, seman- good, and 70-79 average. An adjudicator is not required to
tic differential scales, paired comparisons, and succes- give a superior rating if no student has earned one."'
sive interval comparisons. Likert scales are used to obtain
The guidelines suggest that "evaluative comments and
a total score by adding individual item scores, but they
ideas are of great value to students and teachers and each
do not offer a reliable scoring of opinion.'" Semantic dif-
judge should make as comprehensive a critique as time
ferential scales are a type of measurement that anchors
allows, acknowledging achievement and encouraging
specific antonyms, for example the terms "bright- dark.'
improvement." They go on to say that
Semantic differential, however, may be reliable only if
the judges agree on the terminology and a factor analy- ... comments that would be embarrassing to the teacher or
sis is offered.' 2 Paired comparisons discriminate among discouraging to the student must be avoided; and no one
similar performances when rankings are desired, but should be criticized to the point of humiliation or defeat. The
are not practical when assessing large numbers of per- real competition is between the student and one's own self-
image or self-imposed goals. The preparation for the Auditions
formers."' Successive interval comparison, considered
and the written constructive criticisms are far more valuable
best by Boyle and Radocy when there are too many per-
than any award or honor given to the students."
formances to make paired comparisons practical, in-
volves a cornplicated mathematical process of ranking While this suggested numerical scoring gives a positive
categories."' The author incorporates aspects of some point of reference to the student, it does not allow for
of these scoring methods in the assessment form offered adjudicators to give completely accurate assessments of
at the end of this article. ability, as the bottom score of 70 implies that no possi-
The National Association of Teachers of Singing of- ble performer could rank lower, which is not realistic.
Iers yearly student auditions where students can com- A scale from 0- 100 would more accurately depict vari-
pete with others of their own age group and vocal level. ability between singers than a score of 70- 100. No other
Each region decides its own format for state and regional specific criteria or categories of evaluation are offered
competitions and uses its own members as judges. The as suggestions for regional competitions.
assessment criteria used by NATS regions in their stu- In a 2003-04 survey of NATS governors, the author
dent auditions have undergone no change since the 1993 found that seven respondents use different formats in
(Committee for Audition Guidelines Revision was ap- adjudicating their student auditions at the state and re-
proved and adopted by the NATS Board of Directors. gional levels. Not all use the same numerical system to
The Committee, chaired by Judith Nicosia Civitano, determine the competition winners, but the criteria for
conducted a "survey of material from virtually every assessment do include similar categories and all allow
chapter and region in NATS" and suggested that during written comments under each category. Three cate-
the sixteen years prior to the first "Guidelines for Student gories common to all include: Vocal technique (also
5
Auditions" development, many changes had occurred.' titled by some as Vocalism or Voice Characteristics),
The 1993 adjudication guidelines are offered as practi- Musicianship, and Artistry (or Artistic Sensitivity).
cal suggestions for evaluating and strengthening the Within these categories, up to thirty-three factors are
Student Auditions program at all levels, but do not require considered in some evaluations, and each region uses its
a specific format or criteria. They suggest numerical own specific criteria.
scoring in the following way: The Mid-Atlantic Region uses scoring based on fif-
teen criteria: "Tone quality, enunciation, breath man-
Scoring should be done against a standard for the classifica-
tion being judged, i.e., what students at that age level and agement, accuracy of rhythm, interpretation, diction,
length of study should be able to do. It should be stressed that dynamics, intonation, tempo, understanding and pro-
SETTMBIZMiER/OCTOBER 2006
37
IIKatherineEberle Fink
jection of text, articulation, posture, accuracy of notes, cal technique, voice characteristics, artistry, diction,
phrasing, stage deportment" as appropriate to the level communication, and additional comments."
of the student. Their explanation of scoring is that "25-30 The Central Region uses no numerical scoring and
is considered superior and (only students within this offers space for written comments under each song title:
score range will be eligible to advance to the Regional "Selection: #1, #2, and #3*' No additional specific crite-
Auditions), 20-24 is Excellent, 15-19 is Good, and be- ria are indicated on their form.
low 15 special attention (is) needed.. :' The Southeast Region uses the following guidelines:
The Texoma Region scores are ranked from 70- 100. "1) Circle only one descriptive evaluation (1. Superior,
Sixteen criteria are offered for evaluation. "Musicianship: 2. Excellent, 3. Good, 4. Fair, 5. Singing for Comments
rhythm, tempo, phrasing, accuracy, dynamics, musical only). 2) Do not indicate any ranking or numerical scores
sensitivity; Vocal Technique: flexibility, breath support, of any kind on this sheet. 3) Place your rankings on the
diction, language accuracy, freedom; Voice Character- additional sheet provided. 4) If any judge feels that a
istics: quality, range, evenness of scale; Artistry: pro- singer should be disqualified, all three judges must re-
jection of the mood and spirit of song, tone color" are port to the tally room to resolve the issues before any
to be ranked as either "above average" with a plus (+) rankings are assigned." Twenty additional criteria are
sign, "average" with a check mark (/), or "below aver- listed on the form with room for written comments un-
age" with a minus sign (-). der each category. "Voice Characteristics (Native qual-
The West Central Region competition form offers ity and range of instrument, evenness of scale, vitality;
these guidelines for judges: "1. Do not lower score if lit- Techniques: Flexibility, breath control, diction, vowel
erature is, in your estimation, not appropriate for the purity, language accuracy, intonation, etc.); Musicianship:
voice type. 2. Do not compare students: score against a stan- accuracy of rhythm, phrasing, etc., control of tempo and
dard of the level you are hearing. 3. Direct your state- dynamics, etc.); Artistry: projection of mood, appropri-
ments to the student, not the teacher. 4. Scores for the ateness of tonal colors, stage presence, style."
level of class: 90-100 excellent, 85-89 very good, 80-84 The North Central Region allows room for written com-
average, 70-79 below average. 5. Allow at least 1/4th of ments with possible constructive comments using these
all the singers in one class to proceed to the semi-finals:' categories of criteria: "Vocalism: tone quality, production/
Their seventeen criteria are: "Musicianship: rhythm, placement, intonation, breath management, etc.;
tempo, phrasing, accuracy, dynamics; Vocal technique: Musicianship: melodic, rhythmic, and dynamic accuracy,
flexibility, breath support, diction, freedom, intonation; phrasing, style, etc.; Communication: diction, projection
Vocal characteristics: quality, range, evenness of scale; of text, interpretation, dramatic effectiveness, etc.; Presen-
Artistry: projection of the mood and spirit of the song, tation: appearance, expression, deportment, posture/bear-
tone color, stage presence, personality:' Each of these cri- ing, personality, etc.' No numerical scoring is offered.
teria is to be given a mark of plus (+) for above average, Because words can mean different things to each per-
check (U) for average, or minus (-) for below average. son, evaluators need to be able to agree on definitions
The Southern Region uses a plus (+) for above aver- of all of these categories and criteria. In Appendix B at
age and a minus (-) for below average for the following the end of this article, the terms mentioned above that
twenty-one characteristics. "Musicianship: rhythm, most frequently appeared on NATS adjudication forms
tempo, phrasing, accuracy, dynamics, sensitivity; Tech- for student auditions are listed. Definitions of these terms
nique: flexibility, breath support, diction, language ac- are included, using the author's interpretations of specific
curacy, freedom, control, intonation; Characteristics: listings found in the American HeritageDictionary of
quality, range, evenness; Artistry: performance practice, the English Language.18 Individual criteria are listed al-
projection/spirit, tone color, stage presence, personal- phabetically rather than in the order found on the adju-
ity.' No numerical scoring is offered. dication forms for ease of reference. These specific cri-
The Northwest Region uses no numerical scoring. It teria are more useful in evaluation than a generalized,
allows room for each adjudicator to write comments un- "How do you like this singer's voice?" The author offers
der the following seven categories: "musicianship, vo- her own standardized form in Appendix A.
38 JOURNAL OF SINGING
PerceptualAcoustic Assessment of Singing I1
In a symposium titled "Sounding the Voice," which the level singer from amateur to advanced. An attempt was
author hosted at the University of Iowa School of Music made to eliminate writing while adjudicating so the lis-
in April 2004, a panel of guest faculty offered new cri- tener could focus on the performance rather than at-
teria for consideration in an evaluation form. The panel tempting to draft criticisms of the performance.
included: Jody Kreiman, PhD, UCLA; Christine Bergan, A number of instructions for using this form need to
C(;C-SLP, MM, MA, University of Iowa; Stephen be considered. Review the definitions in Appendix B
Swanson, MM, University of Iowa; Donald Simonson, before beginning. Adjudicators should listen to the en-
DM, Iowa State University; and Leanne Freeman-Miller, tire performance of the singer before writing any com-
MM, Drake University. Their suggestions for additional ments on the lines. One song or aria should be rated at
criteria to include on a standardized form were: vocal a time. While timings will vary, at least three minutes
efficiency, maximized output/minimized effort, ease of of music should be assessed. The performance should
production, the "it" or intense desire to communicate, be measured against that of the best professional singer
beauty, power, flexibility, appropriateness of repertoire of that gender, age, cultural background, and length of
for that singer, size of voice, level of difficulty of reper- study. Adjudicators should indicate on the visual ana-
toire, style of interpretation, appropriate vocal health, log scale the degree of perceived deviance with a check
and performance practice. They added that a form must mark: 0 = severely deviant, 50 = moderately deviant,
be certain not to reward the voice more than the tech- 100 = no deviance or the best professional singer of that
nique demonstrated and have sufficient room for aes- gender, age, background, and length of study. Hence,
thetic evaluation. the higher the score, the better the performance. One
The CAPENV format that speech pathologists currently should report the score again for each category in the
use does not fit the needs of singer evaluation. Speech blank space to the right (x/100) so that the adjudicator
pathologists evaluate overall severity, roughness, breath- can easily see the overall scoring. In the criteria under each
iness, strain, pitch, and loudness in three tasks on the scale are two letters, "C" which represents "consistent"
CAPENV form. The tasks are: sustained vowels, [a] and and "'T indicating "intermittent:' The adjudicator will
[i] for three to five seconds duration each, sentence pro- circle the letter that best describes the consistency of the
duction of six specific sentences, and spontaneous speech performance. Any other noteworthy characteristics can
in response to: "Tell me about your voice problem" To be listed on each line using descriptive labels, for ex-
modify this concept to make it applicable to singing, all ample, clear, resonant, shrill, nasal, or breathy. Performers
singers would have to perform the same song or aria. This should not be given these evaluations, as they are in-
would not be practical because voice types, abilities, and tended only for the use of the observer and should be
individual repertoire preference vary considerably. The retained for comparison purposes. If written responses
CAPENV form uses a visual analog scale measuring mildly are required to be returned to the performers, it is rec-
deviant, moderately deviant, and severely deviant voice ommended that an additional blank sheet be provided
characteristics. It critiques whether or not a certain qual- so that constructive criticisms can help the students im-
ity is consistently or intermittently present, and offers prove. Students tend to want to know how they rank,
space to write in any additional characteristics noted. but delicate egos can be discouraged when comparisons
The intent of the form offered in Appendix A is to are made numerically.
open a dialogue about auditory perception for singing Ralph Appelman and James McKinney published cas-
evaluation. The author chose to adopt the visual analog sette tapes of sample vocal sounds that supplement their
scale and the consistent and intermittent comments from pedagogy books, and these tapes can be used as anchors
the CAPENV form while using criteria better suited to to train auditors' ears. However, Appelman and McKinney
evaluation of the healthy singing voice. This Perceptual- used their own mature voices to demonstrate tone qual-
Auditory Assessment form offers a minimal set of mean- ities rather than typical student singers. Ideally, anchors
ingful terms that can be used quickly and is applicable or training tapes of representative sounds need to become
to all voice types. The one page assessment form is in- commercially available to train listeners' perceptions to
tended to be used for any type adjudication and for any accurately assess voices. Professionally recorded tapes or
39
SiPTrEMBER/OCTOBER 2006
SKatherine Eberle Fink
CDs need to demonstrate absolute standards of accept- ultimately make the evaluation process fairer and more
able performance and should be used in conjunction with accurate and help to train young teachers to assess sing-
the PAA form to hone adjudicators' listening skills. ing tone in a more scientific manner rather than purely
A standardized assessment model for singing will subjectively.
APPENDIX A
Eberle' s Perceptual Acoustic Assessment (PAA)
Copyright 2004 by Katherine Eberle
Vocal Characteristics
/100
0 10 20 30 40 50 60 70 80 90 100
Tone Quality
Evenness of Scale
C I
C I
Vocal Efficiency (freedom)
C I
Intonation (pitch)
C I
Projection (loudness)
C I
C I
0 10 20 30 40 50 60 70 80 90 100
Posture
C I
Breath Management
C I
Vibrato
C I
Attack & Release
Legato
C I
Agility
C I
C I
Register Transitions
C I
Diction Accuracy
C I
C I
Musicianship
/100
0 10 20 30 40 50 60 70 80 90 100
Melodic Accuracy
C I
Dynamic Accuracy
C I
Rhythmic Accuracy
C
Phrasing
Tempo
C I
C I
C I
Artistry
/100
0 10 20 30 40 50 60 70 80 90 100
Stage Presence C I
Communication C I
Performance Practice C I
Appropriateness of Repertoire C I
C 1
40 JOURNAL OF SINGING
PerceptualAcoustic Assessment of Singing I
APPENDIX B
Terms and Definitions
SHITEMBER/OCTOBER 2006 41
1 Katherine Eberle Fink
Musicianship Tempo
Skill, taste, or artistry in a performance. The relative speed at which a composition is to be sung, as indi-
cated by a descriptive or metronomic direction to the performer.
Performance practice
A presentation of appropriate musical style according to the tra- Tone quality
ditions of certain time periods or compositional standards. Timbre as determined by overtones; a characteristic or attribute
of singing; the essential character, excellence, or superiority of
Personality
specific pitches.
The state or quality of being a person; the dynamic character,
self, or psyche that constitutes and animates the individual per- Understanding and projection of text
son; the pattern of collective character, behavioral, temperamen- The act of putting forth the text of a song so that the audience
tal, emotional, and mental traits of an individual; the distinctive perceives and comprehends the nature and significance of it.
qualities of an individual.
Vibrato
Phrasing A tremulous or pulsating effect produced in a vocal tone by per-
The manner in which a phrase is rendered or interpreted. ceptible minute and rapid variations in pitch.
42 JOURNAL OF SINGING
PerceptualAcoustic Assessment of Singing I
9. Ibid., 171. Katherine Eberle is an active and versatile mezzo soprano soloist. Her
10. Ibid., 321-322. career includes a wide array of professional and collegiate engagements
inboth the United States and abroad. She specializes inart song, ora-
11. Ibid., 178-179.
torio, chamber music, and opera, performing in both international and
12. Ibid., 179-180. regional venues with more than eighty professional engagements to
13. Ibid., 180. Paired comparisons are preferred for assessment date with various orchestras, choral organizations, and chamber music
of speaking voice quality by perception researchers at UCLA groups. Concert credits include solo performances with the symphonies
because of their reliability. Jody Kreiman, interview with au- of Detroit, Lansing, and Saginaw (Michigan), and Atlanta, Macon, Rome,
thor, Iowa City, Iowa, March 13, 2004. and Valdosta (Georgia). She has given over one hundred solo recitals as
a guest artist in eighteen states as well as inBrazil, the Netherlands,
14. Ibid., 183.
Russia, St. John and St. Thomas, Virgin Islands. She made her New York
IS. Richard Berry, Myra Brand, Jerry Daniels, Edward Deckard, debut at Weill Recital Hall, Carnegie Hall in 1994. She was an Artistic
and Judith Nicosia Civitano, Guidelinesfor Student Auditions Ambassador for the United States Information Agency doing solo con-
(Tampa, FL: National Association of Teachers of Singing, cert tours inSouth America (in1995 she appeared inArgentina, Ecuador,
1993), 3. Peru, Trinidad, and Tobago) and in Korea in 1997. Eberle's recording
16. Ibid., 9. From a Woman's Perspective is available through Albany Records on
17. Ibid.
the Vienna Modern Masters label. Eberle earned degrees from Baldwin-
Wallace Conservatory (BME), the University of Cincinnati (MM), and the
18. William Morris, ed., The American Heritage Dictionary of
University of Michigan (DMA). Indemand as an adjudicator, she was
the English Language (Boston: Houghton Mifflin, 1981). State President of the National Association of Teachers of Singing Iowa
19. Glenda Maurice, "Some Personal Thoughts on Artistry,' chapter for three years, and has adjudicated for numerous vocal com-
JournaloJ Singing 53, no. 4 (March/April 1997): 7. petitions. She has taught voice at the Interlochen Center for the Arts
20. Richard Miller, The Structure of Singing(New York: Schirmer Summer Music Camp and voice pedagogy at the Summer Vocology Insti-
Books, 1986), 312. tute inDenver, CO. Eberle is currently a professor at The University of
Iowa, where she chaired the School of Music voice area for seven years.
21. Clifton Ware, Basics of Vocal Pedagogy (Boston: McGraw-
She joined the faculty of the University of Iowa in1990 after five years
Ifill, 1998), 280.
of teaching at the University of Georgia. See www.keberle.com for more
22. Miller, 312. information.
Houghton College
a higherpurpose in mind Scolo usic
Integratingthe Christianfaith
with professional music-making
and fine scholarship.
800.777.2556 or 585.567.9400
One Willard Ave, Houghton, New York 14744-0128
E-n-ail: nisic@houghton.edu
MEMBER
www. houghton.edu/greatbatch
Si I"
FEMBER/OCTOBER 2006 43
COPYRIGHT INFORMATION