Facial Expressions of Emotion - Stimuli and Tests (FEEST) : Psychology Manual v1.0 Thames Valley Test Company

Facial Expressions of
Emotion Stimuli and

Tests (FEEST)
Psychology manual v1.0
Thames Valley Test Company
Facial Expressions of Emotion Stimuli and Tests (FEEST).

Thames Valley Test Company January 2002.
Psychology manual v 1.0

Contents: page 1 of 4
Facial Expressions of Emotion Stimuli and Tests (FEEST)

Psychology manual v1.0
Andrew Young, University of York, England
David Perrett, University of St. Andrews, Scotland
Andrew Calder, MRC CBU, Cambidge, England
Reiner Sprengelmeyer, University of St. Andrews, Scotland
Paul Ekman, University of California, USA
Thames Valley Test Company 2002
Bury St Edmunds, England.
http://www.tvtc.com
ISBN: 1 874261 96 2
Designed and produced by Boag Associates,
34 Bartholomew Place, London EC1A 7NJ.
http://www.boag.co.uk


Contents
Overview ................................................................................................................2
About the FEEST...................................................................................................2
The FEEST facial expression recognition tests ....................................................4
The Ekman 60 Faces test...................................................................................5
The Emotion Hexagon test ................................................................................6
The FEEST stimuli ................................................................................................7
Emotion Megamixes ..........................................................................................7
Caricatures and anti-caricatures .......................................................................8
Morphed and caricatured Continua ..................................................................9
Uses of the FEEST...............................................................................................10
References ............................................................................................................11
Section 1: The Ekman 60 Faces test......................................................................2
Stimuli ...................................................................................................................2
Procedure .............................................................................................................4
Performance norms...............................................................................................6
Overall scores.....................................................................................................6
Scores for each emotion.....................................................................................8
Validity and reliability.........................................................................................10
Previous use of the Ekman 60 Faces test ........................................................... 12
References ........................................................................................................... 14
Appendix 1: Files used in the Ekman 60 Faces test .............................................. 15
Filenames ............................................................................................................ 15
Test files ........................................................................................................... 15
Duplicate files .................................................................................................. 17
Section 2: The Emotion Hexagon test...................................................................2
Stimuli ...................................................................................................................2
Procedure ..............................................................................................................7
Performance norms...............................................................................................8
Graphical representation of results...................................................................8
Overall scores...................................................................................................10
Scores for each emotion................................................................................... 12
Validity and reliability......................................................................................... 14
Previous use of the Emotion Hexagon test......................................................... 16
References ........................................................................................................... 19
Appendix 2: Files used in the Emotion Hexagon test ........................................... 19
Filenames ............................................................................................................ 19
Test files .......................................................................................................... 20
Duplicate files .................................................................................................. 21
Section 3: Emotion Megamixes.............................................................................2
The Emotion Megamix stimuli .............................................................................2
Properties of the Emotion Megamix images ........................................................6
References ........................................................................................................... 14
Appendix 3: Organisation of the Emotion Megamix images on the CD-ROM .... 15
Section 4: Caricatures and anti-caricatures..........................................................2
The Caricature stimuli ..........................................................................................2
Properties of the Caricature images .....................................................................5


References .............................................................................................................9
Appendix 4: Organisation of the Caricature images on the CD-ROM....................9
Section 5: Morphed and caricatured Continua.....................................................2
The Continua.........................................................................................................2
Properties of the Continua....................................................................................4
References .............................................................................................................6
Appendix 5: Organisation of the Continua on the CD-ROM ..................................6
Section 6: Origin of the photographs in FEEST ...................................................2
References .............................................................................................................8
Section 7: Computer morphing and caricaturing procedures..............................2
Caricaturing...........................................................................................................2
Morphing...............................................................................................................4
References .............................................................................................................7


Overview .............................................................................................................................2
About the FEEST ............................................................................................................2
The FEEST facial expression recognition tests..............................................................4
The Ekman 60 Faces test............................................................................................5
The Emotion Hexagon test .........................................................................................6
The FEEST stimuli.......................................................................................................... 7
Emotion Megamixes ................................................................................................... 7
Caricatures and anti-caricatures ................................................................................8
Morphed and caricatured Continua ...........................................................................9
Uses of the FEEST ........................................................................................................ 10
References......................................................................................................................11


Overview: page 1 of 11
Overview
The ability to interpret the moods and feelings of other people is an important social
skill. Misinterpretation of affect can lead to misunderstanding and inappropriate
social behaviour.
For humans, facial expressions provide important indicators of emotion. Basic
emotions are expressed and recognised in similar ways throughout the world.
Functional imaging studies have uncovered some of the mechanisms involved in
neural responses to perceived emotion, and impaired recognition of facial expressions
has been documented after a number of types of brain disease.
The Facial Expressions of Emotion: Stimuli and Tests (FEEST) makes available a
range of high-quality materials for testing recognition of facial expressions of
emotion, including standard tasks with data for comparison groups and
supplementary stimuli which can be used to create new tests and experiments. The
stimuli include the six basic emotions from the Ekman and Friesen (1976) series
(anger, disgust, fear, happiness, sadness, and surprise) and neutral expressions.
Computer-morphing and computer-caricaturing procedures are used to create
continua varying from one expression to another, or differing in intensity. Clinicians
and researchers can therefore create tasks which can be graded in difficulty ranging
from subtle to intensely expressed emotions.
About the FEEST
The FEEST comprises the FEEST facial expression recognition tests (the Ekman 60
Faces test and the Emotion Hexagon test), and the FEEST stimuli (Emotion
Megamixes, Caricatures, and Continua).
All of the images used in FEEST are derived from pictures of facial expressions in the
Ekman and Friesen (1976) series of Pictures of Facial Affect, which has been the most
widely used and validated series of photographs in facial expression research. From
this series, 10 models (6 female, 4 male) were chosen, using the criterion that wellrecognised expressions corresponding to each of six basic emotions (anger, disgust,
fear, happiness, sadness, and surprise) were available, as well as a neutral pose. In
this Manual, we refer to these pictures from the Ekman and Friesen series as
prototype expressions, because they form the basis from which the computermanipulated expression images were created.


Figure 0.1: Photographs of facial expressions from the Ekman and Friesen (1976)
Pictures of Facial Affect used in FEEST. There are emotional expressions of anger
(A), disgust (D), fear (F), happiness (H), sadness (S) and surprise (U), and a neutral
(N) pose for 6 female and 4 male models. The labels used to identify each of the
models locate them in the Ekman and Friesen series (F2 = second female model in
the series, M1 = first male model, etc.).
The prototype expressions used in FEEST are shown in Figure 0.1. The identifiers of
the facial expressions in the Ekman and Friesen (1976) series are given in Tables 0.1a
and 0.1b.


F2
F4
F5
F6
F7
F8
Anger
10 C-2-12
53 MF-2-07
61 MO-2-11
69 NR-2-07
89 PF-2-04
96 SW-4-09
Disgust
12 C-1-04
55 MF-2-13
64 MO-2-18
71 NR-3-29
91 PF-1-24
98 SW-1-30
Fear
9 C-1-23
50 MF-1-26
59 MO-1-23
68 NR-1-19
88 PF-2-30
95 SW-2-30
Happiness
7 C-2-18
48 MF-1-06
57 MO-1-04
66 NR-1-06
85 PF-1-06
93 SW-3-09
Sadness
8 C-1-18
49 MF-1-30
58 MO-1-30
67 NR-2-15
86 PF-2-12
94 SW-2-16
Surprise
11 C-1-10
54 MF-1-09
63 MO-1-14
70 NR-1-14
90 PF-1-16
97 SW-1-16
Neutral
13 C-2-03
56 MF-1-02
65 MO-1-05
72 NR-1-03
92 PF-1-02
99 SW-3-03
Table 0.1a: Identifiers from the Ekman and Friesen (1976) series for the facial
expressions of the 6 female models shown in Figure 0.1.
M1
M4
M5
M6
Anger
18 EM-5-14
38 JJ-3-12
80 PE-2-21
105 WF-3-01
Disgust
20 EM-4-17
40 JJ-3-20
82 PE-4-05
108 WF-3-11
Fear
16 EM-5-21
37 JJ-5-13
79 PE-3-21
104 WF-3-16
Happiness
14 EM-4-07
34 JJ-4-07
74 PE-2-12
101 WF-2-12
Sadness
15 EM-4-24
36 JJ-5-05
75 PE-2-31
102 WF-3-28
Surprise
19 EM-2-11
39 JJ-4-13
81 PE-6-02
107 WF-2-16
Neutral
21 EM-2-04
41 JJ-3-04
83 PE-2-04
110 WF-2-05
Table 0.1b: Identifiers from the Ekman and Friesen (1976) series for the facial
expressions of the 4 male models shown in Figure 0.1.
The origins of the photographs in the Ekman and Friesen series are described in
Section 6 of this Manual, which lists the Action Units (facial muscle movements) for
each picture. The computer image-manipulation techniques used to create
caricatured and morphed variants of each expression are described in Section 7.
The FEEST facial expression recognition tests
Two tests of facial expression recognition are included in FEEST, each using a
different procedure. The tests can be run from software included on the FEEST CDROM.


Testing of facial expression recognition is notoriously tricky. Stimulus sets that

predated the Ekman and Friesen (1976) series often led to very variable responses by
human perceivers (see Woodworth and Schlosberg, 1954). Our vocabulary of emotion
words is rich and to some extent personal, so that some emotion words can mean
slightly different things to different people, which makes it hard to interpret results
from tasks in which perceivers are simply asked to describe a person's feelings on the
basis of their facial expressions.
The procedure adopted in many modern studies of facial expression recognition has
therefore been to restrict stimuli to basic emotions which are likely to have significant
evolutionary histories, and to use a forced-choice procedure in which participants
assign expressions to a limited range of categories. Both tests developed for FEEST
are of this type.
The Ekman 60 Faces test
The Ekman 60 Faces test is described in detail in Section 1 of the FEEST Manual. It
uses a range of photographs from the Ekman and Friesen series (the top 6 rows of
Figure 0.1) to test recognition of facial expressions of basic emotions (anger, disgust,
fear, happiness, sadness, and surprise). The test yields a score out of a maximum of
60 correct for recognition of all six emotions, or scores out of 10 for recognition of
each basic emotion. The computer software on the CD-ROM presents the stimuli in
random order for 5 seconds each, records responses made from mouse clicks to onscreen buttons or user-defined key presses, and summarises these as test scores.
Comparison data are available for 227 individuals aged 2070 years with IQs of 90
and above. Full details of the scores of the comparison group are given in Section 1 of
the FEEST Manual, but for convenience mean correct recognition rates and cut-off
scores defining the border between normal-range and impaired performance (p = .05)
for the entire group and for sub-groupings based on age are reproduced in Table 0.2.


Total
score
50.64
Anger
Disgust
Fear
Happiness
Sadness
Surprise
7.86
8.59
7.19
9.87
8.33
8.55
Cut-off
42
Mean
51.43
8.21
8.38
7.82
9.90
8.59
8.54
Cut-off
45
Mean
51.20
8.17
8.77
7.23
9.84
8.53
8.61
Cut-off
43
Mean
49.41
7.33
9.00
6.47
9.93
8.03
8.66
Cut-off
41
Entire group Mean
Age 2040
Age 4160
Age 6170
Table 0.2: Mean correct recognition rates and cut-off scores indicating the
boundary between normal-range and impaired total scores (max = 60) and scores
for recognition of each emotion (max = 10) in the Ekman 60 Faces test.
The Emotion Hexagon test
Whereas the Ekman 60 Faces test involves recognition of a number of facial
expressions of each emotion, the Emotion Hexagon uses computer image
manipulation techniques to test facial expression recognition with stimuli of graded
difficulty.
The Emotion Hexagon test is described in detail in Section 2 of the FEEST Manual. It
uses 30 computer-manipulated images of faces from the Ekman and Friesen series to
test recognition of basic emotions (anger, disgust, fear, happiness, sadness, and
surprise). The test's results can be shown as performance graphs, or converted to a
score out of a maximum of 120 correct for recognition of all six emotions, or scores
out of 20 for recognition of each basic emotion. The computer software on the CDROM presents the stimuli in random order for 5 seconds each across one practice and
5 test blocks of 30 trials each, records responses made from mouse clicks to on-screen
buttons or user-defined key presses, and summarises these as test scores.
Comparison data are available for 125 individuals aged 2075 years with IQs of 90
and above. Full details of the comparison group scores are given in Section 2 of the
Manual. Mean correct recognition rates and cut-off scores defining the border
between normal-range and impaired performance (p = .05) for the entire group and
for sub-groupings based on age are reproduced in Table 0.3.


Total
score
107.97
Anger
Disgust
Fear
Happiness
Sadness
Surprise
17.84
18.01
16.56
19.64
18.38
17.69
Cut-off
92
13
12
10
18
13
14
Mean
109.16
18.38
16.88
17.74
19.74
18.66
18.10
Cut-off
94
14
11
12
18
13
15
Mean
108.10
17.63
18.71
16.15
19.63
18.31
17.65
Cut-off
92
13
13
10
18
13
14
Mean
105.09
17.13
18.83
14.91
19.43
17.91
16.87
Cut-off
90
13
13
18
12
13
Entire group Mean
Age 2040
Age 4160
Age 6175
Table 0.3: Mean correct recognition rates and cut-off scores indicating the
boundary between normal-range and impaired total scores (max = 120) and scores
for recognition of each emotion (max = 20) in the Emotion Hexagon test.
The FEEST stimuli
The FEEST stimuli comprise more than 1,000 images of faces derived from
photographs in the Ekman and Friesen (1976) series. The stimuli include prototype
(unmodified) facial expressions and computer-manipulated versions. In the
computer-manipulated images, morphing and caricaturing techniques (described in
Section 7 of the FEEST Manual) are used to systematically change the images in ways
that allow the creation of novel tests and experiments suited to a wide range of
purposes. Morphing is used to create images that fall along regularly graded
transitions from one prototype expression to another, whereas caricaturing is used to
increase or decrease the intensity of a particular expression.
The FEEST stimuli are arranged in directories on the CD-ROM. The filenames of the
individual images have been created in a way that will sort the images and allow an
image to be returned to its correct directory if it is accidentally moved.
The image files are encrypted on the CD-ROM, but can be copied to a hard disk using
the software supplied. Instructions for locating files are given as appendices to the
relevant sections of the FEEST Manual.
Emotion Megamixes
The Emotion Megamixes use computer morphing to blend the shapes and surface
tone (regional brightness values) of prototype expressions from the Ekman and
Friesen (1976) series, creating continua showing transitions between one expression
and another. The Emotion Megamix stimuli include continua between all possible
pairs of expressions (anger, disgust, fear, happiness, sadness, surprise, and neutral)
for two separate models (F5 and M4 in Figure 0.1; these are models MO and JJ in the


Ekman and Friesen series). Examples of these morphed continua are given in Figure
0.2, which shows the continua for pairwise transitions between three basic emotions
(happiness, fear, and disgust). There are 21 such continua for each model in the
Emotion Megamix series, with 9 images in each continuum.
Figure 0.2: Examples of continua of morphed facial expressions for three basic
emotions (happiness, fear, and disgust) with model F5/MO. The top row shows the
happiness fear continuum, the centre row fear disgust, and the bottom row
disgust happiness. The prototype expressions from the Ekman and Friesen (1976)
series are not shown in this Figure (they can be seen in Figure 0.1); all of the stimuli
are morphed to a greater or lesser degree.
The Emotion Megamix stimuli are described in detail in Section 3 of the FEEST
Manual, where the full set of 21 continua is illustrated for M4/JJ's face (Figures 3.1a,
3.1b, and 3.2). The use of computer morphing creates a smooth transition from one
prototype expression to another, but the perception of such continua is not linear,
and tends to show clearly demarcated category boundaries (Young, Rowland, Calder,
Etcoff, Seth and Perrett, 1997).
Caricatures and anti-caricatures
Computer caricaturing procedures can be used to change the underlying shape of a
facial expression away from a reference norm, exaggerating any differences in shape
to create a caricatured expression that is seen as more intense and is easier to
recognise. It is also possible to create an anti-caricatured representation that is closer
to the norm. Anti-caricatures are less intense and less easy to recognise.
In the FEEST Caricature stimuli (described in Section 4 of this Manual), the
emotional expressions of models F5/MO and M4/JJ are caricatured and anticaricatured relative to a Neutral expression norm or to an Average expression norm.
Examples of Caricature series for expressions of happiness, fear, and disgust using
model F5/MO's face relative to a Neutral norm are shown in Figure 0.3. Each series
(represented as a row in Figure 0.3) uses 4 levels of anti-caricature and 4 levels of
caricature which, together with the prototype expression, creates a continuum of 9
images of increasing intensity of expressed emotion.


Figure 0.3: Use of anti-caricature (left of the centre column) and caricature (right of
centre column) methods to modify the apparent intensity of happiness (top row),
fear (centre row), and disgust (bottom row) in prototype (unmodified) facial
expressions (centre column) of model F5/MO. The caricatures shown here were
prepared relative to a Neutral expression norm.
The FEEST Caricature stimuli include six continua for each model (one series of 9
images for each of anger, disgust, fear, happiness, sadness, and surprise) prepared
relative to a Neutral expression norm, and a further six continua for each model
prepared relative to an Average expression norm. The full set of images for M4/JJ's
face using an Average expression norm is shown in Figure 4.2, and using a Neutral
expression norm in Figure 4.3.
Whereas the morphed stimuli from the Emotion Megamixes change from one
emotional expression to another (see Figure 0.2), in each Caricature series there is a
change in the expressed intensity of a particular emotion rather than a shift in the
emotion itself.
Morphed and caricatured Continua
The morphed and caricatured Continua are described in detail in Section 5 of the
FEEST Manual. They use a combination of morphing and caricaturing methods to
create Continua from a neutral pose to an intensely expressed emotion. Examples for
F5/MO's face and the emotions of happiness, fear, and disgust are shown in Figure
0.4.


Figure 0.4: Morphed and caricatured Continua for neutral happiness (top row),
neutral fear (centre row), and neutral disgust (bottom row), with model F5/MO.
In each continuum, the neutral expression is shown at the left of the row (first
column), and the prototype expression in the fifth column. The images in the second
to fourth columns are morphed, and the images in the sixth and seventh columns are
caricatured. The combination of image manipulation techniques creates Continua
ranging from a neutral pose to an intensely expressed emotion.
Morphed and caricatured Continua of 7 images are available for all of the 10 models
shown in Figure 0.1, and all six basic emotions (anger, disgust, fear, happiness,
sadness, and surprise), yielding 60 Continua in total. A full set of Continua for face
F2/C is shown in Figure 5.1.
The Continua are useful in circumstances requiring more than one or two models or a
set of expressions that cover a range of intensities from neutral through to highly
emotional, provided that it is not essential that a consistent method (morphing or
caricature) was used to create the changes in expression. The hairlines are masked to
make it necessary to base any decision about the model's sex on facial features. This
allows an incidental task of classifying the faces as female or male to be used if
needed; for example, this is a common requirement in functional imaging studies
(Morris, Frith, Perrett, Rowland, Young, Calder and Dolan, 1996).
Uses of the FEEST
The FEEST has a wide range of potential applications.
Testing of facial expression recognition is important in many neuropsychological and
clinical contexts. The Ekman 60 Faces test and the Emotion Hexagon test can each be
used on its own, or alongside the other test. The Ekman 60 Faces test uses a range of
expressions and is shorter to administer than the Emotion Hexagon test, but the
Emotion Hexagon usually has better reliability. Taken together, the two tests can
point strongly to the existence of deficits affecting facial expression recognition, and
allow examination of whether such deficits affect recognition of all emotions or


have a differentially severe impact on the recognition of certain emotions (see

Sections 1 and 2 of the FEEST manual).
The FEEST stimuli can be used to create supplementary tests for specific purposes.
For example, it is possible to explore emotion recognition deficits in detail by seeing
whether a person who does not recognise certain emotional expressions can match
different pictures as representations of the same underlying emotion, perceive
changes in the intensity of an unrecognised expression, and so on.
A particular application where the stimuli have already been regularly used (see
Section 5 of this Manual) involves functional imaging studies to investigate the neural
responses of the brain to different emotions. The ability to vary the level of intensity
of an expression whilst keeping other factors tightly controlled is very useful in this
type of research.
The FEEST stimuli are also well-suited for creating psychological experiments to
examine the basis of facial expression perception. They can be used to investigate
dimensional and category-based accounts, priming effects from expressions of
different intensities, interference from distractor expressions on incidental tasks, and
many other questions.
The information about the Action Units (AUs) underlying each prototype expression
(Section 6) offers added scope for authoritative studies investigating how AUs are
involved in facial expression perception, in neuropsychological deficits, and in
different types of neural response all possibilities that have yet to be systematically
explored.
References
Ekman, P. and Friesen, W.V. (1976). Pictures of facial affect. Palo Alto, California:
Consulting Psychologists Press.
Morris, J.S., Frith, C.D., Perrett, D.I., Rowland, D., Young, A.W., Calder, A.J. and
Dolan, R.J. (1996). A differential neural response in the human amygdala to
fearful and happy facial expressions. Nature, 383, 812815.
Woodworth, R.S. and Schlosberg, H. (1954). Experimental psychology: revised
edition. New York: Henry Holt.
Young, A.W., Rowland, D., Calder, A.J., Etcoff, N.L., Seth, A. and Perrett, D.I. (1997).
Facial expression megamix: tests of dimensional and category accounts of emotion
recognition. Cognition, 63, 271313


Section 1: The Ekman 60 Faces test...................................................................................2

Stimuli.............................................................................................................................2
Procedure........................................................................................................................4
Performance norms ........................................................................................................6
Overall scores ..............................................................................................................6
Scores for each emotion..............................................................................................8
Validity and reliability .................................................................................................. 10
Previous use of the Ekman 60 Faces test..................................................................... 12
References..................................................................................................................... 14
Appendix 1: Files used in the Ekman 60 Faces test ........................................................ 15
Filenames...................................................................................................................... 15
Test files .................................................................................................................... 15
Duplicate files............................................................................................................ 17


Section 1: page 1 of 20
Section 1: The Ekman 60 Faces test

The Ekman 60 Faces test uses a range of faces from the Ekman and Friesen (1976)
series to test recognition of facial expressions of emotion. The Ekman and Friesen
(1976) faces have been the most widely used and extensively validated set of facial
expressions in research studies, but have not previously been used in published tests
of facial expression recognition.
The test involves recognition of facial expressions of six basic emotions anger,
disgust, fear, happiness, sadness, and surprise. There are ten examples of facial
expressions of each emotion, leading to a score out of a maximum of 60 for overall
performance, or scores out of 10 for recognition of the six emotions.
The FEEST CD includes a computer program for presenting the test, recording and
scoring responses, and saving these as an Excel-compatible spreadsheet file.
Comparison data are available for a group of 227 participants.
Stimuli
Photographs of the faces of 10 people (6 female, 4 male) have been selected from the
Ekman and Friesen (1976) series. For each face, there are poses corresponding to
each of 6 emotions (anger, disgust, fear, happiness, sadness, and surprise), giving a
total of 60 photographs. The 10 models were chosen so that each emotion was as well
recognised as possible in Ekman and Friesen's (1976) norms. Mean percentage
recognition rates across all 10 faces for Ekman and Friesen's (1976) participants, and
standard deviations, are shown in Table 1.1, which also lists the picture numbers and
the faces' identifiers in the Ekman and Friesen (1976) series. Details of the muscle
movements (Action Units) involved in each expression can be found in Section 6 of
this Manual.


Mean
SD
89.50
11.39
Picture numbers and identifiers used by Ekman and

Friesen (1976)
Female
Anger
Male
Female
Disgust
93.10
5.20
Male
Female
Fear
89.50
5.91
Male
Female
Happiness
99.10
2.51
Male
Female
Sadness
89.70
7.87
Male
Female
Surprise
90.70
7.78
Male
10 C-2-12; 53 MF-2-07; 61 MO-2-11; 69 NR-2-07; 89 PF2-04; 96 SW-4-09

18 EM-5-14; 38 JJ-3-12; 80 PE-2-21; 105 WF-3-01
12 C-1-04; 55 MF-2-13; 64 MO-2-18; 71 NR-3-29; 91 PF-124; 98 SW-1-30
20 EM-4-17; 40 JJ-3-20; 82 PE-4-05; 108 WF-3-11
9 C-1-23; 50 MF-1-26; 59 MO-1-23; 68 NR-1-19; 88 PF-230; 95 SW-2-30
16 EM-5-21; 37 JJ-5-13; 79 PE-3-21; 104 WF-3-16
7 C-2-18; 48 MF-1-06; 57 MO-1-04; 66 NR-1-06; 85 PF-106; 93 SW-3-09
14 EM-4-07; 34 JJ-4-07; 74 PE-2-12; 101 WF-2-12
8 C-1-18; 49 MF-1-30; 58 MO-1-30; 67 NR-2-15; 86 PF-212; 94 SW-2-16
15 EM-4-24; 36 JJ-5-05; 75 PE-2-31; 102 WF-3-28
11 C-1-10; 54 MF-1-09; 63 MO-1-14; 70 NR-1-14; 90 PF-116; 97 SW-1-16
19 EM-2-11; 39 JJ-4-13; 81 PE-6-02; 107 WF-2-16
Table 1.1: Faces used in the Ekman 60 Faces test. There are 6 female and 4 female
models expressing each emotion. The final column in the table lists each face's
number and identifier in the Ekman and Friesen (1976) series. Means and standard
deviations calculated from Ekman and Friesen's (1976) data for percentage
recognition as each emotion across the 10 models are shown in the second and third
columns.
An additional set of 6 expressions posed by a single model is used for practice, to
introduce the test.
The complete set of faces used in the Ekman 60 Faces Test is shown in Figure 1.1.
Further details concerning the origin of these pictures and the different facial muscle
movements (Action Units) seen in each picture can be found in Section 6 of this
Manual.


Figure 1.1: Photographs of facial expressions used in the Ekman 60 Faces Test. The
leftmost column shows practice (P) stimuli. Columns in the main matrix show the 6
female (F) and 4 male (M) models used in the test, numbered to reflect their position
in the Ekman and Friesen (1976) Pictures of Facial Affect series (F2 = second female
model in the series, etc.). Rows show facial expressions of anger (A), disgust (D),
fear (F), happiness (H), sadness (S), and surprise (U).
Procedure
A computer program for running the Ekman 60 Faces Test is included on the FEEST
CD-ROM, and separate instructions are provided for this software. The faces are
presented one at a time for 5 seconds each, followed by a blank screen. The
participant is asked to decide which of the emotion names (anger, disgust, fear,
happiness, sadness, and surprise) best describes the facial expression shown. The
names of these six emotions are visible on the computer screen throughout the test,
with the order in which the emotion names are shown on the screen randomised each
time the test is given. Before commencing the test, you should satisfy yourself that
your participant understands the meanings of these emotion words sufficiently
accurately for the results to be meaningful (for example, by asking for examples of


circumstances in which people would experience anger, disgust, fear, etc.). Responses
can be recorded from mouse-clicks to the on-screen buttons, or via the computer
keyboard. The test is not timed participants can take as long as they wish to decide
on the emotion.
The 6 practice trials are followed by 60 test trials. The software randomises the order
of presentation of trials within each block, and allows responses to be saved to an
Excel-compatible spreadsheet file that records the trial number, the stimulus
filename, and the response made.
The 60 test trials (one for each of the 6 emotions across the 10 models) can be used to
derive an overall total score out of a possible maximum of 60 expressions correctly
recognised, or accuracy scores out of a possible maximum of 10 for each of the six
emotions. The CD-ROM software will do this automatically.
When performance is impaired, it may also be desirable to examine the nature of the
errors made. The filenames for the stimuli used in the Ekman 60 Faces test contain
the information needed to interpret responses to particular expressions. Each
filename is constructed in the following way:
ET_emoxx_nnn_id_ekid_per.jpg
where ET designates that this is an image from the Ekman 60 Faces test, emo
indicates the emotion (Ang = anger, Dis = disgust, Fea = fear, Hap = happiness,
Sur = surprise), xx is used to order the pictures of each emotion from 01 to 10, nnn
is the picture number in the Ekman and Friesen (1976) series, id gives the
identifier for each model used at the top of Figure 1.1, ekid is the identifier in the
Ekman and Friesen series, and per is the percentage recognition as the intended
emotion in Ekman and Friesen's data.
For example, the filename ET_Dis08_040_M4_JJ-3-20_088.jpg shows the 8th
picture of disgust in the appropriate row of Figure 1.1. The model is identified as M4
in Figure 1.1. The picture is number 40 in the Ekman and Friesen series, where it has
the identifier JJ-3-20. It was recognised as disgust by 88% of Ekman and Friesen's
participants.
The filename and participant's response for each stimulus is recorded by the program
software. This information can be combined with the information from Table 6.2 in
Section 6 of the FEEST Manual, allowing the user to explore:
whether errors are mainly to expressions that are not so easily recognised by
controls (this can also be done by using the percentage recognition rate at the end
of each filename)
whether errors are of the same type as those made by normal perceivers
whether errors consistently involve certain Action Units.

A complete list of filenames for the stimuli used in the Ekman 60 Faces test is given
as an Appendix to the present Section of the FEEST Manual. These files cannot be
removed from the CD-ROM, but the Appendix gives the location of a duplicate set of
files suitable for use by those who prefer to program their own version of the test. This
can be useful if it is necessary to adjust the procedure for example, to get more
detailed information by recording reaction times as well as accuracy.


Performance norms
We tested an opportunity sample of 241 individuals aged 2070. Where intelligence
test scores were not known from existing records, the NART-R (Nelson, 1991) was
used to estimate intelligence.
Data for 14 people with IQs less than 90 were omitted because the sample did not
contain sufficient numbers to estimate performance reliably for this sub-grouping.
This left 227 people with IQs of 90 and above in the main sample.
As already noted the Ekman 60 Faces test yields an overall score out of a possible
maximum of 60 expressions correctly recognised, or accuracy scores out of a possible
maximum of 10 for each of the six emotions. We will consider these in turn.
Overall scores
An overall (total) score out of a maximum possible of 60 can be derived by summing
correct responses across the 60 test trials.
To determine the effects of age, intelligence and sex on total score, participants were
divided into sub-groups based on five age bands (2030 years, 3140 years, 4150
years, 5160 years, and 6170 years), four levels of intelligence (IQ 90100, IQ
101110, IQ 111120, and IQ over 120), and female or male sex. Means and standard
deviations of total scores across age, IQ, and sex are shown in Table 1.2, together with
the numbers of participants in each sub-grouping.


Mean
SD
Entire group
50.64
5.04
227
Age 2030
51.52
3.83
73
Age 3140
51.22
4.48
32
Age 4150
51.48
4.80
29
Age 5160
50.89
5.03
35
Age 6170
49.41
4.88
58
IQ 90100
49.38
5.83
27
IQ 101110
50.81
4.45
68
IQ 111120
49.65
5.61
85
IQ > 120
51.80
4.22
47
Female
49.98
5.33
124
Male
50.81
4.85
103
Table 1.2: Means and standard deviations (SD) of total scores (Max = 60) on the
Ekman 60 Faces Test for the entire comparison group of 227 individuals, and for
sub-groupings based on age, intelligence and sex. Numbers (N) of participants in
each group are also shown.
A three-factor analysis of variance of the effects of Age (5 levels), Intelligence (4
levels) and Sex (female or male) revealed a borderline effect of Age (F = 2.39, df 4,
190, p = .052). There were no other significant main effects or interactions (all other
probabilities > .1).
Because of the borderline effect of age, it was considered prudent to subdivide the
control participants by age for the purpose of establishing cut-off scores. Table 1.2
shows that age did not have any dramatic effect on overall means, so to achieve
reasonable numbers in each group, the data were subdivided into age bands of 2040
years, 4160 years, and 6170 years, as shown in Table 1.3.


Mean
SD
cut-off score
Entire group
50.64
5.04
227
42
Age 2040
51.43
4.02
105
45
Age 4160
51.20
4.90
64
43
Age 6170
49.41
4.88
58
41
Ekman 60 Faces Test for the entire comparison group of 227 individuals, and for
sub-groupings based on age. Numbers (N) of participants in each sub-group are
shown. A level of performance falling 1.65 SDs below the mean has been used to
derive a cut-off score indicating the boundary between normal-range and impaired
performance for each group.
Cut-off scores to define the border between normal-range and impaired performance
for each sub-group were established using the nearest integer score to a z value of 1.65
(p = .05). These cut-off scores are shown in Table 1.3. They are also reproduced in
Table 0.2, which may be an easier place to find them if they need to be consulted
frequently.
Scores for each emotion
The Ekman 60 Faces test also yields scores out of a maximum possible 10 correct
responses for recognition of each of the 6 emotions (anger, disgust, fear, happiness,
sadness, surprise).
Because of the borderline effect of age on overall scores, a two-factor analysis of
variance was used to investigate the effects of Age (5 levels: 2030 years, 3140
years, 4150 years, 5160 years, and 6170 years) and Emotion (6 levels: anger,
disgust, fear, happiness, sadness, surprise). Mauchly's test of sphericity showed a
significant violation of the assumption of sphericity (W = .532, Chi-square = 138.99,
df 14, p < .001), due in large part to the near-ceiling performance for recognition of
happiness, so degrees of freedom for tests involving the Emotion factor were adjusted
using the Greenhouse-Geisser correction.
The analysis of variance showed a significant main effect of Emotion (F = 69.67, df 4,
933, p < .001), with some emotions being easier to recognise than others (as would be
expected from Ekman and Friesen's original data). The main effect of Age was again
borderline (F = 2.07, df 4, 222, p = .085).
Both of these main effects were qualified by a significant Age x Emotion interaction (F
= 2.75, df 17, 933, p < .001). This interaction is shown in Figure 1.2, which makes
clear that recognition of some emotions (especially fear) declines across age. The data
in Figure 1.2 are given as percentages, to facilitate comparison with Figure 2.4
(Section 2), which charts the equivalent interaction for the Emotion Hexagon test.


Percent correct
recognition
Ekman 60 Faces test

100
Anger
80
Disgust
60
Fear
40
Happiness
20
Sadness
Surprise
Age 20-30 Age 31-40
Age 41-50
Age 51-60 Age 61-70
Figure 1.2: Ekman 60 Faces test: Mean percent correct recognition of facial
expressions of anger, disgust, fear, happiness, sadness and surprise at different
ages.
Because of the Age x Emotion interaction, data for recognition of each emotion in the
Ekman 60 Faces test were subdivided into age bands of 2040 years, 4160 years, and
6170 years, as had been done with the overall total scores. Table 1.4 gives means and
standard deviations of scores for recognition of each emotion for the entire comparison
group of 227 individuals, and mean scores for the sub-groupings based on age.
Anger
Disgust
Fear
Happiness
Sadness
Surprise
Mean
7.86
8.59
7.19
9.87
8.33
8.55
SD
1.90
1.62
2.03
0.42
1.66
1.44
Age 2040
Mean
8.21
8.38
7.82
9.90
8.59
8.54
Age 4160
Mean
8.17
8.77
7.23
9.84
8.53
8.61
Age 6170
Mean
7.33
9.00
6.47
9.93
8.03
8.66
Entire group
Table 1.4: Means and standard deviations (SD) of scores for recognition of each
emotion (Max = 10) in the Ekman 60 Faces test for the entire comparison group of
227 individuals, and means for sub-groupings based on age.
for each sub-group were established at the nearest integer score to a z value of 1.65 (p
= .05), using the standard deviation of the entire group to estimate the degree of
variability of recognition of each emotion. These cut-off scores are shown in Table 1.5,
and reproduced in Table 0.2.


Anger
Disgust
Fear
Happiness
Sadness
Surprise
Entire group
Age 2040
Age 4160
Age 6170
Table 1.5: Cut-off scores indicating the boundary between normal-range and
impaired recognition of each emotion in the Ekman 60 Faces test.
Validity and reliability
Reliability of items in the Ekman 60 Faces test was assessed with split-half
reliabilities, using data from 50 participants. The 10 models whose facial expressions
are used as test items were arbitrarily assigned to one of two groups, each comprising
3 female and 2 male models. The scores of the 50 participants were then subdivided
into scores out of 5 for recognition of each of the 6 emotions from the photographs
posed by models in each group, and overall total scores out of 30 across the 6
emotions.
Correlations between participants' recognition of expressions posed by models from
each group are shown in Table 1.6. These split-half reliabilities are statistically
significant for total scores and for recognition of anger, disgust, fear, sadness, and
surprise. Recognition of happiness does not correlate significantly across the two sets
of faces because scores are at ceiling.
Total score
Anger
Disgust
Fear
Happiness
Sadness
Surprise
0.62
0.62
0.66
0.53
0.21
0.60
0.61
5.47
5.46
6.12
4.38
1.50
5.22
5.31
df
48
48
48
48
48
48
48
< .001
< .001
< .001
< .001
> .1
< .001
< .001
Table 1.6: Split-half reliabilities for scores on the Ekman 60 Faces test (r =
correlation, t = equivalent t-value, df = degrees of freedom, p = probability).
The validity of the items used in the Ekman 60 Faces test does not need to be
established here, since the Ekman and Friesen (1976) series has been the most
extensively used set of stimuli in research on recognition of facial expressions. We
did, however, examine the relation between our performance norms and Ekman and
Friesen's (1976) data.


To compare recognition rates for different emotions from the performance norms
collected for the FEEST to Ekman and Friesen's (1976) data, we converted scores in
the Ekman 60 Faces test to percentage recognition rates. Figure 1.3 shows the mean
percentage correct recognition for the six different emotions by Ekman and Friesen's
(1976) American college student participants and for FEEST control participants aged
2030 years.
Ekman and Friesen (1976)

FEEST age<30 years
100
80
60
40
20
0
Happiness
Surprise
Fear
Sadness
Disgust
Figure 1.3: Mean percentage recognition rates for different emotions by Ekman and
Friesen's (1976) participants and FEEST control participants aged 2030 years.
The FEEST data show good recognition of all emotions (over 80% correct in 6-way
forced-choice for all emotions except fear, whose recognition rate is 78% correct). The
patterns of relative difficulty of the different emotions are comparable across FEEST
and Ekman and Friesen's (1976) results, with happiness being the most easily
recognised. The overall level of performance is slightly higher in the Ekman and
Friesen (1976) data, but this observation should be qualified by noting that Ekman
and Friesen used a different procedure to establish recognition rates for each
photograph (see Section 6 of this Manual). In addition, the expressions used in the
Ekman 60 Faces test were initially selected to have high recognition rates. Choosing
stimuli in this way means that retesting will always be likely to lead to some degree of
downward performance regression due to any error variance in the original data. The
most important observation is that recognition rates are good in both sets of data.
Looking at recognition rates for each of the 60 photographs used as test items in the
Ekman 60 Faces test, there was a strong correlation between recognition rates from
the FEEST performance norms and the recognition rates reported by Ekman and
Friesen (r = 0.81, t = 10.35, df 58, p < .001).
Further evidence of the validity of the Ekman 60 Faces test as a test of facial
expression recognition comes from its proven usefulness in previous research (see
below).


Anger
Previous use of the Ekman 60 Faces test

A number of previously published studies have made use of variants of the Ekman 60
Faces test. In some studies the faces were presented for 3 seconds each
(Sprengelmeyer, Young, Calder, Karnat, Lange, Hmberg, Perrett and Rowland,
1996) or for unlimited time (Calder, Young, Rowland, Perrett, Hodges and Etcoff,
1996), rather than the 5 seconds used in the computer software version of the task on
the FEEST CD-ROM. Timing of control participants' responses has shown that more
than 99.5% of responses are made within 3 seconds, so the 5 seconds duration of each
stimulus is equivalent to unlimited presentation for neurologically normal
individuals.
The Ekman 60 Faces test has been used in group studies (Sprengelmeyer, Young,
Pundt, Sprengelmeyer, Calder, Berrios, et al., 1997; Evangeli and Broks, 2000) and in
studies of individual cases. We will provide some examples of case studies.
Table 1.7 illustrates two of the different ways of using overall scores on the Ekman 60
faces test, taken from studies of Mbius syndrome (Calder, Keane, Cole, Campbell
and Young, 2000a) and frontal variant frontotemporal dementia (Keane, Calder,
Hodges and Young, 2002).
Aetiology
Case
Age
Overall score
BC
LP
NC
23
36
27
47
47
46
Case K2
Case K3
Case K4
59
55
54
17
30
30
Mbius syndrome (Calder et al., 2000a)
Frontal variant frontotemporal dementia (Keane et al., in press)
Table 1.7: Overall total scores on the Ekman 60 Faces test for three participants
with Mbius syndrome and three with frontal variant frontotemporal dementia.
Scores marked in bold type are at or below the cut-off for the appropriate age
group.
Mbius syndrome is a rare congenital disorder that produces paralysis of the facial
muscles sufferers have an immobile, mask-like face. The research question
addressed in Calder et al.'s (2000a) study concerned whether individuals who are
unable to produce facial expressions themselves would none the less be able to
recognise the facial expressions made by other people. Although none of the three
participants with Mbius syndrome was particularly good at recognising facial
expressions, they all scored above the appropriate cut-off on the Ekman 60 Faces test,
demonstrating that inability to produce facial expressions does not result in complete
inability to recognise them.
Frontal variant frontotemporal dementia (fvFTD) is a progressive disorder in which
the typical focus of pathology appears to be the ventromedial frontal lobe. Patients are


typically brought to medical attention by their close relatives because of changes in

personality and social behaviour, of which the patients themselves appear to be
unaware. In some cases of fvFTD, testing with standard 'frontal' tests does not reveal
evidence of intellectual deterioration during the early stages of the disease. Table 1.7
shows the performance of three participants with fvFTD from Keane et al.'s (2002)
case series who showed severely impaired ability to recognise communicative signals
of emotion. In such circumstances, testing of facial expression recognition can
provide information pertinent to understanding deficits in everyday behaviour.
Table 1.8 shows how scores for recognising each of the six basic emotions in the
Ekman 60 Faces test can be used to separate emotion-specific forms of impairment in
which the recognition of certain emotions is differentially severely affected.
Aetiology
Case
Age
Anger
Disgust
Fear
Happiness Sadness
Surprise
Post-encephalitic without amygdala damage (Broks et al., 1998)

RS
48
10
3
4
5
10
10
10
8
7
8
9
10
9
Post-encephalitic with amygdala damage (Broks et al., 1998)

SE
YW
RB
64
53
61
10
7
6
10
7
9
Selective amygdala damage (Calder et al., 1996; Sprengelmeyer et al., 1999)

DR
NM
early 50s
50
7
7
8
10
5
2
10
10
9
5
9
9
10
10
8
4
0
8
10
10
10
9
10
8
8
Insula and basal ganglia lesion (Calder et al. 2000b)

NK
25
OCD cases (Sprengelmeyer et al., 1997)

Case S5
Case S7
34
40
Table 1.8: Scores for recognition of emotion in the Ekman 60 Faces test for
participants with brain dysfunction linked to different aetiologies. Scores marked in
bold type are at or below the cut-off for the appropriate age group.
Five of the research participants tabulated in Table 1.8 suffered amygdala damage
(SE, YW, RB, DR, and NM). In three cases (SE, YW, RB) this was associated with
medial temporal lobe pathology resulting from viral encephalitis (Broks, Young,
Maratos, Coffey, Calder, et al., 1998), but in two other cases (DR and NM) the lesions
were relatively circumscribed (Calder et al., 1996; Sprengelmeyer, Young, Schroeder,
Grossenbacher, Federlein, Bttner and Przuntek, 1999). For three of the five people
with amygdala damage, scores for the recognition of fear are at or below the


appropriate cut-off in the FEEST norms, and for the other two cases (DR and RB) the
scores for fear recognition are low even though they do not enter the region that can
be regarded as significantly impaired on the basis of this test alone.
These examples of poor recognition of fear after amygdala damage can be contrasted
with case RS (Broks et al., 1998), whose viral encephalitis spared the region of the
amygdala. RS showed good recognition of fear and the other basic emotions,
strengthening the putative link between amygdala function and fear recognition.
Table 1.8 contrasts the cases with amygdala damage and poor recognition of fear with
three research participants who showed impaired recognition of disgust (NK, Case S5,
Case S7).
NK (Calder, Keane, Manes, Antoun and Young, 2000b) was selected for testing
because his lesions involved the insula and part of the basal ganglia regions
identified in fMRI studies of neurologically normal participants as important for
processing facial expressions of disgust (Phillips, Young, Scott, Calder, Andrew,
Giampetro, et al., 1998; Phillips, Young, Senior, Brammer, Andrew, Calder, Bullmore
and Perrett, 1997). The Ekman 60 Faces Test data show selectively impaired
recognition of disgust.
The two cases of obsessive-compulsive disorder (OCD Case S5 and Case S7) are
taken from a group study in which people with OCD were recruited as participants
because neurophysiological and neuropsychological studies of OCD have highlighted
abnormalities in fronto-striatal regions (Sprengelmeyer et al., 1997). These two cases
also show clear selective deficits of disgust recognition.
These examples show some of the ways in which the Ekman 60 Faces test has already
been used, but there are many other circumstances in which testing of facial
expression recognition may provide useful information.
References
Broks, P., Young, A.W., Maratos, E.J., Coffey, P.J., Calder, A.J., Isaac, C.L., Mayes,
A.R., Hodges, J.R., Montaldi, D., Cezayirli, E., Roberts, N. and Hadley, D. (1998).
Face processing impairments after encephalitis: amygdala damage and
recognition of fear. Neuropsychologia, 36, 5970.
Calder, A.J., Keane, J., Cole, J., Campbell, R. and Young, A.W. (2000a). Facial
expression recognition by people with Mbius syndrome. Cognitive
Neuropsychology, 17, 7387.
Calder, A.J., Keane, J., Manes, F., Antoun, N. and Young, A.W. (2000b). Impaired
recognition and experience of disgust following brain injury. Nature
Neuroscience, 3, 10771078.
Calder, A.J., Young, A.W., Rowland, D., Perrett, D.I., Hodges, J.R. and Etcoff, N.L.
(1996). Facial emotion recognition after bilateral amygdala damage: differentially
severe impairment of fear. Cognitive Neuropsychology, 13, 699745.
Evangeli, M. and Broks, P. (2000). Face processing in schizophrenia: parallels with
the effects of amygdala damage. Cognitive Neuropsychiatry, 5, 81104.
Keane, J., Calder, A.J., Hodges, J.R. and Young, A.W. (2002). Face and emotion
processing in frontal variant frontotemporal dementia. Neuropsychologia, 40,
655665.


Nelson, H.E. (1991). National Adult Reading Test (NART): test manual (revised).
Windsor: NFER-Nelson.
Phillips, M.L., Young, A.W., Scott, S.K., Calder, A.J., Andrew, C., Giampetro, V.,
Williams, S.C.R., Bullmore, E.T., Brammer, M. and Gray, J.A. (1998). Neural
responses to facial and vocal expressions of fear and disgust. Proceedings of the
Royal Society: Biological Sciences, 265, 18091817.
Phillips, M.L., Young, A.W., Senior, C., Brammer, M., Andrew, C., Calder, A.J.,
Bullmore, E.T., Perrett, D.I., Rowland, D., Williams, S.C.R., Gray, J.A. and David,
A.S. (1997). A specific neural substrate for perceiving facial expressions of disgust.
Nature, 389, 495498.
Sprengelmeyer, R., Young, A.W., Calder, A.J., Karnat, A., Lange, H.W., Hmberg, V.,
Perrett, D.I. and Rowland, D. (1996). Loss of disgust: perception of faces and
emotions in Huntington's disease. Brain, 119, 16471665.
Sprengelmeyer, R., Young, A.W., Pundt, I., Sprengelmeyer, A., Calder, A.J., Berrios,
G., Winkel, R., Vollmoeller, W., Kuhn, W., Sartory, G. and Przuntek, H. (1997).
Disgust implicated in obsessive-compulsive disorder. Proceedings of the Royal
Society: Biological Sciences, B264, 17671773.
Sprengelmeyer, R., Young, A.W., Schroeder, U., Grossenbacher, P.G., Federlein, J.,
Bttner, T. and Przuntek, H. (1999). Knowing no fear. Proceedings of the Royal
Society: Biological Sciences, 266, 24512456.
Appendix 1: Files used in the Ekman 60 Faces test

This Appendix explains the system used to create names for the JPEG image files
used in the Ekman 60 Faces test, and how to locate files needed for the test on the
FEEST CD-ROM.
Filenames
Each filename is constructed in the following way:
ET_emoxx_nnn_id_ekid_per.jpg
where the ET prefix designates that this is an image from the Ekman 60 Faces test
(Practice images are prefixed ETP), emo indicates the emotion (Ang = anger, Dis =
disgust, Fea = fear, Hap = happiness, Sur = surprise), xx is used to order the
pictures of each emotion from 01 to 10, nnn is the picture number in the Ekman
and Friesen (1976) series, id gives the identifier for each model used at the top of
Figure 1.1, ekid is the identifier in the Ekman and Friesen series, and per is the
percentage recognition as the intended emotion in Ekman and Friesen's data.
Test files
The test begins with 6 practice trials. Files for these are:
ETP_Ang_025_M2_GS-2-08_070.jpg
ETP_Dis_027_M2_GS-2-25_084.jpg
ETP_Fea_024_M2_GS-1-25_077.jpg
ETP_Hap_022_M2_GS-1-08_096.jpg
ETP_Sad_023_M2_GS-2-01_071.jpg
ETP_Sur_026_M2_GS-1-16_100.jpg
The complete set of 60 files used for the test trials, and correct responses, is as
follows:


Filename
Correct response
ET_Ang01_010_F2_C-2-12_074.jpg
ET_Ang02_053_F4_MF-2-07_100.jpg
ET_Ang03_061_F5_MO-2-11_100.jpg
ET_Ang04_069_F6_NR-2-07_100.jpg
ET_Ang05_089_F7_PF-2-04_079.jpg
ET_Ang06_096_F8_SW-4-09_100.jpg
ET_Ang07_018_M1_EM-5-14_083.jpg
ET_Ang08_038_M4_JJ-3-12_076.jpg
ET_Ang09_080_M5_PE-2-21_083.jpg
ET_Ang10_105_M6_WF-3-01_100.jpg
ET_Dis01_012_F2_C-1-04_096.jpg
ET_Dis02_055_F4_MF-2-13_090.jpg
ET_Dis03_064_F5_MO-2-18_100.jpg
ET_Dis04_071_F6_NR-3-29_083.jpg
ET_Dis05_091_F7_PF-1-24_096.jpg
ET_Dis06_098_F8_SW-1-30_094.jpg
ET_Dis07_020_M1_EM-4-17_097.jpg
ET_Dis08_040_M4_JJ-3-20_088.jpg
ET_Dis09_082_M5_PE-4-05_090.jpg
ET_Dis10_108_M6_WF-3-11_097.jpg
ET_Fea01_009_F2_C-1-23_088.jpg
ET_Fea02_050_F4_MF-1-26_088.jpg
ET_Fea03_059_F5_MO-1-23_088.jpg
ET_Fea04_068_F6_NR-1-19_084.jpg
ET_Fea05_088_F7_PF-2-30_100.jpg
ET_Fea06_095_F8_SW-2-30_079.jpg
ET_Fea07_016_M1_EM-5-21_092.jpg
ET_Fea08_037_M4_JJ-5-13_096.jpg
ET_Fea09_079_M5_PE-3-21_092.jpg
ET_Fea10_104_M6_WF-3-16_088.jpg
ET_Hap01_007_F2_C-2-18_099.jpg
ET_Hap02_048_F4_MF-1-06_100.jpg
ET_Hap03_057_F5_MO-1-04_100.jpg
ET_Hap04_066_F6_NR-1-06_092.jpg
ET_Hap05_085_F7_PF-1-06_100.jpg
ET_Hap06_093_F8_SW-3-09_100.jpg
ET_Hap07_014_M1_EM-4-07_100.jpg
ET_Hap08_034_M4_JJ-4-07_100.jpg
ET_Hap09_074_M5_PE-2-12_100.jpg
ET_Hap10_101_M6_WF-2-12_100.jpg
ET_Sad01_008_F2_C-1-18_090.jpg
ET_Sad02_049_F4_MF-1-30_090.jpg
ET_Sad03_058_F5_MO-1-30_088.jpg
ET_Sad04_067_F6_NR-2-15_094.jpg
ET_Sad05_086_F7_PF-2-12_100.jpg
ET_Sad06_094_F8_SW-2-16_092.jpg
ET_Sad07_015_M1_EM-4-24_097.jpg
ET_Sad08_036_M4_JJ-5-05_093.jpg
Anger
Anger
Anger
Anger
Anger
Anger
Anger
Anger
Anger
Anger
Disgust
Disgust
Disgust
Disgust
Disgust
Disgust
Disgust
Disgust
Disgust
Disgust
Fear
Fear
Fear
Fear
Fear
Fear
Fear
Fear
Fear
Fear
Happiness
Happiness
Happiness
Happiness
Happiness
Happiness
Happiness
Happiness
Happiness
Happiness
Sadness
Sadness
Sadness
Sadness
Sadness
Sadness
Sadness
Sadness


ET_Sad09_075_M5_PE-2-31_074.jpg
ET_Sad10_102_M6_WF-3-28_079.jpg
ET_Sur01_011_F2_C-1-10_094.jpg
ET_Sur02_054_F4_MF-1-09_096.jpg
ET_Sur03_063_F5_MO-1-14_090.jpg
ET_Sur04_070_F6_NR-1-14_081.jpg
ET_Sur05_090_F7_PF-1-16_093.jpg
ET_Sur06_097_F8_SW-1-16_100.jpg
ET_Sur07_019_M1_EM-2-11_091.jpg
ET_Sur08_039_M4_JJ-4-13_097.jpg
ET_Sur09_081_M5_PE-6-02_074.jpg
ET_Sur10_107_M6_WF-2-16_091.jpg
Sadness
Sadness
Surprise
Surprise
Surprise
Surprise
Surprise
Surprise
Surprise
Surprise
Surprise
Surprise
On each of the test trials, participants score 0 if they make the wrong response, 1 for
the correct response. The 6 practice trials are not scored.
The 60 test trials yield seven scores, which are automatically totalled by the software
supplied with the CD-ROM:
Anger a score from 010 for the number of anger faces correctly identified.
Disgust a score from 010 for the number of disgust faces correctly identified.
Fear a score from 010 for the number of fear faces correctly identified.
Happiness a score from 010 for the number of happiness faces correctly
identified.
Sadness a score from 010 for the number of sadness faces correctly identified.
Surprise a score from 010 for the number of surprise faces correctly identified.
Total the sum of the above 6 scores.
Duplicate files
The image files listed above are only available to the program supplied with the CDROM, and cannot be copied from the CD. However, a duplicate set is provided for
those who wish to program their own test. These images are encrypted on the CDROM, but can be copied to a hard disk using the software supplied. Their filenames
are the same as those used by the Ekman 60 Faces test program software, except that
they begin with the prefix Ek instead of ET.
The path to the image directory is:
FEEST_Stimuli/Images/Section_1_Ekman_60_faces_test/ Ekman_60_images/
The image directory has seven subdirectories:
Ek60P_Practice images
This directory contains the six practice images:
EkP_Ang_025_M2_GS-2-08_070.jpg
EkP_Dis_027_M2_GS-2-25_084.jpg
EkP_Fea_024_M2_GS-1-25_077.jpg
EkP_Hap_022_M2_GS-1-08_096.jpg
EkP_Sad_023_M2_GS-2-01_071.jpg
EkP_Sur_026_M2_GS-1-16_100.jpg
Ek60T_Anger
This directory contains the ten anger images:
Ek_Ang01_010_F2_C-2-12_074.jpg
Ek_Ang02_053_F4_MF-2-07_100.jpg


Ek_Ang03_061_F5_MO-2-11_100.jpg
Ek_Ang04_069_F6_NR-2-07_100.jpg
Ek_Ang05_089_F7_PF-2-04_079.jpg
Ek_Ang06_096_F8_SW-4-09_100.jpg
Ek_Ang07_018_M1_EM-5-14_083.jpg
Ek_Ang08_038_M4_JJ-3-12_076.jpg
Ek_Ang09_080_M5_PE-2-21_083.jpg
Ek_Ang10_105_M6_WF-3-01_100.jpg
Ek60T_Disgust
This directory contains the ten disgust images:
Ek_Dis01_012_F2_C-1-04_096.jpg
Ek_Dis02_055_F4_MF-2-13_090.jpg
Ek_Dis03_064_F5_MO-2-18_100.jpg
Ek_Dis04_071_F6_NR-3-29_083.jpg
Ek_Dis05_091_F7_PF-1-24_096.jpg
Ek_Dis06_098_F8_SW-1-30_094.jpg
Ek_Dis07_020_M1_EM-4-17_097.jpg
Ek_Dis08_040_M4_JJ-3-20_088.jpg
Ek_Dis09_082_M5_PE-4-05_090.jpg
Ek_Dis10_108_M6_WF-3-11_097.jpg
Ek60T_Fear
This directory contains the ten fear images:
Ek_Fea01_009_F2_C-1-23_088.jpg
Ek_Fea02_050_F4_MF-1-26_088.jpg
Ek_Fea03_059_F5_MO-1-23_088.jpg
Ek_Fea04_068_F6_NR-1-19_084.jpg
Ek_Fea05_088_F7_PF-2-30_100.jpg
Ek_Fea06_095_F8_SW-2-30_079.jpg
Ek_Fea07_016_M1_EM-5-21_092.jpg
Ek_Fea08_037_M4_JJ-5-13_096.jpg
Ek_Fea09_079_M5_PE-3-21_092.jpg
Ek_Fea10_104_M6_WF-3-16_088.jpg
Ek60T_Happiness
This directory contains the ten happiness images:
Ek_Hap01_007_F2_C-2-18_099.jpg
Ek_Hap02_048_F4_MF-1-06_100.jpg
Ek_Hap03_057_F5_MO-1-04_100.jpg
Ek_Hap04_066_F6_NR-1-06_092.jpg
Ek_Hap05_085_F7_PF-1-06_100.jpg
Ek_Hap06_093_F8_SW-3-09_100.jpg
Ek_Hap07_014_M1_EM-4-07_100.jpg
Ek_Hap08_034_M4_JJ-4-07_100.jpg
Ek_Hap09_074_M5_PE-2-12_100.jpg
Ek_Hap10_101_M6_WF-2-12_100.jpg
Ek60T_Sadness
This directory contains the ten sadness images:
Ek_Sad01_008_F2_C-1-18_090.jpg
Ek_Sad02_049_F4_MF-1-30_090.jpg
Ek_Sad03_058_F5_MO-1-30_088.jpg


Ek_Sad04_067_F6_NR-2-15_094.jpg
Ek_Sad05_086_F7_PF-2-12_100.jpg
Ek_Sad06_094_F8_SW-2-16_092.jpg
Ek_Sad07_015_M1_EM-4-24_097.jpg
Ek_Sad08_036_M4_JJ-5-05_093.jpg
Ek_Sad09_075_M5_PE-2-31_074.jpg
Ek_Sad10_102_M6_WF-3-28_079.jpg
Ek60T_Surprise
This directory contains the ten surprise images:
Ek_Sur01_011_F2_C-1-10_094.jpg
Ek_Sur02_054_F4_MF-1-09_096.jpg
Ek_Sur03_063_F5_MO-1-14_090.jpg
Ek_Sur04_070_F6_NR-1-14_081.jpg
Ek_Sur05_090_F7_PF-1-16_093.jpg
Ek_Sur06_097_F8_SW-1-16_100.jpg
Ek_Sur07_019_M1_EM-2-11_091.jpg
Ek_Sur08_039_M4_JJ-4-13_097.jpg
Ek_Sur09_081_M5_PE-6-02_074.jpg
Ek_Sur10_107_M6_WF-2-16_091.jpg


Section 2: The Emotion Hexagon test ...............................................................................2

Stimuli.............................................................................................................................2
Procedure........................................................................................................................ 7
Performance norms ........................................................................................................8
Graphical representation of results............................................................................8
Overall scores ............................................................................................................ 10
Scores for each emotion............................................................................................ 12
Validity and reliability .................................................................................................. 14
Previous use of the Emotion Hexagon test .................................................................. 16
References..................................................................................................................... 19
Appendix 2: Files used in the Emotion Hexagon test ..................................................... 19
Filenames...................................................................................................................... 19
Test files ....................................................................................................................20
Duplicate files............................................................................................................ 21


Section 2 page 1 of 22
Section 2: The Emotion Hexagon test

The Emotion Hexagon test uses stimuli of graded difficulty, created using computer
image manipulation techniques. The rationale underlying the Emotion Hexagon is
that whereas the Ekman and Friesen (1976) faces were developed to provide relatively
unambiguous examples of each emotion, in some circumstances it is useful to be able
to test recognition of less clear-cut expressions. In the Emotion Hexagon test,
computer morphing is used to modify photographs from the Ekman and Friesen
(1976) series, creating examples that lie close to or more distant from the prototype
expression.
The test involves recognition of six basic emotions; anger, disgust, fear, happiness,
sadness, and surprise. Results can be represented in graphical form, or converted to
scores out of a maximum of 120 for overall performance and scores out of 20 for
recognition of the six emotions.
The FEEST CD-ROM includes a computer program for presenting the Emotion
Hexagon test, recording and scoring responses, and saving these as an Excelcompatible spreadsheet file. Comparison data are available for a group of 125
participants.
Stimuli
The Ekman and Friesen (1976) series contains examples of facial expressions of six
different basic emotions (anger, disgust, fear, happiness, sadness, and surprise).
Using Ekman and Friesen's (1976) norms, we plotted a confusion matrix for the
different emotions, and then ordered them in a series based on their maximum
confusabilities placing each emotion adjacent to the one it was most likely to be
confused with. The result ran happiness surprise fear sadness disgust
anger, with mean percentage confusabilities for each pair of expressions in this
sequence being happiness and surprise 0.8%, surprise and fear 5.8%, fear and
sadness 2.4%, sadness and disgust 2.7%, disgust and anger 6.4%. The ends of the
sequence (anger and happiness) were then joined to create a hexagonal
representation, as shown in Figure 2.1.


Figure 2.1: Hexagonal representation of confusabilities of basic emotions. Emotions

most likely to have their facial expressions mistaken for each other are placed at
adjacent points on the perimeter.
This hexagonal representation is related to the schemas used in two-dimensional
accounts of facial expression perception. For example, a similar representation of the
confusability of facial expressions was developed by Woodworth and Schlosberg
(1954), though it reverses the relative positions of anger and disgust. However, we do
not wish to imply here that the hexagonal representation directly reflects the
processes used in perceiving facial expressions it is simply a convenient way of
showing what tends to be confused with what.
Morphed images were created for the six continua that lie around the perimeter of
this hexagon (happiness surprise, surprise fear, fear sadness, sadness disgust,
disgust anger, anger happiness). These continua therefore include those between
maximally confusable emotions.
Face JJ from the Ekman and Friesen (1976) series was chosen for morphing because
the photographs of all six emotional facial expressions (happiness, surprise, fear,
sadness, disgust, and anger) were of consistent quality, with reasonably standardised
pose and lighting. To describe the morphing procedure, we refer to these pictures as
the prototype expressions of Happiness, Surprise, Fear, Sadness, Disgust, and Anger.
The numbers and identifiers of the selected prototype expression images in the
Ekman and Friesen (1976) series are given in Table 2.1, which also shows percentage
recognition rates for these six expressions by Ekman and Friesen's (1976) college
student sample. Details of the muscle movements (Action Units) involved in each
prototype expression can be found in Section 6 of this Manual.


Number E&F id
Intended Percentage recognition rates

emotion
Happiness
Surprise
Fear
Sadness
Disgust
Anger
100
31
34
JJ-4-07 Happiness
39
JJ-4-13 Surprise
97
30
37
JJ-5-13 Fear
96
25
36
JJ-5-05 Sadness
93
30
40
JJ-3-20 Disgust
12
88
33
38
JJ-3-12 Anger
15
76
33
Table 2.1: Numbers and identifiers (E&F id) in the Ekman and Friesen (1976) series
for the prototype images used to create the Emotion Hexagon, with percentage
recognition rates for these six expressions by Ekman and Friesen's (1976) college
student sample (N = number of judges for each face).
Photographic-quality continua were made, with five morphed images for each
continuum. These were prepared by blending between two prototype expressions
posed by JJ (e.g., Happiness and Surprise) in proportions 90:10 (i.e., 90% Happiness
10% Surprise for the happiness surprise continuum), 70:30 (70% Happiness 30%
Surprise), 50:50 (50% Happiness 50% Surprise), 30:70 (30% Happiness 70%
Surprise), and 10:90 (10% Happiness 90% Surprise). These correspond to 90%, 70%,
50%, 30%, and 10% morphs along the appropriate continuum (in our example,
happiness surprise).
The preparation of each continuum was done in the same way. Preparation of the
happiness surprise continuum will be described, to illustrate the process. The
procedure involved three stages. Further technical details can be found in Section 7 of
this Manual.
Stage 1: delineation: One hundred and eighty six reference points were positioned
manually onto a digitised image of the Happiness prototype expression photograph of
JJ's face from the Ekman and Friesen (1976) series. The locations of these reference
points were specified in terms of anatomical landmarks, with each facial feature
represented by a set number of points; for example, the mouth was represented by 22
points, and each eyebrow by 8 points. These reference points were then joined to
produce a delineated representation comprising 50 feature contours. Exactly the
same method was applied to a digitised image of JJ's Surprise prototype photograph.
Hence, across the two prototype expressions (Happiness and Surprise) there was
conformity with respect to the anatomical positioning of the 186 reference points on
each face, but not always their exact spatial positions; for example, the eyebrows were
raised in the Surprise but not the Happiness prototype, whereas the shape of the
hairline was the same in both.
Stage 2: shape interpolation: A continuum of face shapes was generated between the
two delineated prototype face shapes (in our example, JJ Happiness and JJ Surprise).
This was achieved by taking the delineation data for the two prototype images and
calculating the vector difference for each landmark. For example, consider the
reference point positioned at the tip of the nose; this has a location on the JJ


Happiness prototype of (x1,y1) and a location on the JJ Surprise prototype of (x2,y2).

Equations describing the vector from (x1,y1) to (x2,y2) were used to obtain positions
for the point at the tip of the nose which moved 10%, 30%, 50%, 70%, and 90% along
a straight line from the location of that reference point in JJ's Happiness prototype
(x1,y1) to the location of that reference point in JJ's Surprise prototype (x2,y2). This
process was repeated for each of the 186 feature reference points, to generate the 5
face shapes which would interpolate at 10%, 30%, 50%, 70%, and 90% distances
between the two original prototype face shapes.
Stage 3: producing a continuous-tone image: The final stage created a continuoustone (photographic quality) image for each of these interpolated face shapes. This was
achieved by taking both of the prototype faces and 'warping' or 'stretching' them (as if
they were printed on a rubber sheet) to the new shape, so that all points representing
the same feature were aligned across images. The two faces, now with the same
intermediary face shape, were then blended with the appropriate weight. For example,
in the 90% Happiness 10% Surprise morph, the pixel intensities in each tessellation
were arrived at by deforming the Happiness prototype face 10% toward the Surprise
prototype, and the Surprise prototype face 90% toward the Happiness prototype, and
then blending the grey levels in these two contributory images in the ratio nine parts
from the Happiness prototype to one part from the Surprise prototype.




Figure 2.2: Expression continua used in the Emotion Hexagon test. From left to
right, the columns show 90%, 70%, 50%, 30% and 10% morphs along each
continuum. In each case, information from the prototype (Ekman and Friesen
series) expression of the emotion at each end of the continuum is blended in the
proportions shown at the top of the Figure. From top to bottom, the continua shown
in each row are happiness (H) surprise (U), surprise (U) fear (F), fear (F)
sadness (S), sadness (S) disgust (D), disgust (D) anger (A), and anger (A)
happiness (H).
The resulting morphed faces are shown in Figure 2.2. In total, there are 30 images (5
from each of 6 continua). Moving from left to right in Figure 2.2, the columns show
90%, 70%, 50%, 30% and 10% morphs along each continuum. Note that a 90%
morph on the happiness surprise continuum would be the same as a 10% morph on
a surprise happiness continuum, and that the prototype expressions are not shown
in Figure 2.2. Moving from top to bottom of Figure 2.2, the rows show the happiness
surprise continuum (top row), surprise fear (second row), fear sadness (third
row), sadness disgust (fourth row), disgust anger (fifth row), and anger
happiness (bottom row).
Procedure
A computer program for running the Emotion Hexagon Test is included on the
FEEST CD-ROM, and separate instructions are provided for this software. The faces
are presented one at a time for 5 seconds each, followed by a blank screen. The
participant is asked to decide which of the emotion names (happiness, sadness,
surprise, disgust, anger, and fear) best describes the facial expression shown. The
names of these six emotions are visible on the computer screen throughout the test,
with the order in which the emotion names are shown on the screen randomised each
time the test is given. Before commencing the test, you should satisfy yourself that
your participant understands the meanings of these emotion words sufficiently
accurately for the results to be meaningful (for example, by asking for examples of
circumstances in which people would experience fear, anger, disgust, etc.). Responses
can be recorded from mouse-clicks to the on-screen buttons, or via the computer
keyboard. The test is not timed participants can take as long as they wish to decide
on the emotion.
The test involves a practice block of 30 trials, followed by 5 test blocks of 30 trials
each. In each block of trials the 30 images shown in Figure 2.2 are presented once
each, in random order. The software allows responses to be saved to an Excelcompatible spreadsheet file that records the trial number, the stimulus filename
(spreadsheet column C), an identifier that allows the results to be sorted (column D),
and the response made.
Data from the practice block of trials are not analysed. This leaves data for 5 blocks of
30 trials. These can be represented in graph form, or analysed numerically using
procedures described below (Performance norms).
The filenames for the stimuli used in the Emotion Hexagon test number the pictures
in Figure 2.2 from 01 to 30, proceeding from left to right and top to bottom of the
Figure. This is equivalent to numbering them from 1 to 30 starting at 12 o'clock
(happiness) and proceeding clockwise around the perimeter of Figure 2.1. Each
filename is constructed in the following way:
HTxx_M4_JJ_ex1_pp1_ex2_pp2.jpg
where HT designates that this is an image from the Emotion Hexagon test, xx is
used to order the pictures from 01 to 30, M4 is the identifier for the model (JJ)


used at the top of Figure 1.1, JJ designates the model in the Ekman and Friesen
series, ex1 and ex2 indicate the blended prototype expressions (Ang = Anger, Dis =
Disgust, Fea = Fear, Hap = Happiness, Sad = Sadness, Sur = Surprise), and pp1
and pp2 indicate the percentages of each prototype expression in the blend.
For example: the filename HT20_M4_JJ_Sad_10%_Dis_90%.jpg designates the
20th picture in Figure 2.2 (the image in the 5th column of the 4th row). The model is
identified as M4 in Figure 1.1, who is JJ in the Ekman and Friesen series. The picture
is a blend of 10% Sadness and 90% Disgust prototypes.
A complete list of filenames for the stimuli used in the Emotion Hexagon test is given
as an Appendix to the present Section of the FEEST Manual. These files cannot be
removed from the CD-ROM, but the Appendix gives the location of a duplicate set of
files suitable for use by those who prefer to program their own version of the test. This
can be useful if it is considered necessary to adjust the procedure for example, to get
more detailed information by recording reaction times as well as accuracy, or to add a
masking stimulus to the end of each trial to inhibit time-consuming strategies (see
below Previous use of the Emotion Hexagon test).
Performance norms
Note that the prototype face for each expression is never shown in the Emotion
Hexagon test; all the stimuli are morphs. Note too that the morphs are presented in
random order in each block of trials; they are not grouped into the underlying
continua.
Data from the Emotion Hexagon test can be represented in graph form, or analysed
numerically using overall total scores or scores for recognition of each emotion. We
will describe these methods in turn.
Graphical representation of results
The 5 blocks of 30 test trials can be used to derive a score out of a maximum possible
5 for a participant's recognition of each of the 30 test images as each of the 6
emotions. For example, if we consider the top left image in Figure 2.2 (image 01 in the
Emotion Hexagon series, filename HT01_M4_JJ_Hap_90%_Sur_10%.jpg), we can
look at how many times a participant classifies it as anger, disgust, fear, happiness,
sadness, or surprise across its 5 presentations (one presentation in each block of test
trials). Then we can do the same with the next image along the top row of Figure 2.2
(image 02 in the Emotion Hexagon series, filename
HT02_M4_JJ_Hap_70%_Sur_30%.jpg), and so on until the data for all 30 images
have been tabulated. To facilitate this, the filenames are constructed so that selecting
the data in columns AE of the output file and then sorting these by Column C
(filenames) will group together responses to each image.
The data can then be represented as a graph in which the 30 images used in the test
are located along the horizontal axis, as if lying along a continuum that traverses the
perimeter of Figure 2.1 (happiness surprise fear sadness disgust anger
happiness), and the vertical axis shows how often these 30 images are identified as
each of the 6 emotions (happiness, surprise, fear, sadness, disgust, or anger) across
the 5 presentations of each image.


Emotion Hexagon: Neurologically normal perceivers

5
4
Happiness
Surprise
Fear
Sadness
Disgust
Anger
3
2
1
0
Figure 2.3: Graph representing performance of the Emotion Hexagon test by 40

participants aged 2060 years. The 30 images used in the test (as read from left to
right and top to bottom of Figure 2.2) are located along the horizontal axis, as if
lying along a continuum that traverses the perimeter of Figure 2.1 (Happiness
Surprise Fear Sadness Disgust Anger Happiness). Because of the number
of images lying along the horizontal axis, not all are labelled; in the horizontal axis
labels, H = Happiness, U = Surprise, F = Fear, S = Sadness, D = Disgust, A = Anger,
and the numbers indicate the percentages of each emotion in the morphed image.
The vertical axis shows means for how often these 30 images were identified as each
of the 6 emotions (happiness, surprise, fear, sadness, disgust, or anger) across the 5
presentations of each image used in the test (i.e. excluding practice trials).
Figure 2.3 shows a graph for the performance of 40 neurologically normal controls
aged 2060 years (Sprengelmeyer, Young, Pundt, Sprengelmeyer, Calder, Berrios, et
al., 1997a). It is clear that each of the six basic emotions has a distinct region of the
Emotion Hexagon in which it is primarily perceived. Interestingly, the 50% images
seem to be perceived as one or other of the emotions from which they are derived, not
as completely indeterminate. For example, around half the participants saw the 50%
Sadness 50% Disgust image (number 18 in the Emotion Hexagon series) as sadness,
and around half saw it as disgust, but no-one saw it as happiness, surprise, fear or
anger.
Although graphing the Emotion Hexagon test results can be informative (see Figures
2.3 and 2.6), it does not lend itself readily to statistical analysis. However, a simple
way of analysing the data follows from the observation that each emotion is perceived
in a distinct region of the Hexagon. In general, morphs that are based on 90% and
70% of a particular prototype emotion are usually recognised as that emotion. Thus in
Figure 2.2, the two top left images (90% Happiness 10% Surprise, and 70%
Happiness 30% Surprise) are usually seen as happiness, and the two bottom right
images (30% Anger 70% Happiness, and 10% Anger 90% Happiness) are seen as
happiness too. Of the 30 Hexagon images, 24 are consistently seen as one of the six
basic emotions (4 images for each emotion), whereas the remaining 6 images (the
50% morphs that form the central column of Figure 2.2) are more variably perceived.
Column D of the software output file adds a code that can be used to group together
the 90% and 70% morphs for Happiness (H), Surprise (U), Fear (F), Sadness (S),


Disgust (D), and Anger (A). In Column D, the 50% morphs are all labelled X, so that
they can be separated from the images that should be consistently identified as a
particular basic emotion. Selecting the data in columns AE of the output file and
then sorting these by Column D will therefore group together responses to images
that should be recognised as happiness, surprise, fear, and so on. The CD-ROM
software automatically calculates a score out of a maximum of 20 (4 images usually
identified as a particular emotion, repeated across 5 blocks of test trials) for
recognition of each of the six emotions, and an overall total score out of a maximum
possible of 120 correctly recognised images.
Overall scores
As explained above, the 120 test trials with unambiguous stimuli (4 pictures for each
of the 6 emotions across the 5 test blocks) can be used to derive an overall (total)
score out of a possible maximum of 120 expressions correctly recognised, or accuracy
scores out of a possible maximum of 20 for each of the six emotions.
To assess performance of the Emotion Hexagon, we tested an opportunity sample of
128 individuals aged 2075. Where intelligence test scores were not known from
existing records, the NART-R (Nelson, 1991) was used to estimate intelligence.
Data for 3 people with IQs less than 90 were omitted because the sample did not
contain sufficient numbers to estimate performance reliably for this sub-grouping.
This left 125 people with IQs of 90 and above in the main sample.
To determine the effects of age, intelligence and sex on overall score, participants
were divided into sub-groups based on five age bands (2030 years, 3140 years,
4150 years, 5160 years, and 6175 years), four levels of intelligence (IQ 90100, IQ
101110, IQ 111120, and IQ over 120), and female or male sex. Means and standard
deviations of total scores across age, IQ, and sex are shown in Table 2.2, together with
the numbers of participants in each sub-grouping.


Mean
SD
Entire group
107.97
9.51
125
Age 2030
109.00
8.75
28
Age 3140
109.36
10.32
23
Age 4150
108.80
9.87
29
Age 5160
107.14
9.35
22
Age 6175
105.09
9.41
23
IQ 90100
106.19
8.14
16
IQ 101110
107.76
10.74
46
IQ 111120
107.78
9.90
45
IQ > 120
110.56
5.73
18
Female
109.49
9.28
63
Male
106.42
9.56
62
Emotion Hexagon Test for the entire comparison group of 125 individuals, and for
sub-groupings based on age, intelligence and sex. Numbers (N) of participants in
each group are also shown.
A three-factor analysis of variance of the effects of Age (5 levels), Intelligence (4
levels) and Sex (female or male) showed no significant main effects or interactions
(all probabilities > .1). However, to maintain comparability with the Ekman 60 Faces
test (Section 1 of the FEEST Manual), the data were subdivided into age bands of
2040 years, 4160 years, and over 60 (6175) years, as shown in Table 2.3.


Mean
SD
cutoff score
Entire group
107.97
9.51
125
92
Age 2040
109.16
9.37
51
94
Age 4160
108.10
9.60
51
92
Age 6175
105.09
9.41
23
90
Emotion Hexagon Test for the entire comparison group of 125 individuals, and for
sub-groupings based on age. Numbers (N) of participants in each sub-group are
shown. A level of performance falling 1.65 SDs below the mean has been used to
derive a cut-off score indicating the boundary between normal-range and impaired
performance for each group.
for each sub-group were established using the nearest integer score to a z value of 1.65
(p = .05). These cut-off scores are shown in Table 2.3. They are also reproduced in
Table 0.3, which may be an easier place to find them if they need to be consulted
frequently.
Scores for each emotion
The Emotion Hexagon test also yields scores out of a maximum possible 20 correct
responses for recognition of each of the 6 emotions (anger, disgust, fear, happiness,
sadness, surprise).
As for the Ekman 60 Faces test, a two-factor analysis of variance was used to
investigate the effects of Age (5 levels: 2030 years, 3140 years, 4150 years, 5160
years, and 6175 years) and Emotion (6 levels: anger, disgust, fear, happiness,
sadness, surprise) on performance of the Emotion Hexagon test. Mauchly's test of
sphericity showed a significant violation of the assumption of sphericity (W = .423,
Chi-square = 101.67, df 14, p < .001), again due in large part to the near-ceiling
performance for recognition of happiness, so degrees of freedom for tests involving
the Emotion factor were adjusted using the Greenhouse-Geisser correction.
The analysis of variance showed a significant main effect of Emotion (F = 16.83, df 4,
487, p < 0.001), with some emotions being easier to recognise than others. The main
effect of Age was non-significant (F < 1), but there was an Age x Emotion interaction
(F = 2.31, df 16, 487, p < .01). This interaction is shown in Figure 2.4, which makes
clear that recognition of some emotions changes across age. In particular, recognition
of fear declines, but recognition of disgust shows a slight improvement. The data in
Figure 2.4 are given as percentages, to facilitate comparison with Figure 1.2 (Section
1), which shows the equivalent interaction for the Ekman 60 Faces test.


Emotion Hexagon test
Percent correct recognition
100
80
Anger
Disgust
Fear
Happiness
Sadness
Surprise
60
40
20
0
Age 20-30
Age 31-40
Age 41-50
Age 51-60
Age 61-75
Figure 2.4: Emotion Hexagon test: Mean percent correct recognition of facial
expressions of anger, disgust, fear, happiness, sadness and surprise at different
ages.
Because of the Age x Emotion interaction, data for recognition of each emotion in the
Emotion Hexagon test were subdivided into age bands of 2040 years, 4160 years,
and 6175 years, as had been done with the overall scores. Table 2.4 gives means and
standard deviations of scores for recognition of each emotion for the entire
comparison group of 125 individuals, and mean scores for the sub-groupings based on
age.
Anger
Disgust
Fear
Happiness
Sadness
Surprise
Mean
17.84
18.01
16.56
19.64
18.38
17.69
SD
2.80
3.65
3.76
0.80
3.42
2.16
Age 2040
Mean
18.38
16.88
17.74
19.74
18.66
18.10
Age 4160
Mean
17.63
18.71
16.15
19.63
18.31
17.65
Age 6175
Mean
17.13
18.83
14.91
19.43
17.91
16.87
Entire group
Table 2.4: Means and standard deviations (SD) of scores for recognition of each
emotion (Max = 20) in the Emotion Hexagon test for the entire comparison group of
125 individuals, and means for sub-groupings based on age.
for each sub-group were again established at the nearest integer score to a z value of
1.65 (p = .05), using the standard deviation of the entire group to estimate the degree
of variability of recognition of each emotion. These cut-off scores are presented in
Table 2.5 and reproduced in Table 0.3.


Anger
Disgust
Fear
Happiness
Sadness
Surprise
Entire group
13
12
10
18
13
14
Age 2040
14
11
12
18
13
15
Age 4160
13
13
10
18
13
14
Age 6175
13
13
18
12
13
Table 2.5: Cut-off scores indicating the boundary between normal-range and
impaired recognition of each emotion in the Emotion Hexagon test.
Validity and reliability
Reliability of items in the Emotion Hexagon test was established with split-half
reliabilities, using data from 40 participants. From the set of 30 images used as
stimuli in the Emotion Hexagon, 4 are reliably recognised as each of the 6 emotions,
and the remaining 6 images (the 50% morphs) are only used for graphing results. By
assigning each of the 4 images reliably recognised as each emotion to one of two
groups, we could calculate scores out of a maximum of 10 (2 images in 5 blocks of test
trials) for recognition of anger, disgust, fear, happiness, sadness and surprise from
images in each group, and total scores out of a maximum of 60.
Correlations between participants' recognition of expressions using images from each
group are shown in Table 2.6. These split-half reliabilities are statistically significant
for total scores and for recognition of anger, disgust, fear, sadness, and surprise.
Recognition of happiness does not correlate significantly across the two sets of faces
because scores are at ceiling.
Total score
Anger
Disgust
Fear
Happiness
Sadness
Surprise
0.92
0.68
0.92
0.88
0.18
0.65
0.33
14.60
5.66
14.56
11.61
1.15
5.23
2.14
df
38
38
38
38
38
38
38
< .001
< .001
< .001
< .001
> .1
< .001
< .001
Table 2.6: Split-half reliabilities for scores on the Emotion Hexagon test (r =
correlation, t = equivalent tvalue, df = degrees of freedom, p = probability).
Evidence of the validity of the Emotion Hexagon test comes from its proven
usefulness in previous research (see below), and from comparing its results to the
Ekman 60 Faces test. The Emotion Hexagon and Ekman 60 Faces tests use different
procedures, so perfect agreement between the results of the tests would not be
expected, but they should be significantly related.
To compare recognition rates for different emotions in the Emotion Hexagon test to
scores from the Ekman 60 Faces test, we calculated percentage recognition rates
across the entire comparison group for each test. These are shown in Figure 2.5.


Ekman 60
Emotion hexagon
100
80
60
40
20
Anger
Disgust
Fear
Happiness Sadness
Surprise
Figure 2.5: Mean percentage recognition rates for different emotions in the Ekman
60 Faces and Emotion Hexagon tests.
The data show good recognition of all emotions. The overall level of performance is
slightly higher in the Emotion Hexagon test, but the patterns of relative difficulty of
the different emotions are comparable across the two tests, with happiness being the
most easily recognised.
Total score
0.68
Anger
0.51
Disgust
0.27
Fear
0.52
Happiness
-0.05
Sadness
0.54
Surprise
0.42
7.48
4.78
2.26
4.91
-0.40
5.17
3.73
df
65
65
65
65
65
65
65
< .001
< .001
< .05
< .001
> .1
< .001
< .001
Table 2.7: Correlation of scores in the Ekman 60 Faces and Emotion Hexagon tests
across 67 participants (r = correlation, t = equivalent t-value, df = degrees of
freedom, p = probability).
Sixty-seven participants from our comparison group had undertaken both tests.
Correlations of their scores in the Ekman 60 Faces test and the Emotion Hexagon test
are shown in Table 2.7. These correlations are statistically significant for total scores
and for all emotions except happiness, for which the correlation is restricted by nearceiling performance on both tests.


Previous use of the Emotion Hexagon test

In its original form, the Emotion Hexagon test presented the morphed image for 2
seconds, after which it was replaced by a picture of JJ posing a neutral expression
(Calder, Young, Rowland, Perrett, Hodges and Etcoff, 1996). The rationale for the
limited presentation time and masking stimulus was to inhibit the use of timeconsuming but potentially effective strategies by participants who have become very
used to forced-choice testing. For example, when an expression is not recognised, a
person who is not very good at recognising a certain emotion may come to realise that
it is a good idea to offer the forced-choice option that doesn't seem to be needed for
the other faces, rather than simply guessing.
Because it is sometimes desirable to minimise the use of such strategies, the neutral
masking stimulus (face 41 JJ-3-04 in the Ekman and Friesen series) is included with
the FEEST emotion hexagon images for those who wish to set up their own version of
the task. However, in most circumstances a masking stimulus is not necessary and the
CD-ROM software adopts the 5 seconds unmasked presentation used in
Sprengelmeyer's studies (Sprengelmeyer, Young, Calder, Karnat, Lange, Hmberg,
Perrett and Rowland, 1996; Sprengelmeyer et al., 1997a; Sprengelmeyer, Young,
Schroeder, Grossenbacher, Federlein, Bttner, and Przuntek, 1999; Sprengelmeyer,
Young, Sprengelmeyer, Calder, Rowland, Perrett, Hmberg and Lange, 1997b).
Like the Ekman 60 Faces test, the Emotion Hexagon test has been used in group
studies and in case studies. As an example of how overall scores in the Emotion
Hexagon test can be used, Table 2.8 shows data for the three participants with
Mbius syndrome (Calder, Keane, Cole, Campbell and Young, 2000) whose overall
scores on the Ekman 60 Faces test were given in Table 1.6. Although none of the three
participants with Mbius syndrome was particularly good at recognising facial
expressions in the Ekman 60 Faces test, Table 1.6 showed that they all scored above
the appropriate cut-off. With the Emotion Hexagon, BP was again just above the cutoff, LP scored close to the mean for the appropriate age group, but NC was clearly
impaired. These results show the value of having data from more than one test in
reaching conclusions about facial expression perception.
Case
Age
Overall score
BC
23
95
LP
36
106
NC
27
83
Table 2.8: Overall total scores on the Emotion Hexagon test for three participants
with Mbius syndrome. Scores marked in bold type are at or below the cut-off for
the appropriate age group.
Figure 2.6 and Table 2.9 show the use of the Emotion Hexagon to investigate different
forms of impairment in which certain emotions are differentially severely affected.
They present data for NM, who has amygdala damage (Sprengelmeyer et al., 1999),
and Cases S5 and S7 from the series of participants with obsessive-compulsive
disorder (OCD) studied by Sprengelmeyer et al. (1997a). Performance of the Ekman
60 Faces test for these participants was shown in Table 1.7; NM was poor at
recognising fear, whereas S5 and S7 were impaired at recognising disgust.


Emotion Hexagon: NM
5
4
Happiness
Surprise
Fear
Sadness
Disgust
Anger
3
2
1
0
5
Happiness
Surprise
Fear
Sadness
Disgust
Anger
4
3
2
1
A30 H70
A70 H30
D10 A90
D50 A50
D90 A10

S30 D70
S70 D30
F10 S90
F50 S50
F90 S10
U30 F70
U70 F30
H10 U90
H50 U50
H90 U10
Number of times identified
Emotion Hexagon: Case S5

Emotion Hexagon: Case S7

5
4
Happiness
Surprise
Fear
Sadness
Disgust
Anger
3
2
1
0
Figure 2.6: Recognition of emotion in the Emotion Hexagon test for participants
with amygdala damage (NM) or obsessive-compulsive disorder (Cases S5 and S7).
Comparable findings were obtained with the Emotion Hexagon. By comparing the
graphs from Figure 2.6 with the findings for neurologically normal participants
shown in Figure 2.3, it is clear that NM is primarily impaired in the region where
neurologically normal perceivers see fear (although he also consistently classifies the
50% Anger 50% Happiness image as surprise), whereas S5 and S7 have problems
seeing disgust (they tend to classify as anger the images normal perceivers see as
disgust).
Aetiology
Case
Age
Anger
Disgust
Fear
Happiness
Sadness
Surprise
20
19
19
19
Selective amygdala damage (Sprengelmeyer et al., 1999)

NM
50
20
OCD cases (Sprengelmeyer et al., 1997)

Case S5
34
20
18
20
20
18
Case S7
40
19
20
20
19
20
Table 2.9: Scores for recognition of emotion in the Emotion Hexagon test for
participants with amygdala damage (NM) or obsessive-compulsive disorder (Cases
S5 and S7). Scores marked in bold type are at or below the cut-off for the
appropriate age group.
The same pattern is clear from the Emotion Hexagon scores for recognition of each
emotion by these participants presented in Table 2.9, where NM scores below the cutoff for fear, and S5 and S7 score poorly at recognising disgust. These consistent
patterns of impairment across the Ekman 60 Faces and Emotion Hexagon tests are

impressive because performance of these tests is not particularly highly correlated in

normal perceivers (see Table 2.7). A common pattern of impairment on tests using
different procedures to investigate the same ability is a rigorous way to demonstrate
an underlying selective impairment (Young, Newcombe, de Haan, Small and Hay,
1993).
The Emotion Hexagon test can therefore be useful in its own right, or to provide an
additional source of information to the Ekman 60 Faces test. Taken together, the two
tests can point strongly to the existence of deficits affecting facial expression
recognition, and allow examination of whether such deficits are relatively global
(affecting recognition of all emotions) or circumscribed (affecting the recognition of
certain emotions with differential severity).
References
Calder, A.J., Keane, J., Cole, J., Campbell, R. and Young, A.W. (2000). Facial
expression recognition by people with Mbius syndrome. Cognitive
Calder, A.J., Young, A.W., Rowland, D., Perrett, D.I., Hodges, J.R. and Etcoff, N.L.
(1996). Facial emotion recognition after bilateral amygdala damage: differentially
severe impairment of fear. Cognitive Neuropsychology, 13, 699745.
Nelson, H.E. (1991). National Adult Reading Test (NART): test manual (revised).
Windsor: NFER-Nelson.
Sprengelmeyer, R., Young, A.W., Calder, A.J., Karnat, A., Lange, H.W., Hmberg, V.,
Perrett, D.I. and Rowland, D. (1996). Loss of disgust: perception of faces and
emotions in Huntington's disease. Brain, 119, 16471665.
Sprengelmeyer, R., Young, A.W., Pundt, I., Sprengelmeyer, A., Calder, A.J., Berrios,
G., Winkel, R., Vollmoeller, W., Kuhn, W., Sartory, G. and Przuntek, H. (1997a).
Disgust implicated in obsessive-compulsive disorder. Proceedings of the Royal
Society: Biological Sciences, B264, 17671773.
Sprengelmeyer, R., Young, A.W., Schroeder, U., Grossenbacher, P.G., Federlein, J.,
Bttner, T. and Przuntek, H. (1999). Knowing no fear. Proceedings of the Royal
Society: Biological Sciences, 266, 24512456.
Sprengelmeyer, R., Young, A.W., Sprengelmeyer, A., Calder, A.J., Rowland, D.,
Perrett, D.I., Hmberg, V. and Lange, H. (1997b). Recognition of facial
expressions: selective impairment of specific emotions in Huntington's disease.
Cognitive Neuropsychology, 14, 839879.
Woodworth, R.S. and Schlosberg, H. (1954). Experimental psychology: revised
edition. New York: Henry Holt.
Young, A.W., Newcombe, F., de Haan, E.H.F., Small, M. and Hay, D.C. (1993). Face
perception after brain injury: selective impairments affecting identity and
expression. Brain, 116, 941959.
Appendix 2: Files used in the Emotion Hexagon test

This Appendix explains the system used to create names for the JPEG image files
used in the Emotion Hexagon test, and how to locate files needed for the test on the
FEEST CD-ROM.
Filenames
Each filename is constructed in the following way:


HTxx_M4_JJ_ex1_pp1_ex2_pp2.jpg
where HT designates that this is an image from the Emotion Hexagon test, xx is
used to order the pictures from 01 to 30, M4 is the identifier for the model (JJ)
used at the top of Figure 1.1, JJ designates the model in the Ekman and Friesen
series, ex1 and ex2 indicate the blended prototype expressions (Ang = Anger, Dis =
Disgust, Fea = Fear, Hap = Happiness, Sad = Sadness, Sur = Surprise), and pp1
and pp2 indicate the percentages of each prototype expression in the blend.
Test files
The test has one block of 30 practice trials and 5 blocks of 30 test trials. Each block
uses the same set of 30 images, presented in random order. The filenames and correct
responses for the complete set of 30 files are as follows:
Filename
Correct response
HT01_M4_JJ_Hap_90%_Sur_10%.jpg
Happiness
Happiness
Surprise
Surprise
HT06_M4_JJ_Sur_90%_Fea_10%.jpg
Surprise
Surprise
Fear
Fear
HT11_M4_JJ_Fea_90%_Sad_10%.jpg
Fear
Fear
Sadness
Sadness
HT16_M4_JJ_Sad_90%_Dis_10%.jpg
Sadness
Sadness
Disgust
Disgust
HT21_M4_JJ_Dis_90%_Ang_10%.jpg
Disgust
Disgust
Anger


Anger
HT26_M4_JJ_Ang_90%_Hap_10%.jpg
Anger
Anger
Happiness
Happiness
The files marked 'X' are not scored, because they do not have an unambiguous correct
answer; they are only used if the data are plotted in a graph. Results from the 30
practice trials are also not scored.
On each of the remaining test trials (i.e., those not marked 'X'), participants score 0 if
they make the wrong response, 1 for the correct response. The test trials yield seven
scores, which are automatically totalled by the software supplied with the CD-ROM:
Anger a score from 020 for the number of anger faces correctly identified.
Disgust a score from 020 for the number of disgust faces correctly identified.
Fear a score from 020 for the number of fear faces correctly identified.
Happiness a score from 020 for the number of happiness faces correctly
identified.
Sadness a score from 020 for the number of sadness faces correctly identified.
Surprise a score from 020 for the number of surprise faces correctly
identified.
Total the sum of the above 6 scores.

Duplicate files
The image files listed above are only available to the program supplied with the CDROM, and cannot be copied from the CD. However, a duplicate set is provided for
those who wish to program their own test. These images are encrypted on the CDROM, but can be copied to a hard disk using the software supplied. Their filenames
are the same as those used by the Emotion Hexagon test program software, except
that they begin with the prefix Hex instead of HT.
FEEST_Stimuli/Images/Section_2_Emotion_hexagon_test/Hexagon_images/
The image directory has seven subdirectories:
Hex_Part_A_Happiness-Surprise
This directory contains the five images for the happiness-surprise continuum:
Hex01_M4_JJ_Hap_90%_Sur_10%.jpg
Hex_Part_B_Surprise-Fear
This directory contains the five images for the surprise-fear continuum:
Hex06_M4_JJ_Sur_90%_Fea_10%.jpg


Hex_Part_C_Fear-Sadness
This directory contains the five images for the fear-sadness continuum:
Hex11_M4_JJ_Fea_90%_Sad_10%.jpg
Hex_Part_D_Sadness-Disgust
This directory contains the five images for the sadness-disgust continuum:
Hex16_M4_JJ_Sad_90%_Dis_10%.jpg
Hex_Part_E_Disgust-Anger
This directory contains the five images for the disgust-anger continuum:
Hex21_M4_JJ_Dis_90%_Ang_10%.jpg
Hex_Part_F_Anger-Happiness
This directory contains the five images for the anger-happiness continuum:
Hex26_M4_JJ_Ang_90%_Hap_10%.jpg
Masking_image
This directory contains the neutral masking image used by Calder et al. (1996). It
is not needed if the test is set up for the standard procedure (5s presentation,
without masking) used by the FEEST software:
Hex00_M4_JJ_Masking_image.jpg


Section 3: Emotion Megamixes..........................................................................................2

The Emotion Megamix stimuli.......................................................................................2
Properties of the Emotion Megamix images..................................................................6
References..................................................................................................................... 14
Appendix 3: Organisation of the Emotion Megamix images on the CD-ROM............... 15


Section 3: Emotion Megamixes

The Emotion Hexagon test shows the usefulness of computer morphing of facial
expressions, but to create an easily administered test it only uses some of the possible
continua between expressions in the Ekman and Friesen (1976) series. The Emotion
Megamix stimuli depict all possible pairwise continua between prototype expressions
in the Ekman and Friesen (1976) series, in 10% steps. Separate sets of morphed
continua are available for two models MO and JJ.
These stimuli can be used to construct new tasks for assessment of different forms of
facial expression recognition impairment, for studies of categorical perception, and
for many other purposes.
The Emotion Megamix stimuli
Morphed continua have been developed from prototype expressions of two models;
MO (designated F5 here, because she is the fifth female model in the Ekman and
Friesen series) and JJ (M4, the fourth male model).
JJ (M4) and MO (F5) were chosen for morphing because the photographs of all six
emotional facial expressions (happiness, surprise, fear, sadness, disgust, and anger)
were of consistent quality for each face, with reasonably standardised pose and
lighting.
The prototype expressions for each of the six basic emotions and a neutral pose for
these individuals are shown in columns F5 (MO) and M4 (JJ) in Figure 0.1. Table 3.1
shows the numbers and identifiers in the Ekman and Friesen (1976) series for the
selected prototype images of emotional facial expressions, and percentage recognition
rates for these six expressions by Ekman and Friesen's college student sample. Details
of the muscle movements (Action Units) involved in each prototype emotional
expression can be found in Section 6 of this Manual. The additional prototype neutral
poses are faces 65 MO-1-05 and 41 JJ-3-04.


Number
E&F id
Intended Percentage recognition rates

emotion
Happiness
Surprise
Fear
Sadness
Disgust
Anger
100
24
57
MO-1-04 Happiness
63
MO-1-14 Surprise
90
31
59
MO-1-23 Fear
13
88
24
58
MO-1-30 Sadness
88
24
64
MO-2-18 Disgust
100
24
61
MO-2-11 Anger
100
24
34
JJ-4-07 Happiness
100
31
39
JJ-4-13
Surprise
97
30
37
JJ-5-13
Fear
96
25
36
JJ-5-05 Sadness
93
30
40
JJ-3-20 Disgust
12
88
33
38
JJ-3-12 Anger
15
76
33
Table 3.1: Numbers and identifiers (E&F id) in the Ekman and Friesen (1976) series
for the prototype images used to create the Emotion Megamixes, with percentage
recognition rates for the expressions by Ekman and Friesen's (1976) college student
sample (N = number of judges for each face).
The procedure used to create morphed images is described in Sections 2 and 7 of this
Manual. It involves determining the shapes of two prototype images, creating a new
shape in which feature locations are located a certain proportion of the distance
between their locations in each prototype, adjusting the surface texture (brightness
values) of each prototype to the new shape (by stretching the image, as if it lay on a
rubber sheet), and then blending the textures in the required proportions.
Continua consisting of morphs at 10% intervals between every possible pairing of the
six basic emotions in the Ekman and Friesen series (anger, disgust, fear, happiness,
sadness, surprise) have been created for JJ's face (M4) and for MO's face (F5). In
total, this involves 15 continua of 9 images for each model. Figures 3.1a and 3.1b show
these continua for JJ's face. Note that there are only 15 continua (rather than 30)
because the continua are bi-directional; a continuum between prototype expressions
X and Y is the same as a continuum between Y and X, but with the images in the
reverse order.


Figure 3.1a: Morphed expression continua for model M4 (JJ). Prototype expressions
(not shown in this Figure) of Anger (A), Disgust (D), Fear (F), Happiness (H),
Sadness (S), and Surprise (U) are blended in proportions 90:10, 80:20, 70:30,
60:40, 50:50, 40:60, 30:70, 20:80, and 10:90 to create the following continua:
Anger Disgust, Anger Fear, Anger Happiness, Anger Sadness, Anger


Surprise, Disgust Fear, Disgust Happiness, Disgust Sadness, and Disgust

Surprise. Note that the continua are bi-directional; for example, a Disgust Anger
continuum involves the same images as the Anger Disgust continuum (top row),
but in the opposite direction of travel.
Figure 3.1b: Morphed expression continua for model M4 (JJ). Prototype expressions
(not shown in this Figure) of Anger (A), Disgust (D), Fear (F), Happiness (H),
Sadness (S), and Surprise (U) are blended in proportions 90:10, 80:20, 70:30,
60:40, 50:50, 40:60, 30:70, 20:80, and 10:90 to create the following continua:
Fear Happiness, Fear Sadness, Fear Surprise, Happiness Sadness,
Happiness Surprise, and Sadness Surprise.
A further six continua have been prepared for each model (M4/JJ and F5/MO),
consisting of morphs at 10% intervals between the Neutral prototype and the
prototype expression of each of the six basic emotions. These continua are shown for
face M4 (JJ) in Figure 3.2.


Figure 3.2: Neutral emotion continua for model M4 (JJ). The prototype Neutral
(N) expression (not shown in this Figure) is morphed with prototype expressions of
the six basic emotions of Anger (A), Disgust (D), Fear (F), Happiness (H), Sadness
(S), and Surprise (U) in proportions 90:10, 80:20, 70:30, 60:40, 50:50, 40:60,
30:70, 20:80, and 10:90 to create the following continua: Neutral Anger, Neutral
Disgust, Neutral Fear, Neutral Happiness, Neutral Sadness, and Neutral
Surprise.
Properties of the Emotion Megamix images
If we return to the hexagonal schematic representation of confusable emotions shown
in Figure 2.1, one way to think of the emotion emotion continua from Figures 3.1a
and 3.1b is that they traverse the inner regions as well as the perimeter of the
hexagon, as shown in Figure 3.3. It is therefore of interest to know whether morphed
images from the inner continua of Figure 3.3 have properties any different from those
of images from continua located around the perimeter. To the extent that the
hexagonal schema represents underlying perceptual mechanisms, morphs from the
inner continua should be less consistently classified (because they will pass through
similar regions and hence become confused with each other) than morphs from
continua lying around the perimeter (Young, Rowland, Calder, Etcoff, Seth and
Perrett, 1997).

Figure 3.3: The 15 emotion emotion continua for the Emotion Megamix. Six
continua (happiness surprise, surprise fear, fear sadness, sadness disgust,
disgust anger, and anger happiness) form the perimeter of the hexagonal
schema (cf. Figure 2.1). Six continua form two inner triangles (happiness fear,
fear disgust, and disgust happiness; surprise sadness, sadness anger, and
anger surprise). Three continua form the diagonals (happiness sadness, fear
anger, and disgust surprise).
Figure 3.4 shows the stimuli used to investigate this issue by Young et al. (1997),
which involve 90%, 70%, 50%, 30%, and 10% morphs along all possible pairwise
continua between prototype expressions for face M4 (JJ). The continua shown at the
top left of Figure 3.4 form the perimeter of the hexagonal schema shown in Figure
3.3, the centre left and lower left continua are the inner triangles of Figure 3.3, and
the top right continua are the diagonals. The bottom right panel of Figure 3.4 shows
the emotion neutral continua.


Figure 3.4: Stimuli used by Young et al. (1997). The continua show 90%, 70%, 50%,
30%, and 10% morphs along continua between prototype expressions (not shown in
Figure 3.4) of Anger (A), Disgust (D), Fear (F), Happiness (H), Sadness (S), Surprise
(U), and a Neutral (N) pose.
In their first experiment, Young et al. (1997) used the 90%, 70%, 50%, 30% and 10%
morphs of JJ's face from all 15 of the emotion emotion continua schematised in
Figure 3.3, to create a set of 75 stimuli. These were presented in random order to


neurologically normal participants, who were asked to classify each morphed image
as most like anger, disgust, fear, happiness, sadness or surprise. Their choices and
response times were recorded across 9 blocks of trials with the set of 75 stimuli
presented in random order, with the first block of trials discarded as practice, as in
the Emotion Hexagon test. Note that, as for the Emotion Hexagon, prototype images
were never shown in the experiment; all of the stimuli were morphs.
Happiness-Surprise-Fear-Sadness-Disgust-Anger-Happiness
6000
Happiness
Surprise
Fear
Sadness
Disgust
Anger
4000
2000
0
100
80
Happiness
Surprise
Fear
Sadness
Disgust
Anger
60
40
20
0
Figure 3.5a: Mean percentage identification (upper panel) and response times
(lower panel) for continua that form the perimeter of Figure 3.3 (shown in the upper
left panel of Figure 3.4).
Figure 3.5a shows results from the continua that form the perimeter of Figure 3.3;
these form the upper left panel in Figure 3.4. They are the same stimuli as used in the
Emotion Hexagon test described in Section 2 of this Manual. As the upper panel of
Figure 3.5a shows, identification of these stimuli is not affected by the presence in the
experiment of morphed images drawn from a wider range of continua; the results
look closely comparable to those for the Emotion Hexagon (Figure 2.3).
Identification data from the other emotion emotion continua are shown in the
upper panels of Figure 3b (one of the inner triangles in Figure 3.3, shown as the
centre left panel in Figure 3.4), Figure 3c (the second inner triangle in Figure 3.3,
shown as the lower left panel in Figure 3.4), and Figure 3d (the diagonals in Figure
3.3, shown as the upper right panel in Figure 3.4). In all cases, there is a clear and
abrupt shift from reporting one category to reporting the other category near the midpoint of each continuum. Where these category shifts occur, there are few intrusions
from categories other than those at either end of the relevant continuum (for

example, a 50% Happiness 50% Surprise morph will be perceived as happiness or

surprise, but not as anger, disgust, fear, or sadness).
Happiness-Fear-Disgust-Happiness
100
80
Happiness
Surprise
Fear
Sadness
Disgust
Anger
60
40
20
0
90 70 50 30 10 90 70 50 30 10 90 70 50 30 10 90
Happiness-Fear-Disgust-Happiness
6000
Happiness
Surprise
Fear
Sadness
Disgust
Anger
4000
2000
0
90 70 50 30 10 90 70 50 30 10 90 70 50 30 10 90
Figure 3.5b: Mean percentage identification (upper panel) and response times
(lower panel) for continua that form an inner triangle of Figure 3.3 (shown in the
centre left panel of Figure 3.4).


Surprise-Sadness-Anger-Surprise
100
80
Happiness
Surprise
Fear
Sadness
Disgust
Anger
60
40
20
0
90 70 50 30 10 90 70 50 30 10 90 70 50 30 10 90
Surprise-Sadness-Anger-Surprise
6000
Happiness
Surprise
Fear
Sadness
Disgust
Anger
4000
2000
0
90 70 50 30 10 90 70 50 30 10 90 70 50 30 10 90
Figure 3.5c: Mean percentage identification (upper panel) and response times
(lower panel) for continua that form an inner triangle of Figure 3.3 (shown in the
lower left panel of Figure 3.4).


Happiness-Sadness : Surprise-Disgust : Fear-Anger

100
80
Happiness
Surprise
Fear
Sadness
Disgust
Anger
60
40
20
0
90 70 50 30 10
90 70 50 30 10
90 70 50 30 10
Happiness-Sadness : Surprise-Disgust : Fear-Anger

6000
Happiness
Surprise
Fear
Sadness
Disgust
Anger
4000
2000
0
90 70 50 30 10
90 70 50 30 10
90 70 50 30 10
Figure 3.5d: Mean percentage identification (upper panel) and response times
(lower panel) for continua that form the diagonals of Figure 3.3 (shown on the
upper right of Figure 3.4).
The identification data therefore show clearly that whilst Figure 3.3 may be a
convenient way of representing the various morphed continua, it does not reflect the
properties of the perceptual mechanisms we use to recognise facial expressions. If
Figure 3.3 were a correct representation of how we perceive faces, morphs from the
continua that traverse the inner regions of the hexagon (the two inner triangles, and
especially the diagonals) should be more variable in how they are classified than
would be morphs from the perimeter. This was not the case all continua effectively
behave in more or less the same way. This pattern of results implies that any twodimensional schema (the hexagon is only one of the possible two-dimensional
arrangements) will not give a fully satisfactory account of mechanisms involved in
facial expression perception.
Figures 3.5a, 3.5b, 3.5c, and 3.5d also show response times (in the lower panels).
These show the interesting property that reaction times are fastest for the morphs
that lie closest to the prototype images, and become progressively slower as the
morphed image moves away from the prototype. The consequent scalloping of RTs is
clearly seen in all of the graphs. In other words, although the identification data show
that we can see which category prototype a morphed image lies most close to (even
though the prototypes were not shown in the experiment), distance from the
prototype affects the time needed to do this.
In a second experiment, Young et al. (1997) added the 90%, 70%, 50%, 30%, and 10%
morphs from the emotion neutral continua shown in the lower right panel of Figure


3.4 to the set of 75 images used in their first experiment. This created a set of 105
morphed images used in an experiment where each had to be classified as most like
anger, disgust, fear, happiness, sadness, surprise, or neutral.
Happiness-Neutral-Surprise : Fear-Neutral-Sad : DisgustNeutral-Anger

100
Happiness
Surprise
Fear
Sadness
Disgust
Anger
Neutral
80
60
40
20
0
Happiness-Neutral-Surprise : Fear-Neutral-Sad : DisgustNeutral-Anger

4000
Happiness
Surprise
Fear
Sadness
Disgust
Anger
Neutral
2000
Figure 3.6: Mean percentage identification (upper panel) and response times (lower
panel) for continua that include a neutral and an emotional expression.
Figure 3.6 shows data for the emotion neutral continua. As can be seen, morphs
derived from the Neutral prototype behave much the same as morphs derived from
prototype expressions representing basic emotions. The data for the emotion
emotion continua recorded in this experiment were comparable to those presented in
Figures 3.5a, 3.5b, 3.5c, and 3.5d, and are not shown here.
These identification data suggest that perception of facial expressions may involve
assigning them to discrete categories (Etcoff and Magee, 1992), rather than
interpreting them in terms of their location on a small number of more general
underlying dimensions.
The hallmark of categorical perception is usually taken to be better ability to
discriminate stimuli that straddle a category boundary compared to ability to
discriminate stimuli that are equally different physically but fall within the same
perceptual category (Harnad, 1987). Colour perception is a case in point; although
wavelength varies along a continuum, perceived colour shifts abruptly in certain
regions of the colour spectrum. In these regions, the effects of changes in wavelength
are more easily seen.


Predicted
Observed
1.80
1.40
1.00
0.60
0.20
-0.20
Figure 3.7: Simultaneous discrimination of adjacent images from continua that

form the perimeter of Figure 3.3 (shown in the upper left panel of Figure 3.4).
Observed discrimination performance is plotted alongside predicted performance
based on a categorical perception model.
Morphing provides a way of exploring the hypothesis of categorical perception of
facial expressions. Although the morphed continua are multidimensional in nature
(unlike wavelength), morphing offers the closest we can at present get to a
mathematically linear transformation (Calder, Young, Perrett, Etcoff and Rowland,
1996; Young et al., 1997). Figure 3.7 shows discrimination performance for adjacent
images from the continua that form the perimeter of Figure 3.3 (shown in the upper
left panel of Figure 3.4; these are the stimuli from the Emotion Hexagon test).
Observed discrimination performance (using the signal detection theory measure d')
is plotted alongside predicted performance based on a categorical perception model.
The fit between predicted and observed performance was assessed by correlating
predicted and observed values, and converting this to a t-value (McNemar, 1962).
There was a significant correlation between predicted and observed discrimination (r
= 0.53, t = 3.29, df 28, p < 0.01).
Although investigating dimensional and category-based accounts of facial expression
recognition was our initial reason for making the Emotion Megamix stimuli (Calder et
al., 1996; Young et al., 1997), it has not been the only way in which they have been
used. In neuropsychological studies, they have allowed the creation of tailor-made
forms of testing to allow close investigation of specific forms of emotion recognition
impairment, for example by systematically testing all possible continua involving a
particular emotion (Sprengelmeyer, Young, Sprengelmeyer, Calder, Rowland, Perrett,
Hmberg and Lange, 1997). We suspect the stimuli have a number of other potential
uses.
References
Calder, A.J., Young, A.W., Perrett, D.I., Etcoff, N.L. and Rowland, D. (1996).
Categorical perception of morphed facial expressions. Visual Cognition, 3, 81117.
Ekman, P. & Friesen, W.V. (1976). Pictures of facial affect. Palo Alto, California:
Etcoff, N.L. & Magee, J.J. (1992). Categorical perception of facial expressions.
Cognition, 44, 227240.


Harnad, S. (Ed.). (1987). Categorical perception: the groundwork of cognition.

Cambridge: Cambridge University Press.
McNemar, Q. (1962). Psychological statistics (Third Edition.). New York: Wiley.
Sprengelmeyer, R., Young, A.W., Sprengelmeyer, A., Calder, A.J., Rowland, D.,
Perrett, D.I., Hmberg, V. and Lange, H. (1997). Recognition of facial expressions:
selective impairment of specific emotions in Huntington's disease. Cognitive
Young, A.W., Rowland, D., Calder, A.J., Etcoff, N.L., Seth, A. and Perrett, D.I. (1997).
Facial expression megamix: tests of dimensional and category accounts of emotion
recognition. Cognition, 63, 271313.
Appendix 3: Organisation of the Emotion Megamix images on the CDROM

This Appendix explains the system used to create names for the Emotion Megamix
JPEG image files, and how to locate these on the FEEST CD-ROM. The image files are
encrypted on the CD-ROM, but can be copied to a hard disk using the software
supplied. The instructions given below will help you to locate the images you need.
FEEST_Stimuli/Images/Section_3_Megamixes/
There are six subdirectories of the FEEST_Stimuli/images/Section_3_Megamixes
directory:
F4_MO_100%_(prototype)_images
This directory contains the seven prototype (100%) images for model MO from the
Ekman and Friesen series. They are labelled using the following convention:
MM_MO_100% _exp.jpg
where MM designates that this is an image from the Megamix series, MO
designates the model, 100% shows it is a prototype image, and exp indicates the
expression (Ang = anger, Dis = disgust, Fea = fear, Hap = happiness, Neu =
neutral, Sad = sadness, SUr = surprise).
Hence the filenames of the images in the directory are:
MM_MO_100% _Ang.jpg
MM_MO_100%_Dis.jpg
MM_MO_100%_Fea.jpg
MM_MO_100%_Hap.jpg
MM_MO_100%_Neu.jpg
MM_MO_100%_Sad.jpg
MM_MO_100%_SUr.jpg
F5_MO_emotion_continua
This directory contains subdirectories for each of the fifteen continua resulting
from morphing between one of the six basic emotions and another basic emotion.
These subdirectories are:
MM_MO_Anger-Disgust
MM_MO_Anger-Fear
MM_MO_Anger-Happiness
MM_MO_Anger-Sadness
MM_MO_Anger-Surprise
MM_MO_Disgust-Fear
MM_MO_Disgust-Happiness


MM_MO_Disgust-Sadness
MM_MO_Disgust-Surprise
MM_MO_Fear-Happiness
MM_MO_Fear-Sadness
MM_MO_Fear-Surprise
MM_MO_Happiness-Sadness
MM_MO_Happiness-Surprise
MM_MO_Sadness-Surprise
In these subdirectory labels, MM indicates that the subdirectory holds images
from the Megamix series, MO designates the model, and the emotion labels
indicate the morphed continuum. Note that there are only fifteen (rather than
thirty) continua because the images needed for a disgust-anger continuum are the
same as those for the anger-disgust continuum, and so on.
Each of these emotion-emotion subdirectories contains nine images, labelled
using the following convention:
MM_MO_AD1_ex1_pp1%_ex2_pp2%.jpg
designates the model, ex1 and ex2 indicate the blended expressions (Ang = anger,
Dis = disgust, Fea = fear, Hap = happiness, Sad = sadness, SUr = surprise), and
pp1 and pp2 indicate the percentages of each prototype expression in the blend
(from 10% to 90% in 10% steps).
For example, the images in the MM_MO_Anger-Disgust directory are:
MM_MO_AD1_Ang_090%_Dis_010%.jpg
F5_MO_neutral continua
This directory contains subdirectories for each of the six continua resulting from
morphing between one of the six basic emotions and a neutral expression. These
subdirectories are:
MM_MO_Neutral-Anger
MM_MO_Neutral-Disgust
MM_MO_Neutral-Fear
MM_MO_Neutral-Happiness
MM_MO_Neutral-Sadness
MM_MO_Neutral-Surprise
In these subdirectory labels, MM indicates that the subdirectory holds images
from the Megamix series, MO designates the model, and the emotion labels
indicate the morphed continuum. Note that there are only six continua because
the images needed for a neutral-anger continuum are the same as those for an
anger-neutral continuum.
Each of these neutral-emotion subdirectories contains nine images, labelled using
the following convention:
MM_MO_AD1_ex1_pp1%_ex2_pp2%.jpg



designates the model, ex1 and ex2 indicate the blended expressions (Ang = anger,
Dis = disgust, Fea = fear, Hap = happiness, Neu = neutral, Sad = sadness, SUr =
surprise), and pp1 and pp2 indicate the percentages of each prototype expression
in the blend (from 10% to 90% in 10% steps).
For example, the images in the MM_MO_Neutral-Anger directory are:
MM_MO_NA1_Neu_090%_Ang_010%.jpg
M4_JJ_100%_(prototype)_images
This directory contains the seven prototype (100%) images for model JJ from the
Ekman and Friesen series. They are labelled in the same way as the images in the
F4_MO_100%_(prototype)_images directory (see above), except that the
identifier JJ designates the model.
M4_JJ_emotion_continua
This directory contains subdirectories for each of the fifteen continua resulting
from morphing between one of the six basic emotions and another basic emotion
for model JJ. The subdirectories and images are labelled in the same way as the
images in the F5_MO_emotion_continua directory (see above), except that the
These are the images shown in Figures 3.1a and 3.1b.
M4_JJ_neutral continua
morphing between one of the six basic emotions and a neutral expression for
model JJ. The subdirectories and images are labelled in the same way as the
images in the F5_MO_neutral continua directory (see above), except that the
These are the images shown in Figure 3.2


Section 4: Caricatures and anti-caricatures.......................................................................2

The Caricature stimuli ....................................................................................................2
Properties of the Caricature images...............................................................................5
References.......................................................................................................................9
Appendix 4: Organisation of the Caricature images on the CD-ROM..............................9


Section 4: Caricatures and anti-caricatures

The difference between pictures of two facial expressions can be thought of as
involving differences in underlying shape (the locations of facial landmarks) and in
surface tone, i.e. texture and pigmentation (regional brightness and colour values).
For example, all the surprised expressions in Figure 0.1 have an open mouth, whereas
all the neutral poses have a closed mouth. So the shape of the mouth region differs
between these expressions (more rounded in surprise, more elongated in the neutral
pose), but opening the mouth also introduces changes in texture and pigmentation (a
dark region from the mouth cavity, and light where the teeth show).
Morphing usually involves simultaneous changes in shape and texture, to create a
smoothly blended continuum (see Section 7 of the Manual), but there are
circumstances under which it can be desirable to be able to manipulate shape and
texture independently. Computer caricature procedures offer one way to do this.
Computer caricaturing works by measuring the differences between image values in a
particular stimulus and a reference norm, and exaggerating these differences to create
a caricatured representation that is further from the norm. Caricatured faces can be
easier to recognise than veridical images, and may also be judged to be a better
likeness (Rhodes, 1996). It is also possible to create an anti-caricatured
representation that is closer to the norm; anti-caricatures are less easy to recognise
and judged to be poorer likenesses.
The computer caricature process can be applied to underlying shape, to surface
texture and pigmentation, or to both at the same time (see Section 7). For FEEST, we
have created shape caricatures of facial expressions from the Ekman and Friesen
(1976) series, using the same models (F5/MO and M4/JJ) as the Emotion
Megamixes.
Caricaturing facial expressions is of value in its own right, because it creates continua
in which the apparent intensity of an expressed emotion can be increased or
decreased in regular steps. In addition, because the Caricature image series only
involves shape changes, it forms a useful adjunct to the morphed Emotion Megamix
series (in which shape and texture are changed at the same time).
The Caricature stimuli
Caricatures were created independently for the two models from the Emotion
Megamix series (F5/MO and M4/JJ). Their prototype expressions for each of the six
basic emotions and a neutral pose are shown in columns F5 (MO) and M4 (JJ) in
Figure 0.1. Further details of the prototype expressions of basic emotions for MO and
JJ can be found in Table 3.1. Details of the muscle movements (Action Units) involved
in each prototype emotional expression can be found in Section 7 of this Manual. The
prototype neutral poses are faces 65 MO-1-05 and 41 JJ-3-04 in the Ekman and
Friesen (1976) series.
Photographs of each model's prototype images of anger, disgust, fear, happiness,
sadness, and surprise were digitised. The anatomical features of each prototype
expression were highlighted with 195 manually positioned reference points, with each
facial feature represented by a set number of points (e.g., 22 points for the mouth).
Information about the shape of each prototype expression was then stored as a 195
point x/y co-ordinate database.
Reference norms were established using the x/y co-ordinates for each model's neutral
pose (the Neutral expression norm) or the average x/y co-ordinate values across the
model's prototype images for the six basic emotions (the Average expression norm).
These reference norms are shown in complete-face form for JJ in Figure 4.1.

Figure 4.1: Average expression and neutral expression reference norms for JJ's face.
The norms are shown in complete-face form, rather than as the x/y co-ordinates
used by the computer-caricature program.
Each prototype expression's x/y co-ordinate database was compared to its Neutral
expression reference norm, and the differences in location between equivalent points
in the two databases were exaggerated by a factor of +15%, +30%, +50% or +75%, or
reduced by -15%, -30%, -50%, or -75%. This procedure was repeated using each
model's Average expression norm.
The result of the procedures described thus far was to produce caricatured and anticaricatured face shapes of the six prototype expressions of basic emotions. Next, a
triangulation was performed on the feature points of the prototype (0%, undistorted)
continuous-tone image for each expression to produce a mesh of triangles with the
shortest possible sides; details of this triangulation procedure can be found in Benson
and Perrett (1991a; 1991b) and Section 7 of this Manual. The feature points in each
caricatured or anti-caricatured face shape were also triangulated so that the vertices
of each triangle were identical to those in the corresponding prototype image. Finally,
the pixel intensity in each of the prototype image triangles was mapped onto the
corresponding triangle in the caricature by altering the spatial distribution to the new
shape.


Figure 4.2: Anti-caricatures and caricatures of facial expressions of basic emotions

of anger (A), disgust (D), fear (F), happiness (H), sadness (S), and surprise (U) for
M4/JJ's face, relative to an Average expression norm. Reading from left to right,
each row shows anti-caricatures of -75%, -50%, -30%, -15%, the prototype (0%
caricature) image from the Ekman and Friesen (1976) series, and then caricatures
of +15%, +30%, +50%, and +75%.
Caricatures and anti-caricatures of M4/JJ's face relative to the Average expression
norm are shown in Figure 4.2, and relative to the Neutral expression norm in Figure
4.3.


Figure 4.3: Anti-caricatures and caricatures of facial expressions of basic emotions

of anger (A), disgust (D), fear (F), happiness (H), sadness (S), and surprise (U) for
M4/JJ's face, relative to a Neutral expression norm. Reading from left to right, each
row shows anti-caricatures of -75%, -50%, -30%, -15%, the prototype (0%
caricature) image from the Ekman and Friesen (1976) series, and then caricatures
of +15%, +30%, +50%, and +75%.
In addition to these series of anti-caricatured and caricatured expressions relative to
Average and to Neutral expression norms for M4/JJ's face, the FEEST CD-ROM
contains equivalent series for F5/MO. Details of how to access these images on the CD
are given in Appendix 4, at the end of this Section of the Manual.
Properties of the Caricature images
Caricaturing facial expressions enhances their recognisability, and anti-caricaturing
reduces it. Figure 4.4 shows reaction times to correctly recognise facial expressions
from prototype (0% caricature) images, +50% caricatures, and -50% anti-caricatures
prepared relative to Average expression or to Neutral expression norms (Calder,
Young, Rowland and Perrett, 1997). The stimuli used in this experiment were


prepared for M4/JJ's face; they form the second, centre, and eighth columns of
Figures 4.2 and 4.3.
Figure 4.4: Mean correct reaction times (in milliseconds) for recognition of facial
expressions at -50%, 0%, and +50% levels of caricature, with images prepared for
M4/JJ's face relative to Average expression and Neutral expression norms.
The data plotted in Figure 4.4 do not show any statistically significant difference in
the size of the caricature effect across the two different reference norms (F < 1 for the
level of caricature x reference norm interaction). For most purposes, it does not seem
to matter greatly which reference norm is used. In fact, subsequent research has
shown that caricaturing can be effective even when a fixed norm is dispensed with. All
that is needed is to increase the differences between a target expression and some
other expression (Calder, Rowland, Young, Nimmo-Smith, Keane and Perrett, 2000).
For example, Calder et al. (2000) found that caricaturing an anger face was just as
effective when the caricatures were created relative to other prototype expressions
(disgust, fear or happiness) as when neutral or average expression norms were used.
The difference between caricatured facial expressions and the more usual type of
caricature which exaggerates distinctive features of an individual face's identity
(Benson and Perrett, 1991a; Brennan, 1985; Rhodes, 1996) is the reference norm
used. For identity caricatures, the norm needs to specify features relevant to identity
(for example, the average of several individuals' faces), whereas for expression
caricatures it needs to specify features relevant to expression (a person's neutral pose,
their average expression, or just another of their expressions).
The consequence of caricaturing a facial expression seems to be to move the
expression along a dimension which corresponds to variations in its intensity. Figure
4.5 shows data from an experiment by Calder et al. (2000) in which participants rated


the intensity of emotion seen in anti-caricatured (-50%), prototype (0% caricature)

and caricatured (+50%) expressions for M4/JJ's face relative to a Neutral expression
norm. The images used for this study were those shown in the second column, centre
column, and eighth column of Figure 4.3. These 18 stimuli were each rated for
intensity of each emotion on 010 scales.


Figure 4.5: Mean ratings (and standard error bars) for intensity of happiness,
surprise, fear, sadness, disgust, and anger for facial expressions of each of these
emotions at three levels of caricature (-50%, 0%, and +50%).
Figure 4.5 shows that the rated intensity of the emotion shown in the prototype
expression is increased in the caricatured image and decreased in the anti-caricatured
image. Interestingly, this is not simply because the caricatured faces look more
'emotional'. For example, caricaturing JJ's prototype happiness face increases its
perceived intensity of happiness, and anti-caricaturing reduces its perceived
happiness, but ratings of the same images for the other emotions (surprise, fear,


sadness, disgust, and anger) do not differ across different degrees of caricature. In
general, it is mainly the rated intensity of the caricatured emotion that is affected;
other emotions usually remain at much the same (low) level of intensity. The main
exception involves the perceptually similar emotions of fear and surprise, whose
intensities covary; caricaturing a fear expression affects its rated intensity of surprise
as well as its rated intensity of fear, and caricaturing a surprise expression affects its
rated intensity of fear as well as its rated intensity of surprise.
The Caricature stimuli thus offer the possibility of using stimuli for which the same
expression occurs at systematically different levels of intensity in neuropsychological
and experimental studies.
References
Benson, P.J. and Perrett, D.I. (1991a). Perception and recognition of photographic
quality facial caricatures: implications for the recognition of natural images.
European Journal of Cognitive Psychology, 3, 105135.
Benson, P.J. and Perrett, D.I. (1991b). Synthesising continuous-tone caricatures.
Image and Vision Computing, 9, 123129.
Brennan, S.E. (1985). Caricature generator: dynamic exaggeration of faces by
computer. Leonardo, 18, 170178.
Calder, A.J., Rowland, D., Young, A.W., Nimmo-Smith, I., Keane, J. and Perrett, D.I.
(2000). Caricaturing facial expressions. Cognition, 76, 105146.
Calder, A.J., Young, A.W., Rowland, D. and Perrett, D.I. (1997). Computer-enhanced
emotion in facial expressions. Proceedings of the Royal Society: Biological
Sciences, B264, 919925.
Rhodes, G. (1996). Superportraits: caricatures and recognition. Hove, East Sussex:
Psychology Press.
Appendix 4: Organisation of the Caricature images on the CD-ROM

This Appendix explains the system used to create names for the Caricature and anticaricature JPEG image files, and how to locate these on the FEEST CD-ROM. The
image files are encrypted on the CD-ROM, but can be copied to a hard disk using the
software supplied. The instructions given below will help you to locate the images you
need.
FEEST_Stimuli/Images/Section_4_Caricatures/
There are four subdirectories of the FEEST_Stimuli/Images/Section_4_Caricatures
directory:
F5_MO_caricatures_average_norm
caricaturing one of the six basic emotions relative to a norm based on the average
of the six prototype expressions (average expression norm), using face F5/MO.
F5_MO_Avgenorm_Anger
F5_MO_Avgenorm_Disgust
F5_MO_Avgenorm_Fear
F5_MO_Avgenorm_Happiness
F5_MO_Avgenorm_Sadness


F5_MO_Avgenorm_Surprise
In these subdirectory labels, F5_MO designates the model (female model 5 in the
Ekman and Friesen series, identifier MO), Avgenorm signifies that the caricatures
are relative to an average expression norm, and the emotion labels indicate the
caricatured expression.
Each of these caricatured expression subdirectories contains nine images, labelled
CA_MO_Avgenorm_expZ_car%.jpg
where CA designates this as an image from the average caricature series, MO
designates the model, Avgenorm signifies that the caricatures are relative to an
average expression norm, exp indicates the caricatured expression (Ang = anger,
Dis = disgust, Fea = fear, Hap = happiness, Sad = sadness, SUr = surprise), Z is an
integer from 1 to 9 used to order each series, and car indicates the percentage of
caricature (-75, -50, -30, -15, +00, +15, +30, +50, +75). Each series contains anticaricatures of -75%, -50%, -30%, and -15%, a prototype (+00% caricature)
expression, and positive caricatures of +15%, +30%, +50%, and +75%.
For example, the images in the F5_MO_Avgenorm_Anger directory are:
CA_MO_Avgenorm_Ang1_-75%.jpg
CA_MO_Avgenorm_Ang5_+00%.jpg
F5_MO_caricatures_neutral_norm
of the six prototype expressions (average expression norm), using face F5/MO.
F5_MO_Neutnorm_Anger
F5_MO_Neutnorm_Disgust
F5_MO_Neutnorm_Fear
F5_MO_Neutnorm_Happiness
F5_MO_Neutnorm_Sadness
F5_MO_Neutnorm_Surprise
In these subdirectory labels, F5_MO designates the model (female model 5 in the
Ekman and Friesen series, identifier MO), Neutnorm signifies that the caricatures
are relative to a neutral expression norm, and the emotion labels indicate the
CN_MO_Neutnorm_expZ_car%.jpg
where CN designates this as an image from the neutral caricature series, MO
designates the model, Neutnorm signifies that the caricatures are relative to a
neutral expression norm, exp indicates the caricatured expression (Ang = anger,


For example, the images in the F5_MO_Neutnorm_Anger directory are:
CA_MO_Neutnorm_Ang1_-75%.jpg
CA_MO_Neutnorm_Ang5_+00%.jpg
M4_JJ_caricatures_average_norm
of the six prototype expressions (average expression norm), using face M4/JJ.
M4_JJ_Avgenorm_Anger
M4_JJ_Avgenorm_Disgust
M4_JJ_Avgenorm_Fear
M4_JJ_Avgenorm_Happiness
M4_JJ_Avgenorm_Sadness
M4_JJ_Avgenorm_Surprise
In these subdirectory labels, M4_JJ designates the model (male model 4 in the
Ekman and Friesen series, identifier JJ), Avgenorm signifies that the caricatures
CA_JJ_Avgenorm_expZ_car%.jpg
where CA designates this as an image from the average caricature series, JJ
designates the model, Avgenorm signifies that the caricatures are relative to an
average expression norm, exp indicates the caricatured expression (Ang = anger,
These are the images shown in Figure 4.2.
M4_JJ_caricatures_neutral_norm
of the six prototype expressions (average expression norm). These subdirectories
are:
M4_JJ_Neutnorm_Anger
M4_JJ_Neutnorm_Disgust
M4_JJ_Neutnorm_Fear

M4_JJ_Neutnorm_Happiness
M4_JJ_Neutnorm_Sadness
M4_JJ_Neutnorm_Surprise
In these subdirectory labels, M4_JJ designates the model (male model 4 in the
Ekman and Friesen series, identifier JJ), Avgenorm signifies that the caricatures
CN_JJ_Neutnorm_expZ_car%.jpg
where CN designates this as an image from the neutral caricature series, JJ
designates the model, Neutnorm signifies that the caricatures are relative to a
neutral expression norm, exp indicates the caricatured expression (Ang = anger,
These are the images shown in Figure 4.3.




Section 5: Morphed and caricatured Continua..................................................................2

The Continua ..................................................................................................................2
Properties of the Continua .............................................................................................4
References.......................................................................................................................6
Appendix 5: Organisation of the Continua on the CD-ROM ............................................6


Section 5: Morphed and caricatured Continua

The morphed and caricatured Continua draw on both of the techniques used to create
the Expression Megamix (Section 3 of this Manual) and Caricature (Section 4) series.
Each continuum shows a series of seven images with increasingly intense emotional
expressions, ranging from a neutral pose to a caricatured expression of a basic
emotion. Continua are available for 10 individuals and all six of the basic emotions
depicted in the Ekman and Friesen (1976) series.
These stimuli are useful in circumstances requiring more than one or two models or a
set of expressions that cover a range of intensities from neutral through to highly
emotional, provided that it is not essential that a consistent method (morphing or
caricature) was used to create the changes in expression.
The Continua
Neutral emotion continua have been developed for all of the ten models shown in
Figure 0.1. Each continuum begins with a model's neutral pose (as shown in the
bottom row of Figure 0.1), which is then morphed in steps of 25%, 50%, and 75%
towards one of the same model's prototype expressions of a basic emotion from the
Ekman and Friesen (1976) series. In each continuum, the prototype expression is
regarded as 100% intensity, and caricatures using the model's neutral pose as a
reference norm are then used to heighten its intensity with +25% caricature (creating
an image of 125% intensity) and +50% caricature (150% intensity). Each continuum
thus runs 0% (neutral), 25% (morph), 50% (morph), 75% (morph), 100% (prototype
expression), 125% (caricature), 150% (caricature).
The morphing and caricaturing techniques used to create the Continua are the same
as those employed for the Emotion Megamixes and Caricatures (see Sections 3, 4, and
7 of this Manual), but the hairlines are masked to make it necessary to base any
decision about the model's sex on facial features rather than hairstyle. This has
proved especially useful in functional imaging studies (see below), where incidental
tasks (such as classifying the faces as female or male) are commonly used to ensure
participants are attending to presented stimuli and to equate levels of performance
(of the incidental task) across different conditions.


Figure 5.1: Continua for face F2/C from the Ekman and Friesen (1976) series
showing neutral anger (A), neutral disgust (D), neutral fear (F), neutral
happiness (H), neutral sadness (S), and neutral surprise (U). Each continuum is
displayed horizontally. The neutral pose (0% intensity of emotion) is shown on the
left of each row, with the prototype (100% intensity) emotional expression in the
fifth column. The second, third, and fourth columns show interpolated 25%, 50%,
and 75% morphed images. The sixth and seventh columns use computer caricature
procedures to enhance the intensity of the expression by +25% (125% intensity
column) and +50% (150% intensity column).
Figure 5.1 shows the six Continua (from a neutral pose to the +50% caricatured
expression of each of the six basic emotions) for face F2 from Figure 0.1 (model C in
the Ekman and Friesen series). An equivalent set of six Continua has been developed


for each of the other 9 models shown in Figure 0.1. The combination of 10 models and
6 emotions leads to 60 Continua in total.
Properties of the Continua
The combination of morphing and caricature in the same sequence makes the
physical intervals between successive stimuli in the Continua somewhat arbitrary, but
as Figure 5.1 shows, they are effective in creating a continuum of increasing intensity
of each expression.
The validity of the caricatured regions of the Continua was demonstrated by Calder,
Young, Rowland and Perrett (1997), who measured reaction times for recognising
prototype (0% caricature; the 100% images in Figure 5.1) and +50% caricature
expressions (the 150% images in Figure 5.1) across 8 models (models F2, F5, F6, F7,
F8, M1, M5, and M6 in Figure 0.1). Data from this experiment are shown in Figure
5.2. Caricaturing enhanced (speeded up) recognition of facial expressions of all six
emotions from the Ekman and Friesen series.


Figure 5.2: Mean correct reaction times (in milliseconds) for recognising facial
expressions of happiness, sadness, anger, fear, disgust, and surprise from prototype
expressions (0% caricature, shown as 100% intensity in Figure 5.1) and +50%
caricatured expressions (shown as 150% intensity in Figure 5.1).
The main use to date of morphed and caricatured Continua has been in functional
imaging studies of the neural substrates of emotion recognition. Morris, Frith,
Perrett, Rowland, Young, Calder and Dolan (1996) began this line of work by
measuring neural activity when participants viewed facial expressions from continua
ranging from 0% (neutral) to 125% (+25% caricatured expression) happiness or fear.
An incidental task was used, in which the participant was asked to classify each face
as male or female. The neural response in the left amygdala was greater to fearful
than to happy expressions.
Subsequent studies that have used morphed and caricatured expressions to
manipulate the intensity of an expressed emotion (Phillips, Young, Senior, Brammer,
Andrew, Calder, Bullmore, Perrett et al., 1997) or to adjust the baseline condition
(Phillips, Young, Scott, Calder, Andrew, Giampetro, et al., 1998) have confirmed the
importance of the amygdala to the recognition of fear, and begun to chart its time
course (Phillips, Medford, Young, Williams, Williams, Bullmore, Gray and Brammer,
2001) and interactions with other neural structures (Morris, Friston, Bchel, Frith,
Young, Calder and Dolan, 1998).


These functional imaging studies of amygdala involvement in the recognition of fear

are consistent with neuropsychological studies using the Ekman 60 Faces and
Emotion Hexagon tests (see Table 1.8, Figure 2.6, and Table 2.8).
Another line of functional imaging work has used morphed and caricatured
expressions to demonstrate involvement of the insula and parts of the basal ganglia in
the recognition of facial expressions of disgust (Phillips et al., 1998; Phillips et al.,
1997). Again, these findings from functional imaging of the neural responses of the
normal brain are consistent with neuropsychological data from the Ekman 60 Faces
and Emotion Hexagon tests (Table 1.8, Figure 2.6, and Table 2.9).
As groundwork for their initial functional imaging study, Morris et al. (1996) carried
out tests of the perception of morphed and caricatured expressions of happiness and
fear in the 0% (neutral) to 125% (+25% caricature) range. Ratings of intensity of
expression correlated well with an image's position on each continuum (r = 0.77 for
fear, r = 0.83 for happiness). When images from the same continuum were presented
in pairs and participants were asked to select the more intense expression, there was
69% agreement for pairs differing by 25% (i.e. one step apart in Figure 5.1), 83%
agreement for pairs differing by 50% (two steps apart in Figure 5.1), and 100%
agreement for pairs differing by more than 50% (three steps or more apart).
The morphed and caricatured Continua thus provide useful stimuli when the prime
requirements are for expressions that vary in intensity, derived from a number of
models.
References
Calder, A.J., Young, A.W., Rowland, D. and Perrett, D.I. (1997). Computer-enhanced
emotion in facial expressions. Proceedings of the Royal Society: Biological
Sciences, B264, 919925.
Morris, J.S., Friston, K.J., Bchel, C., Frith, C.D., Young, A.W., Calder, A.J. and
Dolan, R.J. (1998). A neuromodulatory role for the human amygdala in processing
emotional facial expressions. Brain, 121, 4757.
Morris, J.S., Frith, C.D., Perrett, D.I., Rowland, D., Young, A.W., Calder, A.J. and
Dolan, R.J. (1996). A differential neural response in the human amygdala to
fearful and happy facial expressions. Nature, 383, 812815.
Phillips, M.L., Medford, N., Young, A.W., Williams, L., Williams, S.C.R., Bullmore,
E.T., Gray, J.A. and Brammer, M.J. (2001). Time courses of left and right
amygdalar responses to fearful facial expressions. Human Brain Mapping, 12,
193202.
Phillips, M.L., Young, A.W., Scott, S.K., Calder, A.J., Andrew, C., Giampetro, V.,
Williams, S.C.R., Bullmore, E.T., Brammer, M. and Gray, J.A. (1998). Neural
responses to facial and vocal expressions of fear and disgust. Proceedings of the
Royal Society: Biological Sciences, B265, 18091817.
Phillips, M.L., Young, A.W., Senior, C., Brammer, M., Andrew, C., Calder, A.J.,
Bullmore, E.T., Perrett, D.I., Rowland, D., Williams, S.C.R., Gray, J.A. and David,
A.S. (1997). A specific neural substrate for perceiving facial expressions of disgust.
Nature, 389, 495498.
Appendix 5: Organisation of the Continua on the CD-ROM

This Appendix explains the system used to create names for the JPEG image files for
the Continua, and how to locate these on the FEEST CD-ROM. The image files are


encrypted on the CD-ROM, but can be copied to a hard disk using the software
supplied. The instructions given below will help you to locate the images you need.
FEEST_Stimuli/Images/Section_5_Continua/
There are ten subdirectories of the FEEST_Stimuli/images/Section_5_Continua
directory:
MC01_F2_C_continua
MC02_F4_MF_continua
MC03_F5_MO_continua
MC04_F6_NR_continua
MC05_F7_PF_continua
MC06_F8_SW_continua
MC07_M1_EM_continua
MC08_M4_JJ_continua
MC09_M5_PE_continua
MC10_M6_WF_continua
Each of these subdirectories holds the continua for one model from the Ekman and
Friesen series. The subdirectories are labelled using the following convention:
MCnn_fi_id_continua
where MC indicates that they hold images from the morphed and caricatured
continua, nn is a number used to designate models 01 to 10, fi is the FEEST
Identifier for the model (as shown in Figure 0.1), and id is the model's identifier in
the Ekman and Friesen series.
These ten subdirectories each have their own six subdirectories, one for a
continuum for each basic emotion. These subdirectories are labelled using the
following convention:
MCnn_emo
where MC indicates that they hold images from the morphed and caricatured
continua, nn is a number used to designate models 01 to 10, and emo indicates
the emotion continuum stored in the subdirectory (Anger, Disgust, Fear,
Happiness, Sadness, or Surprise).
For example, the subdirectories of
FEEST_Stimuli/Images/Section_5_Continua/MC01_F2_C_continua are as
follows:
MC01_Anger
MC01_Disgust
MC01_Fear
MC01_Happiness
MC01_Sadness
MC01_Surprise
These are the subdirectories that contain the actual images, labelled in the
following way:
MCnn_exp_per%.jpg
where MC indicates that the image is from the morphed and caricatured
Continua, nn is a number used to designate models 01 to 10, exp indicates the
expression (Ang = anger, Dis = disgust, Fea = fear, Hap = happiness, Sad =


sadness, SUr = surprise), and per indicates the percentage of morphing or

caricature (as in Figure 5.1).
For example, the images stored in the subdirectory
FEEST_Stimuli/ImagesSection_5_Continua/MC01_F2_C_continua/MC01_
Anger are as follows:
MC01_Ang_000%.jpg
MC01_Ang_025%.jpg
MC01_Ang_050%.jpg
MC01_Ang_075%.jpg
MC01_Ang_100%.jpg
MC01_Ang_125%.jpg
MC01_Ang_150%.jpg
These are the images shown in the top row of Figure 5.1.


Section 6: Origin of the photographs in FEEST ................................................................2

References.......................................................................................................................8


Section 6: Origin of the photographs in FEEST

Paul Ekman
University of California, San Francisco
In the mid 1960s we began a series of cross cultural studies, in which we showed
photographs to people in literate and preliterate cultures, asking them to judge what
emotion was shown in each picture (see Ekman, 1999, for a review and discussion of
these studies, which extended over more than thirty years). Although the initial work
was done with photographs provided by Silvan S. Tomkins, we needed more
photographs for many of the studies we wanted to do, and acquired new images
ourselves.
For our cross cultural studies we had studied six emotions, those which had been
most often found to emerge from our analysis of all previous research on facial
expression (Ekman, Friesen and Ellsworth, 1972). These six emotions are anger,
disgust, fear, happiness, sadness, and surprise. Rather than asking subjects to pose
each emotion, we instructed them to move their facial muscles into particular
patterns. These instructions were based on our knowledge of the mechanics of facial
movement that was emerging as we were, concurrently, developing the Facial Action
Coding System (FACS) (Ekman and Friesen, 1976b; 1978). The only spontaneous
photographs contained in the series are those showing happiness, most of which were
taken when we joked with the person showing the expression.
FACS was not completed, however, at the time we made our final selection of pictures
from the hundreds we had taken. Instead, three steps were followed to choose the
expressions that were subsequently published as a series the Pictures of Facial
Affect (POFA) (Ekman and Friesen, 1976). First, we selected only those photographs
that we thought best represented each emotion, based on our theories about the
muscular movements that signify each emotion. Then we showed these pictures to
U.S. born college students, asking them to select a single term from among the six
provided that best represented the emotion shown in these faces. With one group of
students the same six emotion terms were provided, but each emotion was on a
seven-point scale, with neutral or no emotion at one end, and the intended emotion at
the other. These observers rated every slide on each of the six emotion scales. To
convert these data to the same format as the single choice judgement task, we reduced
the judgements to a single choice, which was the emotion that had received the
highest score on the seven-point scales, with the additional requirement that there
had to be a difference of at least two points between this score and the score of the
next highest emotion.
We had not originally intended to publish the photographs, but the number of
requests for copies of the pictures quickly grew and by 1976 POFA was published. The
110 photographs in POFA were judged by at least 70% of the students as showing the
emotion it was intended to represent, all but 11 of them were judged to be the
intended emotion by at least 80% of the students. The complete set of 110 POFA
photographs can be ordered from my website paulekman.com, as can a more recent
set JACFEE that is in colour, includes half male and half female faces, and in which
no person appears more than once. The JACFEE photographs are much more
uniform in facial configuration than the POFA, as they were selected using FACS.
From the 110 POFA, 70 were selected as the basis for the computer-manipulated
images to be used in the FEEST. These are shown in Figure 6.1. They comprise
pictures of six emotional expressions (happiness, sadness, fear, anger, surprise,
disgust) and a neutral pose across 10 different models. The main selection principle
was to choose expressions from persons for whom there was a POFA photograph for

all six emotions, and if there was more than one candidate, to take the one that had
obtained the highest agreement by the student judges.


Figure 6.1: Photographs of facial expressions from the Ekman and Friesen (1976)
Pictures of Facial Affect (POFA) used in FEEST, arranged with a different model in
each row. There are emotional expressions of anger (A), disgust (D), fear (F),
happiness (H), sadness (S) and surprise (U), and a neutral (N) pose for 6 female and
4 male models. The labels used to identify each of the models show the identifier in
POFA (C, EM, JJ, etc.) and number them by their positions in the POFA series (F2 =
second female model in the series, M1 = first male model, etc.).
Table 6.1 summarises the original judgement data, for the 60 photographs showing
emotional (non-neutral) expressions.
Recognition
rate:
71%80%
Happiness
Sadness
Fear
Anger
Surprise
Disgust
81%90%
91%100%
10
Table 6.1: Summary of original judgement data from Ekman and Friesen (1976),
showing the numbers of photographs from the 60 emotional (non-neutral)
expressions used in FEEST that achieved recognition rates of 71%80%, 81%90%,
and 91%100%.
Ten neutral pose photographs, one for each of the ten persons, are also used in
FEEST. The judgement of neutral facial expressions is influenced by two factors: (1)
the face that just preceded it, so that a neutral is likely to be judged as happy if the
preceding photograph was of a negative emotion or sad if the preceding expression
was of happiness; and (2) physiognomy, such that a person with brows that sit low on
the face is likely to be judged as angry. Both of those effects were observed in the
judgement of the neutral pictures. Recently performed FACS scoring of these ten
neutral photographs revealed there is no scorable activity they are in that sense
neutral poses.
Table 6.2 presents the original judgement data for each of the photographs, organised
by emotion, and then by person within emotion, with the FACS scoring recently
performed. For those unfamiliar with FACS, let me explain that it is an objective,
comprehensive measurement technique for scoring any observed facial behavior, in
still photographs, film or video. FACS decomposes any expression into the muscular
actions that produced the expression. The scores are called Action Units (AUs), rather
than using the names of the muscles, because we had to subdivide the actions of
muscles that are capable of producing more than one distinctive change in
appearance, and combine muscles which produce appearance changes that
overlapped. Each AU has a number that stands for the particular appearance change
produced by the underlying muscle(s). The score, then, for an observed expression, is
the string of AUs that produced it. For example, 1+15+17 is an expression in which the
inner corners of the eyebrows have been pulled up by AU 1, the lip corners depressed
by AU 15, and the lower lip pushed up by AU 17. Where necessary, intensity is
indicated on a five point scale, using the letters A through E following the AU number.
Unilaterality is marked by a L or R before the AU number.


No.
EF id
Emot
Hap Sad
Fea Ang
Sur
Dis
Neu
F2
C-2-18
HAP
99
M1
14
EM-4-07
HAP
100
M4
34
JJ-4-07
HAP
100
F4
48
MF-1-06
HAP
F5
57
MO-1-04
F6
66
M5
Action Units
147 6C 12D 26
32 6D 12D 25
31
6C 12C 26
100
31
6D 12D 25
HAP
100
24 6C 12C 26
NR-1-06
HAP
92
24 1A 2A 6C 12D 26
74
PE-2-12
HAP
100
31
6E 12E 26
F7
85
PF-1-06
HAP
100
31
6B 12B
F8
93
SW-3-09
HAP
100
24 6C 12C 25
M6
101
WF-2-121
HAP
100
31
6E 12D 25
Table 6.2a: Percentage recognition rates and AUs happy faces. The first column
numbers number the models by their positions in the POFA series (F2 = second
female model in the series, M1 = first male model, etc.). In the other columns, No. =
number of photograph in the Ekman and Friesen (1976) series, EF id = photograph
identifier from Ekman and Friesen (1976) series, Emot = intended emotion, Hap =
happiness, Sad = sadness, Fea = fear, Ang = anger, Sur = surprise, Dis = disgust,
Neu = neutral, N = number of judges.
No.
Id
Emot
Hap Sad
Fea Ang
Sur
Dis
Neu
F2
C-1-18
SAD
90
M1
15
EM-4-24
SAD
97
M4
36
JJ-5-05
SAD
F4
49
MF-1-30
SAD
F5
58
MO-1-30
F6
67
M5
145 1C 4C 6B 7C 26
31
93
30 1C 4D L11A 64A
90
31
SAD
88
24 1B 4A
NR-2-15
SAD
94
31
1B 17B 24A
75
PE-2-31
SAD
74
16
31
1D 4D 26 38
F7
86
PF-2-12
SAD
100
24 4B 62B 64B
F8
94
SW-2-16
SAD
92
24 1C 4C
M6
102
WF-3-28
SAD
79
29 1A 4C 6A 25 64A
Table 6.2b: Percentage recognition rates and AUs sad faces.


Action Units
1B 4C 17A 25
1D 4D L11A 25 56B
No.
Id
Emot
Hap Sad
Fea Ang
Sur
Dis
Neu
F2
C-1-23
FEA
88
M1
16
EM-5-21
FEA
M4
37
JJ-5-13
FEA
F4
50
MF-1-26
FEA
F5
59
MO-1-23
F6
68
M5
Action Units
13
24 1D 2D 5D 20B 31
92
24 4B 5C 11A 25
96
25
88
24 1B 2B 4B 5D L10A 25
FEA
88
13
24 1C 2C 5E R20B 26
NR-1-19
FEA
10
84
31
79
PE-3-21
FEA
92
25
F7
88
PF-2-30
FEA
100
31
F8
95
SW-2-30
FEA
79
24 1B 2B 5C 25
M6
104
WF-3-16
FEA
88
25
1B 2B 4B 5B 20A 25
Sur
Dis
Neu
Action Units
4D 7C 17D 24D
1B 2B 4C 5C 20B 26
L1A R1A L2A R2B 4A

5A 26
1D 2C 4B 5E 16A 25
38
1C 2C 5D 20B 26
Table 6.2c: Percentage recognition rates and AUs fear faces.

No.
Id
Emot
Hap Sad
Fea Ang
F2
10
C-2-12
ANG
74
19
31
M1
18
EM-5-14
ANG
83
13
30 4D 5B 10B 25
M4
38
JJ-3-12
ANG
15
76
33 4D 5C 20A 26
F4
53
MF-2-07
ANG
100
24 4E 5C L10A T23C 26
F5
61
MO-2-11
ANG
100
24 4C 5C T23B
F6
69
NR-2-07
ANG
100
31
M5
80
PE-2-21
ANG
83
30 4D 5C 15A 17D 24E 38
F7
89
PF-2-04
ANG
79
21
F8
96
SW-4-09
ANG
100
24 4D 5B 7B 17B T23B
24D
30 4C 7C 16 25 29
M6
105
WF-3-01
ANG
100
30 4E 5A 16 26
Table 6.2d: Percentage recognition rates and AUs angry faces.


4D 5B 23C 25 38A
No.
Id
Emot
Hap
Sad
Fea Ang
Sur
Dis
Neu
F2
11
C-1-10
SUR
M1
19
EM-2-11
SUR
M4
39
JJ-4-13
SUR
F4
54
MF-1-09
SUR
F5
63
MO-1-14
F6
70
M5
Action Units
94
147 1E 2E 5C 26
91
32 1C 2C 5B 26
97
30 1C 2C 5C 26
96
24 1D 2D 5B L12A 26
SUR
90
31
1C 2C 5C 26
NR-1-14
SUR
16
81
31
1D 2D 5C 26
81
PE-6-02
SUR
23
74
31
1C 2C 5B 26
F7
90
PF-1-16
SUR
93
30 1C 2C 5C 26
F8
97
SW-1-16
SUR
100
31
M6
107
WF-2-16
SUR
91
69 1C 2C 5C 26
1C 2C 5C 26
Table 6.2e: Percentage recognition rates and AUs surprised faces.

No.
Id
Emot
Hap Sad
Fea Ang
Sur
Dis
Neu
F2
12
C-1-04
DIS
M1
20
EM-4-17
DIS
M4
40
JJ-3-20
DIS
F4
55
MF-2-13
DIS
F5
64
MO-2-18
F6
71
M5
96
147 9D 17C
97
30 4A 7B 10C 17D
12
88
33 9D 10C 15B 17C
10
90
30 4C 7D 9D 10D 25
DIS
100
24 7D 9C 10A 17B
NR-3-29
DIS
17
83
24 4D 7B 9D 10A 25
82
PE-4-05
DIS
10
90
31
F7
91
PF-1-24
DIS
96
24 4B 7C 9D 17B 24C
F8
98
SW-1-30
DIS
94
31
M6
108
WF-3-11
DIS
97
29 4D 7B 9C 25
Table 6.2f: Percentage recognition rates and AUs disgusted faces.

Many investigators have treated all the photographs for a specific emotion as if they
were identical, differing only in terms of which person is showing the expression, but
the FACS scoring shows there are differences in the expressions themselves.

Action Units
10C 17B
4C 7C 9D
Among the happiness photographs, all show AUs 6+12, qualifying as what we have
called Duchenne smiles (Ekman, Davidson and Friesen, 1990), but they differ in the
intensity of the muscular actions. The lips are parted in all but one expression (AUs
25 or 26). One photograph also shows AUs 1+2 (brows raised), and the judgement
data show that it is the only happiness expression for which a few observers had
judged it to be fear or surprise.
Among the sadness photographs there is much greater variation. Two photographs
show sadness only in the raising of the inner corners of the eyebrows and brows
drawn together (AUs 1+4); six others show this action plus other actions in the lower
part of the face, but not the same actions. There is also variability in the intensity of
the actions. There are no sadness expressions that are identical.
Among the fear photographs there is again a great deal of variation. Three different
eyebrow configurations are shown, and eight different configurations in the lower
face. All the stimuli show raised upper eyelid (AU 5), but there are differences in the
intensity of these actions. Again there are no expressions that are identical.
Among the anger photographs there is variation, but not as much as among sad and
fear faces. All photographs show brow lowered (AU 4) and eight of them show upper
eyelid raised (AU 5), although there are differences in the intensity of these two
actions. There are eight different configurations in the lower face.
There is nearly as much homogeneity among surprise photographs as there is for the
happiness photographs. All the pictures show the brows raised (AUs 1+2) and the
upper eyelid raised (AU 5), but the intensity of these two actions varies. They all show
the mouth open. One photograph also includes a slight hint of a smile on one side of
the face (AU L12a).
There is about as much variation among disgust as there was among the anger
photos. Eight of the photographs show nose wrinkled (AU 9), two show upper lip
raise only (AU 10). Six photographs show brow lowered(AU 4). In three of the
photographs the mouth is open.
With this information now available investigators can select photographs that are
most similar in muscular configuration, or select photographs that differ in specified
ways in their configuration. Those who already have data collected, could re-examine
their findings to see if any variations within an emotion can be explained by the
differences in FACS scoring.
References
Ekman, P. (1999). Facial expressions. In T. Dalgleish and M. Power (Eds.), The
Handbook of Cognition and Emotion. Pp. 301320. Sussex, U.K.: John Wiley and
Sons, Ltd.
Ekman, P., Davidson, R.J. and Friesen, W.V. (1990). Emotional expression and brain
physiology II: the Duchenne smile. Journal of Personality and Social Psychology,
58, 342353.
Ekman, P. and Friesen, W.V. (1976). Pictures of Facial Affect. Palo Alto, CA:
Ekman, P. and Friesen, W. V. (1976b). Measuring facial movement. Environmental
Psychology and Nonverbal Behavior, 1, 5675.
Ekman, P. and Friesen, W. V. (1978). Facial Action Coding System: a technique for
the measurement of facial movement. Palo Alto, California: Consulting
Psychologists Press.
Ekman, P., Friesen, W. V. and Ellsworth, P. (1972). Emotion in the human face:
guidelines for research and an integration of findings. New York: Pergamon Press


Section 7: Computer morphing and caricaturing procedures...........................................2

Caricaturing ....................................................................................................................2
Morphing ........................................................................................................................4
References....................................................................................................................... 7


Section 7: Computer morphing and caricaturing procedures

David Perrett
University of St. Andrews
Two different computer image-manipulation procedures have been used to create
images for FEEST caricaturing and morphing. Their essential features are explained
here.
Caricaturing
Caricaturing can be used for enhancing or diminishing the intensity of expression.
The technique utilises an algorithm formulated by Brennan (1985) simulating that
which caricature artists achieve by skill and intuition. Brennan defined the landmark
positions of a particular face (as indicated by the delineation stage of Figure 7.1) and
then exaggerated the differences in the positions of landmarks between this face and
those of an average face. Brennan calculated the average face shape from a
representative sample of faces from one class (e.g. 40 European adult males).
To create a caricature that increases the distinctiveness in shape of a target face by
50%, Brennan computed the destination caricature shape as the original target shape
plus 50% of the difference between target and average face shapes.
We have used caricaturing to exaggerate expressions in a controlled manner. Starting
with the natural expression pose (e.g. a prototype expression in the Ekman and
Friesen series), one can construct a sequence of images in which the expression is
exaggerated by fixed percentage steps. Note that the exaggeration can be made away
from any reference face, thus a happy face could be contrasted with a reference norm
based on a neutral face, an average of all expressions, or another particular
expression. Figure 7.1 illustrates the exaggeration of a happy expression (of individual
JJ, top middle) away from a Neutral expression (top left). Here, original prototype
expressions from the Ekman and Friesen series are defined as 100% expressions.


neutral
100% happy
150% happy caricature
Figure 7.1: Caricature exaggeration of the shape difference between happy and
neutral facial expressions for individual JJ.


In the first step, a large number of landmarks are placed on the features of the
reference neutral facial expression and the target (100%) happy facial expression.
This process is referred to as delineation and is performed manually; one point is
allocated to the tip of the nose, one to the left corner of the eye and so on. Each
landmark has two co-ordinate values (x and y; the position left to right and top to
bottom relative to one corner of the image). This delineation phase results in maps
detailing the shape of each expression. For illustration, lines (b-spline curves) can be
used to join up relevant landmarks around major facial features (Figure 7.1, middle
row). A +50% caricature of the happy expression increases the differences between
the x and y values of the corresponding landmarks in facial shapes of happy and
neutral expressions by 50%. The new caricatured happy expression shape can also be
defined as 150% happy (Figure 7.1 middle row, right).
The St. Andrews Perception lab extended Brennans caricaturing process (which had
been developed for line drawings) by distorting the shape of digital images to produce
photographic quality facial caricatures (Benson and Perrett, 1991; Tiddeman, Duffy
and Rabey, 2001). The construction of photographic quality caricature expressions
(Figure 7.1 top row, right) involves warping the digital image of the original
expression from its original shape to that of the new exaggerated shape (Figure 7.1
middle row, mid to right columns).
To achieve this the image space is tessellated; that is, it is divided into a series of
triangular areas (each triangle linking three adjacent facial landmarks; Figure 7.1
lower row). The tessellation structures define how the original image must be
reshaped to take on the new image configuration. The digitised pixels in each triangle
of the original image are repositioned and stretched so that they fit appropriately
within the new triangle shape in the product image. Tessellation structures, however,
form only the vehicle for computing the shape change of the digital image.
The whole process of image reshaping can be envisaged as painting the image of the
original facial expression on an elastic sheet and then stretching the elastic sheet into
the new exaggerated shape. If one imagines sticking pins through the sheet at each
original facial landmark and then moving the pins to their new caricatured coordinates, the elastic sheet would distort accordingly into the new caricatured facial
shape.
Caricaturing is usually thought of as a process of exaggeration. It should be realised
that the process can also be worked in reverse to diminish an expression's intensity.
For this, the differences between the positions of landmarks in the original facial
expression and the neutral facial pose are diminished rather than exaggerated. For
example, a -50% anti-caricatured happy expression is formed by moving the shape of
the original happy expression 50% of the way to a neutral shape.
The caricaturing of expressions for FEEST involves manipulation of image shape
alone. More recently, we extended the caricaturing process to include the
manipulation of image colour (Burt and Perrett, 1995; Lee and Perrett 1997, 2000;
Tiddeman, Perrett and Burt, 2001). The caricaturing of colour essentially enhances
the distinctive colouration and intensity of the target face image relative to the
reference face.
Morphing
This technique is widely used in films and advertising to change one face or object
gradually into a second. The technique is comprised of two components. First, within
the morph sequence, the image of the first face fades out and the second image
gradually increases in contrast. This part of the technique is effectively a crossdissolve that has a long history in film. During the 1980s the Perception Lab at St.

Andrews added a second component to improve the continuity within morph

sequences. The second component was simply a gradual change from the shape of the
first face to the shape of the second; the shape change proceeded at the same rate as
the change in contrast. An image half of the way through a morph sequence between
two faces is thus midway between the start and end faces both in terms of shape and
in terms of colouration.
For a morph between two faces of different identity (e.g. Prime Minister Tony Blair
and President George Bush) the mid-point in the morph is thus the average of the two
individuals, neither Blair nor Bush but 50% of both.
We have used morphing to manipulate expressions quantitatively. Starting with one
expression, it is possible to construct a sequence of images changing by fixed
percentage steps to a second expression (e.g. from a sad face to a happy face).


100% sad
100% happy
50% sad 50% happy

shape
100% sad image

50% sad 50% happy shape
50% Sad 50% Happy

100% happy image
50% sad 50% happy shape
Figure 7.2: Morphing to blend happy and sad expressions of individual JJ.


For illustration a morph has been constructed between two expressions (sad and
happy) of one individual (JJ). Figure 7.2 shows the computer graphic operations
implemented to create one image in a morph sequence; the formation of an image
midway between a sad and happy expression. In the first step, a large number of
landmarks are again placed on the prototype (100%) happy and (100%) sad facial
expressions. To form the facial shape that is the average of happy and sad face
expressions, the x and y co-ordinates of equivalent landmarks in the two faces are
added in the proportion 50% happy 50% sad. This forms the destination facial shape
(Figure 7.2, top, centre column). Each original image is then warped by the difference
between the original and destination delineation structures, using the methods
described for caricaturing (above).
The results of the warping are visible in the bottom row of the left and right columns.
Note that the teeth present in the 100% happy prototype facial expression remain
visible after this image has been reshaped into the 50% happy 50% sad shape (bottom
right). By contrast the image on the bottom left has no teeth visible even though it has
the same 50% happy 50% sad shape. This is because the image derived from the
original 100% sad prototype expression in which the mouth was closed and teeth
occluded from sight.
The final stage of the morphing process is to blend the two warped images together
(left and right column, bottom row). This is done digitally, so that the intensity value
of each point in the product image (centre column, bottom row) is 50% x the intensity
of the corresponding point in the warped sad image (left column, bottom row) and
50% x the intensity of the point in the warped happy image (right column, bottom
row). Note that, in this process, the product image contains 50% of the shape and
50% of the colour (intensity) from each of the two starting images. For other images
in a morph sequence there is a different percentage contribution of the two starting
images. To create a 75% happy: 25% sad facial mix, the shape of the destination face
is first calculated as 75% of the original happy and 25% of the original sad shape; the
happy and sad images are warped into this shape and finally the intensity of the two
warped images is mixed in the proportion 75:25, happy to sad.
References
Benson, P. J. and Perrett, D. I. (1991). Perception and recognition of photographic
quality facial caricatures: implications for the recognition of natural images.
European Journal of Psychology, 3, 105135.
Brennan, S. E., (1985). The caricature generator. Leonardo, 18, 170178.
Burt, D.M., and Perrett, D.I. (1995). Perception of age in adult Caucasian male faces:
computer graphic manipulation of shape and colour information. Proceedings of
the Royal Society of London, B259, 137143.
Lee, K. and Perrett, D.I. (1997). The role of colour in face identification. Perception,
26, 733752.
Lee, K.J. and Perrett, D.I. (2000). Manipulation of colour and shape information and
its consequences upon recognition and best likeness judgements. Perception, 29,
12911312.
Tiddeman, B., Duffy, N. and Rabey, G. (2001). A general method for overlap control
in image warping. Computers and Graphics, 25.
Tiddeman B.P, Perrett, D.I. and Burt, D.M. (2001). Prototyping and transforming
facial textures for perception research. IEEE Computer Graphics and
Applications, Research, 21, 4250



Facial Expressions of Emotion - Stimuli and Tests (FEEST) : Psychology Manual v1.0 Thames Valley Test Company

Diunggah oleh

Informasi Dokumen

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Facial Expressions of Emotion - Stimuli and Tests (FEEST) : Psychology Manual v1.0 Thames Valley Test Company

Diunggah oleh

Hak Cipta:

Format Tersedia

Facial Expressions of

Emotion Stimuli and

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST)

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Testing of facial expression recognition is notoriously tricky. Stimulus sets that

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Entire group Mean

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Entire group Mean

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

have a differentially severe impact on the recognition of certain emotions (see

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Section 1: The Ekman 60 Faces test...................................................................................2

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Section 1: The Ekman 60 Faces test

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Picture numbers and identifiers used by Ekman and

10 C-2-12; 53 MF-2-07; 61 MO-2-11; 69 NR-2-07; 89 PF2-04; 96 SW-4-09

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

whether errors consistently involve certain Action Units.

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Ekman 60 Faces test

Age 51-60 Age 61-70

Facial Expressions of Emotion Stimuli and Tests (FEEST).

Psychology manual v 1.0

Facial Expressions of Emotion Stimuli and Tests (FEEST).