Pamela Fox

ENG 449
Viet Nguyen

Creative Project: Can Someone’s Race be Predicted by their Facial

Features?

The purpose of this project is to discover whether a person’s race

can be determined solely by looking at a black&white photo of their
face, and if so, determine which facial features are the most important
in determining their race. Both because this project is ultimately for an
Asian-American class and because of my own bias in believing East
Asians have particularly characteristic faces, the project focuses on
contrasting East Asians with other races.

Methodology:
I setup a quiz that, when taken by multiple participants, would
yield statistics on the likelihood of a particular facial feature
determining a race or just the general ability to determine race. There
are four sections to the quiz, and an introduction screen to obtain
demographic information about the respondents.

Screen 1:
In the introduction, I explained the terms that would be used
throughout the quiz. Racial terms are always controversial, so I found
those definition that satisfied the distributions of photos I was using
and instructed users to use that definition. This is a tricky matter
however – many people already have a definition in their brain
corresponding to these terms, and it is difficult in timed quizzes to not
follow instinct. I considered instructing the users to use their own
definitions, but then I wouldn’t be able to verify a correct answer
versus an incorrect answer when analyzing the results. I also asked for
their race and gender in case I wanted to compare their stats to their
ability to determine race in the results.

Quiz 1:
This is one of two quizzes to test the respondent’s ability to
determine race based on particular facial features. All of the quizzes
are based on a database of 20 photos: 8 East Asian, 4 Hispanic, 4
Indian, and 4 Caucasian. Because of the ease of finding female photos
online, all of the photos are of females from that racial group. For this
quiz and the second quiz, I cut the photos up into 3 sections: eyes,
nose, mouth. The respondent is randomly presented with a face
section and asked to, as quickly as possible, choose the correct race
for the photo that the section came from. The user will see 60 face
sections by the end of the quiz, covering all 20 faces.

Quiz 2:
This is the exact same as the last quiz, except it is not timed. I
added in the timing factor to see if it would make a difference in
number of correct answers for the respondent to have unlimited
response time. This would perhaps indicate another level of reasoning
occurring, perhaps one resulting from social education of some kind.
Quiz 3:
This is the first of two quizzes to test the respondent’s ability to
determine race determined on seeing the complete face. Since it is the
first, it is also timed.

Quiz 4:
This is the exact same as the last one, but it is not timed.
Results:

The quizzes were advertised mostly to students through an

online social network. There were a total of 230 respondents, but 100
of those were thrown out due to problems in the data or their failure to
complete the entire survey. All the data shown reflects the 130
remaining respondents. The graph below shows the race of the
respondents. The gender variable was lost due to a technical difficulty.
This shows mostly white respondents, so we can take their responses
to represent those of the white community.

Demographics of Respondents

80

70

60
Number of Respondents

50

40 Series1

30

20

10

0
Hispanic Other White East Asian Southeast Pacific African South Asian American
Asian Islander or American Indian or
Native Alaska
Hawaiian Native
Races
Question: Can facial features be used to determine the race of a
black&white photo?
Answer: Yes, depending on the feature. There were consistently
significantly more correct answers by respondents when presented
with eye segments. The nose and mouth segments had approximately
the same percentage of correct answers. The graphs below illustrate
this result (photos in order of Asian, Caucasian, Hispanic, Other):
Eyes, Timed - FRACTION OF CORRECT GUESSES PER PHOTO

AVG: .79 AVG: .86 AVG: .49 AVG: .23

1

0.9

0.8

0.7
% Correct Guesses

0.6

0.5

0.4

0.3

0.2

0.1

0
A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4
Race

Noses, Timed - FRACTION OF CORRECT GUESSES PER PHOTO Mouths, Timed - FRACTION OF CORRECT GUESSES PER PHOTO

AVG: .54 AVG: .66 AVG: 33 AVG: .1 AVG: .55 AVG: .64 AVG: .32 AVG: .1
1 1

0.9 0.9

0.8 0.8
Fraction of Correct Guesses

0.7 0.7
% Correct Guesses

0.6 0.6

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1

0 0
A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-3 H-4 O-1 O-2 O-3 O-4 A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4
 Question: Are particular races easier to distinguish by facial feature
than others?
Answer: Yes. According to our results, both East Asians and Caucasians
are significantly easier to distinguish by facial features than others.
This may be because Hispanic is a rather broad category (and not
actually a race according to our census anymore), and because the
other category was “Other,” which doesn’t imply any particular
characteristics. People may be inclined to categorize as non-Other. If I
were to re-do the quiz I would probably replace Hispanic with Mexican,
or something more equivalent in geographic scope to East Asians and
Caucasians.
Noses, Untimed Mouths, Untimed

AVG: .54 AVG: .55 AVG: .34 AVG: .11 AVG: .52 AVG: .62 AVG: .31 AVG: .12
1 1

0.9 0.9

0.8 0.8

0.7 0.7
% Correct Guesses

% Correct Guesses
0.6 0.6

0.5 0.5

0.4 0.4

0.3 0.3

0.2 0.2

0.1 0.1

0 0
A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4 A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4
Race Race

Eyes, Untimed

AVG: .71 AVG: .81 AVG: .49 AVG: .23

1

0.9

0.8

0.7
% Correct Guesses

0.6

0.5

0.4

0.3

0.2

0.1

0
A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4
Race
Question: Can the race of a particular person be determined by a
complete black&white photo of their face?
Answer: Yes, with a more than 50/50 chance for the races chosen.
Once again, East Asians & Caucasians were significantly easier to
guess correctly than Hispanic. It was difficult for people to categorize
the Indian photos as Other, so it may help to have a subset of options
to choose from when deciding the race. It seemed that makeup made
it more difficult to determine race, as some of the lowest correct
answer percentages came for photos of women in makeup. It may be
that women sometimes use makeup to transform themselves into the
current stylish race, creating a confused set of visual cues. The graphs
below illustrate these results (notice A5, H1, O4):
Faces, Timed - ANSWERS A C H O

140

120

100
Race Guesses

80

60

40

20

0
A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-3 H-4 O-1 O-2 O-3 O-4
Races
Question: How does timing affect a person’s ability to correctly
determine race?
Answer: It has negligible effect. There were not significant differences
between timed and untimed quizzes. Occasionally people did better,
occasionally worse. This would indicate that race can actually be
determined quickly on a day-to-day basis- it can be thought of as a
snap judgment. The graphs below illustrate two similar results, timed
vs. untimed:

Mouths, Untimed

AVG: .52 AVG: .62 AVG: .31 AVG: .12

1

0.9

0.8

0.7
% Correct Guesses

0.6

0.5

0.4

0.3

0.2

0.1 Mouths, Timed - FRACTION OF CORRECT GUESSES PER PHOTO

AVG: .55 AVG: .64 AVG: .32 AVG: .1

0 1
A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4
0.9 Race

0.8

0.7
% Correct Guesses

0.6

0.5

0.4

0.3

Conclusions:
0.2
When I
proposed the 0.1
project, I
expected to find 0
A-1 A-2 A-3 A-4 A-5 A-6 A-7 A-8 C-1 C-2 C-3 C-4 H-1 H-2 H-3 H-4 O-1 O-2 O-3 O-4
that only East
Asians would be easy to recognize based on facial features. However,
I’ve been proved wrong. East Asians and Caucasians showed
comparable results throughout. This surprises me, as Caucasians is
really a much bigger category than East Asians, and I always think of
us as the unmarked race. Of course, this is typical ethnocentric
behavior. But now, seeing these results, I realize that to East Asians,
they may think of themselves as unmarked and us as marked. In fact,
it appears that both of us are marked, in our particular way. I also
believed at the beginning I would discover that eyes would be the best
determiner of race, and I was correct on that hypothesis. It is
significant to note that overall accuracy did increase between viewing
just eyes and viewing complete faces however – this indicates that the
nose and mouth, though not distinguishing on their own, can combine
with eyes in a recognizable way. There is a synergy in facial features
that enhances visual cues for race.

Future Work:
This was an amateur quiz created by someone not well versed in
social experiments. It would be interesting to see it redone at a larger
scale, with more careful attention paid to control factors. In this quiz I
mixed up celebrities and non-celebrities, made-up women vs. plain-
faced women, etc. These should be separated out to test the actual
effect of them. Also, I eliminated color from the photos to prove that it
was not necessary to determine race, and that race is not just a result
of color. There could be a final quiz with color to see how greatly that
affects the accuracy of the answers. There are many interesting
questions that quizzes like these could answer about our visual
perception of race.