Research Report

How Medical School Applicant Race, Ethnicity,

and Socioeconomic Status Relate to Multiple
Mini-InterviewBased Admissions Outcomes:
Findings From One Medical School
Anthony Jerant, MD, Tonya Fancher, MD, MPH, Joshua J. Fenton, MD, MPH,
Kevin Fiscella, MD, MPH, Francis Sousa, MD, Peter Franks, MD, and Mark Henderson, MD

Abstract
Purpose
To examine associations of medical
school applicant underrepresented
minority (URM) status and socioeconomic
status (SES) with Multiple Mini-Interview
(MMI) invitation and performance and
acceptance recommendation.
Method
The authors conducted a correlational
study of applicants submitting secondary
applications to the University of
California, Davis, School of Medicine,
20112013. URM applicants were black,
Southeast Asian, Native American,
Pacific Islander, and/or Hispanic. SES
from eight application variables was
modeled (01 score, higher score=

The Multiple Mini-Interview (MMI)

is replacing traditional interviews at

medical schools.15 In the MMI, trained
raters evaluate applicants in a series of
brief, timed, structured stations, and
station ratings are pooled to yield a
summary score. Stations are designed to
assess skills that are difficult to ascertain
from medical school applications, such as
interpersonal communication, teamwork,
ability to handle stress, problem solving,
and integrity/ethics. The MMI is well
accepted by applicants, reasonably
reliable, and predictive of medical school
and subsequent performance.13
Little studied is how underrepresented
racial/ethnic minority (URM) and lower
Please see the end of this article for information
about the authors.
Correspondence should be addressed to Anthony
Jerant, Department of Family and Community
Medicine, University of California, Davis, School of
Medicine, 4860 Y Street, Suite 2300, Sacramento,
CA 95817; telephone: (916) 734-7081; e-mail:
afjerant@ucdavis.edu.
Acad Med. 2015;90:16671674.
First published online May 20, 2015
doi: 10.1097/ACM.0000000000000766

lower SES). Regression analyses

examined associations of URM status
and SES with MMI invitation (yes/no),
MMI score (mean of 10 station ratings,
range 03), and admission committee
recommendation (accept versus not),
adjusting for age, sex, and academic
performance.

CI 0.08 to 0.08), or acceptance

recommendation (OR 1.08; CI 0.69 to
1.68). Lower SES applicants were more
likely to be invited to an MMI (OR 5.95;
CI 4.76 to 7.44) and recommended for
acceptance (OR 3.28; CI 1.79 to 6.00),
but had lower MMI scores (0.12 points,
CI 0.23 to 0.01).

Results
Of 7,964 secondary-application
applicants, 19.7% were URM and 15.1%
self-designated disadvantaged; 1,420
(17.8%) participated in the MMI and
were evaluated for acceptance. URM
status was not associated with MMI
invitation (OR 1.14; 95% CI 0.98 to
1.33), MMI score (0.00-point difference,

Conclusions
MMI-based admissions did not disfavor
URM applicants. Lower SES applicants
had lower MMI scores but were
more likely to be invited to an MMI
and recommended for acceptance.
Multischool collaborations should
examine how MMI-based admissions
affect URM and lower SES applicants.

socioeconomic status (SES) applicants

may be affected by adoption of the MMI.
This is a key issue given that U.S. medical
schools admit disproportionately few
URM and lower SES individuals.68 URM
admissions declined at many schools
in the wake of legislation restricting
consideration of race and ethnicity in
acceptance decisions,9,10 a trend that may
accelerate following a Supreme Court
decision upholding the constitutionality
of such restrictions.11 Concurrently,
medical education and health policy
groups call for a more diverse physician
workforce.1216 In such a climate, it is
necessary to evaluate how evolving
admissions trends like MMI adoption
may influence the diversity of medical
school classes.

among physicians.24 The effects of

bias during interviews can be reduced
by increasing structure (removing
ambiguity and, therefore, the tendency
to rely on stereotype-driven judgments)
and pooling evaluations from multiple
raters (potentially diluting or offsetting
individual biases).20,2527 In being
structured and incorporating multiple
raters perspectives, the MMI may
thus be less susceptible to implicit bias
effects than traditional medical school
interviews.

A long-recognized problem with

traditional nonstructured interviews
is vulnerability to interviewer biases
triggered by various applicant
characteristics.1722 Implicit (i.e.,
unconscious) biases disfavoring racial/
ethnic minority and lower SES persons
are common in U.S. society,23 including

Academic Medicine, Vol. 90, No. 12 / December 2015

Only three studies to our knowledge

have explored the associations of medical
school applicants racial/ethnic minority
status or SES with MMI performance.
In one study, involving six Canadian
schools, aboriginal status was negatively
correlated with MMI scores, whereas
family income level was not significantly
associated with MMI performance.28
The analyses included few aboriginal
participants (< 3%) and did not explore
other race/ethnicity categories. Further,
a robust indicator of SES would consider
factors beyond income, such as parental

1667

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Research Report

education.29 A single-school U.S. study

using a limited dichotomous (yes/no)
single-item self-report indicator of
disadvantaged status found no evidence
of an association between disadvantaged
status and MMI performance.30 Similarly,
a single-school United Kingdom study,
using a geographic area-based measure
of deprivation, found no association
between applicants deprivation score
and MMI performance.31 However,
ecological measures such as geographic
area deprivation scores have significant
limitations.32
To our knowledge, no studies have
examined whether applicants race/
ethnicity influences acceptance following
MMI participation, or whether race/
ethnicity or SES influences the likelihood
of being invited to an MMI. Such
outcomes are likely to be more strongly
influenced by parochial concerns (e.g.,
institutional mission focus) than the
MMI process,12,33 which is relatively
similar across schools.34,35 Nonetheless, it
is important to consider MMI invitation
and acceptance decisions to provide
context for evaluating the MMI-based
admissions process.
We examined the associations of
applicants URM status and SES with
MMI invitation, MMI performance, and
post-MMI acceptance recommendation
among applicants to the University
of California, Davis (UCD), School
of Medicine (SOM) in Sacramento,
California, over three admission cycles
(20112013), adjusting for other
applicant demographic characteristics and
postsecondary academic performance.
Method

We employed data collected as part of

the routine admissions processes during
the 2011, 2012, and 2013 application
cycles. The admissions office provided
relevant application data in an electronic
spreadsheet with personal identifiers
removed. The study was conducted
from April 18, 2014, through the end
of August 2014. The UCD institutional
review board reviewed the protocol and
determined it was exempt.
Application, screening, and MMI
invitation and scheduling
Applicants initially applied to the
UCD SOM via the American Medical
College Application Service (AMCAS).

1668

Following initial screening based on

cumulative grade point average (GPA)
and Medical College Admission Test
(MCAT) scores, admissions committee
members reviewed all applications and
invited a subset of applicants to submit a
secondary application. Faculty evaluated
secondary applications for invitation
to an MMI based on cumulative
GPA and MCAT scores, personal
statements, extracurricular activities,
recommendation letters, and other
characteristics that could contribute to
fulfilling the educational and service
missions of the school. Invited applicants
self-scheduled their MMI sessions via an
online portal.
MMI process and scoring
The MMI consisted of 10 individual
10-minute stations. At each station,
applicants had 2 minutes to read a brief
set of instructions, and 8 minutes to
address the assigned tasks on entering
the room. Nine stations assessed skills
in the following domains: integrity/
ethics, professionalism, interpersonal
communication, diversity/cultural
awareness, teamwork, ability to handle
stress, and problem solving. An additional
station asked applicants to explain their
choice to pursue a career in medicine.
Most stations were adapted from content
developed at McMaster University and
marketed by ProFitHR.34
A single trained rater, blinded to
participants AMCAS application
information, attended each station. In
some stations, raters interacted directly
with applicants. At others, raters observed
applicant interactions with actors or
other applicants. There were 216 different
raters during the study period; the mean
number of MMI stations that each
evaluated was 104 (standard deviation
[SD] 61.9; range 8276). Women made
up 61% of raters. Rater professional
backgrounds were as follows: physicians,
31%; medical students, 15%; other
clinicians (e.g., nurses), 11%; basic
science faculty, 6%; patients, 2%; and
various nonclinician leaders (e.g., deans),
professionals (e.g., lawyers), and highlevel administrative staff (e.g., curriculum
manager), 35%. The range of rater
backgrounds reflected the conviction
that diverse perspectives are helpful in
selecting future physicians who will be
able to work effectively with people from
all walks of life. Mandatory rater training
included a one-hour course reviewing the

admissions process, rater roles and duties,

and the need to avoid pursuing protected
class issues (e.g., race/ethnicity, gender).36
At each station, raters scored overall
applicant performance using an
anchored four-point scale: 0, < 25th
percentile performance (relative to other
applicants); 1, 25th50th percentile;
2, 51st75th percentile; or 3, > 75th
percentile. Raters were instructed
to consider both the applicants
communication abilities and the content
(e.g., comprehensiveness) of their
statements in assigning ratings. The
total MMI score was the mean of each
applicants individual station scores.
Scale internal consistency (Cronbach
alpha=0.67) was comparable to that
observed in other MMI studies.2,18,3741
Acceptance recommendation
The admissions committee met weekly
during the admission cycle to review each
participants MMI performance, AMCAS
application, and secondary application.
Subsequently, the committee made one
of the following recommendations:
reject, low waitlist, high waitlist, or offer
acceptance. For the current analyses, we
dichotomized the recommendation (offer
acceptance versus not).
URM status
We determined URM status (URM
[black, Southeast Asian, Native
American, or Pacific Islander race and/
or Hispanic ethnicity] versus not [all
other responses]) from self-reported
race/ethnicity information in the
AMCAS application. These groups
remain underrepresented in the medical
profession relative to the general
population.42
Socioeconomic disadvantage
We developed a composite measure of
SES using self-reported information
in the AMCAS application, screening
candidate indicators for inclusion using
logistic regression analyses and based
on their contribution to predicting
applicants self-designated disadvantaged
status. The following predictors (yes/
no items except where indicated) were
significant and maximized the area under
the receiver operating characteristic curve
(0.95): fee assistance received for medical
school application (yes/no); childhood
spent in an underserved area; family
recipients of family assistance program;

Academic Medicine, Vol. 90, No. 12 / December 2015

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Research Report

Table 1
Characteristics of 7,964 Applicants to the University of California, Davis, School of Medicine
Submitting Secondary Applications, by MMI Participation Status, From a Study of Applicant
Race and Ethnicity, Socioeconomic Status, and MMI-Based Admissions Outcomes, 20112013
Characteristica

Not invited to an
MMI, n = 6,389

Participated
in an MMI, n = 1,420

All screened applicants,b

N = 7,964

Demographics
Age category (years), no. (%)
< 22

1,113 (17.4)

213 (15.0)

1,363 (17.1)

22

1,664 (26.0)

306 (21.5)

2,011 (25.3)

23

1,388 (21.7)

284 (20.0)

1,699 (21.3)

> 24

2,224 (34.8)

617 (43.5)

2,891 (36.3)

Female, no. (%)

2,884 (45.1)

715 (50.4)

3,678 (46.2)

Underrepresented minority, no. (%)c

1,172 (18.3)

368 (25.9)

1,570 (19.7)

742 (11.6)

425 (29.9)

1,202 (15.1)

627 (9.8)

274 (19.3)

927 (11.6)

662 (10.4)

329 (23.2)

1,019 (12.8)

Socioeconomic factors
Self-designated disadvantaged, no. (%)
Fee assistance, no. (%)
Underserved childhood, no. (%)
Family on assistance, no. (%)
Any need-based scholarship, no. (%)

981 (15.4)

403 (28.4)

1,416 (17.8)

1,406 (22.0)

493 (34.7)

1,941 (24.4)

Highest parental education level, no. (%)

Less than high school

61 (1.0)

39 (2.7)

105 (1.3)

High school graduate

259 (4.1)

117 (8.2)

386 (4.8)

Some college

440 (6.9)

115 (8.1)

563 (7.1)

5,629 (88.1)

1,149 (80.9)

6,910 (86.8)

< $25,000

303 (4.7)

163 (11.5)

480 (6.0)

$25,000 to < $50,000

531 (8.3)

190 (13.4)

735 (9.2)

$50,000 to < $75,000

530 (8.3)

132 (9.3)

670 (8.4)

5,025 (78.7)

935 (65.8)

6,079 (76.3)

College graduate
Family income category, no. (%)

 $75,000
Contributed to family income, no. (%)

436 (6.8)

220 (15.5)

656 (8.4)

% Family contributed to college costs, mean (SD)

29.1 (39.1)

22.7 (35.1)

27.8 (38.4)

SES score, mean (SD)d

0.12 (0.23)

0.25 (0.34)

0.14 (0.26)

Academic factors
Cumulative GPA category, no. (%)
< 3.4

1,070 (16.7)

201 (14.2)

1,283 (16.1)

3.4 to 3.6

1,827 (28.6)

293 (20.6)

2,146 (26.9)

> 3.6 to 3.8

2,060 (32.2)

404 (28.5)

2,510 (31.5)

> 3.8

1,432 (22.4)

522 (36.8)

2,025 (25.4)

Total MCAT score category, no. (%)

19 to 26

277 (4.3)

136 (9.6)

416 (5.2)

27 to 30

1,818 (28.5)

305 (21.5)

2,145 (26.9)

31 to 32

1,517 (23.7)

262 (18.5)

1,797 (22.6)

33 to 34

1,370 (21.4)

246 (17.3)

1,644 (20.6)

35 to 44

1,407 (22.0)

471 (33.2)

1,962 (24.6)

Abbreviations: MMI indicates Multiple Mini-Interview; SD, standard deviation; SES, socioeconomic status; GPA, grade point
average; MCAT, Medical College Admission Test.
a
For all of the characteristics listed, P < .001 for the difference between those not invited to an MMI and those who
participated in an MMI (chi-square test for categorical variables and t test for continuous variables).
b
Of the 15,844 people who applied to the medical school during the three-year study period, 8,933 (56.4%) were invited
to submit secondary applications. Of these invitees, 7,964 (89.2%) submitted secondary applications, all of which were
individually screened by the admissions committee to determine MMI invitations.
c
Applicants indicating black, Southeast Asian, Native American, or Pacific Islander race and/or Hispanic ethnicity.
d
Scaled 01.0, higher score meaning lower SES. Derived using logistic regression predicting self-designated disadvantaged status
based on responses to the following application items: fee assistance received for application (yes/no); underserved childhood
(yes/no); family on assistance (yes/no); contributed to family income as child (yes/no); total family income (in dollars); parents
highest level of education (< high school, high school graduate, some college, college graduate); received financial-need-based
scholarship(s) in college (yes/no); and percentage of college costs contributed by family. See Method for details.

Academic Medicine, Vol. 90, No. 12 / December 2015

1669

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Research Report

continuous score), and application year

(2011, 2012, or 2013). The acceptance
recommendation model additionally
included the total MMI score (03).

Table 2
Results of a Logistic Regression Model Examining Associations of Characteristics
of 7,964 Applicants to the University of California, Davis, School of Medicine
Submitting a Secondary Application With MMI Invitation, From a Study of
Applicant Race and Ethnicity, Socioeconomic Status, and MMI-Based Admissions
Outcomes, 20112013
Adjusted OR
(95% CI)a

Variable

Results

P value

Age category (reference < 22)

22

1.10 (0.91, 1.33)

.30

23

1.33 (1.09, 1.62)

< .01

24+

1.87 (1.56, 2.24)

< .001

Female

1.36 (1.21, 1.53)

< .001

Underrepresented minorityb

1.14 (0.98, 1.33)

.08

SES score

5.95 (4.76, 7.44)

< .001

< 3.4

0.32 (0.26, 0.40)

< .001

3.43.6

0.38 (0.32, 0.45)

< .001

> 3.63.8

0.52 (0.45, 0.60)

< .001

GPA (reference > 3.8)

MCAT (reference 35)

< 27

0.69 (0.53, 0.90)

< .01

2730

0.37 (0.31, 0.44)

< .001

3132

0.43 (0.36, 0.51)

< .001

3334

0.50 (0.43, 0.60)

< .001

Abbreviations: MMI indicates Multiple Mini-Interview; OR, odds ratio; CI, confidence interval; SES,
socioeconomic status; GPA, grade point average; MCAT, Medical College Admission Test.
a
The logistic regression model also included admission year (2011, 2012, or 2013).
b
Applicants indicating black, Southeast Asian, Native American, or Pacific Islander race and/or Hispanic ethnicity.
c
Scaled 01.0, higher score meaning lower SES. Derived using logistic regression predicting self-designated
disadvantaged status based on responses to the following application items: fee assistance received for
application (yes/no); underserved childhood (yes/no); family on assistance (yes/no); contributed to family income
as child (yes/no); total family income (in dollars); parents highest level of education (< high school, high school
graduate, some college, college graduate); received financial-need-based scholarship(s) in college (yes/no); and
percentage of college costs contributed by family. See Method for details.

income level category of applicants

family (< $25,000; $25,000 to < $50,000;
$50,000 to < $75,000; or > $75,000);
applicant contributed to family income;
any financial-need-based scholarship(s)
in paying for postsecondary education;
percentage of postsecondary education
costs contributed by the family; and
parents highest level of educational
attainment (< high school, high school
graduate, some college, or college
graduate). The model yielded a predicted
probability of being self-designated
disadvantaged, ranging continuously
from 0 to 1.0 (higher predicted
probability = lower SES). We employed
the score, which correlated 0.91 with a
factor-analytic-derived score, as the study
measure of applicant SES. The SES score
was preferable to alternatives29 because
the continuous scale acknowledges
that socioeconomic disadvantage is
not a binary characteristic and reduces
misclassification (false positives and
negatives).

1670

During the three study application

cycles, 15,844 people applied, and
8,933 (56.4%) were invited to submit
secondary applications. Of the invitees,
7,964 (89.2%) submitted secondary
applications, and 1,575 (19.8%) were
invited to an MMI. Of the MMI invitees,
1,420 (90.2%) attended an MMI.
Applicant characteristics
Table1 summarizes the characteristics
of screened applicants. Compared with
those not invited to an MMI, those who
participated in an MMI were older and
more likely to be female, from an URM
group, disadvantaged (based on both selfdesignation and the continuous SES score),
and had higher GPAs and MCAT scores.
MMI invitation
Although adjusted URM status was not
associated with MMI invitation, lower
SES was associated with receiving an
MMI invitation, as were older age, female
gender, and higher GPA and MCAT score
(Table2).
MMI score

The admissions office also provided

information from AMCAS regarding
applicant age, sex, cumulative
postsecondary GPA, and total MCAT score.

Mean MMI score was 1.30 (SD 0.62). In

adjusted analyses, URM status was not
associated with MMI score, but lower SES
was associated with lower MMI scores
(Table3). Older age, female gender, and
lower GPA were also associated with
MMI score (Table3).

Data analysis

Acceptance recommendation

We analyzed the data using Stata version

13.1 (Stata Corporation Inc., College
Station, Texas). We modeled MMI
invitation (yes/no) and medical school
acceptance recommendation (accept
versus not) using logistic regression (a
separate regression for each dependent
variable). We modeled total MMI score
using linear regression. All models
included the following characteristics:
age category (< 22 [reference], 22, 23,
or > 24 years); female gender (yes/no);
URM status (versus not); cumulative
GPA category (< 3.4, 3.43.6, > 3.63.8,
or > 3.8 [reference]); total MCAT
score category (1926, 2730, 3132,
3334, or > 34 [reference]); SES (01.0

Of the 1,420 MMI participants, 334

(23.5%) were recommended for acceptance.
Lower SES was associated with being
recommended for acceptance, whereas
URM status was not (Table4). Of other
factors examined, mean MMI score was
the most strongly associated with being
recommended for acceptance. Older age,
female gender, and higher GPA and MCAT
scores were also associated with being
recommended for acceptance (Table4).

Other characteristics

Discussion

In a diverse sample of applicants over

three admission cycles at UCD, superior
MMI performance was strongly linked

Academic Medicine, Vol. 90, No. 12 / December 2015

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Research Report

Table 3
Results of a Linear Regression Model Examining Associations of Characteristics
of 1,420 Applicants to the University of California, Davis, School of Medicine
Completing an MMI With Total MMI Score, From a Study of Applicant Race and
Ethnicity, Socioeconomic Status, and MMI-Based Admissions Outcomes, 20112013
Adjusted PE
(95% CI)a

P value

Age category (reference < 22)

22

0.15 (0.05, 0.25)

< .01

23

0.14 (0.04, 0.25)

< .01

24+

0.28 (0.18, 0.38)

< .001

Variable

Female

0.14 (0.08, 0.20)

< .001

0.00 (0.08, 0.08)

.97

0.12 (0.23, 0.01)

.03

Underrepresented minorityb
SES scorec
GPA (reference > 3.8)
< 3.4
3.43.6
> 3.63.8

0.07 (0.05, 0.18)

.25

0.15 (0.05, 0.24)

< .01

0.06 (0.02, 0.14)

.12

MCAT (reference 35)

< 27

0.00 (0.13, 0.14)

.96

2730

0.03 (0.07, 0.13)

.56

3132

0.03 (0.06, 0.12)

.54

3334

0.05 (0.04, 0.14)

.31

rater implicit biases regarding racial/

ethnic minorities, such biases have been
documented to be pervasive in U.S.
society, including among physicians and
other professionals,23,24 and can affect the
outcomes of employment interviews in
various fields including medicine.17,1922
Thus, it is likely that implicit biases were
present among our raters; however, they
did not exert a significant net influence,
given that mean MMI scores did not
differ between URM and non-URM
applicants. Because lack of URMs in
medicine is a widely acknowledged
problem,6,7,13,33,4244 it is possible that
biases against URM applicants were
offset by ratings biased in favor of URM
applicants, made by raters seeking to
address limited racial/ethnic diversity in
the physician workforce.

Although the reasons for the lower MMI

performance among lower SES applicants
are unclear, poorer postsecondary
academic preparation and performance
are unlikely explanations because we
adjusted for GPA and MCAT score.
Lower SES applicants may have fewer
life experiences bolstering skills assessed
by the MMI. Similar reasoning has
been suggested to explain the lower
MCAT scores among such applicants.45
Although less affluent applicants are
more likely to report paid employment
during postsecondary education, their
financial circumstances may require
taking jobs that do not require MMI-type
preemployment screening. Lack of prior
experience with MMI-type screening
may be a disadvantage in the medical
school MMI because prior experience
with a particular interview format is
associated with better future performance
with that format.46 Lower-level jobs
also may not facilitate the higher-level
communication, critical thinking, and
problem-solving skills the MMI assesses,
and the time required for such jobs may
limit participation in pursuits that build
such skills (e.g., scholarly presentations,
volunteer clinic work).

Our findings regarding MMI

performance may have the broadest
applicability, given the relatively high
standardization of the MMI across
institutions.34,35 By contrast, MMI

Rater implicit bias could also help to

explain the lower MMI scores among lower
SES applicants. Raters were not provided
any information about participants,
and only one MMI station afforded

Abbreviations: MMI indicates Multiple Mini-Interview; PE, parameter estimate; CI, confidence interval; SES,
socioeconomic status; GPA, grade point average; MCAT, Medical College Admission Test.
a
The linear regression model also included admission year (2011, 2012, or 2013).
b
Applicants indicating black, Southeast Asian, Native American, or Pacific Islander race and/or Hispanic ethnicity.
c
Scaled 01.0, higher score meaning lower SES. Derived using logistic regression predicting self-designated
disadvantaged status based on responses to the following application items: fee assistance received for
application (yes/no); underserved childhood (yes/no); family on assistance (yes/no); contributed to family income
as child (yes/no); total family income (in dollars); parents highest level of education (< high school, high school
graduate, some college, college graduate); received financial-need-based scholarship(s) in college (yes/no); and
percentage of college costs contributed by family. See Method for details.

with being recommended for acceptance.

Further, URM applicants were no less
likely than non-URM applicants to
receive an MMI invitation, performed
similarly on the MMI, and were just as
likely to be recommended for acceptance.
In the only prior study to exploring
this issue, MMI performance at six
Canadian medical schools was worse
for the < 3% of applicants with selfreported aboriginal status, whereas
income was not significantly associated
with MMI scores.28 Our findings provide
some reassurance that adoption of the
MMI-based admissions process at U.S.
medical schools need not adversely affect
admission prospects for URM applicants.
The similar MMI scores for URM
and non-URM participants support
the notion that structured interview
processes that incorporate the
perspectives of multiple evaluators like
the MMI may be less vulnerable to the
effects of individual evaluator implicit
biases.20,2527 Although we did not measure

invitation and acceptance decisions

are shaped more by parochial concerns
such as institutional mission and local
workforce needs.12,33 In this context, our
finding that lower SES applicants had
worse adjusted MMI performance may
be cause for concern. Although three
prior studies reported no association of
applicant SES with MMI performance,
all relied on less robust measures of SES.
Nonetheless, the decrement in MMI
performance with decreasing SES in
our study was small: The MMI score
(scale of 03 points) declined by a mean
of 0.12 points across the 01 range of
the SES score. Further, the lower MMI
scores among lower SES applicants
were more than offset by their greater
likelihood of being invited to an MMI
and recommended for acceptance. These
findings may reflect the ongoing shift
from a purely metric-based applicant
review process toward the more holistic
process advocated by the Association of
American Medical Colleges.12,15

Academic Medicine, Vol. 90, No. 12 / December 2015

1671

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Research Report

Limitations

Table 4
Results of a Logistic Regression Model Examining Associations of Characteristics
of 1,420 Applicants to the University of California, Davis, School of Medicine
Completing an MMI With Admissions Committee Recommendation to Accept, From
a Study of Applicant Race and Ethnicity, Socioeconomic Status, and MMI-Based
Admissions Outcomes, 20112013
Variable

Adjusted OR
(95% CI)a

P value

Age category (reference < 22)

22

1.14 (0.62, 2.11)

.66

23

2.13 (1.15, 3.95)

.02

24+

1.78 (1.01, 3.15)

.05

Female

1.42 (1.02, 1.99)

.04

Underrepresented minorityb

1.08 (0.69, 1.68)

.74

SES scorec

3.28 (1.79, 6.00)

< .001

< 3.4

0.14 (0.07, 0.29)

< .001

3.43.6

0.51 (0.32, 0.83)

< .01

> 3.63.8

0.84 (0.56, 1.26)

.39

< 27

0.25 (0.12, 0.54)

< .001

2730

0.28 (0.16, 0.48)

< .001

3132

0.69 (0.43, 1.11)

.12

3334

0.56 (0.34, 0.90)

.02

35.24 (23.14, 53.67)

< .001

GPA (reference > 3.8)

MCAT (reference > 35)

MMI scored

Abbreviations: MMI indicates Multiple Mini-Interview; OR, odds ratio; CI, confidence interval; SES,
socioeconomic status; GPA, grade point average; MCAT, Medical College Admission Test.
a
The logistic regression model also included admission year (2011, 2012, or 2013).
b
Applicants indicating black, Southeast Asian, Native American, or Pacific Islander race and/or Hispanic ethnicity.
c
Scaled 01.0, higher score meaning lower SES. Derived using logistic regression predicting self-designated
disadvantaged status based on responses to the following application items: fee assistance received for
application (yes/no); underserved childhood (yes/no); family on assistance (yes/no); contributed to family income
as child (yes/no); total family income (in dollars); parents highest level of education (< high school, high school
graduate, some college, college graduate); received financial need-based scholarship(s) in college (yes/no); and
percentage of college costs contributed by family. See Method for details.
d
Mean of 10 individual station scores assigned by application-blinded raters; score range 03.

applicants the opportunity to describe

their backgrounds. Nonetheless, subtle
information apparent to raters could have
led to implicitly biased ratings, possibly
triggered by the generally relatively
large social distance between lower SES
applicants and the typical rater.47,48 Most
of our MMI raters were well educated and
relatively affluent. Prior work indicates that
applicant factors such as use of language
unfamiliar to the typical rater could trigger
a biased low rating.20,21 Applicants verbal
skills have been shown to determine
immediate interviewer impressions and,
in turn, final appraisals.49 The issue of
SES-based physician workforce disparities
has received less attention than race/
ethnicity-based disparities.6 Thus, it is less
likely that raters consciously biased their
evaluations in favor of lower SES applicants
to address SES-based physician workforce
disparities. If our findings are replicated,
it would suggest the need to consider rater

1672

training to minimize the influence of

SES-basedbiases.
We found that older applicants and
women performed better than their
younger and male counterparts both in the
MMI (consistent with prior studies28,37)
and throughout the admissions process.
Older applicants are more likely to have
had life experiences requiring effective
communication. Women, more than men,
have been noted to communicate in ways
that quickly build rapport in novel social
situations such as the MMI. Previously
underrepresented in medicine, women
now constitute well over half of all U.S.
medical students.50 Given the increasing
adoption of the MMI and strong influence
of MMI performance on medical school
acceptance, our findings suggest the
potential for a disparity to develop
disfavoring male applicants, and warrant
monitoring.

Our study was correlational, limiting

causal inferences. Our data were derived
from applicants at a single school. How
the findings generalize to other schools
is unclear. We lacked sociodemographic
information regarding MMI raters
and admissions committee personnel,
precluding examination of how such
characteristics may have influenced the
study outcomes. We employed a novel
composite measure of SES, albeit one
with theoretical advantages over other
indicators.29 Because schools vary widely
in institutional mission,33 applicant
pools, admissions personnel, and other
attributes, multi-institution studies
are needed to better gauge the impact
of MMI-based admissions processes
on URM and socioeconomically
disadvantaged applicants. Studies ideally
should account for the characteristics of
admissions personnel and applicants, and
examine the relative utility of different
SES indicators. It is also unknown
whether URM or lower SES applicants
would have fared differently in oneon-one interview-based admissions
processes. Ideally, randomized trials
would compare traditional interview
processes with MMI-based processes,
examining their impact on the prospects
of URM and lower SES applicants.
Conclusion
In conclusion, in analyses of data from
one California medical school, an
MMI-based admissions process did not
disfavor racial/ethnic minority groups
underrepresented in the physician
workforce. Applicants from lower SES
backgrounds, also underrepresented in
medicine, had lower MMI scores but
were more likely to receive an MMI
invitation and be recommended for
acceptance. Multischool collaborations
are needed to further evaluate the
impact of MMI-based medical school
admissions processes on URM and lower
SES applicants.
Funding/Support: None reported.
Other disclosures: None reported.
Ethical approval: On April 18, 2014, the University
of California Davis (UCD) institutional review
board (IRB) reviewed the study protocol (UCD
IRB reference number 599704-1) and concluded
that it is not human subjects research, based on
the use of completely anonymous data that were
collected for a purpose other than the research,
and is therefore exempted from IRB approval and
oversight.

Academic Medicine, Vol. 90, No. 12 / December 2015

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Research Report
A. Jerant is professor, Department of Family and
Community Medicine, Center for Healthcare Policy
and Research, University of California, Davis, School
of Medicine, Sacramento, California.
T. Fancher is associate professor, Division of
General Internal Medicine, Department of Internal
Medicine, University of California, Davis, School of
Medicine, Sacramento, California.
J.J. Fenton is associate professor, Department
of Family and Community Medicine, Center for
Healthcare Policy and Research, University of
California, Davis, School of Medicine, Sacramento,
California.
K. Fiscella is professor, Department of Family
Medicine, University of Rochester School of Medicine
and Dentistry, Rochester, New York.
F. Sousa is assistant dean, Admissions and Student
Development, and volunteer clinical professor,
Department of Internal Medicine, University of
California, Davis, School of Medicine, Sacramento,
California.
P. Franks is professor, Department of Family and
Community Medicine, Center for Healthcare Policy
and Research, University of California, Davis, School
of Medicine, Sacramento, California.
M. Henderson is associate dean, Admissions
and Outreach, and professor, Division of General
Medicine, Department of Internal Medicine,
University of California, Davis, School of Medicine,
Sacramento, California.

References
1 Eva KW, Reiter HI, Rosenfeld J, Norman GR.
The ability of the multiple mini-interview to
predict preclerkship performance in medical
school. Acad Med. 2004;79(10 suppl):S40
S42.
2 Eva KW, Reiter HI, Trinh K, Wasi P, Rosenfeld
J, Norman GR. Predictive validity of the
multiple mini-interview for selecting medical
trainees. Med Educ. 2009;43:767775.
3 Reiter HI, Eva KW, Rosenfeld J, Norman GR.
Multiple mini-interviews predict clerkship
and licensing examination performance. Med
Educ. 2007;41:378384.
4 Griffin B, Wilson I. Associations between
the big five personality factors and multiple
mini-interviews. Adv Health Sci Educ Theory
Pract. 2012;17:377388.
5 Harris G. New for aspiring doctors, the
people skills test. NY Times. July 10, 2011:A1.
http://www.nytimes.com/2011/07/11/health/
policy/11docs.html?pagewanted=al. Accessed
April 4, 2015.
6 Magnus SA, Mick SS. Medical schools,
affirmative action, and the neglected
role of social class. Am J Public Health.
2000;90:11971201.
7 Saha S, Shipman SA. Race-neutral versus
race-conscious workforce policy to improve
access to care. Health Aff (Millwood).
2008;27:234245.
8 Association of American Medical Colleges.
Minorities in Medical Education: Facts and
Figures 2005. Washington, DC: Association
of American Medical Colleges; 2005.
9 Steinecke A, Terrell C. After affirmative
action: Diversity at California medical
schools. AAMC Analysis in Brief. 2008;8(6).
https://www.aamc.org/download/102358/
data/aibvol8no6.pdf. Accessed April 4, 2015.

10 Association of American Medical Colleges.

Assessing Medical School Admissions
Policies: Implications of the U.S. Supreme
Courts Affirmative-Action Decisions.
Washington, DC: Association of American
Medical Colleges; 2003. http://www.aacp.
org/resources/studentaffairspersonnel/
admissionsguidelines/Documents/
AAMCassessmedschooladmissions.pdf.
Accessed April 4, 2015.
11 Liptak A. Supreme Court upholds Michigans
affirmative action ban. NY Times. April 22,
2014. http://www.nytimes.com/2014/04/23/us/
supreme-court-michigan-affirmative-actionban.html?hp&_r=1. Accessed April 4, 2015.
12 Association of American Medical Colleges.
Holistic review. https://www.aamc.org/
initiatives/holisticreview/. Accessed April 4,
2015.
13 Saha S, Guiton G, Wimmers PF, Wilkerson L.
Student body racial and ethnic composition
and diversity-related outcomes in US medical
schools. JAMA. 2008;300:11351145.
14 Walker KO, Moreno G, Grumbach K. The
association among specialty, race, ethnicity,
and practice location among California
physicians in diverse specialties. J Natl Med
Assoc. 2012;104:4652.
15 Addams AN, Bletzinger RB, Sondheimer
HM, White SE, Johnson LM. Roadmap
to Diversity: Integrating Holistic Review
Practices Into Medical School Admissions
Processes. Washington, DC: Association
of American Medical Colleges; 2010.
https://members.aamc.org/eweb/upload/
Roadmap%20to%20Diversity%20
Integrating%20Holistic%20Review.pdf.
Accessed April 4, 2015.
16 Committee on Institutional and Policy-Level
Strategies for Increasing the Diversity of the
U.S. Health Care Workforce, Board on Health
Sciences Policy. In the Nations Compelling
Interest: Ensuring Diversity in the Health
Care Workforce. Washington, DC: National
Academies Press; 2004.
17 Salvatori P. Reliability and validity of
admissions tools used to select students for
the health professions. Adv Health Sci Educ
Theory Pract. 2001;6:159175.
18 Eva KW, Rosenfeld J, Reiter HI, Norman GR.
An admissions OSCE: The multiple miniinterview. Med Educ. 2004;38:314326.
19 Fiske ST, Markus HR. Facing Social Class:
How Societal Rank Influences Interaction.
New York, NY: Russell Sage Foundation;
2012.
20 Macan T. The employment interview: A
review of current studies and directions for
future research. Hum Resour Manage R.
2009;19:203218.
21 Purkiss SLS, Perrew PL, Gillespie TL,
Mayes BT, Ferris GR. Implicit sources of
bias in employment interview judgments
and decisions. Organ Behav Hum Dec.
2006;101:152167.
22 Cuddy AJ, Fiske ST, Kwan VS, et al.
Stereotype content model across cultures:
Towards universal similarities and some
differences. Br J Soc Psychol. 2009;48(pt
1):133.
23 Greenwald AG, Poehlman TA, Uhlmann EL,
Banaji MR. Understanding and using the
Implicit Association Test: III. Meta-analysis
of predictive validity. J Pers Soc Psychol.
2009;97:1741.

Academic Medicine, Vol. 90, No. 12 / December 2015

24 Chapman EN, Kaatz A, Carnes M. Physicians

and implicit bias: How doctors may
unwittingly perpetuate health care disparities.
J Gen Intern Med. 2013;28:15041510.
25 Kutcher EJ, Bragger JD. Selection interviews
of overweight job applicants: Can structure
reduce the bias? J Appl Soc Psychol.
2004;34:19932022.
26 Brecher E, Bragger J, Kutcher E. The
structured interview: Reducing biases toward
job applicants with physical disabilities.
Employ Responsib Rights J. 2006;18:155170.
27 Levashina J, Hartwell CJ, Morgeson FP,
Campion MA. The structured employment
interview: Narrative and quantitative review
of the research literature. Pers Psychol.
2014;67:241293.
28 Reiter HI, Lockyer J, Ziola B, Courneya CA,
Eva K; Canadian Multiple Mini-Interview
Research Alliance (CaMMIRA). Should
efforts in favor of medical student diversity
be focused during admissions or farther
upstream? Acad Med. 2012;87:443448.
29 Grbic D, Jones DJ, Case ST. Effective Practices
for Using the AAMC Socioeconomic Status
Indicators in Medical School Admissions.
Washington, DC: Association of American
Medical Colleges; 2013.
30 Uijtdehaage S, Doyle L, Parker N. Enhancing
the reliability of the multiple mini-interview
for selecting prospective health care leaders.
Acad Med. 2011;86:10321039.
31 Taylor CA, Green KE, Spruce A. Evaluation
of the effect of socio-economic status on
performance in a Multiple Mini Interview for
admission to medical school [published online
July 2, 2014]. Med Teach. 2015;37:5963.
32 Morgenstern H. Uses of ecologic analysis in
epidemiologic research. Am J Public Health.
1982;72:13361344.
33 Mullan F, Chen C, Petterson S, Kolsky G,
Spagnola M. The social mission of medical
education: Ranking the schools. Ann Intern
Med. 2010;152:804811.
34 Advanced Psychometrics for Transitions Inc.
Welcome to ProFitHR. http://www.profithr.
com/. Accessed April 4, 2015.
35 Pau A, Jeevaratnam K, Chen YS, Fall AA,
Khoo C, Nadarajah VD. The Multiple MiniInterview (MMI) for student selection in
health professions traininga systematic
review. Med Teach. 2013;35:10271041.
36 U.S. Equal Employment Opportunity
Commission. Title VII of the Civil Rights Act
of 1964. http://www.eeoc.gov/laws/statutes/
titlevii.cfm. Accessed April 4, 2015.
37 Jerant A, Griffin E, Rainwater J, et al. Does
applicant personality influence multiple
mini-interview performance and medical
school acceptance offers? Acad Med.
2012;87:12501259.
38 Dowell J, Lynch B, Till H, Kumwenda B,
Husbands A. The multiple mini-interview
in the U.K. context: 3 years of experience at
Dundee. Med Teach. 2012;34:297304.
39 Harris S, Owen C. Discerning quality: Using
the multiple mini-interview in student
selection for the Australian National University
Medical School. Med Educ. 2007;41:234241.
40 Till H, Myford C, Dowell J. Improving
student selection using multiple miniinterviews with multifaceted Rasch modeling.
Acad Med. 2013;88:216223.
41 Sebok SS, Luu K, Klinger DA. Psychometric
properties of the multiple mini-interview

1673

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Research Report
used for medical admissions: Findings
from generalizability and Rasch analyses.
Adv Health Sci Educ Theory Pract.
2014;19:7184.
42 Association of American Medical Colleges.
Underrepresented in medicine definition.
https://www.aamc.org/initiatives/urm/.
Accessed April 4, 2015.
43 Ko M, Heslin KC, Edelstein RA, Grumbach
K. The role of medical education in reducing
health care disparities: The first ten years of
the UCLA/Drew medical education program.
J Gen Intern Med. 2007;22:625631.
44 Whitla DK, Orfield G, Silen W, Teperow C,
Howard C, Reede J. Educational benefits

of diversity in medical school: A survey of

students. Acad Med. 2003;78:460466.
45 Davis D, Dorsey JK, Franks RD, Sackett PR,
Searcy CA, Zhao X. Do racial and ethnic
group differences in performance on the
MCAT exam reflect test bias? Acad Med.
2013;88:593602.
46 Huffcutt AI, Van Iddekinge CH, Roth
PH. Understanding applicant behavior in
employment interviews: A theoretical model
of interviewee performance. Hum Resour
Manage R. 2011;21:353367.
47 Dohrenwend BS, Colombotos J, Dohrenwend
BP. Social distance and interviewer effects.
Milbank Q. 1969;41:213226.

48 Howard JL, Ferris GR. The employment

interview context: Social and situational
influences on interviewer decisions. J Appl
Soc Psychol. 1996;26:112136.
49 Barrick MR, Dustin SL, Giluk TL, Stewart
GL, Shaffer JA, Swider BW. Candidate
characteristics driving initial impressions
during rapport building: Implications for
employment interview validity. J Occup
Organ Psychol. 2012;85:330352.
50 Association of American Medical Colleges.
U.S. medical school applicants and
students 19821983 to 20112012. https://
www.aamc.org/download/153708/data/
charts1982to2012.pdf. Accessed April 4, 2015.

Teaching and Learning Moments

Confessions of a Medical Voluntourist

On November 8, 2013, Typhoon Haiyan
displaced 4 million Filipinos. Over 6,000
people drowned, including three of my
relatives. Hundreds of unidentified bodies
were burned a month later. I believe that my
6-year-old cousin was among them; he was
never found. My 80-year-old grandmother
was found, under the family jeep; they
buried her properly. Most of my other
relatives survived. They clung to coconut
trees despite 195 mph winds, floated up to
the attic as water levels rose, and ate Spam
out of the can for days to stay alive.
Their suffering reached me in Rochester,
New York. I was eager to apply what I
had learned in my first year of medical
training to the recovery efforts. I arrived
in Tacloban and commuted to my
designated rural health clinic, only to
discover that my physician supervisor
would be gone the following week. I
was frustrated and upset. I had traveled
halfway across the world to spend the
last free summer of my life like this? To
be told to come back tomorrow and only
have three days to help? On my way out,
people hailing from over 65 barangays
(neighborhoods) waited under the
unrelenting sun to be seen at the clinic.
Their patient, amiable smiles met my
fuming eyes; it gave me pause. The next
day, I delivered the two bags of syringes I
had brought from Rochester, since I knew
the clinic had run out. With my shortened
stay, this was the only tangible way I
could contribute. Unfortunately, they
needed syringes with needles to administer

1674

tuberculosis and Depo-Provera shots. In

that moment, I realized how little I knew.
My assigned physician supervisor was
the sole physician responsible for the
care of thousands, so she had no time
to mentor me. Instead, I accompanied
nurses and midwives during their pediatric
vaccination community outreach. En route
to a neighboring barangay, a nurse pointed
out the unfinished bridge that caused
many of the motorcycle accident injuries
they saw in the clinic. A year earlier, the
president of the Philippines had designated
1.2 million pesos for its completion; no
one knew where the money had gone.
The region had a history of poverty,
corrupt politics, and the stigmatization
of its islanders as lazy. In addition to
Haiyans destruction, Tacloban was a site of
structural violence, the sociological term
Paul Farmer uses to highlight how social
institutions disempower people and cause
negative health consequences.
Shadowing nurses and midwives, I learned
how the quality and delivery of care hinged
on this cohesive health care team and
their fluid navigation of unmarked roads,
family/land politics, and relationships
with community officials, in addition
to their medical knowledge and skills.
Observing them in their element helped
me appreciate the approach, skills, and
knowledge I hope to bring to my future
patients: a multidisciplinary, team-based
understanding of social context grounded
in forging community relationships.

I brought back many lessons from

Tacloban to Rochester. I became the
education coordinator of my schools
Street Medicine chapter, which reaches
out to the homeless and connects them
with social services. In this role, I surveyed
clients about their priorities and found
that frostbite prevention, wound care, and
opiate overdose reversal kits were pressing
needs. Moreover, the relationships we
built during street rounds, with clients
and with local outreach teams of nurses,
deacons, and social workers, are critical
to being able to comprehensively address
the complex needs the clients identified.
Becoming a participantobserver taught
me that doctors are not the only source
of valuable health information. Indeed,
valuable health information extends
beyond biomedicine to the social map
of the community as well as that of
the health care team. My mistakes as a
medical voluntourist abroad taught me
my first lessons in patient-centered care
and health advocacy, lessons I hope to
continue to apply here at home.
Acknowledgments: The author wishes to thank
Cheryl Kodjo, Sonia Mendoza, and Helena Hansen
for their mentorship and support of this work.
Nichole Roxas
N. Roxas is a third-year medical student, University
of Rochester School of Medicine and Dentistry,
Rochester, New York; e-mail: nichole_roxas@urmc.
rochester.edu.
An AM Rounds blog post on this article is available
at academicmedicineblog.org.

Academic Medicine, Vol. 90, No. 12 / December 2015

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.