Anda di halaman 1dari 31

The Effectiveness of

Competency-Based Education

Stephen R. Smith, M.D.

Associate Dean for Medical Education
Brown Medical School
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Calls for Reform
• AMA Future Directions in Med Ed (1982)
• GPEP Report (1984)
• Macy Conference Report (1989)
• Bloom (1988)
• Robert Wood Johnson Foundation Program
in Medical Education (1989)
Flexnerian Model
➀ ② ③ ➃
Define Teach the Test for Hope for
“Fundamental Fundamentals Knowledge of the Best
Knowledge” Fundamentals

Competency-Based Model

Develop Define the
Measures and
Learning Successful
Standards of
Experiences Graduate
Brown’s 9 Abilities
Effective Communication
Basic Clinical Skills
Using Basic Science in the Practice of Medicine
Diagnosis, Management, and Prevention
Lifelong Learning
Professional Development and Personal Growth
Social and Community Context of Medicine
Moral Reasoning and Clinical Ethics
Problem Solving
ACGME Outcome Project
General Competencies for Residents
• Patient Care
• Medical Knowledge
• Practice-based learning and improvement
• Interpersonal and communication skills
• Professionalism
• Systems-based practice
The Scottish Doctor
• What the doctor is able to do
– Clinical skills
– Practical procedures
– Patient investigations
– Patient management
– Health promotion and disease prevention
– Communication
– Medical informatics
The Scottish Doctor (2)
• How the doctor approaches practice
– Attitudes, ethical understanding, and legal
– Decision-making skills and clinical reasoning
and judgement
– Basic, social and clinical sciences and
underlying principles
The Scottish Doctor (3)
• The doctor as a professional
– Role of the doctor within the health care system
– Personal development
Lifelong Learning

The competent graduate is aware of the limits

of his/her personal knowledge and experience
and has an intellectual interest in general
education and medical science. The graduate
actively sets clear learning goals, pursues
them, and applies the knowledge gained to
the practice of his/her profession.
1.Recognizes personal limits in knowledge
and experience
• initiates steps to rectify gaps in knowledge
• researches, reviews, and extracts data from
multiple sources
• confers with knowledgeable colleagues,
advisors, or experts
• Beginner: demonstrate proficiency in a
structured setting (e.g., medical school
• Intermediate: demonstrate proficiency in a
clinical or research setting
• Advanced: submission for publication or
acceptance of abstract, poster, or oral
presentation at an academic meeting
Performance-based Assessment
• Authentic
• Increasing level of challenge
• Formative feedback
• P=KxA
QuickTime™ and a
DV/DVCPRO - NTSC decompressor
are needed to see this picture.
Evidence of Effectiveness
• Carraccio et al. (2002) reviewed 469 articles
• “revealed little scientific evidence evaluating the
outcomes of competency-based education”
• “What evidence that does exist clearly favors
competency-based education over the current
structure- and process-based model”
Clinical Skills

A competency-based curriculum
introduced into three medical schools in
China rapidly improved the clinical skills
of students compared to the traditional

From Stillman PL. Teaching and assessing clinical skills: a competency-based programme in
China. Med. Educ. 1997; 31:33–40.
Neurosurgical Training
20 Traditional
15 CBE

Laminectomy Spinal fusion Pterional Suboccipital
craniotomy craniotomy

From Long DM. Competency-based Residency Training: The Next Advance in

Graduate Medical Education. Acad. Med. 2000; 75:1178–1183.
Epidemiological Proof
• Strength of the Association
• Dose–Response Relationship
• Consistency of the Association
• Temporally Correct Association
• Specificity of the Association
• Coherence with Existing Information
(Biological Plausibility)
Educational Proof
• Theory and Principle (Educational
• Presence of Desired Effects
• Absence of Adverse Effects
• Qualitative Assessment
• Near-term Quantitative Assessment
“The only way to get somewhere, you know, is
to figure out where you’re going before you go
——John Updike, Rabbit Run, 1960

“It is a highly questionable practice to label

someone as having achieved a goal when you
don’t even know what you would take as
evidence of achievement.”
——R. F. Mager, Preparing Educational
Objectives, 1962
Desired Effects
• Changes in learning behavior in desired
directions (e.g., clinical skills)
• More explicit guidance for students
• More authentic and relevant teaching and
Avoid Adverse Effects
• No diminution in USMLE/NBME scores
• Theoretical criticisms not realized
– Competencies not representative of tasks
– Reductionistic, trivializing, decontextualized
– Devalues knowledge
– Constrains teaching and creativity
• Not too expensive or resource intensive
Qualitative Assessment
• Concept valued by students
• Initial skepticism by faculty, but willingness
to go along
• Credibility gap created when only lip
service paid to competencies
• Resistance to jumping through more hoops
• Submerged into the culture
Ability VII Fulfillment
Independent Studies

33.7 Brown
40 U.S.
Evaluation Approaches
• Near Term
– Qualitative assessments
– Intermediate goals
• Long Range
– Comprehensive studies (à la Tamblyn)
– Long-term goals