Anda di halaman 1dari 31

The Effectiveness of

Competency-Based Education

Stephen R. Smith, M.D.


Associate Dean for Medical Education
Brown Medical School
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Courtesy of Ronald Harden
Calls for Reform
• AMA Future Directions in Med Ed (1982)
• GPEP Report (1984)
• Macy Conference Report (1989)
• Bloom (1988)
• Robert Wood Johnson Foundation Program
in Medical Education (1989)
Flexnerian Model
➀ ② ③ ➃
Define Teach the Test for Hope for
“Fundamental Fundamentals Knowledge of the Best
Knowledge” Fundamentals

?
Competency-Based Model

Design
Develop Define the
Measures and
Learning Successful
Standards of
Experiences Graduate
Performance
Brown’s 9 Abilities
Effective Communication
Basic Clinical Skills
Using Basic Science in the Practice of Medicine
Diagnosis, Management, and Prevention
Lifelong Learning
Professional Development and Personal Growth
Social and Community Context of Medicine
Moral Reasoning and Clinical Ethics
Problem Solving
ACGME Outcome Project
General Competencies for Residents
• Patient Care
• Medical Knowledge
• Practice-based learning and improvement
• Interpersonal and communication skills
• Professionalism
• Systems-based practice
The Scottish Doctor
• What the doctor is able to do
– Clinical skills
– Practical procedures
– Patient investigations
– Patient management
– Health promotion and disease prevention
– Communication
– Medical informatics
The Scottish Doctor (2)
• How the doctor approaches practice
– Attitudes, ethical understanding, and legal
responsibilities
– Decision-making skills and clinical reasoning
and judgement
– Basic, social and clinical sciences and
underlying principles
The Scottish Doctor (3)
• The doctor as a professional
– Role of the doctor within the health care system
– Personal development
Definition
Lifelong Learning

The competent graduate is aware of the limits


of his/her personal knowledge and experience
and has an intellectual interest in general
education and medical science. The graduate
actively sets clear learning goals, pursues
them, and applies the knowledge gained to
the practice of his/her profession.
Criteria
1.Recognizes personal limits in knowledge
and experience
• initiates steps to rectify gaps in knowledge
• researches, reviews, and extracts data from
multiple sources
• confers with knowledgeable colleagues,
advisors, or experts
Levels
• Beginner: demonstrate proficiency in a
structured setting (e.g., medical school
class)
• Intermediate: demonstrate proficiency in a
clinical or research setting
• Advanced: submission for publication or
acceptance of abstract, poster, or oral
presentation at an academic meeting
Performance-based Assessment
• Authentic
• Increasing level of challenge
• Formative feedback
• P=KxA
QuickTime™ and a
DV/DVCPRO - NTSC decompressor
are needed to see this picture.
Evidence of Effectiveness
• Carraccio et al. (2002) reviewed 469 articles
• “revealed little scientific evidence evaluating the
outcomes of competency-based education”
• “What evidence that does exist clearly favors
competency-based education over the current
structure- and process-based model”
Clinical Skills

A competency-based curriculum
introduced into three medical schools in
China rapidly improved the clinical skills
of students compared to the traditional
curriculum.

From Stillman PL. Teaching and assessing clinical skills: a competency-based programme in
China. Med. Educ. 1997; 31:33–40.
Neurosurgical Training
40
35
30
25
20 Traditional
15 CBE

10
5
0
Laminectomy Spinal fusion Pterional Suboccipital
craniotomy craniotomy

From Long DM. Competency-based Residency Training: The Next Advance in


Graduate Medical Education. Acad. Med. 2000; 75:1178–1183.
Epidemiological Proof
• Strength of the Association
• Dose–Response Relationship
• Consistency of the Association
• Temporally Correct Association
• Specificity of the Association
• Coherence with Existing Information
(Biological Plausibility)
Educational Proof
• Theory and Principle (Educational
Plausibility)
• Presence of Desired Effects
• Absence of Adverse Effects
• Qualitative Assessment
• Near-term Quantitative Assessment
“The only way to get somewhere, you know, is
to figure out where you’re going before you go
there.”
——John Updike, Rabbit Run, 1960

“It is a highly questionable practice to label


someone as having achieved a goal when you
don’t even know what you would take as
evidence of achievement.”
——R. F. Mager, Preparing Educational
Objectives, 1962
Desired Effects
• Changes in learning behavior in desired
directions (e.g., clinical skills)
• More explicit guidance for students
• More authentic and relevant teaching and
assessment
Avoid Adverse Effects
• No diminution in USMLE/NBME scores
• Theoretical criticisms not realized
– Competencies not representative of tasks
– Reductionistic, trivializing, decontextualized
– Devalues knowledge
– Constrains teaching and creativity
• Not too expensive or resource intensive
Qualitative Assessment
• Concept valued by students
• Initial skepticism by faculty, but willingness
to go along
• Credibility gap created when only lip
service paid to competencies
• Resistance to jumping through more hoops
• Submerged into the culture
Ability VII Fulfillment
Independent Studies

72.5
80
70
60
50
33.7 Brown
40 U.S.
30
20
10
0
Evaluation Approaches
• Near Term
– Qualitative assessments
– Intermediate goals
• Long Range
– Comprehensive studies (à la Tamblyn)
– Long-term goals