Critical Appraisal

27 Januari 2016
ProDi S3 FK UGM
http://rossisanusi.wordpress.com
 Critical Appraisal (CA)?
• CA makalah penelitian empirik (PE) = memeriksa
apakah kesimpulannya valid (berbobot).
• Kesimpulan PE: hipotesis penelitian didukung/
tidak?
• Hipotesis Penelitian: Effect Size bermakna (kuat)
a. r ≥ rmin
b. d ≥ dmin
c. OR ≥ ORmin
 Manafsirkan ES
• Cohen’s “Rules-of-Thumb”
– standardized mean difference effect size
• small = 0.20
• medium = 0.50
• large = 0.80
– correlation coefficient
• small = 0.10
• medium = 0.25
• large = 0.40
– odds-ratio
• small = 1.50
• medium = 2.50
• large = 4.30
Sumber: http://mason.gmu.edu/~dwilsonb/downloads/interpretation.ppt

Practical Meta-Analysis -- D. B. Wilson 3

 Kesimpulan PE berbobot jika
1. Kerangka konsep valid.
2. Rancangan penelitian kokoh (robust).
3. Pelaksanaan penelitian seksama (high fidelity).

CA laporan PE: No 1, 2 dan 3 dipenuhi? 

“YA”, “TIDAK” atau “TIDAK DAPAT DINILAI”.

Bagaimana jika kesimpulan yg berbobot mengatakan

bahwa Hipotesis Penelitian tidak didukung?
 PENGGUNA
(Praktisi, Perencana, PENGHASIL
Pembuat Keputusan, (Peneliti)
Anggota Masyarakat)
Menghadapi
Menghadapi Masalah Masalah Penelitian
Praktis dan dalam menyediakan
membutuhkan bukti bukti yg valid ttg
yg valid ttg intervensi intervensi efikasius
efikasius pengendalian
pengendalian penyakit2 prioritas
penyakit2 prioritas
Sumber:
http://evidencebasedliving.human.cornell.edu/2010/04/20/evidence-based-systematic
 Hirarki
Bukti

Sumber: http://www.mededirect.org/faculty.cfm
Contoh Systematic Review:
Klinik: Houwert, R. M., Wijdicks, F. J., Bisschop, C. S., Verleisdonk, E. J., & Kruyt, M. (2012). Plate fixation
versus intramedullary fixation for displaced mid-shaft clavicle fractures: a systematic review. International
Orthopaedics, 36(3), 579-585.
1. Kerangka Kosep Valid? Definisi operasional konstruk2 valid menurut para ahli
dan hasil penelitian sebelumnya.

Mediator?
(Mengapa,
Bagaimana)

Prediktor? Kriterion?
(Intervensi) (Hasil)

Moderator?
(Ciri-ciri &
keadaan Unit
Analisis
9
2. Rancangan Penelitian Kuat?
a. Rancangan Pengumpulan Data:
• Menggunakan alat atau prosedur yang
valid? (berdasarkan hasil penelitian
sebelumnya dan hasil validasi penelitian
sekarang).
• Alat/prosedur digunakan secara
konsisten? (berdasarkan hasil uji
reliabilitas intra- dan inter- pengumpul
data)
 Pengumpulan Data
Konstruk Variabel2 Skala Unit Perlakuan
Penga-
matan

Prediktor
Kriterion
Mediator
Moderator:
a. Spesifik
b. Confound-
ing.
b. Rancangan Pengolahan data:
• Menggunakan metoda statistik diskriptif
dan statistik inferensial yg tepat?
(memenuhi syarat-syarat penggunaan
metoda yg bersangkutan)
c. Rencangan Penafsiran data:
• Validitas Dalam rendah? (Effect Size
kebetulan karena sampling error besar;
Effect Size semu karena moderator2 spesifik
dan variabel2 confounding diabaikan).
• Validitas Luar rendah? (banyak moderator2
spesifik dikendalikan)
 Penafsiran Data
Pop Sasaran

Pop yg
Disampel
Validitas
Luar
Inferensi statistik

Validitas Dalam Sampel

3. Pelaksanaan Penelitian Seksama?
(penggunaan sumberdaya sesuai dengan
metoda pengumpulan, pengolahan dan
penafsiran data yang dirancang)
Contoh Makalah SR:
Bergs, J., Hellings, J., Cleemput, I., Zurel, O., De
Troyer, V., Van Hiel, M., Demeere, J.-L, Claeys, D.,
& Vandijck, D. (2014). Systematic review and
meta-analysis of the effect of the World Health
Organization surgical safety checklist on
postoperative complications. British Journal of
Surgery, 101: 150-158.
Masalah penelitian (awal):
Apakah penerapan SSC WHO pada pembedahan
mayor menurunkan komplikasi dan kematian pasca
operasi?

Karena diperlukan bukti yg lebih kuat, maka Tujuan

Penelitian (awal):
Melakukan RCT – pembedahan mayor ditempatkan
secara acak ke Kelompok SSC dan ke kelompok
kendali (prosedur biasanya/tanpa SSC atau checklist
yg lain).
Untuk telaah makalah2 PE selanjutnya peneliti
melacak dan mengCA makalah2 PE yg terbit setelah
makalah PE terakhir yg direview penulis2 SR tsb.

Contoh:
Molina, G., Jiang, W., Edmondson, L., Gibbons, L.,
Huang, L. C., Kiang, M. V., ... & Singer, S. J. (2016).
Implementation of the Surgical Safety Checklist in
South Carolina Hospitals Is Associated with
Improvement in Perceived Perioperative Safety.
Journal of the American College of Surgeons.
Background
Prior research suggests surgical safety checklists
(SSCs) are associated with reductions in postoperative
morbidity and mortality as well as improvement in
teamwork and communication. These findings stem
from evaluations of individual or small groups of
hospitals. Studies with more hospitals have assessed
the relationship of checklists with teamwork at a
single point in time. The objective of this study was to
evaluate the impact of a large-scale implementation
of SSCs on staff perceptions of perioperative safety in
the operating room.
Study Design
As part of the Safe Surgery 2015 initiative to
implement SSCs in South Carolina hospitals, we
administered a validated survey designed to
measure perception of multiple dimensions of
perioperative safety among clinical operating
room personnel before and after
implementation of an SSC.
Results
Thirteen hospitals administered baseline and follow-up
surveys, separated by one to two years. Response rates
were 48.4% at baseline (929/1921) and 42.7% (815/1909)
at follow-up. Results suggest improvement in five of the
five dimensions of teamwork (relative percent
improvement ranged from +2.9% for coordination to
+11.9% for communication). These were significant after
adjusting for respondent characteristics, hospital fixed-
effects, and multiple comparisons, and clustering robust
standard errors by hospital (all p<0.05). More than half of
respondents (54.1%) said their surgical teams always used
checklists effectively; 73.6% said checklists had averted
problems or complications.
Conclusions
A large-scale initiative to implement SSCs is
associated with improved staff perceptions of
mutual respect, clinical leadership, assertiveness
on behalf of safety, team coordination and
communication, safe practice, and perceived
checklist outcomes.