CRITICAL

APPRAISAL
(TELAAH KRITIS)
PENELITIAN
KEDOKTERAN
TELAAH KRITIS = CRITICAL APPRAISAL

PENILAIAN HASIL PENELITIAN

SECARA SISTEMATIS

Menambah pemahaman
Mengikuti perkembangan
Melakukan penelitian
LANGKAH AWAL TELAAH KRITIS

PENILAIAN STRUKTUR DAN

KELENGKAPAN MAKALAH

PENILAIAN JUDUL,
ABSTRAK DAN RUJUKAN

CHECK LIST UMUM

STRUKTUR DAN ISI MAKALAH

PEMBAHASAN
TELAAH KRITIS
 Penilaian struktur & kelengkapan
makalah
Judul
Nama pengarang & institusinya
Abstrak
Pendahuluan
Metodologi penelitian
Hasil
Diskusi/pembahasan
Ucapan terima kasih
Daftar rujukan
 PENILAIAN TERHADAP JUDUL

Sesuai dgn keperluan pembaca

Tidak terlalu panjang atau pendek
Masalah yang diteliti jelas / efisien
Menarik
Tanpa singkatan, selain yang baku
 ABSTRAK

Informasi menyeluruh/ringkas
Latar belakang/tujuan
Rancangan penelitian
Metodologi
Hasil
Kesimpulan
Saran
Kurang dari 250 kata
RUJUKAN

Rujukan terbaru
(Terutama jurnal hasil
penelitian)
Penulisan yang tepat
CHECK LIST UMUM STRUKTUR DAN
ISI MAKALAH

CHECK LIST
YA
TIDAK
TIDAK RELEVAN
 PENDAHULUAN

1. Pendahuluan tdd 2 paragraf atau 2 bagian

2. Paragraf atau bagian pertama mengemukakan alasan
dilakukannya penelitian
3. Paragraf atau bagian kedua menyatakan hipotesis atau
tujuan penelitian
4. Paragraf kedua menyebutkan design yg digunakan
5. Pendahuluan didukung oeh pustaka yg kuat dan relevan
6. Pendahuluan lebih dari 1 halaman
 METODE

1. Desain, tempat, dan waktu disebutkan

2. Populasi sumber disebutkan
3. Kriteria pemilihan inklusi dan eksklusi dijelaskan
4. Tehnik Sampling disebutkan
5. Perkiraan besar sampel disebutkan dan alasannya
6. Perkiraan besar sampel dihitung dgn rumus yg sesuai
7. Komponen-komponen rumus besar sampel diisi
dengan angka yang masuk akal
 METODE

8. Observasi, pengukuran serta intervensi dirinci sehingga orang lain

dapat mengulanginya
9. Rujukan disebutkan bila teknik pengukuran tidak dirinci
10. Pengukuran dilakukan secara tersamar
11. Uji keandalan pengukuran (kappa) dilakukan
12. Definisi istilah dan variabel penting dikemukakan
13. Ethical clearance diperoleh
14. Persetujuan subjek diperoleh
15. Disebutkan rencana analisis, batas kemaknaan, &power penelitian
16. Program komputer yang dipakai disebutkan
 HASIL

1. Tabel deskripsi subyek penelitian disebutkan

2. Pada uji perbandingan, karakteristik subyek yang penting
sebelum intervensi dibandingkan kesetaraannya
3. Uji hipotesis untuk kesetaraan dilakukan
4. Jumlah subyek penelitian disebutkan
5. Subyek yang droup out dijelaskan beserta alasannya
6. Ketepatan numerik dinyatakan dengan benar
7. Penulisan tabel dilakukan dengan tepat
8. Tabel dan iliustrasi bersifat informatif
9. Tabel dan ilustrasi memang diperlukan
 HASIL

10. Semua hasil dalam tabel disebutkan didalam nas.

11. Semua outcome yang penting disebutkan dalam hasil
12. Subyek yang drop out diikutkan dalam analisis
13. Analisis dilakukan dengan uji yang sesuai
14. Hasil uji statistik disertakan, derajat kebebasan dan
nilai p disertakan
15. Tidak dilakukan analisis yang semula tidak
direncanakan
16. Interval kepercayaan disertakan
17. Komentar dan pendapat tidak disertakan dalam hasil
 DISKUSI

1. Semua hasil yang relevan dibahas

2. Hal yang dikemukakan dalam hasil sering diulang
3. Dibahas keterbatasan hasil penelitian, dan kemungkinan
dampaknya terhadap hasil
4.Diskusi menyebutkan kesulitan penelitan, penyimpangan dari
protokol, dan kemungkinan dampaknya terhadap hasil
5.Pembahasan dihubungkan dengan pertanyaan penelitian
6.Pembahasan dihubungkan dengan teori dan hasil peneltian
terdahulu
7. Dibahas hubungan hasil dengan praktek klinis
8. Disertakan kesimpulan utama penelitian.
 DISKUSI

9. Kesimpulan didasarkan pada data penelitian

10. Kesimpulan bersifat shahih
11. Diskusi mengemukakan dan membahas efek samping
12. Hasil tambahan selama observasi, disebutkan
13. Hasil tambahan dianalisis secara statistik
14. Generalisasi hasil penelitan, disebutkan
15. Diskusi disertakan saran penelitian selanjutnya, dengan
anjuran metodologis yang tepat
 UCAPAN TERIMA KASIH

1. Ucapan terima kasih ditujukan kpd orang-orang

yang tepat
2. Ucapan terima kasih dinyatakan secara wajar
Critical Appraisal
On Article of Diagnostic Test
(EBM-Diagnostic)
 Critical appraisal is one step in the process of evidence-
based clinical practice.
To determine what is the best evidence, we need critical
appraisal skills that will help us to understand the
methods and results of research and to assess the
quality of the research.
Most research is not perfect, and critical appraisal is not
an exact science - it will not give us the right answer.
But it can help us to decide whether we think a
reported piece of research is good enough to be used
in decision making.
Critical Appraisal - Worksheet for critical appraisal
- Software : CAT Maker
Main area of clinical objectives:
1. Diagnosis

2. Prognosis

3. Therapy/Treatment

4. Risk/Harm

Others:
Systematic Review and Meta-analysis

Clinical Guidelines

Clinical Decision Making etc.

THREE MAIN ASPECTS TO BE APPRAISED: VIA

1. VALIDITY :
VALID (CLOSENESS TO THE TRUTH) IN THE
METHODOLOGY SECTION

2. IMPORTANCE :
IMPORTANT (USEFULNESS) IN THE RESULTS SECTION

3. APPLICABILITY :
APPLICABLE (CAN BE APPLIED IN CLINICAL PRACTICE)
IN THE DISCUSSION SECTION
DIAGNOSIS WORKSHEET
Citation:

Are the results of this diagnostic study valid?

Was there an independent, blind comparison with a

reference (gold) standard of diagnosis?
Was the diagnostic test evaluated in an appropriate
spectrum of patients (like those in whom it would be
used in practice)?
Was the reference standard applied regardless of the
diagnostic test result?
Was the test (or cluster of tests) validated in a second,
independent group of patients?
Are the valid results of this diagnostic study important?
SAMPLE CALCULATIONS
Target disorder Totals
(iron deficiency anemia)
Present Absent
Diagnostic Positive 731 270 1001
test result (< 65 mmol/L) a b a+b
(serum Negative 78 1500 1578
ferritin) ( 65 mmol/L) c d c+d
Totals 809 1770 2579
a+c b+d a+b+c+d
Sensitivity = a/(a+c) = 731/809 = 90%
Specificity = d/(b+d) = 1500/1770 = 85%
Likelihood ratio for a positive test result = LR+ = sens/(1-spec) = 90%/15% = 6
Likelihood ratio for a negative test result = LR - = (1-sens)/spec = 10%/85% = 0.12
Positive Predictive Value = a/(a+b) = 731/1001 = 73%
Negative Predictive Value = d/(c+d) = 1500/1578 = 95%
Pre-test probability (prevalence) = (a+c)/(a+b+c+d) = 809/2579 = 32%
Pre-test odds = prevalence/(1-prevalence) = 31%/69% = 0.45
Post-test odds = pre-test odds LR
Post-test probability = post-test odds/(post-test odds +1)
YOUR CALCULATIONS

Totals
Target disorder

Present Absent
Diagnostic Positive a b a+b
test result
Negative c d c+d
Totals a+c b+d a+b+c+d
 SnNout

Diagnostic test with a very high sensitivity ,

a negative result effectively rules out the diagnosis

SpPin

Diagnostic test with a very high specificity ,

a positive result effectively rules in the diagnosis
Can you apply this valid, important evidence about
a diagnostic test in caring for your patient?

Is the diagnostic test available, affordable, accurate, and precise in your

setting?
Can you generate a clinically sensible estimate of your patients pre-test
probability (from personal experience, prevalence statistics, practice
databases, or primary studies)?
Are the study patients similar to your own?
Is it unlikely that the disease possibilities or probabilities have changed
since the evidence was gathered?
Will the resulting post-test probabilities affect your management and help
your patient?
Could it move you across a test-treatment threshold?
Would your patient be a willing partner in carrying it out?
Would the consequences of the test help your patient?

Additional notes:
 Apakah ada perbandingan, independen buta dengan referensi ("emas")
standar diagnosis?
Apakah tes diagnostik dievaluasi dalam spektrum yang sesuai pasien (seperti di
yang akan digunakan dalam praktek)?
Apakah standar referensi diterapkan tanpa hasil uji diagnostik?
Apakah tes (atau kelompok tes) divalidasi dalam kelompok, kedua pasien
independen?

Apakah tes diagnostik yang tersedia, terjangkau, akurat, dan tepat dalam
pengaturan Anda?
Dapatkah Anda menghasilkan perkiraan klinis yang masuk akal dari
probabilitas pre-test pasien Anda (dari pengalaman pribadi, statistik
prevalensi, database praktek, atau studi utama)?
Apakah pasien penelitian serupa dengan Anda sendiri?
Apakah tidak mungkin bahwa kemungkinan penyakit atau
probabilitas telah berubah sejak bukti itu dikumpulkan?
Apakah post-test sehingga mempengaruhi probabilitas manajemen
Anda dan membantu pasien Anda?
Mungkinkah menggerakkan Anda melintasi ambang tes- perawatan?
Apakah pasien Anda bersedia menjadi mitra dalam menjalankan test?
Apakah konsekuensi dari uji membantu pasien Anda?
Example : an article of diagnostic test, entitle :

Bedside Diagnosis of
Influenzavirus Infections
in Hospitalized Children

Katherine A. Poehling, et al
American Academy of Pediatrics
Background:
Influenzavirus has a significant impact on the pediatric
population, with school-aged children having the highest
infection rates.
For preventing nosocomial influenza infections and to
facilitate prompt antiviral therapy, an accessible, rapid
diagnostic method for influenzavirus is needed.

Objective:
To compare the performance of a rapid diagnostic test
(QuickVue Influenza Test; Quidel Corp, San Diego, CA)
completed at the bedside of hospitalized children to viral
culture and/or polymerase chain reaction (PCR) for
influenzavirus.
Method:
Study population:
1) younger than 19 years and hospitalized with respiratory
symptoms or 2) younger than 3 years and hospitalized with
fever.
Sampel: 1) a primary admission diagnosis of an acute
respiratory illness characterized by rhinorrhea, sore throat,
cough, shortness of breath, or apnea or 2) a primary
admission diagnosis consistent with a febrile illness and a
temperature of 100.4F.
Broad inclusion criteria were chosen such that all children
who were hospitalized with symptoms potentially related to
influenza infections were eligible.
Sample in
this study
 Study design:
prospective, cross sectional study from each child, 2 nasal
swabs of the turbinates were obtained1 for influenzavirus
culture and PCR and the other for the rapid diagnostic test.
The rapid test results were compared with that of culture and
PCR for influenzavirus.

Influenza infection was defined as any sample with 1) a

positive culture for influenzavirus or 2) 2 consecutive positive
PCRs for influenza A or B.
Each researcher was trained to perform and interpret the
rapid diagnostic test at the bedside according to the
manufacturers instructions.
The laboratory technician who performed the culture and
PCR was masked to the rapid diagnostic test results.
 Results:
Sens: 74%
Spec: 98%
PPV: 74%
NPV: 98%
DIAGNOSIS WORKSHEET
Citation:

Are the results of this diagnostic study valid?

Was there an independent, blind comparison with a Yes

reference (gold) standard of diagnosis?
Was the diagnostic test evaluated in an appropriate Yes
spectrum of patients (like those in whom it would be
used in practice)?
Was the reference standard applied regardless of the Yes
diagnostic test result?
Was the test (or cluster of tests) validated in a second, Yes
independent group of patients?
 Are the valid results of this diagnostic study important?

SAMPLE CALCULATIONS
Totals
Target disorder

Present Absent
Diagnostic Positive a b a+b
test result
Negative c d c+d
Totals a+c b+d a+b+c+d
 + -
+ 14 5
- 5 209
 Culture or PCR Totals
Present Absent
Quick Vue Positive 14 5 19
Influenza
Negative 5 209 214
test
Totals 19 214 233

Sensitivity = a/(a+c) = 14/19 = 74%

Specificity = d/(b+d) = 209/214 =98%

Likelihood ratio for a positive test result = LR+ = sens/(1-spec) = 74%/2% = 37

Likelihood ratio for a negative test result = LR - = (1-sens)/spec = 26%/98% = 0.27

Positive Predictive Value = a/(a+b) = 14/19 = 74%

Negative Predictive Value = d/(c+d) = 209/214 = 98%

Pre-test probability (prevalence) = (a+c)/(a+b+c+d) = 19/233 = 8%

Pre-test odds = prevalence/(1-prevalence) = 8%/92% = 0.087

Post-test odds = pre-test odds LR = 0.087 x 37 = 3.22
Post-test probability = post-test odds/(post-test odds +1)= 3.22/4.22=76%
Just to remind you:

Sensitivity: the percentage of persons with the

disease of interest who have positive test results.
= a/(a+c) x 100

Specificity: the percentage of persons without

the disease of interest who have negative results.
= d/(d+b) x 100
 PPV: the percentage of persons with positive
test results who actually have the disease of
interest.
= a/(a+b) x 100

NPV: the percentage of persons with negative

test results who do not have the disease of
interest.
= d/(d+c) x 100
Likelihood Ratio
Likelihood is the probability of a particular test
result for a person with the disease of interest
divided by the probability of that test result for a
person without the disease of interest.

Likelihood Ratio for a positive test result (LR+)

Likelihood Ratio for a negative test result (LR-)
 (LR+) is the probability of a positive test result for a
person with the disease of interest divided by the
probability of a positive test result for a person without
the disease.

LR+ = Sensitivity / (1-Specificity)

LR+ > 1: persons affected with disease of interest are
more likely to have a positive test result than unaffected
persons.
The larger the value of the LR, the stronger the
association between having a positive test result and
having the disease of interest.
LR+ value of 10 or greater is perceived as indication of
a test of high diagnostic value.
 LIKELIHOOD RATIO
LRs >10 or <0.1 cause large changes in
likelihood.
LRs 5-10 or 0.1-0.2 cause moderate changes.
LRs 2-5 or 0.2-0.5 cause small changes.
LRs between <2 and 0.5 cause little or no
change
 Pretest probability of disease: the probability that a
person has the disease of interest before the test is
performed.
= prevalence

Pretest odds of disease: the estimate before diagnostic

testing of the probability that a patient has the disease
of interest divided by the probability that the patient
does not have the disease of interest.
Pretest odds= pretest probability/(1-pretest probability)
 Posttest odds of disease: as the estimate after diagnostic
testing of the probability that a patient has the disease of
interest divided by the probability that the patient does not
have the disease of interest.
Posttest odds= pretest odds x LR +

Posttest probability = posttest odds/ (1+posttest odds)

a result of obtaining a positive test result, the estimated

probability of the presence of disease has risen from 0.08
(pretest probability) to 0.76 (posttest probability).

Diagnostic tests that produce big changes from pretest to

post-test probabilities are important and likely to be useful
to us in our practice
 Software: CAT Maker
Can you apply this valid, important evidence about
a diagnostic test in caring for your patient?

Is the diagnostic test available, affordable, accurate, and precise in your

setting? No
Can you generate a clinically sensible estimate of your patients pre-test
probability (from personal experience, prevalence statistics, practice Yes
databases, or primary studies)?
Are the study patients similar to your own?
Is it unlikely that the disease possibilities or probabilities have changed
Yes
since the evidence was gathered?
Will the resulting post-test probabilities affect your management and help
your patient? Yes
Could it move you across a test-treatment threshold?
Would your patient be a willing partner in carrying it out?
Would the consequences of the test help your patient?
Yes
Additional notes:
Thank you..