ANALISIS JURNAL

“RISK PREDICTION MODELS FOR ATHEROSCLEROTIC

CARDIOVASCULAR DISEASE IN PATIENTS WITH CHRONIC KIDNEY
DISEASE: THE CRIC STUDY’’

Disusun oleh :
Resta Saputri. Yn
2110070170001

Dosen Pengampu :
Ns.Fatimah, S.Kep, M.Kep

PROGRAM STUDI KEPERAWATAN ANESTESIOLOGI

PROGRAM SARJANA TERAPAN FAKULTAS VOKASI
UNIVERSITAS BAITURRAHMAH PADANG
TAHUN AJARAN 2022/2023

33: 601–611, 2022 1533-3450/1046-601

 Judul jurnal : “Risk Prediction Models For Atherosclerotic Cardiovascular Disease
In Patients With Chronic Kidney Disease: The Cric Study’’
Penulis : Joshua D. Bundy ,1,2 Mahboob Rahman,3 Kunihiro Matsushita,4
Byron C. Jaeger,5Jordana B. Cohen ,6 Jing Chen,1,2,7 Rajat Deo,8
Mirela A. Dobre,3 Harold I. Feldman ,9 John Flack,10 Radhakrishna
R. Kallem,6 James P. Lash ,11 Stephen Seliger ,12Tariq Shafi ,13
Shoshana J. Weiner ,12 Myles Wolf,14 Wei Yang,9 Norrina B.
Allen,15 Nisha Bansal,16 Jiang He,1,2,7 and the CRIC Study
Investigators
Tahun : 2022
Penerbit : Journal of the American Society of Nephrology
Reviewer : Resta Saputri Yn

A. Latar Belakang Riset

Peneliti melakukan Riset ini karena melihat perkembangan Model Prediksi Risiko
Penyakit Kardiovaskular Aterosklerotik pada Pasien Penyakit Ginjal Kronis. Individu
dengan CKD mungkin berisiko tinggi untuk penyakit kardiovaskular aterosklerotik
(ASCVD). Namun, tidak ada model prediksi risiko ASCVD yang dikembangkan pada
populasi CKD untuk menginformasikan perawatan klinis dan pencegahan.

B. Tujuan Penelitian
Penelitian ini bertujuan untuk mengembangkan dan memvalidasi model prediksi
risiko ASCVD 10 tahun pada pasien dengan CKD yang menyertakan peserta tanpa
penyakit kardiovaskular yang dilaporkan sendiri dari studi Chronic Renal Insufisiensi
Cohort (CRIC).
C. Pertanyaan Penelitian
Apakah perkembangan Model Prediksi Risiko Penyakit Kardiovaskular
Aterosklerotik pada Pasien Penyakit Ginjal Kronis dapat dilakukan dengan baik ?

Berdasarkan masalah pada penelitian ini maka di dapat disusun PICO sebagai berikut :

P (problem) Mengembangkan dan memvalidasi model prediksi risiko ASCVD.

menghasilkan 2604 peserta
I (intervention) Pasien Penyakit Ginjal Kronis
C (compuration) Analisis dilakukan menggunakan SAS versi 9.4 (SAS Institute,
33: 601–611, 2 022 Cary, NC) dan R versi 4.1.1 (R Project for 1533-3450/1046-601
 Statistical Computing). Dua sisiPnilai 0,05 dianggap
signifikan secara statistik untuk semua analisis.
O (outcame) perkembangan Model Prediksi Risiko Penyakit
D. Bahan dan Metode
1. Lokasi penelitian
Penelitian ini dilakukan pusat klinis di Amerika Serikat.
2. Popupasi dan Sampel
Populasi pada penelitian ini adalah kelompok ras dan etnis yang beragam dari 3939
pria dan wanita berusia 21-74 tahun. Serta sampel yang digunakan riwayat CVD yang
dilaporkan sendiri pada awal (n51335), menghasilkan 2604 peserta. Studi ini disetujui
oleh dewan peninjau institusional di setiap institusi dan semua peserta memberikan
persetujuan tertulis.
3. Desain Penelitian
Penelitian ini Studi CRIC adalah studi kohort prospektif longitudinal dari peserta
dengan CKD ringan sampai sedang berdasarkan criteria masuk eGFR 20-70 ml/menit per
1,73 m2.2. Sebuah kelompok ras dan etnis yang beragam dari 3939 pria dan wanita
berusia 21-74 tahun terdaftar dari tujuh pusat klinis di Amerika Serikat (Ann Arbor, MI;
Baltimore, MD; Chicago, IL; Cleveland, OH; New Orleans, LA; Philadelphia , PA; dan
San Francisco, CA) dalam fase I, antara Mei 2003 dan Agustus 2008.
4. Variabel yang Diukur
Dari jurnal ini dalam variable Variabel prediktor kandidat yang dipertimbangkan
untuk dimasukkan dalam model prediksi ditampilkan pada Gambar Tambahan 1. Variabel
kandidat yang termasuk dalam ACC/AHA PCE6: usia, jenis kelamin, ras dan etnis,
kolesterol total, kolesterol HDL, tekanan darah sistolik, penggunaan obat penurun
tekanan darah, riwayat diabetes, dan merokok saat ini. Variabel kandidat tambahan dipilih
berdasarkan asosiasi dengan ASCVD di antara pasien dengan CKD yang diidentifikasi
dala mlaporan sebelumnya, dan termasuk faktor metabolisme, penyaki ginjal,
metabolisme lipid, metabolisme mineral, dan factor peradangan, dan biomarker jantung.
5. Cara Pengumpulan Data
Pengumpulan data Titik akhir utama untuk model prediksi adalah insiden ASCVD:
gabungan dari insiden stroke fatal atau nonfatal atau infark miokard.Peristiwa dipastikan
setiap 6 bulan dari catatan medis, dan diagnosis dikonfirmasi oleh dua peninjau buta
sebagai peristiwa yang mungkin, kemungkinan, atau pasti. Waktu tindak lanjut disensor

33: 601–611, 2022 1533-3450/1046-601

 pada saat kematian paling awal, mangkir (n5203; 7,8%, atau 10 tahun tindak lanjut
lengkap, konsisten dengan model sebelumnya yang dikembangkan untuk ASCVD.
6. Analisis Data
Dari jurnal ini peneliti menggunakan tabel dan gambar yan jelas dalam menjelaskan
data yang digunakan. Analisis dilakukan menggunakan SAS versi 9.4 (SAS Institute,
Cary, NC) dan R versi 4.1.1 (R Project for Statistical Computing). Dua sisiPnilai 0,05
dianggap signifikan secara statistik untuk semua analisis.
E. Hasil
Hasil penelitian ini digambarkan jelas dalam bentuk tabel Pengembangan Model dan
Validasi Internal Hasil dari 10310 kali lipat validasi silang internal disajikan dalam Tabel
Tambahan. 2604 peserta (usia rata-rata 55,8 tahun; 52,0% laki-laki) termasuk dalam
analisis, 252 memiliki insiden ASCVD dalam waktu 10 tahun dari awal. Dibandingkan
dengan persamaan kohort gabungan American College of Cardiology/ American Heart
Association (area di bawah kurva karakteristik operasi penerima [AUC]50,730), model
dengan koefisien yang diperkirakan dalam sampel CRIC memiliki diskriminasi yang
lebih tinggi (P50,03), mencapai AUC 0,736 (interval kepercayaan 95% [CI], 0,649 hingga
0,826). Model CRIC yang dikembangkan menggunakan variabel yang tersedia secara
klinis memiliki AUC 0,760 (95% CI, 0,678 hingga 0,851).
F. Pembahasan
Berdasarkan hasil JBI yang dinilai pada jurnal didapatkan kriteria yang sesuai dengan
kaidah jurnal dan pembahasan yang sistematis terhadap hasil yang ditemukan. model
prediktif untuk risiko 10 tahun ASCVD. Kami menemukan bahwa PCE ACC/AHA yang
diterbitkan memiliki kemampuan prediksi sedang untuk risiko 10 tahun ASCVD di antara
peserta studi CRIC.
G. Kesimpulan
Pada penelitian ini terdapat hasil kesimpulannya kami telah mengembangkan model
baru untuk prediksi risiko 10 tahun ASCVD di antara pasien dengan CKD, termasuk
model yang terdiri dari variabel yang tersedia secara klinis dan model yang diperkaya
biomarker. Model-model ini secara signifikan meningkatkan kinerja prediksi di luar PCE
ACC/AHA pada populasi pasien dengan CKD tahap 2-4 ini, terutama bagi mereka yang
memiliki risiko ASCVD 10 tahun, 20%.
H. Kelebihan dan Kekurangan
1. Kelebihan
33: 601–611, 2022 1533-3450/1046-601
 Dalam jurnal ini membahas secara sistematis bagaimana pengembangan dan
memvalidasi model prediksi risiko ASCVD . Data yang dipaparkan juga lengkap dan
penjelasan yang ada pada jurnal bisa dipahami serta memiliki bahan data yang jelas
seperti adanya tabel.
2. Kekurangan
Penelitian ini mungkin memakan waktu yang sedikit lama karena menggunakan
metode kohor. serta kata yang sedikit sulit dipahami.
I. Implikasi Keperawatan
Untuk kasus yang dijelaskan pada jurnal hanya ada implikasi klinis dan penelitian dari
temuan kami. CKD adalah kondisi yang heterogen38dan penyakit jantung iskemik yang
ditemukan pada CKD sering melibatkan arteri kecil distal yang tidak dapat menerima
revaskularisasi korone. model baru kami memberikan prediksi yang lebih akurat
dibandingkan dengan model prediksi risiko sebelumnya, terutama pada model dengan
risiko ASCVD 10 tahun, 20%. Oleh karena itu, model kami secara signifikan
meningkatkan reklasifikasi nonevent dibandingkan dengan PCE ACC/AHA standar.

33: 601–611, 2022 1533-3450/1046-601

 Lampiran
1. Jurnal
INICAL EPIDEMIOLOGY www.jasn.org

Risk Prediction Models for Atherosclerotic

Cardiovascular Disease in Patients with Chronic
KidneyDisease: The CRIC Study
Joshua D. Bundy ,1,2 Mahboob Rahman,3 Kunihiro Matsushita,4 Byron C.
Jaeger,5 Jordana B. Cohen,6 Jing Chen,1,2,7 Rajat Deo,8 Mirela A. Dobre,3
Harold I. Feldman ,9John Flack,10 Radhakrishna R. Kallem,6 James P.
Lash,11 Stephen Seliger,12Tariq Shafi,13 Shoshana J. Weiner,12 Myles
Wolf,14 Wei Yang,9 Norrina B. Allen,15Nisha Bansal,16 Jiang He,1,2,7 and
the CRIC Study Investigators*
Due to the number of contributing authors, the affiliations are listed at the end of this article.

ABSTRACT
Background Individuals with CKD may be at high risk for atherosclerotic cardiovascular
disease (ASCVD). However, there are no ASCVD risk prediction models developed in CKD
populations to inform clinical careand prevention.
Methods We developed and validated 10-year ASCVD risk prediction models in patients with
CKD that included participants without self-reported cardiovascular disease from the

ICAL EPIDEMIOLOGY
Chronic Renal Insufficiency Cohort (CRIC) study. ASCVD was defined as the first
occurrence of adjudicated fatal and nonfatal stroke ormyocardial infarction. Our models
used clinically available variables and novel biomarkers. Model perfor- mance was
evaluated based on discrimination, calibration, and net reclassification improvement.
Results Of 2604 participants (mean age 55.8 years; 52.0% male) included in the analyses, 252
had incidentASCVD within 10 years of baseline. Compared with the American College of
Cardiology/American Heart Association pooled cohort equations (area under the
receiver operating characteristic curve [AUC]50.730), a model with coefficients
estimated within the CRIC sample had higher discrimination (P50.03), achieving an AUC
of 0.736 (95% confidence interval [CI], 0.649 to 0.826). The CRIC model devel-oped using
clinically available variables had an AUC of 0.760 (95% CI, 0.678 to 0.851). The
CRIC
biomarker-enriched model had an AUC of 0.771 (95% CI, 0.674 to 0.853), which was
significantly higher than the clinical model (P50.001). Both the clinical and biomarker-
enriched models were well-calibrated and improved reclassification of nonevents
compared with the pooled cohort equations (6.6%; 95% CI, 3.7% to 9.6% and 10.0%; 95%
CI, 6.8% to 13.3%, respectively).
Conclusions The 10-year ASCVD risk prediction models developed in patients with CKD,
including novel kidney and cardiac biomarkers, performed better than equations
developed for the general population using only traditional risk factors.

JASN 33: 601–611, 2022. doi: https://doi.org/10.1681/ASN.2021060747

CKD affects more than 10% of the population
globally and is strongly associated with cardiovas-
cular disease (CVD) morbidity and mortality.1–3
T3 3h: e6r0e1f–o6 r1 e1 ,, 2i0n2d2ividuals with CKD are typically
assumed to be at high risk for atherosclerotic CVD (ASCVD).2
However, the accurate classifica-tion of ASCVD risk among individuals
with CKDis still controversial. The Kidney Disease:
1533-3450/1046-601
Improving Global
Outcomes (KDIGO) 2013
lipid guideline considers
CKD patients aged 50
years or older as high risk
regardless of CKD severity
or the status of traditional
CVD risk factors.4 The
European Society of
Cardiology guideline
catego- rizes eGFR 30–59
ml/min per 1.73 m2 as high
risk, and eGFR ,30
ml/min per 1.73 m2 or
patients with diabetes and
albuminuria as very

33: 601–611, 2022 1533-3450/1046-601

 NICAL EPIDEMIOLOGY ww w.jasn.org

high risk.5 The 2013 American Heart Association (AHA)

nificance Statement
and American College of Cardiology (ACC) guideline did
not include CKD measures in ASCVD prediction using the ients with CKD are typically considered to be at high risk for
pooled cohort equations (PCEs). 6 atherosclerotic cardiovascular disease, but CKD is a heteroge-
neous condition and there are no validated atherosclerotic car-
Recent primary prevention guidelines classify CKD as a vascular disease risk stratification tools for this population. Our
risk-enhancing factor.7 Although the KDIGO guideline rec- analysis of 2604 participants in the Chronic Renal Insuffi-
ommends predicting ASCVD risk in CKD patients younger ncy Cohort study found that newly developed risk prediction
than 50 years old, there are few established prediction tools models, using clinically available variables and novel biomarkers,
8 improved discrimination, calibration, and reclassification of none-
developed for this clinical population. Overall, inconsisten-
vents compared with the traditional American College of Cardi-
cies in major clinical guidelines are mainly due to inconsis- ology/American Heart Association pooled cohort equations
tent previous findings.9–12 The Framingham equations and developed for the general population. The new equations may
the ACC/AHA PCEs may inadequately predict ASCVD risk improve risk stratification in patients with CKD and improve
in individuals with CKD.10,13 Furthermore, it is unclear shared decision making for preventive therapy to reduce athero-
sclerotic cardiovascular disease incidence and mortality.
whether incorporating additional novel risk factors will
14,15
improve risk prediction for ASCVD.
The availability of accurate ASCVD risk prediction equa- with a self-reported history of CVD at baseline (n51335),
tions for individuals with CKD could be informative for yielding 2604 participants. The study was approved by the
patients and clinicians and improve shared decision making institutional review board at each institution and all partici-
for preventive therapy to reduce ASCVD incidence and pants provided written informed consent.
mortality.16 Therefore, we developed novel ASCVD risk pre-
diction models for use among patients with CKD using dataCandidate Predictor Variables
from the Chronic Renal Insufficiency Cohort (CRIC) study. A more detailed description of the methods is available in
the Supplemental Methods. Candidate predictor variables
considered for inclusion in the prediction models are dis-
METHODS played in Supplemental Figure 1. Candidate variables
included those in the ACC/AHA PCEs6: age, sex, race and
Study Design and Participants ethnicity, total cholesterol, HDL cholesterol, systolic BP,
The CRIC study is a prospective, longitudinal cohort study use of BP-lowering medications, history of diabetes, and
of participants with mild-to-moderate CKD based on an current smoking. Additional candidate variables were cho-
eGFR entry criterion of 20–70 ml/min per 1.73 m2. A sen based on associations with ASCVD among patients
racially and ethnically diverse group of 3939 men and with CKD identified in prior reports, and included meta-
women aged 21–74 years was enrolled from seven clinical bolic factors, kidney disease, lipid metabolism, mineral
centers in the United States (Ann Arbor, MI; Baltimore, metabolism, and inflammation factors, and cardiac bio-
MD; Chicago, IL; Cleveland, OH; New Orleans, LA; Phila- markers. Because self-reported race is a controversial and
delphia, PA; and San Francisco, CA) in phase I, between ill-defined social construct,18 we used the CKD Epidemiol-
May 2003 and August 2008.17 Patients with New York Heart ogy Collaboration cystatin C equation as a measure of
Association class 3 and 4 heart failure, cirrhosis, HIV infec- eGFR, which does not include a race adjustment factor. 19
tion, polycystic kidney disease, renal cell carcinoma, those We considered two sets of variables for development of the
receiving chronic dialysis or an organ transplant, and those prediction models: one set including only readily clinically
taking immunosuppressive medications were excluded from available variables (Supplemental Figure 1; black text) and
the CRIC study. For the current analysis, we excluded those another set containing all candidate predictor variables,
including biomarkers that are less likely to be measured in
Received June 5, 2021. Accepted December 20, 2021. routine clinical practice (Supplemental Figure 1; blue text).
*The CRIC Study Investigators are Lawrence J. Appel, Jing Chen, Harold I. In a secondary analysis, we evaluated the performance of
Feldman, Alan S. Go, James P. Lash, Robert G. Nelson, Mahboob Rahman, the “CKD patch,” which incorporates eGFR and urinary
Panduranga S. Rao, Vallabh O. Shah, Raymond R. Townsend, and Mark L.
Unruh.
albumin-creatinine ratio (ACR) measurements into the
ACC/AHA PCEs. In brief, the CKD patch equations were
Published online ahead of print. Publication date available at www.jasn.org.
applied to the PCEs recalibrated to the CRIC sample, with
See related editorial, “Cardiovascular Risk Prediction Scores in CKD: What Are We expected eGFR and urinary ACR centered at CRIC study-
Missing?,” on pages 462–464. specific averages, as recommended.8
Correspondence: Dr. Jiang He, Department of Epidemiology, Tulane University Natural logarithm and nonlinear (e.g., polynomial,
School of Public Health and Tropical Medicine, 1440 Canal Street, Suite 2000, square root) transformations of continuous variables were
New Orleans, LA 70118. Email: jhe@tulane.edu
evaluated and retained if they improved model fit. Potential
Copyright © 2022 by the American Society of Nephrology interactions between all candidate predictors were assessed.

33: 601–611, 2022


However, the inclusion of interaction terms was consis-
tently associated with poorer cross-validated model perfor-
mance and they were not retained in the final models. We
could not evaluate some potentially important variables
(e.g., coronary artery calcium, pulse-wave velocity, and elec-
trocardiogram variables) because these variables were not
available among the majority of CRIC participants. Missing
data for other candidate predictors was ,5% for all varia-
bles except homocysteine (11.3%) and neutrophil
gelatinase-associated lipocalin ( 5.7%). Missing values were
imputed using a nearest neighbors model. 20
Outcome Assessment
The primary end point for the prediction models was inci-
dent ASCVD: a composite of incident fatal or nonfatal
stroke or myocardial infarction.6 Events are ascertained
every 6 months from medical records, and diagnosis wasPerformance Metrics
confirmed by two blinded reviewers as possible, probable, or
definite events. Follow-up time was censored at the earliest
21
occurrence of death, loss to follow-up (n5203; 7.8%), or 10
years of complete follow-up, consistent with prior models
developed for ASCVD. All deaths were confirmed by death
6 the outcome and was assessed using the time-dependent
certificate.
Statistical Analyses
Model Development
A priori, we evaluated the ACC/AHA PCEs in the CRIC
sample using two approaches: (1) predicted risk from the
published model coefficients6; and (2) variables included in
the ACC/AHA PCEs with coefficients recalculated using
the CRIC sample. Furthermore, we evaluated several
modeling algorithms and selection procedures to develop
novel models. Our primary approach used Cox propor-
tional hazards regression models to predict the probability
of having an incident ASCVD event within 10 years of
baseline.6 Additionally, we used the Fine and Gray subdis-
tribution hazards model to account for the competing risk
of death.22 Within both modeling algorithms, we evaluated
three predictor selection procedures: (1) backward elimina-
tion with a P,0.05 criterion; (2) backward elimination
with an Akaike information criterion; and (3) LASSO
(“least absolute shrinkage and selection operator”) method
with a regularization penalty (l) one SEM higher than that
which minimized cross-validated error.23
Model Validation
All modeling strategies were internally validated using
10 3 10-fold cross-validation, similar to the derivation pro-
cedure used to develop the ACC/AHA PCEs.6 The best-RESULTS
performing (see “Performance Metrics” section) modeling
procedure was used on the full CRIC sample to developBaseline Characteristics
final clinical and biomarker-enriched models. In a sensitiv-
ity analysis, we evaluated the best-performing modeling
procedure from the 10 3 10-fold cross-validation using an
33: 601–611, 2022
www.jasn.orN
g ICAL EPIDEMIOLOGY
internal-external validation approach. In this approach, one
clinical center is left out at a time to cross-validate models
developed in the other centers, which provides additional
information on the potential performance of the models in
external datasets similar in composition to the CRIC study
sample.24
We validated our models externally using data from two
independent, community-based prospective cohort studies:
the Atherosclerosis Risk in Communities (ARIC) study25
and the Multi-Ethnic Study of Atherosclerosis (MESA).26
We pooled individual-level data among 2222 participants
with CKD, based on an eGFR ,60 ml/min per 1.73 m2 or
urinary ACR $30 mg/g. The equations developed in the
CRIC study were applied directly to the external validation
data, and model performance was assessed.
We evaluated the performance of developed models using
measures of discrimination, calibration, and overall good-
ness of fit.27 Discrimination refers to the ability of a model
to correctly identify those who will or will not experience
(i.e., 10-year) area under the receiver operating characteris-
tic curve (AUC).28 Calibration refers to the agreement
between observed outcomes and model predictions, and
was assessed by plotting the observed versus predicted risk
across deciles of predicted risk. Observed risk was esti-
mated using the cumulative incidence function accounting
for the competing risk of death. The scaled Brier score, or
index of prediction accuracy (IPA), incorporates informa-
tion on both discrimination and calibration, and was used
to evaluate overall goodness of fit.29 A higher IPA indicates
a better performing model, an IPA of 1.0 indicates a perfect
model, and an IPA #0 indicates a useless model. Perfor-
mance results from the 10 3 10-fold cross-validation were
summarized as the mean for point estimates and the 2.5th
and 97.5th percentiles for 95% confidence intervals (CIs).
We used the categorical net reclassification improvement
(NRI) statistic to evaluate the clinical utility of the mod-
els,30,31 using predicted risk thresholds used in current clin-
ical practice guidelines (i.e., 7.5% and 20% per the ACC/
AHA primary prevention guideline).7 Bootstrapping with
1000 replicates was used to generate 95% CIs for the NRI.
Analyses were conducted using SAS version 9.4 (SAS
Institute, Cary, NC) and R version 4.1.1 (R Project for Sta-
tistical Computing). Two-sided P values ,0.05 were con-
sidered statistically significant for all analyses.
Among 2604 participants included in the analyses, the
mean (SD) age was 55.8 (11.7) years, 52.0% were male,
41.6% had diabetes, and the mean (SD) eGFR was 56.0
D Risk Prediction in CKD
 NICAL EPIDEMIOLOGY www.jasn.org

(24.7) ml/min per 1.73 m2. During the first 10 years of were slightly higher and IPAs were slightly lower
follow-up from baseline, 252 incident ASCVD events (Supplemental Table 2).
occurred (11.9 events per 1000 person-years). Participants
who had an ASCVD event within 10 years were more likelyModel Characteristics and Risk Calculations
to be older, Black race, currently smoke, have a history of Table 2 provides the hazard ratios (HRs) with 95% CIs,
diabetes, and use BP-lowering medications compared with b coefficients, and information for calculating predicted 10-
those who did not have an ASCVD event (Table 1). On year risk of ASCVD for the final models. Model 1 included
average, those who had an ASCVD event had higher levels the ACC/AHA PCEs variables with coefficients
of glycated hemoglobin (HbA1c), systolic and diastolic BP, recalculated in the CRIC study sample. Model 2 (CRIC
urinary ACR, troponin-T, and N-terminal prohormone of clinical model) included age, HDL cholesterol, systolic BP,
brain natriuretic peptide (NT-proBNP); and lower levels of current smoking, urinary ACR, HbA1c, and hemoglobin.
HDL cholesterol, eGFR, and hemoglobin compared with Model 3 (CRIC enriched model) included age, total choles-
those who did not have an event. Within 10 years, 411 par- terol, HDL cholesterol, current smoking, urinary ACR,
ticipants died from noncardiovascular causes. HbA1c, apolipoprotein B, high-sensitivity C-reactive pro-
tein (hsCRP), troponin-T, and NT-proBNP. Example pre-
Model Development and Internal Validation dicted risk calculations using each of the three models are
The results from the 10 3 10-fold internal cross-validation available in Supplemental Tables 3–5.
are presented in Supplemental Table 1. Across all candi-
date variables and selection procedures, the Cox propor-Relative Variable Importance
tional hazards regression models obtained higher Figure 1 shows the relative importance of variables
estimates of AUC and IPA than the Fine and Gray subdis- included in models 2 and 3. In both models, age accounted
tribution hazards models, whereas the Fine and Gray for the greatest proportion of the overall model chi-
model improved calibration in the highest predicted prob- squared. The top five predictors in model 2 were age,
ability decile (Supplemental Figure 2). According to the urinary ACR, current smoking, HbA1c, and HDL choles-
10 3 10-fold cross-validation, Cox proportional hazards terol. The top five predictors in model 3 were age, HbA1c,
regression with backward elimination using a criterion of current smoking, NT-proBNP, and urinary ACR.
P,0.05 had the highest AUC and IPA. In a sensitivity anal-
ysis using internal-external validation leaving one CRICDiscrimination and Calibration
clinical center out at a time, prediction performance was All models were well-calibrated, but each overestimated
similar to 10 3 10-fold cross-validation, although AUCs risk within the highest decile of predicted risk (Figure 2

Table 1. Baseline characteristics of CRIC study participants

Variables Overall (n52604) ASCVD Event within 10 Years

No (n52352) Yes (n5252)
Age, mean (SD), yr 55.8 (11.7) 55.5 (11.8) 59.0 (9.8)
Male, No. (%) 1354 (52.0) 1212 (51.5) 142 (56.3)
Black race, No. (%) 1013 (38.9) 898 (38.2) 115 (45.6)
Body mass index, mean (SD), kg/m2 31.7 (7.8) 31.7 (7.9) 32.0 (7.1)
Current smoking, No. (%) 314 (12.1) 262 (11.1) 52 (20.6)
HbA1c, mean (SD), % 6.4 (1.5) 6.4 (1.5) 7.0 (1.6)
Diabetes mellitus, No. (%) 1082 (41.6) 931 (39.6) 151 (59.9)
Use of glucose-lowering medications, No. (%) 620 (24.0) 534 (22.9) 86 (34.3)
Systolic BP, mean (SD), mm Hg 127.0 (21.0) 126.0 (20.7) 136.1 (21.6)
Diastolic BP, mean (SD), mm Hg 72.7 (12.4) 72.5 (12.2) 74.3 (13.7)
Use of BP-lowering medications, No. (%) 2282 (87.6) 2041 (86.8) 241 (95.6)
Total cholesterol, mean (SD), mg/dl 188.4 (44.5) 188.2 (43.7) 190.2 (51.3)
LDL cholesterol, mean (SD), mg/dl 106.6 (35.3) 106.5 (34.9) 107.6 (38.8)
HDL cholesterol, mean (SD), mg/dl 48.8 (16.1) 49.1 (16.2) 45.8 (15.0)
Apolipoprotein B, mean (SD), mg/dl 85.6 (24.4) 85.4 (24.1) 87.7 (27.1)
Use of statin medications, No. (%) 1184 (45.8) 1045 (44.8) 139 (55.4)
eGFR, mean (SD), ml/min per 1.73 m2 56.0 (24.7) 56.9 (25.0) 47.2 (20.4)
Urine ACR, median [IQR], mg/g 39.1 [7.3, 391.2] 34.7 [6.7, 344.7] 224.0 [18.3, 1062.3]
Hemoglobin, mean (SD), g/dl 12.7 (1.8) 12.8 (1.8) 12.3 (1.9)
hsCRP, median [IQR], mg/L 2.4 [1.0, 6.1] 2.3 [1.0, 6.0] 3.0 [1.2, 7.9]
Troponin-T, median [IQR], pg/ml 9.7 [4.5, 18.9] 9.3 [4.1, 17.4] 16.0 [8.9, 31.6]
NT-proBNP, median [IQR], pg/ml 106.5 [49.3, 249.7] 99.3 [46.5, 229.5] 207.9 [95.7, 457.5]
IQR, interquartile range.

33: 601–611, 2022

 www.jasn.orN
g ICAL EPIDEMIOLOGY

Table 2. Novel CKD ASCVD risk prediction models developed in the CRIC study

Model Parameters Model 1: ACC/AHA PCEs Model 2: CRIC Clinical Model 3: CRIC Enriched
Variables Model Model
Variables HR (95% CI) b, per 1-unit HR (95% CI) b, per 1-unit HR (95% CI) b, per 1-unit
Age, per 5 yr 1.15 (1.09 to 1.21) 0.0275 1.23 (1.16 to 1.31) 0.0420 1.19 (1.12 to 1.26) 0.0347
Male sex 1.09 (0.87 to 1.38) 0.0900
Black race 1.21 (0.96 to 1.51) 0.1885
Total cholesterol, 1.02 (1.00 to 1.04) 0.0020 1.08 (1.01 to 1.15) 0.0073
per 10 mg/dl
HDL cholesterol, 0.87 (0.80 to 0.95) 20.0134 0.88 (0.81 to 0.95) 20.0127 0.85 (0.77 to 0.94) 20.0164
per 10 mg/dl
Systolic BP, 1.13 (1.08 to 1.19) 0.0126 1.08 (1.02 to 1.14) 0.0077
per 10 mm Hg
Use of BP-lowering 1.54 (0.97 to 2.46) 0.4342
medications
History of diabetes 1.88 (1.49 to 2.37) 0.6300
Current smoking 2.09 (1.57 to 2.78) 0.7382 2.03 (1.53 to 2.69) 0.7086 1.94 (1.46 to 2.57) 0.6615
Log (urine ACR), mg/g 1.17 (1.11 to 1.23) 0.1553 1.12 (1.05 to 1.19) 0.1127
HbA1c, per 0.5% 1.09 (1.05 to 1.12) 0.1634 1.08 (1.05 to 1.12) 0.1616
Hemoglobin, per 5 g/dl 0.64 (0.46 to 0.88) 20.0903
Apolipoprotein B, 0.89 (0.79 to 0.99) 20.0121
per 10 mg/dl
Log (hsCRP), mg/L 1.11 (1.02 to 1.21) 0.1045
Log (troponin-T), pg/ml 1.23 (1.09 to 1.39) 0.2090
Log (NT-proBNP), pg/ml 1.22 (1.11 to 1.35) 0.2011
Score calculations Baseline
survival at 0.9045 0.9075 0.9100
10 years, S(10) Predicted
probability of 1– S(10)exp(SbX–3.6963) 1– S(10)exp(SbX–3.3283) 1– S(10)exp(SbX–4.5580)
ASCVD at 10 yearsa
aThe predicted 10-year probability for each model can be calculated as 1– S(10)exp(SbX–betaavg) where b is the regression coefficient (b), X is the individual patient’s
level for each risk factor, and betaavg is the sample mean of the b3value sum.

and Supplemental Figure 3). The models developed in the validation data, the ACC/AHA PCEs had an AUC of 0.702
CRIC data were better calibrated across the range of pre- and IPA of 0.053 (Supplemental Figure 6). When the CRIC
dicted risks compared with the published ACC/AHA PCEs. equations were applied to the external validation data, dis-
Applying the published ACC/AHA PCEs coefficients to the crimination and calibration were adequate and improved
CRIC sample yielded an AUC of 0.730 and IPA of 0.058. upon the ACC/AHA PCEs, achieving AUCs of 0.709 and 0.
Model 1 including the ACC/AHA PCEs variables recalcu- 747, and IPAs of 0.087 and 0.117, for the clinical and
lated in the CRIC sample achieved an AUC of 0.736 (95% enriched models, respectively (Supplemental Figure 6).
CI, 0.649 to 0.826) and IPA of 0.072 (95% CI, 20.016 to 0.
137), which reflect improved performance compared withNRI
the ACC/AHA PCEs using published coefficients (Table 3). Model 1 did not improve reclassification of events nor non-
Model 2 (AUC, 0.760; 95% CI, 0.678 to 0.851; and IPA, 0. events compared with the published ACC/AHA PCEs
095; 95% CI, 0.013 to 0.173) and model 3 (AUC, 0.771; (Table 4). Model 2 significantly improved reclassification of
95% CI, 0.674 to 0.853; and IPA, 0.105; 95% CI, 0.033 to 0. nonevents compared with the published ACC/AHA PCEs
168) both further improved prediction performance. Model (6.6%; 95% CI, 3.7% to 9.6%). Model 3 significantly
3 performed best, and was significantly better than model 2 improved reclassification of nonevents compared with both
in terms of AUC (P50.001) and IPA (P50.007). Applying the published ACC/AHA PCEs (10.0%; 95% CI, 6.8% to
the CKD patch to the CRIC study sample in a secondary 13.3%) and model 2 (3.7%; 95% CI, 1.4% to 6.0%), and was
analysis yielded an AUC of 0.752 and IPA of 0.096 the best-performing model in terms of NRI. The numbers
(Supplemental Figure 4). Prediction performance was simi- of participants reclassified according to each model are pre-
lar in participants with CKD stage 2–3 versus stage 4 sented in Supplemental Tables 7 and 8.
(Supplemental Figure 5).
Among the 2222 participants from the external valida-
tion cohorts, 337 ASCVD events occurred during the firstDISCUSSION
10 years of follow-up, and the observed 10-year risk of
ASCVD was higher compared with the CRIC study (15.5% Using a sample of 2604 participants with CKD stages 2–4
versus 10.5%) (Supplemental Table 6). In the external and free of clinical CVD, we developed and validated a set

33: 601–611, 2022 D Risk Prediction in CKD

 NICAL EPIDEMIOLOGY www.jasn.org

A CRIC Clinical Model B CRIC Enriched Model

Age
Age
HbA1c

Urinary ACR
Current Smoking

Current Smoking NT-proBNP

Urinary ACR
HbA1c
Troponin-T

HDL Cholesterol HDL Cholesterol

hsCRP
Systolic BP
Total Cholesterol

Hemoglobin Apolipoprotein B

0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.00 0.05 0.10 0.15 0.20 0.25 0.30
Proportion of Overall χ2 Proportion of Overall χ2

Figure 1. Relative importance of variables included in CKD ASCVD risk prediction models. Proportion of overall chi-squared for
(A) CRIC clinical model and (B) CRIC enriched model. Higher values for the proportion of overall chi-squared indicate more important
variables relative to other variables in the model.

CRIC Clinical Model CRIC Enriched Model

0.50 0.50

0.40 0.40

0.30 0.30
rved Probability

rved Probability

0.20 0.20

0.10 0.10
AUC = 0.760 (0.678, 0.851) AUC = 0.771 (0.674, 0.853)
IPA = 0.095 (0.013, 0.173) IPA = 0.105 (0.033, 0.168)

0.00 0.00
0.00 0.10 0.20 0.30 0.40 0.50 0.00 0.10 0.20 0.30 0.40 0.50
Predicted Probability Predicted Probability

Figure 2. Discrimination and calibration of risk prediction models for ASCVD in CKD. Results were obtained by aggregating pre-dicted
probabilities from 10 3 10-fold cross-validation and then assessing their discrimination (AUC), calibration (predicted probabil-ity versus
observed events, with each circle representing a decile of predicted probability), and overall goodness of fit (IPA). Observed
probability was estimated using the cumulative incidence function accounting for the competing risk of death. Higher val-ues for AUC
(95% CI) and IPA (95% CI) indicate better performing models.

33: 601–611, 2022

 www.jasn.orN
g ICAL EPIDEMIOLOGY

Table 3. Performance characteristics of novel CKD ASCVD risk prediction models developed in the CRIC study
Model 1: ACC/AHA PCEs
Performance Characteristics Model 2: CRIC Clinical Model Model 3: CRIC Enriched Model
Variables
Discrimination
AUC (95% CI)a 0.736 (0.649 to 0.826) 0.760 (0.678 to 0.851) 0.771 (0.674 to 0.853)
P value comparing with 0.03 ,0.001 ,0.001
ACC/AHA PCEsb
P value comparing – 0.01 0.001
successive modelsc
Overall goodness of fit
IPA (95% CI)a 0.072 (-0.016 to 0.137) 0.095 (0.013 to 0.173) 0.105 (0.033 to 0.168)
P value comparing with 0.01 ,0.001 ,0.001
ACC/AHA PCEsb
P value comparing – 0.004 0.007
successive modelsc
aHigher values for AUC and IPA indicate better performing models. bThe AUC
and IPA for the published ACC/AHA PCEs in the CRIC study are 0.730 and 0.058, respectively.cComparison
of model 2 with model 1, and model 3 with model 2.

of predictive models for 10-year risk of ASCVD. We found
that the published ACC/AHA PCEs had moderate predic-
tive ability for 10-year risk of ASCVD among participants
of the CRIC study. However, novel models developed from
a pool of .50 candidate predictors afforded significant pre-
diction improvements. Both a model developed using clini-
cally available variables and a biomarker-enriched model
significantly improved discrimination and calibration
beyond the ACC/AHA PCEs, and significantly improved
reclassification of nonevents. These findings support the
use of CKD-specific tools to predict the long-term risk of
ASCVD among patients with CKD, who may have different
risk profiles compared with the general population, for
whom prior models were developed.
Patients with CKD are typically considered to be at high
risk for ASCVD, irrespective of other individual patient
characteristics or laboratory values.4,5 As such, current clin-
ical practice guidelines do not have specific recommenda-
tions among patients with CKD other than to consider
kidney function markers, including eGFR and albuminuria,
as “risk-enhancing factors.”7 Our findings confirm the value
of CKD markers for prediction, with urinary ACR in par-
ticular being a strong predictor in both our clinical and
enriched models. Notably, eGFR was not included in our
final models, suggesting that urinary ACR has a relatively
stronger prognostic value for development of ASCVD
among patients with CKD compared with eGFR.
We found that several other variables were particularly
important for prediction of ASCVD among patients with
CKD. In both our clinical and enriched models, age was the
most important predictor of ASCVD at 10 years, which is an
important consideration in an aging US population.32 Beyond
age, we note the inclusion in our models of several important
predictors including current smoking, HbA1c, hsCRP, and car-
diac biomarkers. Diabetes is highly associated with both CKD
and CVD.33 Although diabetes status was included in the
ACC/AHA PCEs,6 absolute values of long-term glycemia were
not incorporated. Our models suggest the information afforded
by the values of a glycemic measure, specifically HbA1c, may
be superior for ASCVD risk prediction in CKD populations.
The cardiac biomarkers NT-proBNP and troponin-T
showed particularly high relative variable importance in

Table 4. Reclassification of events and nonevents in the CRIC study

Proportion of Participants Correctly Reclassifiedb
a
Model Comparisons Events Reclassified, % Nonevents Reclassified, % Net Reclassification Index
(95% CI) (95% CI) (95% CI)
Comparing with ACC/AHA PCEs
Model 1 versus ACC/AHA PCEs 210.0 (22.5 to 216.5) 1.5 (20.9 to 3.9) 28.5 (21.0 to 215.3)
Model 2 versus ACC/AHA PCEs 24.8 (212.2 to 2.8) 6.6 (3.7 to 9.6) 1.8 (26.2 to 10.0)
Model 3 versus ACC/AHA PCEs 21.4 (28.9 to 6.4) 10.0 (6.8 to 13.3) 8.6 (0.2 to 16.6)
Comparing with model 1
Model 2 versus 1 6.7 (20.3 to 13.5) 5.2 (2.8 to 8.0) 11.9 (4.6 to 18.9)
Model 3 versus 1 11.7 (3.6 to 19.4) 8.7 (5.9 to 11.7) 20.4 (12.3 to 28.6)
Comparing with model 2
Model 3 versus 2 5.8 (20.1 to 11.2) 3.7 (1.4 to 6.0) 9.5 (3.3 to 15.3)
aModel 1 indicates the ACC/AHA PCEs variables recalculated in the CRIC study; model 2 indicates the CRIC clinical model; and model 3 indicates the CRIC
enriched model.
bModels are compared for reclassification of events to a higher predicted risk category and nonevents to a lower predicted risk category, using cut points of 7.5%

and 20%. More detailed information can be found in Supplemental Tables 7 and 8.

33: 601–611, 2022 D Risk Prediction in CKD

NICAL EPIDEMIOLOGY www.jasn.org
our biomarker-enriched model. These biomarkers are
increasingly recognized as being strongly associated with
risk for ASCVD among patients with CKD, and were more
predictive of CVD than other nontraditional kidney bio-
markers among 8622 participants from the ARIC study. 14
Both NT-proBNP and troponin-T are circulating proteins
that become elevated in response to cardiac stretch and
injury.34,35 Although typically associated with heart failure over prior analyses that selected CKD patients from general
among patients with CKD,36 these biomarkers may also
provide information on a greater burden of subclinical car-
diac damage that contributes directly to the risk of ASCVD.
It should be acknowledged that elevated levels of NT-
proBNP or troponin-T may reflect, at least in part, impaired
renal clearance among CKD patients, but our findings and
the findings of others suggest these biomarkers are valuable
for ASCVD risk prediction in this popula- tion.14,37
Collection of nontraditional, although widely available,
laboratory tests could improve clinical evaluation and
management of ASCVD among patients with CKD.
There are several clinical and research implications of
our findings. CKD is a heterogeneous condition38 and the
ischemic heart disease encountered in CKD often involves
distal, small arteries that are not amenable to coronary
revascularization.39 Frequently CKD patients undergo inva-
sive diagnostic procedures because they are considered to
have high CVD risk. Using a prediction tool better
equipped to discern ASCVD events could help reduce the
number of unnecessary cardiac angiography procedures
and associated morbidity. Other recent advances in this
area include the development of a CKD patch to incorpo-
rate eGFR and ACR measurements into established risk
prediction equations.40 An analysis conducted among
approximately 9 million participants included in the CKD
Prognosis Consortium identified the utility of the CKD
patch in improving performance beyond the ACC/AHA
PCEs and the Systematic COronary Risk Evaluation
(SCORE) model.8 In the current analysis, the CKD patchDISCLOSURES
improved discrimination of ASCVD events compared with
the ACC/AHA PCEs. However, the CKD patch did not
consider potential improvements afforded by additional
biomarkers, nor models specifically developed amongN. Bansal reports associate editor for Kidney360. J.B. Cohen reports honorariafrom
patients with CKD. In the current CRIC study sample, our
new models provided more accurate predictions compared
with previous risk prediction models, particularly in those
with 10-year ASCVD risk ,20%. Therefore, our models
significantly improved reclassification of nonevents com-
pared with the standard ACC/AHA PCEs. Given the het-
erogeneity of risk among patients with CKD, it may be
warranted to employ prediction models specifically devel-
oped for this population. For those with predicted risk of
20% or more, it may be prudent to conduct further evalua-
tion, such as coronary artery calcium scoring. 41 In addition, Disease. J.P. Lash reports scientific advisor or membership with Kidney360. K.
our models could be used in future clinical trials to more
accurately recruit high risk patients, allowing for smaller
sample sizes and more efficient conduct of trials.
There are several strengths in the current analysis. First,
we considered .50 candidate predictor variables, measured
using standardized methodology in the CRIC study, which
improves accuracy and minimizes bias. Second, ASCVD
events are rigorously adjudicated. Third, we have a relatively
large population of patients with CKD stages 2–4 which was
enrolled based on CKD status, which is an improvement
population cohorts based on one assessment of kidney func-
tion. However, there are several limitations. First, although
we externally validated our models in two population-based
cohort studies, the individuals in these datasets were selected
based on only one assessment of kidney function and had a
higher baseline risk of ASCVD, on average. Our results sug-
gest adequate discrimination and calibration of our models,
but future analyses should externally validate and refine our
models among large and diverse populations with CKD and
varying baseline risks for ASCVD. Second, these models
could not incorporate potentially important imaging varia-
bles including coronary artery calcium scores and electrocar-
diogram data. Finally, our models overestimated risk in the
highest predicted probability groups in the CRIC study.
However, this finding is common to most CVD prediction
models, and the overpredictions occur at probabilities well
above clinical decision-making cutoff points.7
In conclusion, we have developed new models for 10-year
risk prediction of ASCVD among patients with CKD, includ-
ing a model composed of readily clinically available variables
and a biomarker-enriched model. These models significantly
improve prediction performance beyond the ACC/AHA
PCEs in this population of patients with CKD stages 2–4,
particularly for those with 10-year ASCVD risk ,20%.
Future analyses should externally validate and refine these
models in multiple diverse CKD cohorts.
N.B. Allen reports research funding with AHA and National Institutes of
Health (NIH)/National Heart, Lung, and Blood Institute (NHLBI).
UpToDate; and research funding with AHA and NIH/NHLBI and NIH/National
Center for Advancing Translational Sciences. R. Deo reports consul- tancy
agreements with Biotronik, Boehringer Ingelheim, Janssen Pharmaceuti- cals,
and Pfizer; research funding with iRhythm Technologies; editorial boardsfor
Circulation and Heart Rhythm O2; and steering committee for Kidney
Disease: Improving Global Outcomes. M.A. Dobre reports honoraria from
Relypsa Inc. and Tricida; and scientific advisor or membership with Tricida
(Metabolic Acidosis Working Group) and Relypsa (Resistant Hypertension
Working Group). H.I. Feldman reports consultancy with InMed, Inc., Kyowa
Hakko Kirin, National Kidney Foundation, and University of California, Los
Angeles; steering committee for the CRIC study; member of advisory board of
the National Kidney Foundation; editor-in-chief of American Journal of Kidney
Matsushita reports consultancy agreements with Akebia, Kyowa Hakko Kirin,
and Mitsubishi Tanabe; research funding from Kyowa Hakko Kirin; honoraria
from Fukuda Denshi; and scientific advisor or membership with American
Journal of Kidney Disease, Circulation Reports, and Kidney International.
33: 601–611, 2022
 www.jasn.orN
g ICAL EPIDEMIOLOGY

M. Rahman reports consultancy with Barologics; research funding from BayerHHSN268201500003I, N01-HC-95159, 75N92020D00005, N01-HC-95160,
Pharmaceuticals and Duke Clinical Research Institute; honoraria from Reata, 75N92020D00002, N01-HC-95161, 75N92020D00003, N01-HC-95162,
Relypsa, and Bayer; associate editor for CJASN; and editorial board member for75N92020D00006, N01-HC-95163, 75N92020D00004, N01-HC-95164,
American Journal of Nephrology. S. Seliger reports consultancy agreements with75N92020D00007, N01-HC-95165, N01-HC-95166, N01-HC-95167, N01-HC-
Tricida, Inc (Endpoint Adjudication Committee); research funding from Kad-95168, and N01-HC-95169), and by the National Center for Advancing
mon Pharmaceuticals, Palladio Biosciences, Reata Pharmaceuticals, Roche Diag- Translational Sciences (UL1TR000040, UL1TR001079, and UL1TR001420).
nostics, and Sanofi US; patents and inventions with University of Maryland,
Baltimore and University of Texas, Southwestern: Methods for assessing
differ- ential risk for developing heart failure; associate editor for CJASN;ACKNOWLEDGMENTS
member of medical review board for ESRD Network 5; chair of Board of
Directors for ESRD Network 5; member of Endpoint AdjudicationThe authors thank the participants, investigators, and staff of the CRIC, ARIC,
Committee and of VALOR CKD trial (Tricida, Inc.); and member of the editorial and MESA studies for their time and commitment. The authorsthank
board for Circu-lation. T. Shafi reports consultancy agreements with Siemens; Ms. Yingying Sang for her technical support.
research funding as clinical trial site investigator with Baxter, CVS, and
Natera; honoraria from Cara Therapeutics, NIH, Siemens, State University of
New York Downstate, and University of Virginia; and scientific advisor or
membership with American Journal of Medicine, American Journal of KidneyAUTHOR CONTRIBUTIONS
Diseases, CJASN, and Kid- ney360. M. Wolf reports consultancy agreements
with Akebia, Amgen, Ardelyx, AstraZeneca, Bayer, Pharmacosmos, Unicycive, J.D. Bundy, J. He, K. Matsushita, and M. Rahman conceptualized the
and Walden; ownership interestwith Akebia, Unicycive, and Walden; honoraria study; J. Chen, H.I. Feldman, J. He, J.P. Lash, M. Rahman were responsible
from Akebia, Amgen, Ardelyx, AstraZeneca, Bayer, Pharmacosmos, Unicycive, for funding acquisition; H.I. Feldman, J. He, J.P. Lash, and M. Rahman
and Walden; and scientific advi- sor or membership with Akebia, Unicycive, were responsible for data curation; J.D. Bundy and B.C. Jaeger were
and Walden. W. Yang reports sta- tistical editor for American Journal of responsible for formal analysis and software; J.D. Bundy, J. He, and
Kidney Disease. J. Flack reports consultancy agreements with TevaB.C. Jaeger were responsible for methodology; J.D. Bundy was responsible for
Pharmaceuticals, Amgen, Sanofi, Fibrogen, and Janssen; ownership interest visualization; N. Allen, N. Bansal, and K. Matsushita were responsible for
with Novocure, Google, Netflix, Albermarle, Alexandria REIT, Amazon, validation; J. He provided supervision; J.D. Bundy wrote the original draft;
Ambarella, Arch Cap Group, Chargepoint, Enphase,Intercontinental Exchange, and all authors reviewed and edited the manuscript.
Johnson Controls, Estee Lauder, Lucid, Marvell Technology, Mastercard,
Microsoft, Nova, Nvidia, Palantir, Paypal, Plains All American Pipeline,
Quantamscape, STMicroelectronics, Reconnaissance Energy, Lightbridge,
Advanced MicroDevices, Apollo Commercial Real Estate, Ballard Power S , UPPLEMENTAL MATERIAL
Emerson Electric, Essential Utils, IDT Corp, Meta Platforms, and Neo-
genomics; research funding from Glaxo Smith Kline, Idorsia, Quantam Geno- This article contains the following supplemental material online at
mics, ReCor Medical, Vascular Dynamics, and Rox Medical; honoraria from http://jasn.asnjournals.org/lookup/suppl/doi:10.1681/ASN.2021060747/-/
Amgen, Sanofi, and Teva; and advisory or leadership roles with American Col- DCSupplemental.
lege of Physicians Board of Regents, American Journal of Hypertension Associate Supplemental Methods.
Editor, and Sanofi. All remaining authors have nothing to disclose. Supplemental Figure 1. Candidate predictor variables considered for
inclusion in prediction models.
Supplemental Figure 2. Calibration slope plots of Cox proportional haz-
ards and Fine and Gray subdistribution hazards modeling algorithms.
FUNDING
Supplemental Figure 3. Calibration of the ACC/AHA PCEs in the CRIC
study.
Funding for the CRIC study was obtained under a cooperative
Supplemental Figure 4. Prediction performance of the CKD predictor
agreement from the National Institute of Diabetes and Digestive and
patch in the CRIC study.
Kidney Diseases (U01DK060990, U01DK060984, U01DK061022,
Supplemental Figure 5. Prediction performance of novel CRIC study
U01DK061021, U01DK061028, U01DK060980, U01DK060963,
models by CKD stage.
U01DK060902, and
Supplemental Figure 6. Prediction performance of novel CRIC study
U24DK060990). In addition, this work was supported in part by the Perelman
models in external validation data.
School of Medicine at the University of Pennsylvania Clinical and Transla-
Supplemental Table 1. Discrimination and IPA for 10-year risk predic-
tional Science Award, National Center for Advancing Translational Sciences
tion of ASCVD.
(UL1TR000003), Johns Hopkins University (UL1TR000424), University of
Supplemental Table 2. Internal-external validation of 10-year risk pre-
Maryland General Clinical Research Center (M01 RR-16500), Clinical and
diction of ASCVD in the CRIC study.
Translational Science Collaborative of Cleveland, the National Center for
Supplemental Table 3. Example calculations of predicted 10-year risk of
Advancing Translational Sciences component of the NIH roadmap for Medical
ASCVD (ACC/AHA PCEs variables).
Research (UL1TR000439), Michigan Institute for Clinical and Health Research
Supplemental Table 4. Example calculations of predicted 10-year risk of
(UL1TR000433), University of Illinois at Chicago Clinical and Translational
ASCVD (CRIC clinical model).
Science Award (UL1RR029879), Tulane COBRE for Clinical and Translational
Supplemental Table 5. Example calculations of predicted 10-year risk of
Research in Cardiometabolic Diseases (P20 GM109036), Kaiser Permanente,
National Center for Research Resources (UCSF-CTSI UL1 RR-024131), and ASCVD (CRIC enriched model).
Department of Internal Medicine, University of New Mexico School of Supplemental Table 6. Baseline characteristics of development (CRIC
Medicine Albuquerque, New Mexico (R01DK119199). J.D. Bundy was sup- study) and external validation (ARIC and MESA) cohort participants.
ported by the Eunice Kennedy Shriver National Institute of Child Health and Supplemental Table 7. Reclassification of events for risk prediction
Human Development career development grant (K12HD043451). The ARIC models for ASCVD in CKD.
study is supported by the National Heart, Lung, and Blood Institute Supplemental Table 8. Reclassification of nonevents for risk prediction
(HHSN268201700001I, HHSN268201700002I, HHSN268201700003I, models for ASCVD in CKD.
33: 601–611, 2022 D Risk Prediction in CKD
HHSN268201700004I, and HHSN268201700005I). MESA is supported by
the National Heart, Lung, and Blood Institute (75N92020D00001,
 NICAL EPIDEMIOLOGY www.jasn.org
REFERENCES
1. Mills KT, Xu Y, Zhang W, Bundy JD, Chen C-S, Kelly TN, et al.: A systematic
analysis of worldwide population-based data on the global burden of
chronic kidney disease in 2010. Kidney Int 88: 950–957, 2015
2. Gansevoort RT, Correa-Rotter R, Hemmelgarn BR, Jafar TH, Heer-
spink HJL, Mann JF, et al.: Chronic kidney disease and cardiovascu- lar
risk: Epidemiology, mechanisms, and prevention. Lancet 382: 339–
352, 2013
3. Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY: Chronic kidney
disease and the risks of death, cardiovascular events, and hospitali-
zation. N Engl J Med 351: 1296–1305, 2004
4. Kidney Disease: Improving Global Outcomes (KDIGO) CKD Work
Group: KDIGO 2012 Clinical Practice Guideline for the Evaluation and
Management of Chronic Kidney Disease. Kidney Int Suppl 3: 1–150,
2013. Available at: https://kdigo.org/wp-content/uploads/
2017/02/KDIGO_2012_CKD_GL.pdf. Accessed December 10, 2021
5. Piepoli MF, Hoes AW, Agewall S, Albus C, Brotons C, Catapano AL,et al.;
ESC Scientific Document Group: 2016 European Guidelines on
cardiovascular disease prevention in clinical practice: The Sixth Joint Task
Force of the European Society of Cardiology and Other
Societies on Cardiovascular Disease Prevention in Clinical Practice
(constituted by representatives of 10 societies and by invited
experts)Developed with the special contribution of the European
Association for Cardiovascular Prevention & Rehabilitation (EACPR).
Eur Heart J 37: 2315–2381, 2016
6. Goff DC, Lloyd-Jones DM, Bennett G, Coady S, D’Agostino RB, Gib-
bons R, et al.: 2013 ACC/AHA guideline on the assessment of car-
diovascular risk: A report of the American College of Cardiology/
American Heart Association task force on practice guidelines. Circu-
lation 129[Suppl 2]: S49–S73, 2014
7. Arnett DK, Blumenthal RS, Albert MA, Buroker AB, Goldberger ZD,
Hahn EJ, et al.: 2019 ACC/AHA guideline on the primary prevention of
cardiovascular disease: A report of the American College of Car-
diology/American Heart Association task force on clinical practice
guidelines. Circulation 140: e596–e646, 2019
8. Matsushita K, Jassal SK, Sang Y, Ballew SH, Grams ME, Surapaneni
A, et al.: Incorporating kidney disease measures into cardiovascular risk
prediction: Development and validation in 9 million adults from 72
datasets. EClinicalMedicine 27: 100552, 2020
9. Matsushita K, Coresh J, Sang Y, Chalmers J, Fox C, Guallar E, et al.:Estimated
glomerular filtration rate and albuminuria for prediction of cardiovascular
outcomes: a collaborative meta-analysis of individual participant data.
Lancet Diabetes Endocrinol 3: 514–525, 2015
10. Ito H, Pacold IV, Durazo-Arvizu R, Liu K, Shilipak MG, Goff DCJ Jr,
et al.: The effect of including cystatin C or creatinine in a cardiovas-cular
risk model for asymptomatic individuals: The multi-ethnic study of
atherosclerosis. Am J Epidemiol 174: 949–957, 2011
11. Clase CM, Gao P, Tobe SW, McQueen MJ, Grosshennig A, Teo KK,
et al.; ONTARGET (ONgoing Telmisartan Alone and in combination with
Ramipril Global Endpoint Trial) and TRANSCEND (Telmisartan
Randomized Assessment Study in Angiotensin-Converting-Enzyme-
Inhibitor Intolerant Subjects with Cardiovascular Disease): Estimated
glomerular filtration rate and albuminuria as predictors of outcomes
in patients with high cardiovascular risk: A cohort study. Ann InternMed
154: 310–318, 2011
12. Matsushita K, Ballew SH, Coresh J, Arima H, A€ rnlo€v J, Cirillo M,
et al.: Measures of chronic kidney disease and risk of incident peripheral
artery disease: A collaborative meta-analysis of individual
participant data. Lancet Diabetes Endocrinol 5: 718–728, 2017
13. Weiner DE, Tighiouart H, Elsayed EF, Griffith JL, Salem DN, Levey
AS, et al.: The Framingham predictive instrument in chronic kidney
disease. J Am Coll Cardiol 50: 217–224, 2007
14. Matsushita K, Sang Y, Ballew SH, Astor BC, Hoogeveen RC, Solomon SD, et
al.: Cardiac and kidney markers for cardiovascular prediction in individuals
with chronic kidney disease: The Atherosclerosis Risk in Communities study.
Arterioscler Thromb Vasc Biol 34: 1770–1777, 2014
15. Matsushita K, Sang Y, Ballew SH, Shlipak M, Katz R, Rosas SE, et al.:
Subclinical atherosclerosis measures for cardiovascular prediction inCKD.
J Am Soc Nephrol 26: 439–447, 2015
16. Fontana M, Asaria P, Moraldo M, Finegold J, Hassanally K, Manisty CH, et
al.: Patient-accessible tool for shared decision making in car- diovascular
primary prevention: Balancing longevity benefits against
medication disutility. Circulation 129: 2539–2546, 2014
17. Lash JP, Go AS, Appel LJ, He J, Ojo A, Rahman M, et al.; Chronic Renal
Insufficiency Cohort (CRIC) Study Group: Chronic Renal Insuffi-ciency Cohort
(CRIC) study: baseline characteristics and associationswith kidney function.
Clin J Am Soc Nephrol 4: 1302–1311, 2009
18. Levey AS, Titan SM, Powe NR, Coresh J, Inker LA: Kidney disease, race,
and GFR estimation. Clin J Am Soc Nephrol 15: 1203–1212,2020
19. Inker LA, Schmid CH, Tighiouart H, Eckfeldt JH, Feldman HI, GreeneT, et al.;
CKD-EPI Investigators: Estimating glomerular filtration rate from serum
creatinine and cystatin C. N Engl J Med 367: 20–29, 2012
20. Beretta L, Santaniello A: Nearest neighbor imputation algorithms: Acritical
evaluation. BMC Med Inform Decis Mak 16[Suppl 3]: 74, 2016
21. Feldman HI, Appel LJ, Chertow GM, Cifelli D, Cizman B, Daugirdas J, et al.;
Chronic Renal Insufficiency Cohort (CRIC) Study Investigators: The Chronic
Renal Insufficiency Cohort (CRIC) study: Design and methods. J Am
Soc Nephrol 14[Suppl 2]: S148–S153, 2003
22. Fine JP, Gray RJ: A proportional hazards model for the subdistribu- tion of a
competing risk. J Am Stat Assoc 94: 496–509, 1999
23. Tibshirani R: The lasso method for variable selection in the Cox
model. Stat Med 16: 385–395, 1997
24. Steyerberg EW, Harrell FEJ Jr: Prediction models need appropriate internal,
internal-external, and external validation. J Clin Epidemiol 69: 245–247, 2016
25. Wright JD, Folsom AR, Coresh J, Sharrett AR, Couper D, Wagen-
knecht LE, et al.: The ARIC (Atherosclerosis Risk In Communities) study:
JACC Focus Seminar 3/8. J Am Coll Cardiol 77: 2939–2959, 2021
26. Bild DE, Bluemke DA, Burke GL, Detrano R, Diez Roux AV, Folsom AR, et al.:
Multi-Ethnic Study of Atherosclerosis: Objectives and design. Am J
Epidemiol 156: 871–881, 2002
27. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchow-
ski N, et al.: Assessing the performance of prediction models: A
framework for traditional and novel measures. Epidemiology 21:
128–138, 2010
28. Blanche P, Dartigues J-F, Jacqmin-Gadda H: Estimating and com-
paring time-dependent areas under receiver operating characteristic
curves for censored event times with competing risks. Stat Med 32:
5381–5397, 2013
29. Kattan MW, Gerds TA: The index of prediction accuracy: An intuitive
measure useful for evaluating risk prediction models. Diagn Progn Res 2:
7, 2018
30. Pencina MJ, D’Agostino RB Sr, D’Agostino RB Jr, Vasan RS: Evaluat-
ing the added predictive ability of a new marker: From area under the
ROC curve to reclassification and beyond. Stat Med 27: 157–172,
discussion 207–212, 2008
31. Uno H, Tian L, Cai T, Kohane IS, Wei LJ: A unified inference proce-
dure for a class of measures to assess improvement in risk prediction
systems with survival data. Stat Med 32: 2430–2442, 2013
32. North BJ, Sinclair DA: The intersection between aging and cardio-
vascular disease. Circ Res 110: 1097–1108, 2012
33. Perkovic V, Agarwal R, Fioretto P, Hemmelgarn BR, Levin A, ThomasMC, et
al.; Conference Participants: Management of patients with
33: 601–611, 2022

diabetes and CKD: Conclusions from a “Kidney Disease: Improving Global
Outcomes” (KDIGO) Controversies Conference. Kidney Int90: 1175–
1183, 2016
34. Yasue H, Yoshimura M, Sumida H, Kikuta K, Kugiyama K, Jougasaki M, et al.:
Localization and mechanism of secretion of B-type natriuretic pep- tide in
comparison with those of A-type natriuretic peptide in normal subjects and
patients with heart failure. Circulation 90: 195–203, 1994
35. de Lemos JA, Drazner MH, Omland T, Ayers CR, Khera A, Rohatgi
A, et al.: Association of troponin T detected with a highly sensitive assay
and cardiac structure and mortality risk in the general popula-tion.
JAMA 304: 2503–2512, 2010
36. Bansal N, Hyre Anderson A, Yang W, Christenson RH, deFilippi CR,
Deo R, et al.: High-sensitivity troponin T and N-terminal pro-B-type
natriuretic peptide (NT-proBNP) and risk of incident heart failure in
patients with CKD: The Chronic Renal Insufficiency Cohort (CRIC) study. J
Am Soc Nephrol 26: 946–956, 2015
37. Scheven L, de Jong PE, Hillege HL, Lambers Heerspink HJ, van Pelt LJ,
Kootstra JE, et al.; PREVEND study group: High-sensitive
33: 601–611, 2022
www.jasn.orN
g ICAL EPIDEMIOLOGY
troponin T and N-terminal pro-B type natriuretic peptide are associ-ated
with cardiovascular events despite the cross-sectional associa-
tion with albuminuria and glomerular filtration rate. Eur Heart J 33:
2272–2281, 2012
38. Levey AS, Coresh J: Chronic kidney disease. Lancet 379: 165–180, 2012
39. Nogueira J, Weir M: The unique character of cardiovascular disease in
chronic kidney disease and its implications for treatment with lipid-
lowering drugs. Clin J Am Soc Nephrol 2: 766–785,
2007
40. Matsushita K, Sang Y, Chen J, Ballew SH, Shlipak M, Coresh J, et al.: Novel
“predictor patch” method for adding predictors using esti- mates from
outside datasets - A proof-of-concept study adding kid-
ney measures to cardiovascular mortality prediction. Circ J 83: 1876–
1882, 2019
41. Chen J, Budoff MJ, Reilly MP, Yang W, Rosas SE, Rahman M, et al.; CRIC
Investigators: Coronary artery calcification and risk of cardio- vascular
disease and death among patients with chronic kidney dis- ease. JAMA
Cardiol 2: 635–643, 2017
D Risk Prediction in CKD
 AFFILIATIONS

1
Department of Epidemiology, Tulane University School of Public Health and Tropical Medicine, New Orleans,
Louisiana 2Tulane University Translational Science Institute, New Orleans,
Louisiana 3Department of Medicine, Case Western Reserve University School of Medicine,
Cleveland, Ohio 4Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Division of
Cardiology, Johns Hopkins School of Medicine,
Welch Center for Prevention, Epidemiology, and Clinical Research, Baltimore, Maryland
5
Department of Biostatistics, University of Alabama at Birmingham, Birmingham,
Alabama 6Renal, Electrolyte, and Hypertension Division, Department of Medicine, University of Pennsylvania Perelman
School of Medicine, Philadelphia,
Pennsylvania
7
Department of Medicine, Tulane University School of Medicine, New Orleans, Louisiana
8
Cardiovascular Division, Department of Medicine, University of Pennsylvania Perelman School of Medicine, Philadelphia,
Pennsylvania 9Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania Perelman School of
Medicine, Philadelphia, Pennsylvania
10
Department of Internal Medicine, Southern Illinois University School of Medicine, Springfield, Illinois
11
Department of Medicine, University of Illinois College of Medicine at Chicago,
Chicago, Illinois 12Department of Medicine, University of Maryland School of
Medicine, Baltimore, Maryland 13Nephrology Division, The University of Mississippi
Medical Center, Jackson, Mississippi 14Department of Medicine, Duke Clinical
Research Institute, Duke University, Durham, North Carolina15Department of
Preventive Medicine, Northwestern Feinberg School of Medicine, Chicago, Illinois
16
Division of Nephrology, University of Washington, Seattle, Washington

33: 601–611, 2022 D Risk Prediction in CKD

 2. Ceklis JBI
JBI CRITICAL APPRAISAL CHECKLIST FOR COHORT STUDIES
Reviewer : Resta Saputri. Yn
Peneliti : Joshua D. Bundy
Tanggal : 16 mei

Tahun : 2023
1. Apakah kedua kelompok serupa dan direkrut √
dari populasi yang sama?

2. Apakah eksposur diukur sama untuk √

menetapkan orang untuk kedua kelompok
terpapar dan tidak terpapar?
3. Apakah eksposur diukur dengan cara yang valid √
dan dapat diandalkan?

4. Apakah faktor perancu diidentifikasi? √

5. Apakah strategi untuk menangani faktor √

perancu dinyatakan?

6. Apakah kelompok/peserta bebas dari hasil pada √

awal penelitian (atau pada saat pemaparan)?

7. Apakah hasil diukur dengan cara yang valid dan √

dapat diandalkan?

8. Apakah waktu tindak lanjut dilaporkan dan √

cukup memadai untuk menghasilkan hasil?

9. Apakah tindak lanjut lengkap, dan jika tidak, √

apakah alasan mangkir dijelaskan dan
dieksplorasi?
10. Apakah strategi untuk mengatasi tindak lanjut √
yang tidak lengkap digunakan?

11. Apakah analisis statistik yang tepat digunakan? √

skor 7
hasil 7/11 : 63%

33: 601–611, 2022 D Risk Prediction in CKD