Accepted Manuscript

Sepsis-3 Septic Shock Criteria and Associated Mortality Among Infected Hospitalized
Patients Assessed by a Rapid Response Team

Shannon M. Fernando, MD, MSc, Peter M. Reardon, MD, Bram Rochwerg, MD, MSc,
Nathan I. Shapiro, MD, MPH, Donald M. Yealy, MD, Andrew J.E. Seely, MD, PhD,
Jeffrey J. Perry, MD, MSc, Douglas P. Barnaby, MD, MS, Kyle Murphy, MD, Peter
Tanuseputro, MD, MHSc, Kwadwo Kyeremanteng, MD, MHA

PII: S0012-3692(18)30741-4
DOI: 10.1016/j.chest.2018.05.004
Reference: CHEST 1730

To appear in: CHEST

Received Date: 13 December 2017

Revised Date: 3 April 2018
Accepted Date: 1 May 2018

Please cite this article as: Fernando SM, Reardon PM, Rochwerg B, Shapiro NI, Yealy DM, Seely AJE,
Perry JJ, Barnaby DP, Murphy K, Tanuseputro P, Kyeremanteng K, Sepsis-3 Septic Shock Criteria
and Associated Mortality Among Infected Hospitalized Patients Assessed by a Rapid Response Team,
CHEST (2018), doi: 10.1016/j.chest.2018.05.004.

This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to
our customers we are providing this early version of the manuscript. The manuscript will undergo
copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please
note that during the production process errors may be discovered which could affect the content, and all
legal disclaimers that apply to the journal pertain.
 ACCEPTED MANUSCRIPT

TEXT WORD COUNT: 2,463

ABSTRACT WORD COUNT: 250

TITLE: Sepsis-3 Septic Shock Criteria and Associated Mortality Among Infected
Hospitalized Patients Assessed by a Rapid Response Team

PT
RUNNING TITLE: Sepsis-3 Criteria and Rapid Response Teams

AUTHORS: Shannon M. Fernando, MD, MSc1,2; Peter M. Reardon, MD1,2; Bram

RI
Rochwerg, MD, MSc3; Nathan I. Shapiro, MD, MPH4; Donald M. Yealy, MD5; Andrew
J.E. Seely, MD, PhD1,6,7; Jeffrey J. Perry, MD, MSc2,6, Douglas P. Barnaby, MD, MS8;
Kyle Murphy, MD1; Peter Tanuseputro, MD, MHSc6,9; Kwadwo Kyeremanteng, MD,

SC
MHA1,6,10

AFFILIATIONS: From the 1Division of Critical Care, Department of Medicine,

University of Ottawa, Ottawa, ON; 2Department of Emergency Medicine, University of

U
Ottawa, Ottawa, ON; 3Department of Medicine, Division of Critical Care, and
Department of Health Research Methods, Evidence, and Impact, McMaster University,
AN
Hamilton, ON; 4Department of Emergency Medicine, Beth Israel Deaconess Medical
Center, Harvard Medical School, Boston, MA; 5Department of Emergency Medicine,
University of Pittsburgh, Pittsburgh, PA; 6Clinical Epidemiology Program, Ottawa
Hospital Research Institute, Ottawa, ON; 7Department of Surgery, University of Ottawa,
M

Ottawa, ON; 8Department of Emergency Medicine, Albert Einstein College of Medicine,

Bronx, NY; 9Bruyere Research Institute, Ottawa, ON; 10Division of Palliative Care,
Department of Medicine, University of Ottawa, Ottawa, ON.
D

CORRESPONDENCE TO: Dr. Shannon M. Fernando, Department of Emergency

TE

Medicine, The Ottawa Hospital, Civic Campus, 1053 Carling Ave., Ottawa, ON. K1Y
4E9.
Email: sfernando@qmed.ca
EP

KEYWORDS: Septic Shock; Infectious Diseases; Sepsis-3; Resuscitation

CONFLICTS OF INTEREST: AJES holds patents related to multiorgan variability

C

analysis, and has shares in Therapeutic Monitoring Systems Inc., a company whose
mission is to help deliver variability-directed clinical decision support products to the
AC

bedside to improve care. None of the other authors report any conflict of interest.

PRIMARY FUNDING SOURCE: None.

 ACCEPTED MANUSCRIPT

ABBREVIATIONS

AUROC = Area under Receiver Operating Characteristic Curve

CI = Confidence Interval
ED = Emergency Department
ICU = Intensive Care Unit

PT
IQR = Interquartile Range
qSOFA = quick Sequential (Sepsis-related) Organ Failure Assessment
RRT = Rapid Response Team
Sepsis-3 = Third International Consensus Definitions for Sepsis and Septic Shock

RI
SD = Standard Deviation
SIRS = Systemic Inflammatory Response Syndrome
SOFA = Sequential (Sepsis-Related) Organ Failure Assessment)

U SC
AN
M
D
TE
C EP
AC

1
 ACCEPTED MANUSCRIPT

ABSTRACT
Background: Rapid Response Teams (RRTs) respond to deteriorating hospitalized

patients, and help determine subsequent management, including Intensive Care Unit

(ICU) admission. In such patients with sepsis and septic shock, the Sepsis-3 clinical

PT
criteria have a potential role in detection, risk-stratification, and prognostication, though

RI
their accuracy in comparison to the Systemic Inflammatory Response Syndrome (SIRS)-

based septic shock criteria are unknown. We sought to evaluate prognostic accuracy of

SC
the Sepsis-3 criteria for in-hospital mortality among infected hospitalized patients with

acute deterioration.

U
Methods: Prospectively collected registry data (2012-2016) from two hospitals, and
AN
including consecutive hospitalized patients with suspected infection seen by the RRT. We
M

compared Sepsis-3 criteria against the SIRS-based criteria for prediction of in-hospital

mortality.
D

Results: Of 1,708 included patients, 418 (24.5%) met Sepsis-3 septic shock criteria,
TE

while 545 (31.9%) met the SIRS-based septic shock criteria. Patients meeting Sepsis-3

septic shock criteria had higher in-hospital mortality (40.9% vs. 33.5%, P<0.0001), ICU
EP

admission (99.5% vs. 89.2%; P<0.001), and discharge rates to long-term care (66.3% vs.

53.7%; P<0.0001), compared to the SIRS-based septic shock criteria. Sensitivity and
C

specificity of quick Sequential Organ Failure Assessment (qSOFA) was 64.9% and
AC

92.2% for prediction of in-hospital mortality, while SIRS had a sensitivity and specificity

of 91.6% and 23.6%, respectively.

Conclusions: Hospitalized patients with deterioration from suspected infection had

higher risk of in-hospital mortality if they met the Sepsis-3 septic shock criteria, as

2
 ACCEPTED MANUSCRIPT

compared to the SIRS-based septic shock criteria. Therefore, use of Sepsis-3 criteria may

be preferable in the prognostication and disposition of these critically ill patients.

PT
RI
U SC
AN
M
D
TE
C EP
AC

3
 ACCEPTED MANUSCRIPT

INTRODUCTION

Severe infection remains a major cause of morbidity and mortality worldwide1,2

and is one of the most common reasons for hospitalization among adult patients3. Up to

PT
50% of these patients are treated at some point on the hospital wards, and are at risk of

acute deterioration, often necessitating escalation of care4,5. Early identification and

RI
management of patients with sepsis and septic shock on the ward is an important

component for improving survival6. Many hospitals have a Rapid Response Team (RRT),

SC
comprised of multi-professional care specialists, who respond to acutely deteriorating

hospitalized patients outside of the ICU. RRT implementation may improve mortality in

U
these patients7,8. Patients with severe infection are commonly assessed by the RRT and
AN
admitted to the ICU9, and accurate recognition and prognostication in these patients helps
M

optimize treatment and disposition10.

Since 1991, diagnostic criteria for sepsis have used the presence of the Systemic
D

Inflammatory Response Syndrome (SIRS) criteria in those with suspected infection11,12,

TE

with presence of two or more criteria being diagnostic of “sepsis”. “Septic shock” was

defined as persistent sepsis-induced hypotension (a systolic blood pressure [SBP] < 90

EP

mmHg), requiring administration of vasopressors/inotropes, or with evidence of perfusion

abnormalities (i.e. lactic acidosis, oliguria, or altered mental status)11.

C

Recently, the Third International Consensus Definitions for Sepsis and Septic
AC

Shock (Sepsis-3) re-evaluated these definitions13. Instead of focusing on SIRS, the

Sepsis-3 taskforce utilized the Sequential (Sepsis-related) Organ Failure Assessment

(SOFA) score, with an increase of ≥ 2 SOFA score points being diagnostic of “sepsis”,

and indicative of increased risk of death14. Sepsis-3 also introduced the quick SOFA

4
 ACCEPTED MANUSCRIPT

(qSOFA); a prognostic risk-stratification tool, with three criteria: low blood pressure

(SBP ≤ 100), increased respiratory rate (≥ 22 breaths/minute), and altered mental status

(Glasgow coma scale [GCS] < 15)15. Presence of two or more of these criteria in patients

with suspected infection indicates an increased risk of death. Finally, Sepsis-3 identified

PT
new, more restrictive criteria for diagnosis of “septic shock”, including the initiation of

RI
vasopressors to maintain a mean arterial pressure (MAP) of ≥ 65 mmHg, coupled with a

serum lactate > 2.0 mmol/L, following “adequate fluid resuscitation”16.

SC
Given their role in detection, risk-stratification and prognostication, the Sepsis-3

clinical criteria have important implications for patients with infection assessed by the

U
RRT for acute deterioration. However, little is known regarding the relative prognostic
AN
accuracy of these criteria in this population, particularly in comparison to the previous
M

SIRS-based criteria. We sought to compare the prognostic accuracy of the Sepsis-3 septic

shock criteria and the SIRS-based septic shock criteria for prediction of in-hospital
D

mortality among patients hospitalized with suspected infection, and receiving RRT
TE

assessment for acute deterioration. We secondarily compared the prognostic accuracy of

qSOFA and SIRS for prediction of in-hospital mortality in this population.

EP

MATERIALS AND METHODS

C
AC

Study Design, Setting and Subjects

The Ottawa Health Science Network Research Ethics Board approved this study.

We studied patients at two individual academic hospitals within The Ottawa Hospital

network. The combined network has 1163-beds, and handles over 160,000 emergency

visits, 50,000 inpatients, and 35,000 surgical cases annually. We prospectively collected

5
 ACCEPTED MANUSCRIPT

data and stored it in the Ottawa Hospital Data Warehouse, a health administrative registry

used in previous reports17-19. Regular quality-assurance initiatives are conducted to ensure

completeness and accuracy of the data17.

We included patients 18 years of age or older, received RRT activation between

PT
May 1, 2012 and May 31, 2016; and had “suspected infection” prior to, or at the time of

RI
RRT assessment. “Suspected infection” was defined as concomitant administration of

oral or parenteral antibiotics, and sampling of body fluid cultures (blood, urine,

SC
cerebrospinal fluid, peritoneal, etc.). If cultures were drawn first, antibiotics had to be

administered within 72 hours, whereas if the antibiotic was administered first, cultures

U
had to be drawn with 24 hours. This operational definition of ‘suspected infection’
AN
matches that used in the Sepsis-3 derivation cohorts13,15,16. We excluded patients with

incomplete demographic or outcome data and those with routine, scheduled RRT follow-
M

up.
D

The RRT only responds to inpatients, outpatients experiencing distress (e.g. in

TE

radiology and endoscopy suites), or patients and family members requiring immediate

care in hospital clinics or waiting rooms. The RRT does not respond to patients being
EP

assessed in the ED (who have not yet been admitted). RRT activation criteria have been

published previously20 (e-Table 1), but healthcare providers are encouraged to activate
C

the RRT for any reason of concern, even in the absence of objective changes in vital signs
AC

or laboratory values.

Data Collection

6
 ACCEPTED MANUSCRIPT

At the time of hospital admission, registration clerical staff collected demographic

data, comorbidities, previous ED visits, previous hospital admissions, and previous ICU

admissions in the year prior to the index admission17. RRT nurses gathered and recorded

data related to RRT activation at the time of patient assessment, including the most recent

PT
laboratory values at the time of RRT activation. Using these data, an investigator unaware

RI
of outcomes calculated SIRS criteria, SOFA and qSOFA scores for each patient. We

estimated PaO2/FiO2 values (utilized for the ‘Respiratory’ value of the SOFA score) by

SC
using the oxygen saturation (SaO2)/FiO2 values, as performed previously21. SOFA scores

at the time of RRT activation were computed in relation to baseline SOFA scores

U
(defined by the most recent laboratory values prior to the index hospital admission), in
AN
order to identify the change in SOFA score. In patients where prior data was unavailable,

the baseline SOFA score was assumed to be zero, as described in the original article13.
M

The primary outcome was in-hospital mortality, comparing the SIRS-based septic shock
D

criteria and Sepsis-3 septic shock criteria. Secondary outcomes included prognostic
TE

accuracy of SOFA, qSOFA and SIRS for prediction of mortality.

EP

Statistical Analysis

We performed all statistical analyses with commercially available statistical

C

packages (R statistical software, Version 3.3.3 and IBM SPSS, Version 24.0). We report
AC

data as mean values (with standard deviation, SD), or as medians (with interquartile

range, IQR), where indicated. We compared Sepsis-3 and SIRS-based septic shock

clinical criteria using the Student’s t-test for continuous variables and χ2 for categorical

variables. We calculated area under the receiver operating characteristics (AUROC)

7
 ACCEPTED MANUSCRIPT

curve to evaluate the performance of qSOFA and SIRS for prediction of in-hospital

mortality. We report odds ratios (OR) with 95% confidence intervals (CI) and P values

with per-comparison alpha set at ≤ 0.05.

PT
RESULTS
RRT was activated for 6,132 discrete adult patients during the study period. Of

RI
these, 109 were excluded due to incomplete data, leaving 6,023 patients with complete

SC
datasets. A total of 1,708 patients met criteria for suspected infection, and 570 of these

patients (33.4%) died in-hospital. The large majority of the overall cohort (1,488, 87.1%)

U
met criteria for “suspected infection” prior to RRT activation. Baseline characteristics of
AN
patients with suspected infection are in Table 1. We depict the distribution of patients

meeting both SIRS-based clinical criteria and Sepsis-3 clinical criteria in e-Figure 1. Of
M

patients with suspected infection, 1,391 (81.4%) had ≥ 2 SIRS criteria, meeting that
D

epoch’s threshold for “sepsis”, as per the SIRS-based criteria. Conversely, 1,045 (61.2%)

patients had ≥ 2 SOFA criteria, meeting the threshold for “sepsis” as per the Sepsis-3
TE

criteria. There were 422 patients (30.3%) with ≥ 2 SIRS criteria who did not meet these
EP

Sepsis-3 criteria for sepsis, and of these, 68 (16.1%) died in-hospital. RRT characteristics

are displayed in Table 2. The most common reasons for RRT activation were respiratory
C

distress (34.3%), hypotension (18.4%), and tachycardia/arrhythmia (14.3%). Prognostic

AC

accuracy of the SOFA criteria for prediction of mortality in patients with suspected

infection is displayed in Table 3. Of those with a SOFA ≥ 2, 38.6% (403/1045) died in-

hospital, compared with 25.2% (167/663) of patients with a SOFA < 2 (OR, 1.86 [95%

CI: 1.51-2.31], P < 0.0001).

8
 ACCEPTED MANUSCRIPT

The prognostic accuracy of the Sepsis-3 septic shock criteria compared with the

SIRS-based septic shock criteria are shown in Table 4. Among all patients with suspected

infection, 418 (24.5%) met the Sepsis-3 septic shock criteria, compared with 545 (31.9%)

who met the SIRS-based septic shock criteria. In-hospital mortality (40.9% vs. 33.5%, P

PT
= 0.02) and need for ICU admission (99.5% vs. 89.2%, P < 0.0001) were higher among

RI
septic shock patients meeting the Sepsis-3 criteria as compared with the SIRS-based

criteria. Of patients not meeting the Sepsis-3 septic shock criteria (n = 1,290), 400

SC
(31.1%) died in hospital, demonstrating increased risk for patients that met the criteria

(OR, 1.55 [95% CI: 1.23-1.94]). By comparison, of patients not meeting the SIRS-based

U
septic shock criteria (n = 1,163), 387 (33.3%) died in-hospital, which was not
AN
significantly different from patients who met the SIRS-based septic shock criteria (OR,

1.01 [95% CI: 0.82-1.22]). Median ICU length of stay was significantly longer among
M

patients meeting the Sepsis-3 septic shock criteria, as compared to the SIRS-based septic
D

shock criteria (5 days vs. 4 days, P < 0.01). There were 15 patients who met the Sepsis-3
TE

septic shock criteria (3.6%), but did not have ≥ 2 SIRS criteria. Sensitivity analysis

excluding patients without limitations on care is presented in e-Table 2.

EP

Finally, we examined the prognostic accuracy of qSOFA in comparison to the

SIRS criteria, for prediction of in-hospital mortality among patients with suspected
C

infection assessed by the RRT (Table 5). Of the eligible infected cohort, 459 patients
AC

(26.9%) met the qSOFA criteria, compared with 1,391 (81.4%) who met the SIRS

criteria. For prediction of in-hospital mortality, qSOFA was found to have a sensitivity of

64.9% (95% CI: 60.8-68.8) and a specificity of 92.2% (95% CI: 90.5-93.7). Among the

200 patients who died despite a negative qSOFA at the time off RRT assessment, the

9
 ACCEPTED MANUSCRIPT

median time from RRT assessment to death was 3 days (IQR, 1-5 days). AUROC for

qSOFA was 0.79 (95% CI: 0.77-0.82). By comparison, SIRS had a sensitivity of 91.6%

(95% CI: 89.0-93.7) and specificity of 23.6% (95% CI: 21.2-26.2) for prediction of in-

hospital mortality, with an AUROC of 0.56 (0.54-0.59). Evaluation of a subgroup of

PT
patients with evidence of bacteremia (e-Table 3) did not reveal substantially different

RI
results.

SC
DISCUSSION

U
We found that patients meeting the Sepsis-3 septic shock criteria were at
AN
significantly higher risk of in-hospital mortality, ICU admission and discharge to long

term care, as compared to the previous SIRS-based septic shock criteria. Patients meeting
M

the SIRS-based septic shock criteria when assessed by the RRT did not have significantly

increased odds of mortality, compared to those not meeting the criteria. When comparing
D

qSOFA and SIRS, qSOFA was found to have poor sensitivity, but high specificity,
TE

whereas SIRS had high sensitivity, but poor specificity for prediction of in-hospital

mortality in this population. Taken together, this work provides insight into the
EP

prognostic accuracy and thus clinical utility of the new Sepsis-3 criteria in assessment of
C

hospitalized patients with acute deterioration from suspected infection, with implications
AC

for clinicians who assess such patients.

When faced with a patient experiencing acute deterioration on the hospital wards,

clinicians must be able to accurately prognosticate risk of mortality, in order to

appropriately escalate therapy, or re-visit goals-of-care. While the traditional SIRS-based

10
 ACCEPTED MANUSCRIPT

criteria have been consistently used for these purposes10,22, this study provides insight

into how the Sepsis-3 criteria perform when used in this context.

Advances in treatment of septic shock have translated into improvements in

survival amongst these patients over the last several decades23,24. However, septic shock

PT
remains a major cause of death among patients with suspected infection, and is often

RI
encountered through RRT assessment9. While previous studies have analyzed the

accuracy of the Sepsis-3 septic shock criteria in the ED25,26, less is known about how it

SC
performs in deteriorating patients on the hospital wards. We found that the Sepsis-3

septic shock criteria were associated with higher risk of mortality than the SIRS-based

U
septic shock criteria. In this cohort, patients meeting the Sepsis-3 septic shock criteria had
AN
a mortality rate of 40.9%, similar to what was seen in the initial derivation cohort16. In

contrast, patients meeting the SIRS-based septic shock criteria had a mortality of 33.5%,
M

similar to that seen in our entire cohort of RRT-assessed patients with suspected infection
D

(33.4%). This suggests prognostic superiority for the Sepsis-3 septic shock criteria among
TE

patients assessed by the RRT, as the use of the SIRS-based septic shock criteria in this

population did not connote an increased risk of mortality above baseline.

EP

While previous work in the ED found that the Sepsis-3 septic shock criteria

missed a significant proportion of deaths that were identified with the SIRS-based septic
C

shock criteria25, that was not the case in this cohort. Regardless, the Sepsis-3 septic shock
AC

criteria were not meant to serve as a prompt for treatment27, but rather to identify a subset

of patients at increased risk of death, in order to properly escalate treatment (e.g. through

ICU admission) or alter goals-of-care. This data supports the usefulness of the Sepsis-3

septic shock criteria in identifying these high-risk deteriorating ward patients.

11
 ACCEPTED MANUSCRIPT

Interestingly, 15 patients in our cohort met the Sepsis-3 septic shock criteria while not

meeting the SIRS criteria at all, further emphasizing the concerns raised regarding the use

of the SIRS criteria in the sepsis population28,29.

Lastly, we compared the prognostic accuracy of qSOFA with SIRS for the

PT
prediction of mortality among RRT-assessed patients with suspected infection. While

RI
several large studies have investigated the prognostic accuracy for qSOFA within and

outside of the ICU30-33, less is known regarding its performance among deteriorating

SC
hospitalized patients receiving RRT assessment. The only other study to investigate

qSOFA in the RRT population utilized a cohort of 97 patients with suspected infection34.

U
They found that qSOFA was associated with moderate sensitivity and specificity, but
AN
with relatively large variability in these estimates, due to the small sample size. We found

that the sensitivity of SIRS was superior to qSOFA in this context, suggesting that it is
M

less likely to miss patients at risk of death. The specificity of qSOFA was substantially
D

better than SIRS, which suggests that the presence of 2 or more of these criteria is an
TE

ominous sign. However, in the assessment of patients with acute deterioration, clinicians

often prioritize sensitivity over specificity, in order to optimize disposition. Therefore, the
EP

use of SIRS might confer an advantage over qSOFA in this context.

This study has several strengths, including a large number of consecutive patients
C

from multiple centres, and data related to various patient and RRT variables. However,
AC

there are several limitations that hinder the generalizability of our results. First, we

utilized the concomitant administration of antibiotics and sampling of body fluids to

broadly represent ‘suspected infection’ in our cohort, in order to replicate the conditions

under which the Sepsis-3 criteria were derived. How well this definition of ‘suspected

12
 ACCEPTED MANUSCRIPT

infection’ truly reflects the population of patients who may ultimately develop sepsis and

septic shock is unclear. Previous work investigating various criteria for ‘suspected

infection’ have found differences in the prevalence of mortality between criteria, but no

differences in accuracy of the Sepsis-3 or SIRS criteria35. Sensitivity analysis of only

PT
patients with evidence of bacteremia did not substantially alter our conclusions.

RI
Furthermore, our population reflects hospitalized patients with acute deterioration,

requiring RRT assessment. Therefore, this work provides important insight into the

SC
management of this important patient population, who are at increased risk of morbidity

and mortality, as compared to other hospitalized patients7,8. However, our findings may

U
not be generalizable to all hospitalized patients with suspected infection. The overall
AN
mortality in our patient population was high, inferring a higher severity of illness among

these patients. Secondly, while our data was gathered from two different hospitals, they
M

exist within the same health network and city. Given the known regional variation in
D

hospital and ICU admission criteria, as well as RRT activation, this may limit the
TE

generalizability of our findings. Finally, we did not have available data regarding when

patients met SIRS-based and Sepsis-3 criteria prior to RRT activation. While the Sepsis-3
EP

criteria have important implications in evaluating prognosis in patients with suspected

infection, they are not actionable for the purposes of clinical decision-making. Future
C

research should focus on the evaluation of existing and novel clinical decision
AC

instruments for prediction of future deterioration in patients with suspected infection.

CONCLUSIONS

13
 ACCEPTED MANUSCRIPT

In deteriorating and infected hospital floor patients, we found that the Sepsis-3

septic shock criteria more accurately predicted in-hospital mortality than SIRS-based

septic shock criteria. While qSOFA has high specificity for mortality in this population,

the SIRS criteria were more sensitive, and therefore may be more appropriate for

PT
screening of these patients. Taken together, these findings have important implications

RI
for clinicians who assess hospitalized patients with acute deterioration from infection.

U SC
AN
M
D
TE
C EP
AC

14
 ACCEPTED MANUSCRIPT

ACKNOWLEDGEMENTS

SMF and KK designed the study. PMR, KM, PT, and KK gathered the data. SMF, BR,
NIS, DMY, AJES, JJP, DPB, and KK analyzed the data. All authors wrote the
manuscript. SMF is guarantor and agrees to be responsible for study contents.

PT
RI
U SC
AN
M
D
TE
C EP
AC

15
 ACCEPTED MANUSCRIPT

REFERENCES

1. Kempker JA, Martin GS. The Changing Epidemiology and Definitions of Sepsis.
Clin Chest Med. 2016;37(2):165-179.
2. Dugani S, Veillard J, Kissoon N. Reducing the global burden of sepsis. CMAJ.
2017;189(1):E2-e3.

PT
3. Lagu T, Rothberg MB, Shieh MS, Pekow PS, Steingrub JS, Lindenauer PK.
Hospitalizations, costs, and outcomes of severe sepsis in the United States 2003 to
2007. Crit Care Med. 2012;40(3):754-761.

RI
4. Esteban A, Frutos-Vivar F, Ferguson ND, et al. Sepsis incidence and outcome:
contrasting the intensive care unit with the hospital ward. Crit Care Med.
2007;35(5):1284-1289.

SC
5. Rohde JM, Odden AJ, Bonham C, et al. The epidemiology of acute organ system
dysfunction from severe sepsis outside of the intensive care unit. J Hosp Med.
2013;8(5):243-247.
6. Bhattacharjee P, Edelson DP, Churpek MM. Identifying Patients With Sepsis on

U
the Hospital Wards. Chest. 2017;151(4):898-907.
7. Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM. Rapid-response
AN
systems as a patient safety strategy: a systematic review. Ann Intern Med.
2013;158(5 Pt 2):417-425.
8. Maharaj R, Raffaele I, Wendon J. Rapid response systems: a systematic review
M

and meta-analysis. Crit Care. 2015;19:254.

9. Jaderling G, Bell M, Martling CR, Ekbom A, Bottai M, Konrad D. ICU
admittance by a rapid response team versus conventional admittance,
D

characteristics, and outcome. Crit Care Med. 2013;41(3):725-731.

10. Chan P, Peake S, Bellomo R, Jones D. Improving the Recognition of, and
Response to In-Hospital Sepsis. Curr Infect Dis Rep. 2016;18(7):20.
TE

11. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and
guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM
Consensus Conference Committee. American College of Chest
EP

Physicians/Society of Critical Care Medicine. Chest. 1992;101(6):1644-1655.

12. Sprung CL, Schein RM, Balk RA. To SIRS With Love-An Open Letter. Crit Care
Med. 2016.
13. Singer M, Deutschman CS, Seymour CW, et al. The Third International
C

Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA.

2016;315(8):801-810.
AC

14. Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure
Assessment) score to describe organ dysfunction/failure. On behalf of the
Working Group on Sepsis-Related Problems of the European Society of Intensive
Care Medicine. Intensive Care Med. 1996;22(7):707-710.
15. Seymour CW, Liu VX, Iwashyna TJ, et al. Assessment of Clinical Criteria for
Sepsis: For the Third International Consensus Definitions for Sepsis and Septic
Shock (Sepsis-3). JAMA. 2016;315(8):762-774.
16. Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a New Definition and
Assessing New Clinical Criteria for Septic Shock: For the Third International

16
 ACCEPTED MANUSCRIPT

Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA.

2016;315(8):775-787.
17. Ronksley PE, McKay JA, Kobewka DM, Mulpuru S, Forster AJ. Patterns of
health care use in a high-cost inpatient population in Ottawa, Ontario: a
retrospective observational study. CMAJ Open. 2015;3(1):E111-118.
18. McIsaac DI, Abdulla K, Yang H, et al. Association of delay of urgent or
emergency surgery with mortality and use of health care resources: a propensity

PT
score-matched observational cohort study. CMAJ. 2017;189(27):E905-e912.
19. Fernando SM, Reardon PM, Dowlatshahi D, et al. Outcomes and Costs of Patients
Admitted to the ICU Due to Spontaneous Intracranial Hemorrhage. Crit Care

RI
Med. 2018.
20. Baxter AD, Cardinal P, Hooper J, Patel R. Medical emergency teams at The
Ottawa Hospital: the first two years. Can J Anaesth. 2008;55(4):223-231.

SC
21. Jones AE, Trzeciak S, Kline JA. The Sequential Organ Failure Assessment score
for predicting outcome in patients with severe sepsis and evidence of
hypoperfusion at the time of emergency department presentation. Crit Care Med.
2009;37(5):1649-1654.

U
22. Cross G, Bilgrami I, Eastwood G, et al. The epidemiology of sepsis during rapid
response team reviews in a teaching hospital. Anaesth Intensive Care.
AN
2015;43(2):193-198.
23. Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality related to
severe sepsis and septic shock among critically ill patients in Australia and New
Zealand, 2000-2012. JAMA. 2014;311(13):1308-1316.
M

24. Stevenson EK, Rubenstein AR, Radin GT, Wiener RS, Walkey AJ. Two decades
of mortality trends among patients with severe sepsis: a comparative meta-
analysis*. Crit Care Med. 2014;42(3):625-631.
D

25. Sterling SA, Puskarich MA, Glass AF, Guirgis F, Jones AE. The Impact of the
Sepsis-3 Septic Shock Definition on Previously Defined Septic Shock Patients.
TE

Crit Care Med. 2017;45(9):1436-1442.

26. Henning DJ, Puskarich MA, Self WH, et al. An Emergency Department
Validation of the SEP-3 Sepsis and Septic Shock Definitions and Comparison
EP

With 1992 Consensus Definitions. Ann Emerg Med. 2017;70(4):544-552.

27. Deutschman CS. Sepsis-3: Seeing the Entire Picture. Crit Care Med.
2017;45(9):1567-1569.
28. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic
C

inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med.

2015;372(17):1629-1638.
AC

29. Vincent JL. Dear SIRS, I'm sorry to say that I don't like you. Crit Care Med.
1997;25(2):372-374.
30. Churpek MM, Snyder A, Han X, et al. Quick Sepsis-related Organ Failure
Assessment, Systemic Inflammatory Response Syndrome, and Early Warning
Scores for Detecting Clinical Deterioration in Infected Patients outside the
Intensive Care Unit. Am J Respir Crit Care Med. 2017;195(7):906-911.
31. Freund Y, Lemachatti N, Krastinova E, et al. Prognostic Accuracy of Sepsis-3
Criteria for In-Hospital Mortality Among Patients With Suspected Infection
Presenting to the Emergency Department. JAMA. 2017;317(3):301-308.

17
 ACCEPTED MANUSCRIPT

32. Raith EP, Udy AA, Bailey M, et al. Prognostic Accuracy of the SOFA Score,
SIRS Criteria, and qSOFA Score for In-Hospital Mortality Among Adults With
Suspected Infection Admitted to the Intensive Care Unit. JAMA.
2017;317(3):290-300.
33. Fernando SM, Tran A, Taljaard M, et al. Prognostic Accuracy of the Quick
Sequential Organ Failure Assessment for Mortality in Patients With Suspected
Infection: A Systematic Review and Meta-analysis. Ann Intern Med.

PT
2018;168(4):266-275.
34. LeGuen M, Ballueer Y, McKay R, Eastwood G, Bellomo R, Jones D. Frequency
and significance of qSOFA criteria during adult rapid response team reviews: A

RI
prospective cohort study. Resuscitation. 2018;122:13-18.
35. Churpek MM, Snyder A, Sokol S, Pettit NN, Edelson DP. Investigating the
Impact of Different Suspicion of Infection Criteria on the Accuracy of Quick

SC
Sepsis-Related Organ Failure Assessment, Systemic Inflammatory Response
Syndrome, and Early Warning Scores. Crit Care Med. 2017.

U
AN
M
D
TE
C EP
AC

18
 ACCEPTED MANUSCRIPT

Table 1 – Baseline Characteristics – Entire Cohort with Suspected Infection (n = 1,708)

Variable
Age, years, mean (SD) 68.0 (16.5)
Male, n (%) 963 (56.4)
Admission Source, n (%)
Home 1187 (69.5)
Acute Care Facility Transfer 129 (7.6)

PT
Long-term Care Facility Transfer 190 (11.1)
Other 202 (11.8)
Comorbidities, n (%)
Congestive Heart Failure 300 (17.6)

RI
Arrhythmia 403 (23.6)
Valvular Disease 59 (3.5)
Peripheral Vascular Disease 94 (5.5)

SC
Hypertension 519 (30.4)
Chronic Obstructive Pulmonary Disease 351 (20.5)
Diabetes Mellitus 868 (50.8)
Renal Failure 225 (13.2)

U
Liver Disease 103 (6.0)
Immunosuppression 61 (3.6)
Metastatic Cancer 159 (9.3)
AN
Elixhauser Comorbidity Score, mean (SD) 8.7 (6.1)
Emergency Department visits in past year, median (IQR) 1 (0-2)
Hospital admissions in past year, median (IQR) 0 (0-1)
M

ICU admissions in past year, median (IQR) 0 (0-0)

Goals-of-Care, n (%)
Full Care 1110 (65.0)
ICU-level Care, No CPR 145 (8.5)
D

Do Not Resuscitate 297 (17.4)

Other/Unknown 157 (9.2)
TE

Service, n (%)
Surgical Services 375 (22.0)
Non-Surgical Services 1333 (78.0)
Suspected Source of Infection, n (%)
EP

Pulmonary 811 (47.5)

Gastrointestinal 450 (26.3)
Urinary Tract 235 (13.8)
Skin/Soft Tissue Infection 68 (4.0)
C

Central Nervous System 42 (2.5)

Other/Unknown 102 (6.0)
AC

Disease Severity, median, IQR

Total SOFA Score 3 (1-5)
Respiratory SOFA Score 1 (0-3)
Coagulation SOFA Score 0 (0-1)
Liver SOFA Score 0 (0-0)
Cardiovascular SOFA Score 1 (1-3)
Central Nervous System SOFA Score 0 (0-1)
Renal SOFA Score 1 (0-2)
Table 1: Abbreviations: SD = Standard deviation; IQR = Interquartile range; ICU =
Intensive Care Unit; CPR = Cardiopulmonary resuscitation; SOFA = Sequential Organ
Failure Assessment

19
 ACCEPTED MANUSCRIPT

Table 2 – RRT Characteristics and Interventions – Entire Cohort with Suspected

Infection (n = 1,708)

Variable
Number of RRT Activations During Admission, median (IQR) 1 (1-1)
Multiple Race Calls, n (%) 294 (17.2)
Time of Initial RRT Activation

PT
Daytime Hours (0800-1659) 1148 (67.2)
Most Recent Vital Signs
Systolic Blood Pressure, mmHg, mean (SD) 138.3 (29.1)
Diastolic Blood Pressure, mmHg, mean (SD) 73.9 (13.2)

RI
Mean Arterial Pressure, mmHg, mean (SD) 94.8 (15.9)
Heart Rate, Beats/Min., mean (SD) 100.3 (28.6)
Temperature, Degrees Celsius, mean (SD) 36.9 (0.8)

SC
Oxygen Saturation, %, median (IQR) 92.6 (7.5)
Most Recent Blood Work
White Blood Cell Count, x109/L, median (IQR) 11.0 (7.1-16.1)
Hemoglobin, g/L, mean (SD) 107 (21.7)
Platelets, x109/L, mean (SD) 220.9 (134.1)

U
Potassium, mmol/L, mean (SD) 4.1 (0.7)
Bilirubin, µmol/L, median (IQR) 9 (6-14)
AN
Creatinine, µmol/L, median (IQR) 93 (62-161)
Urea, mmol/L, median (IQR) 2.2 (1.6-3.0)
Lactate, mmol/L, median (IQR) 2.2 (1.7-3.2)
Albumin, g/L, mean (SD) 25.7 (6.6)
M

INR, median (IQR) 1.2 (1.1-1.4)

Primary Reason for RRT Call, n (%)
Respiratory Distress 586 (34.3)
Tachycardia/Bradycardia/Arrhythmia 244 (14.3)
D

Altered Level of Consciousness 202 (11.8)

Hypotension 315 (18.4)
TE

Hypertension 21 (1.2)
Airway Concern 37 (2.2)
Seizure 4 (0.2)
Worried About Patient 191 (11.2)
EP

Other/Unknown 109 (6.4)

Latency to first RRT Activation from Onset of Concerning Symptoms/Signs
< 1 hour, n (%) 1422 (83.3)
RRT Interventions
C

Initiation of Vasopressors, n (%) 525 (30.7)

Mechanical Ventilation, n (%) 378 (22.1)
AC

Total Fluids, L, median (IQR) 2.0 (1.3-3.8)

ICU Admission, n (%) 600 (35.1)

Table 2: Abbreviations: RRT = Rapid Response Team; SD = Standard deviation; IQR =

Interquartile range; ICU = Intensive Care Unit

20
 ACCEPTED MANUSCRIPT

Table 3 – Prevalence of Sepsis (i.e. Change in SOFA ≥ 2) by Sepsis-3 Criteria Among

RRT Patients with Suspected Infection (n = 1,708)
Variable Sepsis (by Infection Odds Ratio P-value
Sepsis-3 without (95% CI)
criteria) Sepsis (by
(n = 1,045) Sepsis-3

PT
criteria)
(n = 663)
In-Hospital Mortality, n 403 (38.6) 167 (25.2) 1.86 <0.0001

RI
(%) (1.51-2.31)
ICU Admission, n (%)a 441 (57.6) 159 (32.5) 2.82 <0.0001
(2.22-3.57)

SC
Survivors Discharged to 252 (57.1) 161 (34.8) 2.32 <0.0001
Long-term Care (1.77-3.04)
Facilities, n (%)b
Hospital Length of Stay, 17 (8-31) 14 (6-28) <0.01

U
days, median (IQR)
ICU Length of Stay, 5 (2-9) 3 (1-5) <0.01
AN
days, median (IQR)
Table 3: aOnly includes patients with goals-of-care allowing for ICU admission (n =
1,255; Change in SOFA ≥ 2 = 766, Change in SOFA < 2 = 489)
M

b
Only includes survivors originally from home setting (n = 903; Change in SOFA ≥ 2 =
441, Change in SOFA < 2 = 462)
Abbreviations: SOFA = Sequential Organ Failure Assessment; CI = Confidence
D

Interval; IQR = Interquartile range; ICU = Intensive Care Unit

TE
C EP
AC

21
 ACCEPTED MANUSCRIPT

Table 4 – Prognostic Accuracy of Septic Shock Criteria Among Patients with Suspected
Infection
Variable “Sepsis-3” SIRS-based P-Value
Septic Shock Septic Shock
Criteria Criteria
(n = 418) (n = 545)
In-Hospital Mortality, n (%) 171 (40.9) 183 (33.5) 0.02

PT
Odds Ratio For In-Hospital 1.55 1.01
Mortality (95% CI) (1.23-1.94) (0.82-1.22)
ICU Admission, n (%)a 416 (99.5) 456 (89.2) <0.001

RI
Survivors Discharged to 106 (66.3) 167 (53.7) 0.01
Long-term Care Facilities, n
(%)b

SC
Hospital Length of Stay, 19 (9-31) 18 (7-29) 0.12
days, median (IQR)
ICU Length of Stay, days, 5 (3-8) 4 (1-7) <0.01
median (IQR)

U
Table 4: SIRS-based septic shock criteria = Sepsis (SIRS > 2) + Hypotension (SBP < 90)
Sepsis-3 septic shock critera = Sepsis (SOFA > 2) + Initiation of Vasopressors for MAP
AN
> 65 + Lactate >2
a
Only includes patients meeting septic shock criteria, with goals of care allowing for ICU
admission (Sepsis-3 septic shock criteria, n = 418; SIRS-based septic shock criteria, n =
M

511)
b
Only includes survivors meeting septic shock criteria and originally from home setting
(Sepsis-3 septic shock criteria, n = 160; SIRS-based septic shock criteria, n = 311)
D

Abbreviations: SIRS = Systemic Inflammatory Response Syndrome; CI = Confidence

Interval; IQR = Interquartile range; ICU = Intensive Care Unit
TE
C EP
AC

22
 ACCEPTED MANUSCRIPT

Table 5 – Prognostic Accuracy of qSOFA vs. SIRS for Prediction of Mortality Among
Patients with Suspected Infection requiring RRT Activation (n = 1,708)
Variable qSOFA Criteria SIRS Criteria

Positive Criteria 459 1391

Negative Criteria 1249 317
True Positive 370 522

PT
False Positive 89 869
True Negative 1049 269
False Negative 200 48

RI
Sensitivity, % (95% CI) 64.9 91.6
(60.8-68.8) (89.0-93.7)
Specificity, % (95% CI) 92.2 23.6

SC
(90.5-93.7) (21.2-26.2)
Positive Predictive Value, 80.6 37.5
% (95% CI) (77.1-83.7) (36.6-38.5)

U
Negative Predictive Value, 84.0 84.9
% (95% CI) (82.4-85.5) (80.7-88.2)
AN
Positive Likelihood Ratio 8.30 1.20
(95% CI) (6.74-10.22) (1.15-1.25)
Negative Likelihood Ratio 0.38 0.36
(95% CI) (0.34-0.43) (0.27-0.48)
M

AUROC (95% CI) 0.790 0.564

(0.765-0.815) (0.537-0.592)
Table 5: Abbreviations: qSOFA = quick Sequential (Sepsis-related) Organ Failure
D

Assessment; SIRS = Systemic Inflammatory Response Syndrome; CI = Confidence

Interval; IQR = Interquartile range; AUROC = Area under Receiver Operating
TE

Characteristics curve
C EP
AC

23
 ACCEPTED MANUSCRIPT

e-Table 1: Rapid Response Team criteria at The Ottawa Hospital.

Airway Threatened stridor; excessive secretions

Breathing Respiratory rate ≤ 8 breaths/minute or ≥ 30
breaths/minute
Circulation Systolic blood pressure ≤ 90 mmHg or ≥ 200

Level of Consciousness
PT
mmHg or ≥ 40 mmHg decrease
Heart rate ≤ 40 beats/minute or ≥ 130
beats/minute
>2 point decrease in Glasgow Coma Scale

RI
Oxygen Saturation <90% on 50% FiO2 or 6 litres/minute
Urine Output <100 mL over four hours
Other Health care worker “worried” about the patient,

SC
needs medical assistance, failure to respond to
treatment

U
e-Figure 1.
AN
M
D
TE
C EP
AC

Online supplements are not copyedited prior to posting and the author(s) take full responsibility for the accuracy of all data.
 ACCEPTED MANUSCRIPT

e-Table 2 – Characteristics and Outcomes of RRT Patients with Suspected Infection and No
Limitations in Critical Care Interventions (n = 1,255)

Variable
Age, years, mean (SD) 67.4 (16.4)

PT
Male, n (%) 682 (54.3)
Admission Source, n (%)
Home 824 (65.7)
Acute Care Facility Transfer 80 (6.4)

RI
Long-term Care Facility Transfer 248 (19.8)
Other 103 (8.2)
Number of RRT Activations During Admission, median 1 (1-1)

SC
(IQR)
Time of Initial RRT Activation
Daytime Hours (0800-1659) 921 (73.4)
Primary Reason for RRT Call, n (%)

U
Respiratory Distress 485 (38.6)
Tachycardia/Bradycardia/Arrhythmia 156 (12.4)
Altered Level of Consciousness
Hypotension
Hypertension
AN 143 (11.4)
256 (20.4)
13 (1.0)
Airway Concern 28 (2.2)
M
Seizure 4 (0.3)
Worried About Patient 114 (9.1)
Other/Unknown 56 (4.4)
In-Hospital Mortality, n (%) 394 (31.4)
D

ICU Admission, n (%) 600 (47.8)

Sepsis-3 Septic Shock Criteria
TE

Patients Meeting Sepsis-3 Septic Shock Criteria, n (%) 418 (33.3)

Mortality Among Patients Meeting Sepsis-3 Septic Shock 170 (40.9)
Criteria, n (%)
Odds Ratio for In-Hospital Mortality (95% CI) 2.04 (1.60-2.61)
EP

SIRS-based Septic Shock Criteria

Patients Meeting SIRS-based Septic Shock Criteria, n (%) 444 (35.3)
Mortality Among Patients Meeting SIRS-based Septic Shock 127 (28.6)
Criteria, n (%)
C

Odds Ratio for In-Hospital Mortality (95% CI) 1.23 (0.95-1.28)

qSOFA
AC

Sensitivity, % (95% CI) 67.8 (62.9-72.4)

Specificity, % (95% CI) 92.7 (90.7-94.3)
SIRS
Sensitivity, % (95% CI) 93.9 (91.0-96.0)
Specificity, % (95% CI) 26.7 (23.8-29.8)

Online supplements are not copyedited prior to posting and the author(s) take full responsibility for the accuracy of all data.
 ACCEPTED MANUSCRIPT

e-Table 3 – Characteristics and Outcomes of RRT Patients with Suspected Infection and
Positive Blood Cultures (n = 245)

Variable
Age, years, mean (SD) 68.8 (16.6)

PT
Male, n (%) 136 (55.5)
Admission Source, n (%)
Home 165 (67.3)
Acute Care Facility Transfer 17 (6.9)

RI
Long-term Care Facility Transfer 28 (11.4)
Other 35 (14.3)
Number of RRT Activations During Admission, median 1 (1-1)

SC
(IQR)
Time of Initial RRT Activation
Daytime Hours (0800-1659) 168 (68.6)
Primary Reason for RRT Call, n (%)

U
Respiratory Distress 90 (36.7)
Tachycardia/Bradycardia/Arrhythmia 37 (15.1)
Altered Level of Consciousness
Hypotension
Hypertension
AN 27 (11.0)
65 (26.5)
3 (1.2)
Airway Concern 2 (0.8)
M
Seizure 0 (0)
Worried About Patient 15 (6.1)
Other/Unknown 6 (2.4)
In-Hospital Mortality, n (%) 87 (35.5)
D

ICU Admission, n (%) 124 (50.6)

Sepsis-3 Septic Shock Criteria
TE

Patients Meeting Sepsis-3 Septic Shock Criteria, n (%) 77 (31.4)

Mortality Among Patients Meeting Sepsis-3 Septic Shock 36 (46.8)
Criteria, n (%)
Odds Ratio for In-Hospital Mortality (95% CI) 2.01 (1.14-3.55)
EP

SIRS-based Septic Shock Criteria

Patients Meeting SIRS-based Septic Shock Criteria, n (%) 88 (35.9)
Mortality Among Patients Meeting SIRS-based Septic Shock 31 (35.2)
Criteria, n (%)
C

Odds Ratio for In-Hospital Mortality (95% CI) 1.02 (0.60-1.79)

qSOFA
AC

Sensitivity, % (95% CI) 73.6 (63.0-82.5)

Specificity, % (95% CI) 94.9 (90.3-97.8)
SIRS
Sensitivity, % (95% CI) 93.1 (85.6-97.4)
Specificity, % (95% CI) 26.0 (19.3-33.5)
“Positive Blood Cultures” are defined as any blood cultures with an isolated organism that
was not deemed to be a contaminant. A “contaminant” was defined by isolation of an
organism commonly considered to be a contaminant, grown only in 1 culture, and without
evidence of other infection from that organism, OR deemed by the attending medical
microbiologist to be consistent with contaminant.

172960
Online supplements are not copyedited prior to posting and the author(s) take full responsibility for the accuracy of all data.