The Journal of Emergency Medicine, Vol. 45, No. 4, pp.

508–519, 2013
Copyright Ó 2013 Elsevier Inc.
Printed in the USA. All rights reserved
0736-4679/$ - see front matter

http://dx.doi.org/10.1016/j.jemermed.2013.03.042

Original
Contributions

DOES THIS CHILD HAVE BACTERIAL MENINGITIS? A SYSTEMATIC REVIEW OF

CLINICAL PREDICTION RULES FOR CHILDREN WITH SUSPECTED BACTERIAL
MENINGITIS

Dina M. Kulik, MD, FRCPC,* Elizabeth M. Uleryk, BA, MLS,† and Jonathon L. Maguire, MSC FRCPC‡§jj{#**
*Division of Paediatric Emergency Medicine, Hospital for Sick Children, Toronto, Ontario, Canada, †Hospital for Sick Children, Hospital Library,
Toronto, Ontario, Canada, ‡Department of Pediatrics, St. Michael’s Hospital, Toronto, Ontario, Canada, §Keenan Research Centre, Li Ka
Shing Knowledge Institute of St. Michael’s Hospital, Toronto, Ontario, Canada, jjDivision of Pediatric Medicine and the Pediatric Outcomes
Research Team (PORT), Hospital for Sick Children, Toronto, Ontario, Canada, {Child Health Evaluative Sciences, Hospital for Sick Children
Research Institute, Toronto, Ontario, Canada, #Department of Health Policy Management and Evaluation, University of Toronto, Toronto,
Ontario, Canada, and **Department of Paediatrics, University of Toronto, Toronto, Ontario, Canada
Reprint Address: Dina M. Kulik, MD, FRCPC, Pediatric Emergency Medicine, The Hospital for Sick Children, 555 University Avenue, Toronto,
Ontario, Canada M5G 1X8

, Abstract—Background: Acute meningitis is a relatively adequate blinding, predictor reproducibility assessment,

common phenomenon in children. Identifying which chil-
dren are most likely to have bacterial meningitis vs. self-
limiting aseptic meningitis is important, as these children
require investigation and antibiotic treatment. Objective:
Our aim was to systematically identify and review the qual-
ity and performance of published clinical prediction rules
(CPRs) for children with suspected bacterial meningitis.
Methods: Medline and Embase were searched for CPRs in-
volving children 0–18 years of age with suspected bacterial
meningitis, with cerebral spinal fluid (CSF) culture used as
the reference diagnostic standard. CPR quality was assessed
using 17 previously published items. CPR performance was
evaluated using sensitivity, negative likelihood ratio, and the
treatment frequency that would result if the rule was used.
Results: Eleven studies involving 6675 children with acute
meningitis fulfilled all inclusion criteria and were entered
in the study. They all describe the derivation or validation of
six unique CPRs. A rigorously developed, high-performing,
and well-validated CPR ready for clinical use to guide which
children with suspected bacterial meningitis should be hos-
pitalized and treated with intravenous antibiotics and which
can be safely discharged home was not identified. Areas for
quality improvement for future CPR studies include pro-
spective validation using standardized inclusion criteria,
and meticulous follow-up of outcomes. The Bacterial Menin-
gitis Score had the highest quality and performance and is
the best candidate for prospective validation. Conclusions:
Until consistently high methodological quality and diagnos-
tic performance are demonstrated through prospective
validation, caution is warranted in the routine clinical use
of existing CPRs for children with suspected bacterial
meningitis. Ó 2013 Elsevier Inc.
, Keywords—meningitis; clinical prediction rule; decision
trees; predictive value of tests; multivariate analysis
INTRODUCTION
Background
Acute meningitis is a relatively common and potentially
severe infection in childhood, with approximately 3000
new cases each year in children younger than 18 years
of age in the United States (1–3). Fortunately, most
infections are aseptic and resolve spontaneously, but
6–18% are bacterial in origin and require i.v. antibiotic
treatment, hospitalization, and close monitoring (4,5).

RECEIVED: 30 March 2012; FINAL SUBMISSION RECEIVED: 3 October 2012;

ACCEPTED: 15 March 2013

508
Does This Child Have Bacterial Meningitis?
Accurately diagnosing bacterial meningitis in children
is imperative to avoid severe outcomes, such as neurolog-
ical impairment or death, and minimize unnecessary
antibiotic use and hospitalization in children with self-
limiting illness (6–13). Although diagnostic techniques,
such as cerebral spinal fluid (CSF) culture and poly-
merase chain reaction (PCR) are reliable, physicians
must decide which patients to treat with i.v. antibiotics
empirically before culture or PCR results are reported
48 h later (14,15). Weighing the risk of missing a
true case of bacterial meningitis with unnecessarily
admitting patients without bacterial meningitis for 48 h
of i.v. antibiotics remains a major challenge (16).
Clinicians often review results of CSF glucose, pro-
tein, white blood cell (WBC) count, Gram stain, and
serum WBC count to guide diagnostic decisions (17).
Gram stain is highly specific but it is falsely negative in
up to 40% of cases and therefore cannot be exclusively re-
lied upon (2,18–20). Elevated CSF WBC count is more
common in aseptic rather than bacterial meningitis
(21–27). Widespread use of vaccines against Strepto-
coccus pneumonia and Hemophilus influenza type B
have decreased the probability of bacterial meningitis in
children, further decreasing the likelihood that elevated
CSF WBC count is the result of bacterial meningitis
(28–31).
Clinical prediction rules (CPRs) are potentially power-
ful evidence-based tools for reducing uncertainty and
improving accuracy in medical decision making by stan-
dardizing the collection and interpretation of clinical data
(32). They can also minimize the use of potentially harm-
ful diagnostic tests, such as lumbar puncture, and reduce
admissions and adverse events from antibiotic use. They
have been defined as clinical decision-making tools that
quantify the relative importance of 3 or more variables
from history, physical examination, or simple tests to pro-
vide the probability of an outcome or suggest a single
diagnostic or therapeutic course of action for an individ-
ual patient (32–34).
This study aimed to systematically identify CPRs for
children with suspected bacterial meningitis and compare
their methodological quality and performance for diag-
nosing bacterial meningitis using a recently developed
framework to evaluate CPRs for children (35).
METHODS
Search Strategy
Potentially relevant studies were identified through elec-
tronic searches of Medline and Embase from January
1950 up to September 2012 (Appendix 1). Because there
is no medical subject heading (MeSH) that specifies clinical
prediction rules, a previously developed electronic search
509
strategy for CPRs was used with the addition of the
MeSH term bacterial meningitis (35,36). The reference
lists of identified clinical prediction rule publications
were searched manually to identify additional studies.
Our search was limited to English publications.
Inclusion Criteria
Only prospective or retrospective studies that derived,
validated, or assessed the impact of CPRs were included.
A CPR was defined as a clinical decision-making tool that
1) includes three or more predictive variables obtained
from the history, physical examination, or simple diag-
nostic tests; 2) provides the probability of an outcome
or suggests a diagnostic or therapeutic course of action
for an individual patient; and 3) is not a decision analysis
or practice guideline (32,33,35–37)
Only studies involving children (term birth–18 years)
with suspected or proven bacterial meningitis were in-
cluded. Studies that did not include positive CSF culture
(defined as a positive bacterial pathogen isolated in the
CSF) as the outcome were not included. Studies involv-
ing both adults and children were included if a separate
analysis was performed for children. Studies requiring
the use of artificial neural networks or that assessed pre-
dictors with no obvious goal of creating a prediction rule
were not included. Studies that did not include positive
CSF culture (defined as a positive bacterial pathogen
isolated in the CSF) as the outcome were not included.
We did not exclude studies that included patients who
received antibiotics; however, no such studies met inclu-
sion criteria.
Selection of Studies
Two reviewers (D. M. Kulik and J. L. Maguire) inde-
pendently assessed the inclusion of potentially relevant
articles through a two-step process. First, the titles and
abstracts from each article identified through the elec-
tronic search were assessed for inclusion. Second, pub-
lications identified as relevant by title or abstract were
reviewed manually. Discrepancies between the two
reviewers were discussed and included by consensus.
Discrepancies occurred <5% of the time and consensus
was reached for every discrepancy.
Assessment of Methodological Quality
The quality of the included studies was assessed using
a 17-item checklist from published guidelines for use in
the derivation or validation of clinical prediction rules for
children (see Appendix 2) (32–36,38,39). Each item was
noted to be present (1) or absent (0). The maximum number
of quality items was 17. Two reviewers (D. M. Kulik and
510 D. M. Kulik et al.

J. L. Maguire) independently abstracted data from in- RESULTS

cluded papers using a standardized data collection form.
Study Selection
Discrepancies between the reviewers were discussed and
resolved by consensus. Given the importance of rule vali-
A total of 6387 titles and abstracts were identified and
dation, hierarchy of rule validation was assessed separately
screened by the search strategy as potentially relevant
(see Assessment of Hierarchy of Rule Validation section).
(Figure 1). Of these, 379 were extracted as full text
articles and assessed for inclusion; 11 studies fulfilled
Assessment of Rule Performance
all inclusion criteria. These studies were derivation or
validation studies of six unique CPRs.
Data were extracted from each publication to construct
a 2  2 table to calculate sensitivity (Sn), specificity, neg-
Description of Studies
ative likelihood ratio (LR) and their 95% confidence in-
tervals (CIs) for the diagnosis of bacterial meningitis
Included studies were published between 2001 and 2010
(33,40). Calculations were performed using SAS 9.1
and involved a total of 6675 children (see Table 1 for
(SAS Institute Inc., Cary, NC). The predicted frequency
population characteristics for each study). The median
of empiric i.v. antibiotic treatment and hospitalization
number of children enrolled in each study was 286 (range
that would result if the rule were applied to every
74–2903). Five studies were published in general pediat-
patient (rule predicted treatment frequency) was also
ric journals, two in infectious disease journals, two in
determined. Rule predicted treatment frequency
general medicine journals, and one in each of emergency
captures both sensitivity and specificity and provides
medicine and epidemiology journals. Three studies were
a sensible quantification of the degree of overtreatment
multicentered and eight were single centered. Five
that the CPR would produce when compared with the
cohorts were from the Unite States, two were from
actual frequency of bacterial meningitis in each study.
France, three were from the Netherlands, and one was
Given that physicians are not likely to feel comfortable
multicentered across Europe (France, Poland, Turkey,
using a rule that misses more than a very small number
Spain, and Switzerland). Eight of 11 studies were per-
of children with bacterial meningitis and that such
formed in the emergency department (ED). Three studies
a rule should not substantially inflate the risk of i.v.
were completed on the pediatric ward. There was consid-
antibiotics and hospital admission, the following 4 rule-
erable variation in the population age (28 days–18 years).
performance benchmarks were used to identify high-
Three studies included patients presenting with the clini-
performing CPRs: Sn > 0.95; lower limit of sensitivity
cal suspicion for meningitis in the ED, and eight studies
95% CI > 0.95; LR < 0.1; and upper limit of the LR
included only children who had a final diagnosis of viral
95% CI < 0.1 (32,33,35–37,40,41). Sensitivity and
or bacterial meningitis. Two studies used post-discharge
LR are also independent of disease prevalence, which
follow-up methods to ensure no missed cases of bacterial
makes them useful measures for comparing CPRs from
meningitis (43,44). The frequency of acute bacterial
different populations with different frequencies of
meningitis in each study population varied between
bacterial meningitis (40,42).
0.04 and 0.48 (median 0.19).
Six studies described derivations of a rule and 5 were
Assessment of Hierarchy of Rule Validation
rule validation studies. None assessed the rule’s clinical
The degree of validation for CPRs that met inclusion cri-
teria was assessed according to the hierarchy of evidence
for CPRs published by the Evidence-Based Medicine
Working Group (32). In this hierarchy, prediction rules
that have been derived but not prospectively validated
are the lowest level of evidence (level 4), rules that
have been prospectively validated in only one sample
are level 3, rules that have been broadly validated in mul-
tiple settings are level 2, and rules that have had impact
analysis performed that demonstrated a change in clini-
cian behavior with beneficial consequences are level 1.
Although performing a meta-analysis would have
been ideal, due to heterogeneity of the populations and
predictors used in many of the rules, we believed that it Figure 1. Selection process for meningitis clinical prediction
would not be appropriate to do. rules.
 Does This Child Have Bacterial Meningitis?
Table 1. Study Characteristics

No. of
Study Setting Rule Name n Study Type Age Population Outcomes Assessment Country Sites

Bonsu, 2008 (47) ED Modified Bonsu 1107 Derivation 4 wk18 y Children diagnosed with acute Bacteria grown from CSF US 1
viral or bacterial meningitis or
children who underwent LP*
Bonsu 2004 (50) ED Bonsu 142 Derivation 5 wk18y Children diagnosed with acute Bacteria grown from CSF US 1
viral or bacterial meningitis or
children who underwent LP*
Chavanet 2007 (51) Ward Meningitest 175 Derivation 3 mos15 y Children hospitalized with LP* and Bacteria grown from CSF France 1
7 WBC/mL in CSF
Dubos 2006* (45) ED Bonsu 161 Validation 28 d16 y Children with acute meningitis Bacteria grown from CSF France 1
(CSF WBC $7/mm3)
Dubos 2006* (45) ED BMS 151 Validation 28 d16 y Children with acute meningitis Bacteria grown from CSF France 1
(CSF WBC $7/mm3)
Dubos 2006* (45) ED Oostenbrink 119 Validation 28 d16 y Children with acute meningitis Bacteria grown from CSF France 1
(CSF WBC $7/mm3)
Dubos 2010* (46) ED Meningitest 198 Validation 29 d18 y CSF WBC $7  106/L and CSF Bacteria grown from CSF 5 European 6
culture/PCR or blood culture countries
positive and procalcitonin level
done
Dubos 2010* (46) ED BMS 198 Validation 29 d18 y CSF WBC $7  106/L and CSF Bacteria grown from CSF 5 European 6
culture/PCR or blood culture countries
positive and procalcitonin level
done
Fine 2007 (27) Ward Modified BMS 696 Derivation 29 d19 y Children with a final diagnosis of Bacteria grown from CSF US 1
meningitis who had LP or CSF meningitis CSF
performed WBC $7/mm with
positive blood culture
Nigrovic 2002 (2) Ward BMS 696 Derivation 29 d19 y Children with a final diagnosis of Bacteria grown from CSF US 1
meningitis who had LP
performed
Nigrovic 2007 (48) ED BMS 2903 Validation 29 d19 y Children with a final diagnosis of Bacteria grown from CSF US 20
meningitis who had LP
performed
Oostenbrink 2001 (43) ED Oostenbrink 286 Derivation 1 mos15 y Children visiting the ED with Bacteria grown from CSF Netherlands 1
meningeal signs
Oostenbrink 2002 (44) ED Oostenbrink 74 Validation 2 mos15 y Children visiting the ED with Bacteria grown from CSF Netherlands 1
meningeal signs
Oostenbrink 2004 (49) ED Oostenbrink 226 Validation 2 mos15 y Children visiting the ED with Bacteria grown from CSF Netherlands 4
meningeal signs

BMS = Bacterial Meningitis Score; CSF = cerebrospinal fluid; ED = emergency department; LP = lumbar puncture; PCR = polymerase chain reaction; WBC = white blood cell count; US =
United States.
* Assessed the performance of more than one rule.

511
512 D. M. Kulik et al.

impact. Five were derivation or validation studies of the
rule created or modified by Bonsu et al. (referred to as
‘‘Bonsu’’ or ‘‘modified Bonsu score,’’ respectively); five
derived or validated the Bacterial Meningitis Score
(BMS) or modified BMS; four derived or validated the
score developed by Oostenbrink et al. (referred to as
‘‘Oostenbrink score’’); and 2 derived or validated the
Meningitest rule. Two studies validated multiple rules
simultaneously (45,46).
Predictors included in the rules varied considerably,
although there were some predictors that were common
to most rules (Table 2). Elevated CSF protein was com-
mon to five rules (BMS, modified BMS, Bonsu, modified
Bonsu, and Meningitest), elevated CSF neutrophil count
was common to four rules (BMS, modified BMS, Bonsu,
and modified Bonsu), history of seizure and positive CSF
Gram stain were common to three rules (BMS, modified
BMS, and Meningitest).
Assessment of Methodological Quality
Studies met between 5 and 12 of the 17 quality items (see
Table 3). All 11 studies resulted in clinically sensible
CPRs that described a course of action to be taken and
provided an adequate description of the mathematical
technique used. The outcome was well described in 10
studies and the population was well described in 9 stud-
ies. The most poorly addressed CPR quality items were
predictor assessor blinding (0 of 11), reproducibility of
predictor variable assessment (0 of 11), prospective
design (1 of 11), adequate follow-up (2 of 11), adequate
description of the study site (3 of 11), and rule application
to all patients at risk (3 of 11). The highest-quality studies
were the Modified Bonsu derivation study (11 of 17) and
the BMS validation study by Nigrovic et al. (12 of 17)
(47,48).
Assessment of Rule Performance
Rule performance varied considerably (Table 4). The
median rule sensitivity was 1 with a range of 0.83 to 1
(median lower limit of the 95% CI for sensitivity was
0.91, range 0.51–0.95) (Table 4). Median LR was 0,
with a range of 0 to 0.23 (median upper limit of the 95%
CI was 0.16, range 0.09–0.83). Median rule-predicted
treatment frequency was 0.48 (range 0.24–0.81).
All of the studies except for the Oostenbrink validation
studies demonstrated sensitivity >0.95 and LR < 0.1
(2,27,43,45,47,49–51). The Oostenbrink derivation
study and BMS and Meningitest validation studies had
the lower limit of the 95% CI for sensitivity > 0.95
(43,46). Only the modified BMS derivation study by
Fine had the upper limit of the 95% CI for LR < 0.1
(27). No derivation or validation studies had 95% confi-
dence in both Sn > 0.95 and LR < 0.1. No studies eval-
uated the performance of a score compared with clinical
judgment alone. The lowest rule-predicted treatment fre-
quencies were 0.33 and 0.41 for the modified BMS study
and BMS study, respectively (27,48).
Assessment of Hierarchy of Rule Validation
Of the six unique CPRs, five were level 4 (two were
derived with no validation and three were derived with

Table 2. Predictors Used in Each Study

Predictor BMS Modified BMS Oostenbrink Bonsu Modified Bonsu Meningitest

Seizure O O O
CSF-positive Gram stain O O O
Elevated CSF protein* O O O O O
CSF neutrophil count $1000 106/L O O O O
Blood neutrophil count $109/L O O
Purpura O
Toxic appearance O
Procalcitonin level $0.5 ng/mL O
Duration of symptoms O
Vomiting O
Cyanosis O
Altered LOC O
Meningeal irritation O
Petechia O
Elevated CRP O
Young age O O
Season
Temperature >38.4 C
WBC >6100/uL
Local disease incidence O

CRP = C-reactive protein; LOC = level of consciousness; WBC = white blood cell count.
* Cerebrospinal fluid (CSF) protein $80 mg/dL (Bacterial Meningitis Score [BMS]), $50 mg/dL (Meningitest).
Does This Child Have Bacterial Meningitis? 513

Power of Results Reported Sensible Use Reported Total

retrospective validation), none was level 3 (prospective

11

10

12
9

8
validation in one setting), and one was level 2 (prospec-
Follow Patients Adequate Reporting Technique Clinically to CI on Sn

tive validation in multiple settings) (2,27,43–47,49–51).

Yes

Yes

Yes

Yes

Yes
yes
No

No

No

No

No

6
No CPR was level 1 (impact analysis performed
demonstrating a change in clinician behavior with
Easy

Yes

Yes

Yes

Yes

Yes
No

No

No

No

No

No

5
beneficial consequences). Of the level 4 CPRs, the
highest quality study was the BMS retrospective
Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

11
validation study, which met 12 of 17 quality items and
also had strong performance (Sn 0.98, 95% CI 0.94–1;
Adequate matical
Mathe-

LR 0.03, 95% CI 0–0.11, rule-predicted treatment fre-

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

11
quency 0.41) (48). The single-level 2 rule met only 8 of
17 quality items and had fair performance (Sn 0.92,
Yes

Yes

Yes

Yes

Yes

95% CI 0.72–1, LR 0.13, 95% CI 0–0.5, rule-

No

No

No

No

No

No

5
predicted treatment frequency 0.45).
Yes

Yes

Yes

Yes

Yes
No

No

No

No

No

No

5 DISCUSSION
Applied

at Risk

This review has identified 11 studies involving the deriva-

Adequate to All

Yes

Yes

Yes
No

No

No

No

No

No

No

No

tion or validation of six different bacterial meningitis

clinical prediction rules for children. Using a recently de-
Yes

Yes
Up

No

No

No

No

No

No

No

No

No

veloped framework for evaluating the quality and perfor-

2

mance of clinical prediction rules for children, we

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured

Not measured
ment of Reproducible

identified the presence or absence of 17 items generally

Predictors

considered to be of importance for high-quality clinical

prediction rules. We also evaluated the diagnostic perfor-
0

mance of these rules based on their sensitivity, LR, and

pective Described Described Defined Predictors Action Defined Outcome
Population Predictors Assess- Course Outcome Assess-

the treatment frequency that would result if the rule were

Blind

Yes

Yes

Yes

Yes

Yes
No

No

No

No

No

No

to be used (2,27,43–46,48–51).
A number of issues made it challenging to compare
Well

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

existing CPRs for children with acute meningitis. First,

No

10

three of the studies included only children who had a dis-

charge diagnosis of meningitis. This situation results in
BMS = Bacterial Meningitis Score; CI = confidence interval; Sn = sensitivity.
Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes
of

11

a circular argument where the derived CPR functions

on a population that will never exist to the clinician trying
ment of
Blind

No

No

No

No

No

No

No

No

No

No

No

to decide ‘‘does this child with acute meningeal signs

0

have bacterial meningitis?’’ We recommend that future

studies include both children who are admitted to hospital
Well

Yes

Yes

Yes

Yes

Yes

Yes
No

No

No

No

No

and those discharged to the community, with follow-up of

6

outcomes several weeks later.

Table 3. Assessment of Methodological Quality

Second, there appears to be considerable variation in the

Well

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes
No

No

frequency of meningitis between the studies (0.04–0.48,

9

median 0.185). It is unclear what accounts for this variabil-

ity, although clearly including only admitted patients, and
Well
Site

Yes

Yes

Yes
No

No

No

No

No

No

No

No

in some cases only including patients with a discharge

diagnosis of meningitis, accounts for the high prevalence
Pros-

Yes

in some of the studies. Children’s age, season of the year,

No

No

No

No

No

No

No

No

No

No

and geographical region might be factors, but comparison

Oostenbrink

Modified BMS

of existing studies shows no clear pattern. This variability

Bonsu, BMS,
Rule Name

Meningitest,

Oostenbrink

Oostenbrink

Oostenbrink
Meningitest
Bonsu

is concerning because CPRs are well-known to perform

Modified

BMS
Bonsu

better at excluding disease when the outcome frequency

BMS

BMS

is lower. We recommend that future CPR studies use con-

2008 (47)

2004 (50)

2007 (51)

2006 (45)

2010 (46)

2007 (27)

2007 (48)

2001 (43)

2002 (44)

2004 (49)
Oostenbrink

Oostenbrink

Oostenbrink
2002 (2)

sistent inclusion criteria, such as including all patients

Chavanet
Study

Nigrovic
Nigrovi
Dubos

Dubos

0–18 years of age presenting to the ED with meningeal

Bonsu

Bonsu

Total
Fine

signs, and that sampling take place throughout the year.

514 D. M. Kulik et al.

Treatment Frequency
The methodological quality of included studies was
Rule Predicted variable. As the consequences of a missed case of acute
bacterial meningitis are high, the methodological quality
0.59

0.24

0.43

0.81
0.73

0.41
0.65
0.66
0.45
0.63

0.34

0.33
0.42
0.5
in the development of a CPR that is to be used in practice
must be high. Although a number of areas of strength
were noted, several methodological weaknesses were
common in the reviewed CPR studies. Most strikingly,
all but one of the studies was retrospective. There is
Bacterial Meningitis

good evidence that retrospectively derived or validated

Frequency

CPRs perform considerably poorer when subsequently

0.24

0.11
0.12
0.13

0.48
0.48

0.04
0.29
0.44
0.42

0.18
0.19
0.1

0.1
prospectively validated (37). The single-level 2 CPR pro-
vides a nice example of this where both derivation and
retrospective validation studies suggested excellent per-
formance (Sn 1 and LR 0) but prospective validation re-
Upper 95% CI for LR

sulted in much lower performance (Sn 0.92, LR 0.13)

(43,44,49).
Other areas of methodological weakness included in-
0.11
0.18

0.83

0.09
0.14

0.5

adequate description of the study site, lack of predictor

*

*
*
*

*
*

*
*

blinding, absence of reproducibility testing of predictor

variables, inadequate follow-up of outcomes, and insuffi-
cient statistical power. There is considerable evidence
that CPRs that fail to meet these methodological stan-
Lower 95% CI on

dards are likely to be biased. Without a clear description

Sensitivity

of the study site, it is unclear to physicians whether the

0.51

0.95

0.94
0.94
0.89
0.79
0.79

0.95

0.93
0.91

0.95
0.75
0.72
0.8

study site is similar enough to their own for the results

to be applicable (35). Deficiencies in predictor blinding
have been shown to result in an overestimation of diag-
nostic performance (52). Assessment of the inter-
0.023
LR

0.23

0.03

0.13
0.02
0.04

observer reliability of predictor variables is necessary to

0
0

0
0

determine whether a rule will perform similarly when

BMS = Bacterial Meningitis Score; CI = confidence interval; LR = likelihood ratio.
Specificity

used by other physicians (33,37). Insufficient power for

0.62

0.57
0.66
0.72

0.52

0.62
0.49

0.61
0.43

0.85

0.36

0.81
0.70

0.42

statistical modeling due to an inadequate number of

outcomes for a given number of predictor variables
increases the possibility of spurious results through
Sensitivity

random effects (42,53). Inadequate follow-up of out-

0.98
0.98

0.83

0.92
0.98
0.98

comes increases the apparent rule performance by poten-

1
1

1
1

1
1

tially missing serious outcomes (37).

For clinicians to feel comfortable using a CPR for di-
Study Type

Derivation
Derivation

Derivation

Derivation
Derivation

Derivation
Validation
Validation
Validation

Validation

Validation
Validation

Validation
Validation

agnosing bacterial meningitis, the performance of such

a rule must necessarily be high. The performance of the
* Could not calculate LR 95% CI as LR = 0.

only level 2 CPR is unlikely to be sufficiently strong to

Table 4. Assessment of Rule Performance

make it a candidate for routine clinical use with both sen-

Modified Bonsu

Modified BMS

sitivity and LR- below existing performance benchmarks.

Rule Name

Oostenbrink

Oostenbrink
Oostenbrink
Oostenbrink
Meningitest

Meningitest

While several level 4 CPRs show promise, the level 4

CPR with the highest quality and performance was the
Bonsu

Bonsu
BMS

BMS

BMS
BMS

BMS, making it the strongest candidate to perform

consistently well in subsequent prospective validation
studies.
Oostenbrink 2001 (43)
Oostenbrink 2002 (44)
Oostenbrink 2004 (49)
Chavanet 2007 (51)

Nigrovic 2007 (48)

Nigrovic 2002 (2)
Dubos 2006 (45)
Dubos 2006 (45)
Dubos 2006 (45)
Dubos 2010 (46)
Dubos 2010 (46)
Bonsu 2008 (47)
Bonsu 2004 (50)

Limitations
Fine 2007 (27)
Study

There are several limitations to this systematic review.

The electronic search strategy may not have identified
all clinical prediction rules for bacterial meningitis in
Does This Child Have Bacterial Meningitis?
children. However, the search strategy was performed on
multiple occasions to ensure all relevant papers were
identified and the reference lists of all clinical prediction
rule publications were manually reviewed, which failed
to reveal any additional studies. The methodological
quality metric that we used treated each quality item
with equal weight and it is possible that some items are
more important than others. Some of the methodological
quality items were subjective, however, two reviewers
reached consensus on every item for each study. Further,
the methodological quality items that we used have been
well described in the literature but have not been rigor-
ously developed or validated (32–34,37,38). However,
they have been previously used to evaluate clinical
prediction rules for children and are widely accepted
for evaluating the quality of studies of diagnostic tests
(35,36,54). Lastly, although the rule-predicted treatment
frequency is illustrative of the degree of overtreatment
that would result from the use of each CPR, it remains dif-
ficult to determine how this compares with clinical judg-
ment alone, as this was not assessed in any of the studies.
CONCLUSIONS
In summary, a rigorously developed, high-performing
and well-validated CPR that is ready for clinical use to
guide which children with suspected bacterial meningitis
should be hospitalized and treated with i.v. antibiotics and
can be safely discharged home was not identified. A de-
tailed analysis of the methodological quality and perfor-
mance of identified CPRs suggested areas for quality
improvement for future CPR studies. These include pro-
spective validation using standardized inclusion criteria
to capture all children at risk of bacterial meningitis,
adequate blinding, predictor reproducibility assessment,
and meticulous follow-up of clinical outcomes after dis-
charge. Although several of the rules that have not yet
undergone prospective validation show promise, the level
4 CPR with the highest quality and performance is the
BMS, making it the best candidate for prospective valida-
tion. Until consistently high methodological quality and
diagnostic performance are demonstrated through pro-
spective validation, caution is warranted in the routine
clinical use of existing CPRs for children with suspected
bacterial meningitis.
REFERENCES
1. Schuchat A, Robinson K, Wenger JD, et al. Bacterial meningitis in
the United States in 1995. Active Surveillance Team. N Engl J Med
1997;337:970–6.
2. Nigrovic LE, Kuppermann N, Malley R. Development and valida-
tion of a multivariable predictive model to distinguish bacterial
from aseptic meningitis in children in the post-Haemophilus influ-
enzae era. Pediatrics 2002;110:712–9.
515
3. Tatara R, Imai H. Serum C-reactive protein in the differential diag-
nosis of childhood meningitis. Pediatr Int 2000;42:541–6.
4. Saez-Llorens X, McCracken GH Jr. Bacterial meningitis in
children. Lancet 2003;361(9375):2139–48.
5. El Bashir H, Laundy M, Booy R. Diagnosis and treatment of bacte-
rial meningitis. Arch Dis Child 2003;88:615–20.
6. Lu CH, Huang CR, Chang WN, et al. Community-acquired bacte-
rial meningitis in adults: the epidemiology, timing of appropriate
antimicrobial therapy, and prognostic factors. Clin Neurol Neuro-
surg 2002;104:352–8.
7. Meyer CN, Samuelsson IS, Galle M, Bangsborg JM. Adult bacterial
meningitis: aetiology, penicillin susceptibility, risk factors, prog-
nostic factors and guidelines for empirical antibiotic treatment.
Clin Microbiol Infect 2004 Aug;10(8):709–17.
8. Tunkel AR, Hartman BJ, Kaplan SL, et al. Practice guidelines for
the management of bacterial meningitis. Clin Infect Dis 2004;39:
1267–84.
9. van de Beek D, de Gans J, Spanjaard L, Vermeulen M, Dankert J.
Antibiotic guidelines and antibiotic use in adult bacterial meningitis
in The Netherlands. J Antimicrob Chemother 2002;49:661–6.
10. Moller K, Skinhoj P. Guidelines for managing acute bacterial men-
ingitis in adults. West J Med 2000;173:223–4.
11. Feigin RD. Use of corticosteroids in bacterial meningitis. Pediatr
Infect Dis J 2004;23:355–7.
12. Proulx N, Frechette D, Toye B, Chan J, Kravcik S. Delays in the ad-
ministration of antibiotics are associated with mortality from adult
acute bacterial meningitis. QJM 2005;98:291–8.
13. Parasuraman TV, Frenia K, Romero J. Enteroviral meningitis. Cost
of illness and considerations for the economic evaluation of poten-
tial therapies. Pharmacoeconomics 2001;19:3–12.
14. Chadwick DR, Lever AM. The impact of new diagnostic methodol-
ogies in the management of meningitis in adults at a teaching hos-
pital. QJM 2002;95:663–70.
15. Poppert S, Essig A, Stoehr B, et al. Rapid diagnosis of bacterial
meningitis by real-time PCR and fluorescence in situ hybridization.
J Clin Microbiol 2005;43:3390–7.
16. Jansen GJ, Mooibroek M, Idema J, Harmsen HJ, Welling GW,
Degener JE. Rapid identification of bacteria in blood cultures by us-
ing fluorescently labeled oligonucleotide probes. J Clin Microbiol
2000;38:814–7.
17. Spanos A, Harrell FE Jr, Durack DT. Differential diagnosis of acute
meningitis. An analysis of the predictive value of initial observa-
tions. JAMA 1989;262:2700–7.
18. Geiseler PJ. Unnecessary duplication of publication. N Engl J Med
1980;22;302:1209–10.
19. Wald ER, Kaplan SL, Mason EO Jr, et al. Dexamethasone therapy
for children with bacterial meningitis. Meningitis Study Group.
Pediatrics 1995;95:21–8.
20. Marton KI, Gean AD. The spinal tap: a new look at an old test. Ann
Intern Med 1986;104:840–8.
21. Sato Y, Ohta Y, Honda Y, Kaji M, Oizumi K. Diagnostic value of
atypical lymphocytes in cerebrospinal fluid from adults with entero-
viral meningitis. J Neurol 1998;245:598–602.
22. Baker RC, Lenane AM. The predictive value of cerebrospinal
fluid differential cytology in meningitis. Pediatr Infect Dis J 1989;
8:329–30.
23. Negrini B, Kelleher KJ, Wald ER. Cerebrospinal fluid findings in
aseptic versus bacterial meningitis. Pediatrics 2000;105:316–9.
24. Berkley JA, Mwangi I, Ngetsa CJ, et al. Diagnosis of acute bacterial
meningitis in children at a district hospital in sub-Saharan Africa.
Lancet 2001;357(9270):1753–7.
25. Freedman SB, Marrocco A, Pirie J, Dick PT. Predictors of bacterial
meningitis in the era after Haemophilus influenzae. Arch Pediatr
Adolesc Med 2001;155:1301–6.
26. Dunbar SA, Eason RA, Musher DM, Clarridge JE 3rd. Microscopic
examination and broth culture of cerebrospinal fluid in diagnosis of
meningitis. J Clin Microbiol 1998;36:1617–20.
27. Fine AM, Nigrovic LE, Reis BY, Cook EF, Mandl KD. Linking
surveillance to action: incorporation of real-time regional data
into a medical decision rule. J Am Med Inform Assoc 2007;14:
206–11.
516
28. Black S, Shinefield H, Fireman B, et al. Efficacy, safety and immu-
nogenicity of heptavalent pneumococcal conjugate vaccine in chil-
dren. Northern California Kaiser Permanente Vaccine Study Center
Group. Pediatr Infect Dis J 2000;19:187–95.
29. Kaplan SL, Mason EO Jr, Wald ER, et al. Decrease of invasive pneu-
mococcal infections in children among 8 children’s hospitals in the
United States after the introduction of the 7-valent pneumococcal
conjugate vaccine. Pediatrics 2004;113(3 Pt 1):443–9.
30. Whitney CG, Farley MM, Hadler J, et al. Decline in invasive pneu-
mococcal disease after the introduction of protein-polysaccharide
conjugate vaccine. N Engl J Med 2003;1(348):1737–46.
31. Kyaw MH, Lynfield R, Schaffner W, et al. Effect of introduction of
the pneumococcal conjugate vaccine on drug-resistant Streptococ-
cus pneumoniae. N Engl J Med 2006;354:1455–63.
32. McGinn TG, Guyatt GH, Wyer PC, Naylor CD, Stiell IG,
Richardson WS. Users’ guides to the medical literature: XXII:
how to use articles about clinical decision rules. Evidence-Based
Medicine Working Group. JAMA 2000;284:79–84.
33. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review
and suggested modifications of methodological standards. JAMA
1997;277:488–94.
34. Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction
rules. Applications and methodological standards. N Engl J Med
1985;313:793–9.
35. Maguire JL, Kulik DM, Laupacis A, Kuppermann N, Uleryk EM,
Parkin PC. Clinical prediction rules for children: a systematic
review. Pediatrics 2011;128:e666–77.
36. Maguire JL, Boutis K, Uleryk EM, Laupacis A, Parkin PC. Should
a head-injured child receive a head CT scan—a systematic review of
clinical prediction rules. Pediatrics 2009;124:e145–54.
37. Stiell IG, Wells GA. Methodologic standards for the development of
clinical decision rules in emergency medicine. Ann Emerg Med
1999;33:437–47.
38. Reilly BM, Evans AT. Translating clinical research into clinical
practice: impact of using prediction rules to make decisions. Ann
Intern Med 2006;144:201–9.
39. Mauer DK, Nolan J, Plaisance P, et al. Effect of active compression-
decompression resuscitation (ACD-CPR) on survival: a combined
analysis using individual patient data. Resuscitation 1999;41:249–56.
40. Hess EP, Thiruganasambandamoorthy V, Wells GA, et al. Diagnos-
tic accuracy of clinical prediction rules to exclude acute coronary
syndrome in the emergency department setting: a systematic review.
CJEM 2008;10:373–82.
D. M. Kulik et al.
41. Stiell IG, Wells GA, Vandemheen K, et al. The Canadian CT Head
Rule for patients with minor head injury. Lancet 2001;357(9266):
1391–6.
42. Concato J, Feinstein AR, Holford TR. The risk of determining risk
with multivariable models. Ann Intern Med 1993;118:201–10.
43. Oostenbrink R, Moons KG, Donders AR, Grobbee DE, Moll HA.
Prediction of bacterial meningitis in children with meningeal signs:
reduction of lumbar punctures. Acta Paediatr 2001;90:611–7.
44. Oostenbrink R, Moons KG, Twijnstra MJ, Grobbee DE, Moll HA.
Children with meningeal signs: predicting who needs empiric anti-
biotic treatment. Arch Pediatr Adolesc Med 2002;156:1189–94.
45. Dubos F, Lamotte B, Bibi-Triki F, et al. Clinical decision rules to
distinguish between bacterial and aseptic meningitis. Arch Dis
Child 2006;91:647–50.
46. Dubos F, Korczowski B, Aygun DA, et al. Distinguishing between
bacterial and aseptic meningitis in children: European comparison
of two clinical decision rules. Arch Dis Child 2010;95:963–7.
47. Bonsu BK, Ortega HW, Marcon MJ, Harper MB. A decision rule for
predicting bacterial meningitis in children with cerebrospinal fluid
pleocytosis when gram stain is negative or unavailable. Acad Emerg
Med 2008;15:437–44.
48. Nigrovic LE, Kuppermann N, Macias CG, et al. Clinical prediction
rule for identifying children with cerebrospinal fluid pleocytosis at
very low risk of bacterial meningitis. JAMA 2007;297:52–60.
49. Oostenbrink R, Moons KG, Derksen-Lubsen AG, Grobbee DE,
Moll HA. A diagnostic decision rule for management of children
with meningeal signs. Eur J Epidemiol 2004;19:109–16.
50. Bonsu BK, Harper MB. Differentiating acute bacterial meningitis
from acute viral meningitis among children with cerebrospinal fluid
pleocytosis: a multivariable regression model. Pediatr Infect Dis J
2004;23:511–7.
51. Chavanet P, Schaller C, Levy C, et al. Performance of a predictive
rule to distinguish bacterial and viral meningitis. J Infect 2007;54:
328–36.
52. Bossuyt PM, Reitsma JB, Bruns DE, et al. Toward complete and
accurate reporting of studies of diagnostic accuracy: the STARD
initiative. Acad Radiol 2003;10:664–9.
53. Moss M, Wellman DA, Cotsonis GA. An appraisal of multivariable
logistic models in the pulmonary and critical care literature. Chest
2003;123:923–8.
54. Pickering A, Harnan S, Fitzgerald P, Pandor A, Goodacre S. Clinical
decision rules for children with minor head injury: a systematic
review. Arch Dis Child 2011;96:414–21.
Does This Child Have Bacterial Meningitis? 517

APPENDIX 1: SEARCH STRATEGY

Topic The following tables and text record the search strate-
gies and terms used.
Search strategies. We ran searches using the OVID search
MEDLINE. The search strategy for MEDLINE was com-
platform in the following databases: MEDLINE, and
pleted from 1948 to September 27, 2012. We used a com-
EMBASE. The initial search was run on August 18,
bination of MeSH and free text terms for:
2011 and AutoAlerts were received and reviewed on an
ongoing basis until article submission on October 3, 2012.

Set History Comments

1 ((derivation or clinical* or predict* or decision*) adj10 (rule* or algorithm* or tool or tools or Prediction Rule Terms
model* or score* or indicator* or validation* or criteria)).ti,ab. or decision support
techniques/
2 limit 1 to (‘‘infant (1 to 23 months)’’ or ‘‘preschool child (2 to 5 years)’’ or ‘‘child (6 to 12 Age Group Limit
years)’’ or ‘‘adolescent (13 to 18 years)’’)
3 Pediatrics/ Pediatric Specialty term
4 2 or (1 and 3) Age Group limits
5 evaluation studies.pt. or evaluation studies as topic/ or validation studies.pt. or validation Diagnostic sensitivity filter terms
studies as topic/ or ‘‘sensitivity and specificity’’/ or predictive value of tests/ or roc
curve/ or diagnostic errors/ or false negative reactions/ or false positive reactions/ or
observer variation/ or likelihood functions/ or (likelihood or likelihood ratio:).mp. or
predictive value of tests/
6 4 and 5 Diagnostic term limits
7 exp Meningitis, Bacterial/ or Meningitis/ or (bacterial adj2 Meningitis).mp. Bacterial Meningitis terms
8 6 and 7 Final results

EMBASE. The search strategy for EMBASE (1980 to 2012 week 39). We used a combination of EMBASE and free text
terms.

Set History Comments

1 (((derivation or clinical* or predict* or decision*) adj10 (rule* or algorithm* or tool or tools or Prediction Rule Terms
model* or score* or indicator* or validation* or criteria)) or (decision adj5 support adj5
techni*)).ti,ab. or decision support system/ or scoring system/
2 limit 1 to (infant <to one year> or child <unspecified age> or preschool child <1 to 6 years> Age Group Limit
or school child <7 to 12 years> or adolescent <13 to 17 years>)
3 pediatrics/ Pediatric Specialty term
4 2 or (1 and 3) Age Group limits
5 ‘‘sensitivity and specificity’’/ or diagnostic accuracy/ or ‘‘prediction and forecasting’’/ or Diagnostic sensitivity filter terms
prediction/ or predictive validity/ or observer variation/ or receiver operating
characteristic/ or roc curve/ or laboratory diagnosis/ or abnormal laboratory result/ or
likelihood functions/ or validation study/ or reproducibility/ or (likelihood or (likelihood
adj2 ratio:)).mp. or ((roc adj2 curve:) or (Receiver adj5 curve*)).mp. or ((roc adj2 curve:)
or (Receiver adj5 curve*)).mp. or ((diagnostic adj5 error:) or (false adj5 negative:) or
(false adj5 positive:)).mp.
6 4 and 5 Diagnostic term limits
7 meningitis/ or bacterial meningitis/ or (bacterial adj2 Meningitis).mp. Bacterial Meningitis terms
8 6 and 7 Final results
518
APPENDIX 2: QUALITY ASSESSMENT SCORE FOR CLINICAL PREDICTION-RULE DERIVATION
Criteria
Data collected prospectively
Study site(s) well described
Study population well described
Predictors well defined
Blinding of predictor assessors
Outcome well defined
Blinding of outcome assessors
Predictors reproducible
Adequate follow-up of outcomes
Rule applied to all patients at risk
Adequate model power
Adequate reporting of results
Course of action described
Clinical sensibility
Ease of bedside use
95% CI of rule properties reported
Mathematical technique reported
CI = confidence interval; CSF = cerebrospinal fluid; NPV = negative predictive value; PPV = positive predictive value.
D. M. Kulik et al.
Score
1 = yes
1 = yes
1 = age, patient presentation reported
1 = predictors were unambiguous
1 = predictor assessors were blind to CSF results
1 = outcome was unambiguous
1 = outcome assessors were blind to predictors
1 = reproducibility in children reported and K < 0.5
1 = outcomes assessed after hospital discharge
1 = rule applied to all patients with features suggestive of bacterial meningitis
1 = number of outcomes to potential predictors at 10 or more
1 = sensitivity, specificity, PPV, and NPV reported
1 = rule suggests CSF analysis and bacterial meningitis treatment or observation with follow-up
1 = predictors and outcomes were clinically meaningful
1 = reviewers could easily apply rule at the bedside
1 = yes
1 = recursive partitioning or logistic regression technique well described
Does This Child Have Bacterial Meningitis? 519

ARTICLE SUMMARY
1. Why is this topic important?
A clinical prediction rule to augment clinical decision
making and distinguish bacterial meningitis from aseptic
meningitis in children would be extremely helpful. Ide-
ally, such a tool would allow physicians to accurately pre-
dict which children are at high risk of bacterial meningitis,
expedite appropriate management, and facilitate with-
holding i.v. antibiotics and hospital admission for those
children at very low risk.
2. What does this study attempt to show?
This study aims to systematically identify clinical pre-
diction rules (CPRs) for children with suspected bacterial
meningitis and compare their methodological quality and
performance for diagnosing bacterial meningitis using
a recently developed framework to evaluate CPRs for
children.
3. What are the key findings?
A rigorously developed, high-performing, and
well-validated CPR ready for clinical use to guide which
children with suspected bacterial meningitis should be
hospitalized and treated with i.v. antibiotics and can be
safely discharged home was not identified. Areas for qual-
ity improvement for future CPR studies include prospec-
tive validation using standardized inclusion criteria,
adequate blinding, predictor reproducibility assessment,
and meticulous follow-up of outcomes. Prospective vali-
dation of the Bacterial Meningitis Score is recommended.
4. How is patient care impacted?
Until consistently high methodological quality and di-
agnostic performance are demonstrated through prospec-
tive validation, caution is warranted in the routine clinical
use of existing CPRs for children with suspected bacterial
meningitis.