+Model

YICCN-2328; No. of Pages 9 ARTICLE IN PRESS

Intensive and Critical Care Nursing (2014) xxx, xxx—xxx

Available online at www.sciencedirect.com

ScienceDirect

journal homepage: www.elsevier.com/iccn

ORIGINAL ARTICLE

Validation of the use of the Critical-Care

Pain Observation Tool (CPOT) with brain
surgery patients in the neurosurgical
intensive care unit
Christine Echegaray-Benites a,b,c, Oxana Kapoustina a,b,c,
Céline Gélinas a,b,d,e,∗

a
McGill University, Ingram School of Nursing, Montréal, Québec, Canada
b
Centre for Nursing Research and Lady Davis Institute, Jewish General Hospital, Montréal, Québec, Canada
c
McGill University Health Centre (MUHC), Montréal, Québec, Canada
d
Quebec Nursing Intervention Research Network (RRISIQ), Montréal, Québec, Canada
e
Alan Edwards Centre for Research on Pain, McGill University, Montréal, Québec, Canada

Accepted 5 April 2014

KEYWORDS Summary Many critically ill patients are unable to self-report their pain. In such situations,
the use of valid behavioral pain scales is recommended.
Adult;
Objective: To validate the use of the Critical-Care Pain Observation Tool (CPOT) with brain
Behavioral scale;
surgery adults in the neurosurgical intensive care unit.
Brain surgery;
Design: Repeated-measure within subject prospective design.
CPOT;
Settings: Forty-three elective brain surgery patients of a Canadian university hospital partici-
Intensive care unit;
pated.
Pain assessment
Method: Participants were video recorded and scored with the CPOT before, during and after
a non-nociceptive (non-invasive blood pressure using cuff inflation) and a nociceptive (turning)
procedure for a total of six assessments. Self-reports of pain were also obtained.
Results: Discriminant validation was supported with higher mean CPOT scores during the noci-
ceptive procedure compared with the non-nociceptive one. More participants reported higher
pain intensity during turning compared with cuff inflation. Criterion validation was supported
with a moderate positive correlation between self-reports of pain intensity and CPOT scores
during turning. Interrater and intrarater reliability of CPOT scores through the viewing of partic-
ipants’ videos by two trained raters was supported with high Intraclass Correlation Coefficients.

∗
Corresponding author at: McGill University, Ingram School of Nursing, 3506 University Street, Wilson Hall, Room 420, Montréal, Québec
H3A 2A7, Canada. Tel.: +1 514 398 6157; fax: +1 514 398 8455.
E-mail address: celine.gelinas@mcgill.ca (C. Gélinas).

http://dx.doi.org/10.1016/j.iccn.2014.04.002
0964-3397/© 2014 Published by Elsevier Ltd.

Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
http://dx.doi.org/10.1016/j.iccn.2014.04.002
+Model
YICCN-2328; No. of Pages 9 ARTICLE IN PRESS
2 C. Echegaray-Benites et al.

Conclusion: The CPOT appears to be valid for the detection of pain in elective brain surgery
patients in the neurosurgical intensive care unit.
© 2014 Published by Elsevier Ltd.

Implications for Clinical Practice

• The CPOT is valid and reliable to detect pain in elective brain surgery ICU patients.
• The CPOT use can be taught using a short standardised training, which is now available online.
• It is urgent that recent clinical guidelines be implemented in ICU practice to achieve improved pain management.

Introduction et al., 2010). Similarly, Thibault et al. (2007) reported

Evidence has shown that brain-injured patients in the neu-
rological intensive care unit (nICU), such as post-craniotomy
patients, experience a higher than expected incidence
of moderate to severe pain (De Benedittis et al., 1996;
Gottschalk, 2009; Hansen et al., 2011; Mordhorst et al.,
2010). Since the gold standard of pain assessment is a
self-report, evaluating pain in non-communicative patients
continues to be a challenge for clinicians (Herr et al., 2011).
In cases when pain cannot be reported verbally or through
signs (e.g. head nodding), the use of behavioral pain scales is
strongly suggested in recent clinical recommendations (Herr
et al., 2011) and guidelines (Barr et al., 2013). The Critical-
Care Pain Observation Tool (CPOT) is one of those scales that
is developed specifically for use with non-communicative
critically ill adult patients (Gélinas et al., 2006). Thus far,
the CPOT has not been tested in brain surgery ICU patients
for the purpose of pain detection.
Background
The International Association for the Study of Pain (IASP)
has defined pain as an unpleasant sensorial and emotional
experience that is associated with actual or potential tis-
sue damage (IASP, 1979). Pain can result from a nociceptive
stimulus that is injurious and that initiates physiologi-
cal mechanisms associated with the experience of pain
(‘‘nociceptive’’, n.d.) (Loeser and Treede, 2008). When the
patient is unable to self-report, IASP also states that: ‘‘the
inability to communicate pain verbally does not negate the
possibility that an individual is experiencing pain’’ (IASP,
2011). Therefore, it is important for clinicians to use pain
assessment methods designed to suit the communication
abilities of the patient.
According to latest studies, nICU patients who undergo
brain surgery experience pain in the first 48 hours post-
surgery with the highest incidence in the first 12 hours (De
Benedittis et al., 1996, Mordhorst et al., 2010). Approxi-
mately 55—80% of post-craniotomy patients rate their pain
intensity as moderate to severe on the first post-operative
day (Flexman et al., 2010; Mordhorst et al., 2010; Thibault
et al., 2007). A recent study of 256 post-craniotomy patients
estimated that up to 87% of them experienced pain in
the nICU during the first 24 hours after surgery (Mordhorst
the prevalence of pain in 76% of the 299 post-craniotomy
patients in the first 48 hours after surgery. Outcomes from
untreated pain include physiologic complications (e.g.,
hypoxia, neuroendocrine stress responses, delirium, psy-
chosis, hyperglycemia, increased risk of developing chronic
pain) and psychological changes (e.g., stress, agitation,
insomnia and demoralisation) (Cousins and Umedaly, 1996;
Greisen et al., 2001; Marik and Zaloga, 2002; Szokol and
Vender, 2001). For example, patients who suffered post-
surgical pain have a 10—50% incidence of new onset chronic
pain (Kehlet et al., 2006).
In a large Canadian survey, 842 critical care nurses
working in adult ICUs including neurological settings were
surveyed to examine their current practices by documenting
knowledge and perceptions of pain assessment and man-
agement practices (Rose et al., 2012). The study findings
showed that nurses were significantly less likely (p < .001) to
use a pain assessment tool for patients unable to commu-
nicate than for patients able to self-report. With patients
able to communicate, self-report tools were used by 89%
of respondents more than 50% of the time. However, with
patients unable to self-report, only 33% of respondents used
a behavioral pain scale more than 50% of the time. These
findings demonstrate the need to strengthen the importance
of using validated pain assessment tools for patients unable
to self-report. Another Canadian study that distributed
questionnaires to 103 neurosurgeons demonstrated gaps in
pain awareness when the most prescribed post-neurosurgery
pain management drug was a weak opioid (i.e., codeine)
and not a more potent such as morphine (Hassouneh et al.,
2011). Neurosurgeons indicated that their prescription was
a personal choice rather than an evidence-based clinical
decision. Taken together, these findings illustrate the need
to implement clinical recommendations and guidelines in
practice to improve pain assessment and management in
nICU patients.
Recommendations by the American Society of Pain Man-
agement Nursing (ASPMN) and recent practice guidelines by
the Society of Critical Care Medicine (SCCM) suggest that
behavioural indicators are a valid option for pain detection
in patients who are unable to self-report (Barr et al., 2013;
Herr et al., 2011). In addition, some behavioral pain scales
were suggested by critical care experts in recent reviews
(Gélinas et al., 2013; Li et al., 2008; Pudas-Tähkä et al.,
2009) and in the SCCM guidelines (Barr et al., 2013) including

Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
http://dx.doi.org/10.1016/j.iccn.2014.04.002
+Model
YICCN-2328; No. of Pages 9 ARTICLE IN PRESS
Validation of the use of the CPOT with brain surgery patients
Table 1 Study sample demographic characteristics (n = 43).
Demographic items n %
Age
Gender
Male 22 51.16
Female 21 48.84
Surgery type
Tumour resection 29 67.44
Trigeminal nerve decompression 5 11.63
Cyst excision-resection 3 6.98
Micro-vascular decompression 2 4.65
More than one type 4 9.30
n Cumulative dose (mg)
Analgesic medication
Morphine 32 7.97
Sedative medication
Propofol 11 24.29
Note: Cumulative doses are calculated based on the total amount of medication participants received for four hours prior to the start
of observations.
the Behavioral Pain Scale (BPS) (Payen et al., 2001) and the
Critical-Care Pain Observation Tool (CPOT) (Gélinas et al.,
2006).
The BPS is a 3-item scale with a total score ranging
from 3 to 12—facial expressions, upper limb movements
and compliance with the ventilator. Its use was validated
with 567 non-verbal surgical (mainly thoracic and abdomi-
nal surgery), medical and trauma ICU patients (Ahlers et al.,
2008, 2010; Aïssaoui et al., 2005; Chen et al., 2011; Juarez
et al., 2010; Payen et al., 2001; Young et al., 2006). These
studies revealed that the BPS could successfully discrimi-
nate between non-nociceptive and nociceptive procedures
in sedated and mechanically ventilated patients with higher
BPS scores during nociceptive procedures (p < .001). Inter-
rater reliability was also supported in most studies with
kappa > 0.60 or ICC > 0.80 (Ahlers et al., 2008, 2010; Aïssaoui
et al., 2005; Juarez et al., 2010; Payen et al., 2001). Cri-
terion validation with the patient’s self-report of pain was
also supported in two studies with samples of 30 partici-
pants or less (Ahlers et al., 2010; Chen et al., 2011). The
ambiguity of the operational definitions of some items of
the BPS has previously been highlighted. More specifically,
the upper limb movement item also includes the concept
of muscle tension, and the compliance with the ventila-
tor does not have a clearly defined description (Li et al.,
2009).
The CPOT was developed for the detection of pain in
non-communicative critically ill adults, mechanically ven-
tilated or not, and includes four behavioural items: facial
expressions, body movements, vocalisation or compliance
with the ventilator and muscle tension. Each item is rated
on a 0—2 response scale for a total score ranging from 0
to 8 (Gélinas et al., 2006). Initially developed in French,
it was translated into English using a back-to-back trans-
lation method. The use of the CPOT has been tested with
524 critically ill adults of various diagnoses (post-operative,
Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
http://dx.doi.org/10.1016/j.iccn.2014.04.002
3
Mean SD Median Range
55.14 14.23 56.00 19—76
SD Median Range
4.40 7.50 2.00—20.00
15.12 30.00 10.00—50.00
medical and trauma) (Gélinas and Arbour, 2009; Gélinas
and Johnston, 2007; Gélinas et al., 2006, 2009; Lee et al.,
2013; Linde et al., 2013; Nürnberg Damström et al., 2011;
Paulson-Conger et al., 2011; Vázquez et al., 2011). Good
discriminant validation was found with higher CPOT scores
during nociceptive procedures (i.e. turning, endotracheal
suctioning) in comparison with baseline or a non-nociceptive
procedure (i.e. non-invasive blood pressure cuff inflation
(NIBP), arm and face wash central catheter dressing change)
(p < 0.001). Criterion validation was supported with moder-
ate correlations of 0.59 and 0.71 (p < .05) between CPOT
scores and self-reports of pain intensity of ICU patients dur-
ing turning (Gélinas and Johnston, 2007; Gélinas et al.,
2006). Additionally, the CPOT was reported to have high
specificity (78%) and sensitivity (86%) with a cut-off score
>2 during nociceptive procedures in post-operative ICU
patients (Gélinas et al., 2009). In different studies, the
CPOT also demonstrated moderate to high interrater reli-
ability with kappa > 0.60 or ICC > 0.80 (Gélinas and Johnston,
2007; Gélinas et al., 2006; Nürnberg Damström et al., 2011;
Vázquez et al., 2011). Although the CPOT has been shown
to be a valid tool for the detection of pain in various ICU
patient groups, its use has not yet been validated with
post-brain surgery patients, which is the purpose of this
study.
Methods
Study aim and objectives
This study aimed to validate the use of the Critical-Care
Pain Observation Tool (CPOT) in elective brain surgery adults
in the nICU. The following specific objectives were to be
examined:
+Model
YICCN-2328; No. of Pages 9 ARTICLE IN PRESS
4 C. Echegaray-Benites et al.
1) Discriminant validation of CPOT scores when exposed to
common procedures (i.e., non-nociceptive and nocicep-
tive).
2) Criterion validation of CPOT scores with the participants’
self-reports of pain.
3) Interrater and intrarater reliability of CPOT scores by two
trained raters.
Design and sample
For this methodological study, a repeated-measure within
subject prospective design was used. A convenience samp-
ling technique that is appropriate for recruiting subjects
from a population with limited accessibility, such as the
elective brain surgery patients, was utilised (Loiselle et al.,
2004). Patients were recruited in a Canadian university affil-
iated neurological hospital, if they: (a) were admitted for
elective craniotomy/craniectomy surgery, (b) were over 18
years of age and (c) were able to self-report, understand
and spoke French or English. Patients with: (a) post-surgical
complications (e.g. haemorrhage, haemodynamic instabil-
ity, swelling of the brain), (b) chronic history of alcohol or
drug use, (c) cognitive deficit or psychiatric disorder diag-
nosis, (d) epilepsy, (e) previous head trauma history, (f)
quadriplegia or (g) receiving neuromuscular blocking agents
were excluded (Aïssaoui et al., 2005; Li et al., 2009; Puntillo
et al., 2004; Siffleet et al., 2007). All these conditions
were judged to have the potential to alter the patients’
behavioural responses to pain.
The sample size was based on the comparison between
a non-nociceptive and a nociceptive procedure and on the
expected correlation with self-reports, as described in pre-
vious studies with critically ill adults using the CPOT (Gélinas
and Johnston, 2007; Gélinas et al., 2006, 2009). A sample
size of 43 was necessary to achieve 90% power and a 0.01
two-tailed significance level to detect a 2-point difference
in CPOT scores using RM-ANOVA, and to reach a correlation
of 0.45 with patients’ self-reports of pain.
Data collection/procedure
Following ethical approval by the institution, eligible
patients were identified and approached on the same day
of admission to the clinic, where they arrived prior to the
surgery. After the patient’s agreement to be approached
about the study was obtained by their nurse, the patients
had the details of the study explained to them by the
research assistants (RA), signed the informed consent form
that was placed in their medical chart, and a copy was kept
in a locked filing cabinet in the researcher’s office. A copy
of the signed consent was also given to the participants.
Data collection took place once the patients were trans-
ferred from the operating room (OR) to the nICU and
were deemed stable by their nurse. Two video cameras
(Sony Handycam HDR-CX110 3.1 Mega Pixels) were set up;
one focusing on the participant’s face and held by the
RA, and another focused on the body and placed on a
tripod at the foot of the bed. The timing of data collec-
tion for the targeted procedures was coordinated with the
responsible nurse according to the patient care needs. ICU
nurses were instructed to give usual care to participants
Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
http://dx.doi.org/10.1016/j.iccn.2014.04.002
including the administration of analgesic and sedative
agents. Participants were assessed during two types of nurs-
ing procedures that are part of routine care in the ICU:
(1) one non-nociceptive procedure, i.e. non-invasive blood
pressure measurement (NIBP) using cuff inflation (Gélinas
and Johnston, 2007) and (2) one nociceptive procedure, i.e.
turning (Puntillo et al., 2001). For each procedure, one-
minute observations were completed at rest (pre-procedure
baseline), during the procedure (which sometimes took
longer during turning; the entire procedure was video
recorded), and fifteen minutes after the procedure (recov-
ery) for a total of six assessments per participant. After the
observation of behaviours with the CPOT, self-report of pain
(presence of pain — yes or no, and its 0—10 intensity) was
obtained whenever possible.
Measurement tools
Critical-Care Pain Observation Tool. The CPOT was used to
score and document the participant’s behaviours (Gélinas
et al., 2009). The research team members were trained
using an established educational training program (Gélinas
et al., 2011) for the use of the CPOT scale by the principal
investigator (CG). The duration of training was 90 minutes,
and started with a description of the CPOT items and the
scoring method. Then, the raters practiced their CPOT sco-
ring method with patient video recordings with the aim to
reach a 100% agreement between raters within an error of
one mark to further ensure internal validity and reliability
of the scores. Finally, halfway into data collection, a booster
session was provided to both raters to ensure consistency in
the scoring method of patient videos with the CPOT (Haidet
et al., 2009).
Self-reports of pain. Participants were first asked ‘‘Do
you have pain?’’ after each assessment to obtain a ‘‘yes’’
or ‘‘no’’ answer verbally or with other signs (e.g. head nod-
ding). Then, all participants were asked to rate their pain
level with the 0—10 Faces Pain Thermometer (FPT) that has
been tested with critically ill adults (Gélinas, 2007). The
tool is a scale from 0 (no pain) to 10 (worst possible pain)
including faces adapted from the work of Prkachin (1992).
The scale showed good convergent validation (r = 0.80—0.86
with a descriptive rating scale) as well as discriminant vali-
dation (t = −5.10, p < 0.001; when scores were compared at
rest and during turning) (Gélinas, 2007).
Socio-demographic and medical variables. Demographic
information (i.e. gender and age) and clinical data from the
patient’s medical chart such as diagnosis, neuroanatomical
location of lesion, surgery procedure, Glasgow Coma Scale
(GCS) score (Teasdale and Jennett, 1974), Richmond Agita-
tion and Sedation Scale (RASS) score (Sessler et al., 2002)
and administration of analgesic/sedative agents within four
hours prior to the procedures (i.e. IV infusions and bolus)
were collected for each participant.
Data analysis
Statistical data analysis was performed with SPSS version
20.0 (SPSS, Inc., Chicago, IL). Prior to data analysis, the
distributions of CPOT scores were examined via normal prob-
ability P—P plots and by the Shapiro—Wilk test for normality
(test statistic range .573—.928, p < .05). Therefore, it was
 Validation of the use of the CPOT with brain surgery patients

YICCN-2328; No. of Pages 9

+Model
http://dx.doi.org/10.1016/j.iccn.2014.04.002
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-

Table 2 CPOT score item frequencies and median total score distributions at each assessment.

CPOT Pre-NIBP NIBP Post-NIBP Pre-turn Turn Post-turn

n % n % n % n % n % n %

ARTICLE IN PRESS
Facial expression
Relaxed 0 32 74.4 36 83.7 35 81.4 31 72.1 19 44.2 36 83.7
Tense 1 8 18.6 4 9.3 8 18.6 12 27.9 19 44.2 7 16.3
Grimace 2 3 7.0 3 7.0 0 0.0 0 0.0 5 11.6 0 0.0
Body movement
Immobile 0 40 93.0 39 90.6 40 93.0 42 97.7 19 44.2 42 97.7
Protection 1 2 4.7 2 4.7 3 7.0 1 2.3 23 53.5 1 2.3
Agitation 2 1 2.3 2 4.7 0 0.0 0 0.0 1 2.3 0 0.0
Vocalisation
No sound/normal 0 38 88.4 39 90.7 42 97.7 42 97.7 21 48.8 43 100.0
Sighing/moaning 1 5 11.6 3 7.0 1 2.3 1 2.3 17 39.6 0 0.0
Crying out/sobbing 2 0 0.0 1 2.3 0 0.0 0 0.0 5 11.6 0 0.0
Muscle tension
Relaxed 0 36 83.7 35 81.4 37 86.0 36 83.7 33 76.7 34 79.1
Tense 1 7 16.3 8 18.6 6 14.0 7 16.3 9 21.0 8 18.6
Very tense 2 0 0.0 0 0.0 0 0.0 0 0.0 1 2.3 1 2.3
Total CPOT scores
Median (min—max) 0.00 (0.00—5.00) 0.00 (0.00—6.00) 0.00 (0.00—2.00) 0.00 (0.00—3.00) 2.00 (0.00—6.00) 0.00 (0.00—2.00)
Quartiles 25/75 0.00/1.00 0.00/1.00 0.00/1.00 1.00/3.00 0.00/1.00 0.00/1.00
Note: NIBP, non-invasive blood pressure; CPOT, Critical-Care Pain Observation Tool.

5
+Model
YICCN-2328; No. of Pages 9 ARTICLE IN PRESS
6 C. Echegaray-Benites et al.
assumed that the study data were not normally distributed,
requiring the use of non-parametric tests. Descriptive anal-
ysis was used to describe general demographic and medical
data. To address the first objective, the CPOT scores of one
of the two raters who is also an experienced ICU nurse were
analysed by Friedman tests for assessments before, dur-
ing and after turning and NIBP procedures; then post hoc
Wilcoxon signed-rank tests were conducted for significant
results. The second objective was achieved by performing
Mann—Whitney tests and bivariate Spearman correlations
for CPOT scores and participants’ self-reports of pain. In
the case of significant tests between the CPOT score and
the participants’ self-report, Receiver Operator Character-
istic (ROC) curve analysis was used to evaluate the ability of
the scale to classify participants with or without pain. The
third objective was achieved by calculating Intraclass Cor-
relation Coefficients (ICCs) of the CPOT scores obtained at
each assessment for both procedures by two trained raters
through the viewing of video recordings for each participant.
ICCs >0.75 were recommended by Laschinger (1992) to show
high agreement between and within raters.
Results
Demographic characteristics of the study sample
Information regarding the demographic characteristics of
the participants is presented in Table 1. The number of
male and female participants was not statistically different
(chi-square = 14.33, p = .99). The ages ranged from 19 to 76,
with a mean of 55 years old. Participants either had cran-
iotomies (n = 34; 79.1%) or craniectomies (n = 9; 20.9%). The
various indications for these surgical interventions included
mainly tumour resection and a smaller proportion of trigem-
inal nerve decompression and cyst excision/resection. Over
10 different locations of brain surgeries were documented
in the study sample, and the most predominant locations
were right suboccipital (n = 7; 16.3%), right frontal (n = 6;
14.0%), left frontal (n = 5; 11.6%) and left suboccipital (n = 5;
11.6%). The majority of participants (n = 32; 74.4%) were
conscious and alert or drowsy (GCS scores = 14—15, and RASS
scores = 0 to −1), and GCS scores and RASS scores’ ranges
were 8—15, and +1 to −2 respectively. Regarding pain man-
agement, most participants received morphine (2.0—5.0 mg
per dose), and only one participant received 1 mg of hydro-
morphone. Some of them also received propofol (10—20 mg
per dose) as a sedative. Cumulative doses of these medica-
tions received by the participants up to four hours before
the assessment period are presented in Table 1.
Discriminant validation of CPOT scores
The frequencies of individual item scores as well as the
median CPOT scores for each assessment are presented in
Table 2. There were no statistically significant differences
in CPOT scores before, during or after NIBP (2 (2) = 2.67,
p = .264). The CPOT scores during different stages of turning
were significant (2 (2) = 57.17, p < .001). Post hoc Wilcoxon
signed rank tests with Bonferroni correction showed a sig-
nificant difference between CPOT scores before and during
turning procedure (Z = 5.14, p < .001) as well as during and
Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
http://dx.doi.org/10.1016/j.iccn.2014.04.002
post-procedure (Z = 5.25, p < .001) but not between pre-
and post-procedure (Z = 0.47, p = .637). Overall, the median
CPOT score increased significantly from 0 during NIBP pro-
cedure to 2 during turning (Z = 4.40, p < .001).
Criterion validation of CPOT scores with
self-reports of pain
Participants’ self-reports of pain (presence/absence and
0—10 intensity) were obtained and are described in Table 3.
As expected, very few participants reported pain during
NIBP compared to the turning procedure (2 (42) = 14.00,
p < .001), and the pain intensity experienced by the partic-
ipants was mild and lower during NIBP (Z = 3.20, p = .001)
by 2.04 on the 10-point FPT. CPOT scores of participants
who reported pain were first compared by Mann—Whitney
U tests to the participants’ self-reports of the presence of
pain (yes or no) for turning and NIBP procedures. Partici-
pants who reported pain during turning showed significantly
higher CPOT scores (U = 53.00, p < .001) compared to those
who reported no pain. Few participants had pain during NIBP
procedures and no significant results were found. Additional
analyses were performed by calculating bivariate Spearman
correlation coefficients using participants’ self-reports of
pain intensity (0—10 FPT). A moderate positive correlation
was observed between self-reports of pain intensity and
CPOT scores (r = 0.571, p < .001) during the turning proce-
dure. As a significant moderate correlation was found, ROC
curve analysis was performed to assess the ability of the
CPOT scale to discriminate between patients who reported
pain and those who reported no pain during the turning pro-
cedure. The area under the curve (AUC) obtained was 0.864,
p < .001 (CI 95% = .757—.971) indicating good discriminative
properties. The cut-off point associated with the maximisa-
tion of the sums of sensitivity (76.9%) and specificity (73.3%)
was found to be >1.5.
Reliability of the raters’ CPOT scores
Interrater reliability between two trained raters
ICC was calculated for the CPOT scores obtained at each
assessment for both procedures by two trained raters via
the viewing of video recordings for each participant within
an interval time of one month after data collection. All ICCs
were >0.75 (Table 4), showing high agreement between the
raters (Laschinger, 1992).
Intrarater reliability of each independent trained rater
ICC was also examined during each of the two procedures via
the viewing of video recordings for each participant by each
rater. The viewing of video recordings for intrarater reli-
ability purposes was undertaken at least one month after
the first viewing of participants’ videos in order to avoid
raters having a close recall of their initial CPOT scorings.
For rater 1, ICCs for NIBP and turning were 0.85, p < .001
(CI 95% = .73—.91) and 0.82, p < .001 (CI 95% = .69—.90)
respectively. For rater 2, ICCs for NIBP and turning were
0.92, p < .001 (CI 95% = .87—.96) and 0.90, p < .001 (CI
95% = .82—.95) respectively.
+Model
YICCN-2328; No. of Pages 9 ARTICLE IN PRESS
Validation of the use of the CPOT with brain surgery patients 7

Table 3 Patients’ self-report of the presence of pain and its intensity.

Patient self-report Total pain intensity rating (0—10)

Yes n (%) No n (%) Median Quartiles (25/75)

NIBP
During 7 (18.9%) 30 (81.1%) 0.00 0.00/0.00
Turning
During 26 (63.4%) 15 (36.6%) 2.00 0.00/5.13
Note: NIBP, non-invasive blood pressure; SD, standard deviation. Six participant reports were missing for NIBP and two for turning
procedures due to sedation and drowsiness. Two participants’ self-report of pain was excluded from the data because external factors
affected the NIBP procedure, confounding the results. As some participants (n = 4, 9.3%) were too drowsy and sedated (as characterised
by the RASS score of −2) to reliably report on either the presence or the intensity of the pain, they were excluded from the following
analysis, if their pain report was missing. For turning, only two participants were not able to report on their pain (RASS of −2).

Discussion (cardiac, thoracic or abdominal), medical and trauma ICU

To our knowledge, this study was the first to examine
if the CPOT successfully captured procedural pain-related
behaviours in elective brain surgery adults during the first
24 hours in the nICU. Consistent findings were obtained with
previous studies in which the CPOT was used in other surgi-
cal, medical and trauma ICU groups (Gélinas and Arbour,
2009; Gélinas and Johnston, 2007; Gélinas et al., 2006;
Nürnberg Damström et al., 2011; Vázquez et al., 2011).
During the nociceptive procedure (i.e. turning) known to
cause pain, participants demonstrated significant changes
in their behavioural CPOT scores as compared to the
non-nociceptive one (i.e., NIBP) supporting discriminant
validation in this post-brain surgery ICU patient group. Cri-
terion validation was also supported as shown in previous
studies (Gélinas and Arbour, 2009; Gélinas and Johnston,
2007; Gélinas et al., 2006) with a moderate positive cor-
relation between the CPOT scores and the gold standard of
pain assessment (i.e., the participants’ self-reports). While
interrater reliability of the CPOT scores between two trained
raters was tested again, intrarater reliability was examined
for the first time. High ICCs were obtained for both reliabil-
ity strategies. Based on these findings, the CPOT as originally
developed appears to be valid for use in this specific nICU
group.
In the current study, CPOT scores during turning were
lower than those previously reported with post-operative
Table 4 Intraclass Correlation Coefficient (ICC) values
between two trained raters’ CPOT scores for non-nociceptive
and nociceptive procedure.
ICC CI 95%
NIBP
Pre-NIBP 0.90* 0.83—0.95
During NIBP 0.93* 0.87—0.96
Post-NIBP 0.77* 0.61—0.87
Turn
Pre-turn 0.76* 0.59—0.86
During turn 0.87* 0.76—0.93
Post-turn 0.84* 0.73—0.91
* p < .001.
patients during various nociceptive procedures such as tur-
ning and endotracheal suctioning (Gélinas and Arbour, 2009;
Gélinas and Johnston, 2007; Gélinas et al., 2006; Lee et al.,
2013). It should be noted that this brain surgery ICU patient
group had a cranial surgical incision, and the nocicep-
tive stimulus used (i.e., turning) only caused them mild
pain, which was lower than expected. More specifically,
the turning procedure involved moving patients from one
side to another while in bed. It is possible that mobil-
isation (i.e., standing up and sitting in a chair) would
have been more painful in these patients due to the
change in intracranial pressure. In previous studies using
the CPOT, critically ill patients had either surgical inci-
sions on their thorax or abdomen, or peripheral sites
of injuries, and turning caused them at least moderate
pain (>4/10) (Gélinas and Johnston, 2007; Gélinas et al.,
2006).
For this brain surgery patient group, we found that the
CPOT cut-off threshold was equal or higher to 2 which is
similar to the cut-off threshold found in post-operative car-
diac surgery ICU patients when exposed to turning (Gélinas
et al., 2009). These two groups were similar in that both
included elective surgery patients with stable ICU course.
Such a threshold seems to vary depending on the patient’s
condition and was shown to be >3 in more critically ill unsta-
ble ICU patients with various diagnoses (i.e. a mixture of
abdominal and thoracic surgery, medical and trauma) and
who were mechanically ventilated (Gélinas and Johnston,
2007).
Our findings indicate that the interrater reliability of
CPOT scores between two trained raters in brain surgery
patients was supported with high ICC scores as obtained
for both NIBP and turning procedures in previous studies
(Gélinas and Johnston, 2007; Nürnberg Damström et al.,
2011). Also, to our knowledge, the consistency of an indi-
vidual rater using the CPOT scale with the same participants
at two different times was evaluated for the first time.
Knowing that pain varies over time in patients, the use
of video cameras to record all procedures allowed for a
systematic examination of all assessments and procedures.
High intrarater reliability was reached for two independent
raters during both NIBP and turning procedures. Additionally,
the reviewing of participant video recordings by each rater
was undertaken at least one month after data collection.

Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
http://dx.doi.org/10.1016/j.iccn.2014.04.002
+Model
YICCN-2328; No. of Pages 9 ARTICLE IN PRESS
8 C. Echegaray-Benites et al.
This time interval prevented raters from remembering orig-
inal CPOT scorings. Thus far, reliability within and between
raters is shown to be high and the CPOT can be consistently
used with minimal standardised training (Gélinas et al.,
2011). Indeed, this training has been shown to provide
the opportunity for clinical practice improvement includ-
ing more frequent assessments of pain, and a better use of
analgesic and sedative agents by ICU nurses (Gélinas et al.,
2011).
Limitations
This study was undertaken in a nICU with a specific patient
subpopulation and cannot be generalised to all brain surgery
patients. The presence of the head bandage may have
limited the observation of the facial expression item in
the CPOT scale, especially of the forehead region for some
patients (n = 15, 34.9%). The nociceptive procedure (tur-
ning) may not have been the optimal procedure to elicit
pain-related behaviours in this patient group because of
the cranial location of the injury that may be too distal
to the centre or periphery of the body. Nociceptive proce-
dures that have higher pain ratings in these patients, such as
wound care (Lee et al., 2013), could be used in future stud-
ies. Another limitation of this study is the impossibility of
blinding the raters to the type of procedure the participants
underwent; however, interrater reliability was examined to
help minimise this potential bias.
Conclusion
Our study findings demonstrated the validity and reliability
of the CPOT for the detection of pain in this elective brain
surgery nICU group, which is new to the literature. Findings
also seemed to support that typical pain-related behaviours
included in existing scales such as the CPOT can be safely
used for pain assessment in post-brain surgery conscious
patients with a stable ICU course. The CPOT can therefore
be useful in assessing pain in these patients when they are
temporarily unable to provide their self-report. A free edu-
cational video is available to ensure standardised training of
ICU nurses to use the CPOT.1 It must be noted that this nICU
patient group is not representative of other brain-injured
ICU patients with altered LOC who appear to exhibit atypical
behaviours (Arbour et al., 2014; Gélinas and Arbour, 2009;
Le et al., 2013), and for which the content of existing val-
idated behavioral pain scales may not be appropriate (Barr
et al., 2013; Gélinas et al., 2013). Further research is still
needed in this latter vulnerable group to better understand
their behavioural responses to pain. For now, it is urgent that
current recommendations and guidelines (Barr et al., 2013;
Herr et al., 2011), such as the use of validated behavioral
pain scales for critically ill patients unable to self-report
including post-brain surgery individuals, be implemented in
ICU practice to ensure that their pain is detected and prop-
erly managed for better and improved clinical outcomes
1 http://pointers.audiovideoweb.com/stcasx/il83win10115/
CPOT2011-WMV.wmv/play.asx (created and funded by Kaiser
Permanente Northern California Nursing Research).
Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
http://dx.doi.org/10.1016/j.iccn.2014.04.002
(Chanques et al., 2006; Gélinas et al., 2011; Rose et al.,
2013; Williams et al., 2008).
Acknowledgments
The authors thank the Ingram School of Nursing at McGill
University in Quebec, Canada for supporting this research
study. In addition to a pilot research grant offered by the
Louise and Alan Edwards Foundation, part of this study is
also funded by a master’s research grant from the Quebec
Nursing Intervention Research Network (RRISIQ) and three
Master’s Degree awards from the Canadian Nursing Founda-
tion (CNF), RRISIQ and by the Quebec Ministry of Education
(MELS program).
Appendix A. Supplementary data
Supplementary data associated with this article can
be found, in the online version, at http://dx.doi.org/
10.1016/j.iccn.2014.04.002.
References
Aïssaoui YS, Zeggwagh AA, Zekraoui AC, Abidi K, Abouqal
R. Validation of a behavioral pain scale in critically ill,
sedated, and mechanically ventilated patients. Anesth Analg
2005;101(5):1470—6.
Ahlers SJGM, Van der Veen AM, Van Dijk M, Tibboel D, Knibbe CAJ.
The use of the behavioral pain scale to assess pain in conscious
sedated patients. Anesth Analg 2010;110(1):127—33.
Ahlers SJ, Van Gulik L, Van der Veen AM, Van Dongen HP, Bruins P,
Belitser SV, et al. Comparison of different pain scoring systems
in critically ill patients in general ICU. Crit Care 2008;12(1):R15.
Arbour C, Choinière M, Topolovec-Vranic J, Loiselle C, Puntillo K,
Gélinas C. Detecting pain in traumatic brain injured patients
exposed to common procedures in the ICU: Typical or atypical
behaviors? Clin J Pain 2014 (in press).
Barr J, Fraser GL, Puntillo K, Ely EW, Gélinas C, Dasta JF, et al. Clin-
ical practice guidelines for the management of pain, agitation,
and delirium in adult patients in the intensive care unit. Crit
Care Med 2013;41:263—306.
Chanques G, Jaber S, Barbotte E, Violet S, Sebbane M, Perrigault
JF, et al. Impact of systematic evaluation of pain and agitation
in an intensive care unit. Crit Care Med 2006;34:1691—9.
Chen YY, Lai YH, Shun SC, Chi NH, Tsai PS, Liao YM. The Chinese
behavior pain scale for critically ill patients: translation and
psychometric testing. Int J Nurs Stud 2011;48(4):438—48.
Cousins MJ, Umedaly HS. Postoperative pain management in the
neurosurgical patient. Int Anesthesiol Clin 1996;34(4):179—93.
De Benedittis G, Lorenzetti A, Migliore M, Spagnoli D, Tiberio F,
Villani R. Postoperative pain in neurosurgery: a pilot study in
brain surgery. Neurosurgery 1996;38(3):466—70.
Flexman AM, Ng JL, Gelb AW. Acute and chronic pain following
craniotomy. Curr Opin Anaesthesiol 2010;23(5):551—7.
Gélinas C. Le thermomètre d’intensité de douleur: Un nouvel outil
pour les patients adultes en phase critique. Perspect Infirm
2007;4(4):12—20.
Gélinas C, Arbour C. Behavioral and physiologic indicators dur-
ing a nociceptive procedure in conscious and unconscious
mechanically ventilated adults: similar or different? J Crit Care
2009;24(4), 628.e7—17.
Gélinas C, Arbour C, Michaud C, Vaillant F, Desjardins S. The imple-
mentation of the critical-care pain observation tool on pain
assessment/management nursing practices in an intensive care
+Model
YICCN-2328; No. of Pages 9 ARTICLE IN PRESS
Validation of the use of the CPOT with brain surgery patients
unit with nonverbal critically ill adults: a before and after study.
Int J Nurs Stud 2011;48:1495—504.
Gélinas C, Fillion L, Puntillo KA. Item selection and content validity
of the critical-care pain observation tool for non-verbal adults.
J Adv Nurs 2009;65(1):203—16.
Gélinas C, Fillion L, Puntillo KA, Viens C, Fortier M. Validation of the
critical-care pain observation tool in adult patients. Am J Crit
Care 2006;15(4):420—7.
Gélinas C, Johnston C. Pain assessment in the critically ill ventilated
adult: validation of the critical-care pain observation tool and
physiologic indicators. Clin J Pain 2007;23(6):497—505.
Gélinas C, Puntillo KA, Joffe AM, Barr J. A validated approach to
evaluating psychometric properties of pain assessment tools for
use in nonverbal critically ill adults. Semin Respir Crit Care Med
2013;34(2):153—68.
Gottschalk A. Craniotomy pain: trying to do better. Anesth Analg
2009;109(5):1379—81.
Greisen J, Juhl CB, Grefte TR, Vilstrup H, Jensen TS, Schmitz O.
Acute pain induces insulin resistance in humans. Anesthesiology
2001;95(3):578—84.
Haidet KK, Tate J, Divirgilio-Thomas D, Kolanowski A, Happ MB.
Methods to improve reliability of video-recorded behavioral
data. Res Nurs Health 2009;32(4):465—74.
Hassouneh B, Centofantil JE, Reddy K. Pain management in post-
craniotomy patients: a survey of Canadian neurosurgeons. Can J
Neurol Sci 2011;38(3):456—60.
Hansen MS, Brennum J, Moltke FB, Dahl JB. Pain treatment after
craniotomy: where is the (procedure specific) evidence? A quali-
tative systematic review. Eur J Anaesthesiol 2011;28(12):821—9.
Herr K, Coyne PJ, McCaffery M, Manworren R, Merkel S. Pain
assessment in the patient unable to self-report: position state-
ment with clinical practice recommendations. Pain Manag Nurs
2011;12(4):230—50.
International Association for the Study of Pain. Pain terms: a list
with definitions and notes on usage. Recommended by the IASP
Subcommittee on Taxonomy. Pain 1979;6:249—52.
International Association for the Study of Pain. IASP pain termi-
nology; 2011, Retrieved February 22, 2011, from http://www.
iasp-pain.org/AM/Template.cfm?Section=General Resource
Links&Template=/CM/HTMLDisplay.cfm&ContentID=3058
Juarez P, Bach A, Baker M, Duey D, Durkin S, Gulczynski B,
et al. Comparison of two pain scales for the assessment of
pain in the ventilated adult patient. Dimens Crit Care Nurs
2010;29(6):307—15.
Kehlet H, Jensen TS, Woolf CJ. Persistent post-surgical pain: risk
factors and prevention. Lancet 2006;367(9522):1618—25.
Laschinger HK. Intraclass correlations as estimates of interrater reli-
ability in nursing research. West J Nurs Res 1992;14(2):246—51.
Le Q, Gélinas C, Arbour C, Rodrigue N. Description of behaviors in
traumatic brain injury patients when exposed to common pro-
cedures in the intensive care unit: a pilot study. Pain Manag Nurs
2013;14(4):e251—61.
Lee KI, Oh HS, Suh YO, Seo WS. Patterns and clinical correlates
of pain among brain injured patients in critical care assessed
with the critical care pain observation tool. Pain Manag Nurs
2013;14(4):259—67.
Li D, Miaskowski C, Burkhardt D, Puntillo K. Evaluations of
physiologic reactivity and reflexive behaviors during noxious pro-
cedures in sedated critically ill patients. J Crit Care 2009;24(3),
472.e9—13.
Li D, Puntillo K, Miaskowski C. A review of objective pain measures
for use with critical care adult patients unable to self-report. J
Pain 2008;9(1):2—10.
Linde SM, Badger JM, Machan JT, Beaudry J, Brucker A, et al. Reeval-
uation of the Critical-Care Pain Observation Tool in intubated
adults after cardiac surgery. Am J Crit Care 2013;22(6):491—7.
Please cite this article in press as: Echegaray-Benites C, et al. Validation of the use of the Critical-Care Pain Observa-
tion Tool (CPOT) with brain surgery patients in the neurosurgical intensive care unit. Intensive Crit Care Nurs (2014),
http://dx.doi.org/10.1016/j.iccn.2014.04.002
9
Loiselle CG, Profetto-McGrath J, Polit DF, Beck CT. Canadian essen-
tials of nursing research. Philadelphia, PA: Lippincott Williams &
Wilkins; 2004. p. 180—4 [chapter 8].
Loeser JD, Treede RD. The Kyoto protocol of IASP basic pain termi-
nology. Pain 2008;137:473—7.
Marik PE, Zaloga GP. Adrenal insufficiency in the critically ill. Chest
2002;122(5):1784—96.
Mordhorst C, Latz B, Kerz T, Wisser G, Schmidt A, Schneider A, et al.
Prospective assessment of postoperative pain after craniotomy.
J Neurosurg Anesthesiol 2010;22(3):202—6.
Nürnberg Damström D, Saboonchi F, Sackey PV, Björling G. A
preliminary validation of the Swedish version of the critical-
care pain observation tool in adults. Acta Anaesthesiol Scand
2011;55(4):379—86.
Payen JF, Bru O, Bosson JL, Lagrasta A, Novel E, Deschaux
I, et al. Assessing pain in critically ill sedated patients by
using a behavioral pain scale. Crit Care Med 2001;29(12):
2258—63.
Paulson-Conger M, Leske J, Maidl C, Hanson A, Dziadulewicz L. Com-
parison of two pain assessment tools in nonverbal critical care
patients. Pain Manag Nurs 2011;12(4):218—24.
Puntillo KA, Morris AB, Thompson CL, Stanik-Hutt J, White CA,
Wild LR. Pain behaviors observed during six common proce-
dures: results from Thunder Project II. Crit Care Med 2004;32(2):
421—7.
Puntillo KA, White CA, Morris AB, Perdue ST, Stanik-Hutt J,
Thompson CL, et al. Patients’ perceptions and responses to pro-
cedural pain: results from Thunder Project II. Am J Crit Care
2001;10(4):238—51.
Pudas-Tähkä SM, Axelin A, Aantaa R, Lund V, Salanterä S.
Pain assessment tools for unconscious or sedated intensive
care patients: a systematic review. J Adv Nurs 2009;65(5):
946—56.
Prkachin KM. The consistency of facial expressions of pain: a com-
parison across modalities. Pain 1992;51:297—306.
Rose L, Haslam L, Dale C, Knechtel L, McGillion M. Behavioral
pain assessment tool for critically ill. Intensive Crit Care Nurs
2013;27:121—8.
Rose L, Smith O, Gélinas C, Haslam L, Dale C, Luk E, et al. Criti-
cal care nurses’ pain assessment and management practices: a
survey in Canada. Am J Crit Care 2012;21(4):251—9.
Sessler CN, Gosnell MS, Grap MJ, Brophy GM, O’Neal PV, Keane KA,
et al. Richmond agitation—sedation scale validity and reliability
in adult intensive care unit patients. Am J Respir Crit Care Med
2002;166:1338—44.
Siffleet J, Young J, Nikoletti S, Shaw T. Patients’ self-report
of procedural pain in the intensive care unit. J Clin Nurs
2007;16(11):2142—8.
Szokol JW, Vender JS. Anxiety, delirium, and pain in the intensive
care unit. Crit Care Clin 2001;17(4):821—42.
Teasdale G, Jennett B. Assessment of coma and impaired conscious-
ness. A practical scale. Lancet 1974;2(7872):81—4.
Thibault M, Girard F, Moumdjian R, Chouinard P, Boudreault D, Ruel
M. Craniotomy site influences postoperative pain following neu-
rosurgical procedures: a retrospective study. Can J Anaesthesiol
2007;54(7):544—8.
Vázquez M, Pardavila MI, Lucia M, Aguado Y, Margall MÁ, Asiain MC.
Pain assessment in turning procedures for patients with invasive
mechanical ventilation. Nurs Crit Care 2011;16(4):178—85.
Williams TA, Martin S, Leslie G, Thomas L, Leen T, Tamaliunas S,
et al. Duration of mechanical ventilation in an adult intensive
care unit after introduction of sedation and pain scales. Am J
Crit Care 2008;17:349—56.
Young J, Siffleet J, Nikoletti S, Shaw T. Use of a behavioral pain
scale to assess pain in ventilated, unconscious and/or sedated
patients. Intensive Crit Care Nurs 2006;22(1):32—9.