M e d i c a l P hy s i c s a n d I n f o r m a t i c s O r i g i n a l R e s e a r c h

Wasser et al.
Electronic Prescribing of CT Oral Contrast Agent

Medical Physics and Informatics

Original Research

Optimizing Radiologist
Downloaded from www.ajronline.org by 36.82.82.63 on 11/29/17 from IP address 36.82.82.63. Copyright ARRS. For personal use only; all rights reserved

e-Prescribing of CT Oral Contrast

Agent Using a Protocoling Portal
Elliot J. Wasser 1,2,3,4 OBJECTIVE. The purpose of this study is to quantify the time expenditure associated
Nicholas J. Galante2,3 with radiologist ordering of CT oral contrast media when using an integrated protocoling
Katherine P. Andriole1,3 portal and to determine radiologists perceptions of the ordering process.
Cameron Farkas1 SUBJECTS AND METHODS. This prospective study was performed at a large aca-
Ramin Khorasani1,2,3 demic tertiary care facility. Detailed timing information for CT inpatient oral contrast orders
placed via the computerized physician order entry (CPOE) system was gathered over a 14-day
Wasser EJ, Galante NJ, Andriole KP, Farkas C, period. Analyses evaluated the amount of physician time required for each component of the
Khorasani R ordering process. Radiologists perceptions of the ordering process were assessed by survey.
Descriptive statistics and chi-square analysis were performed.
RESULTS. A total of 96 oral contrast agent orders were placed by 13 radiologists during the
study period. The average time necessary to create a protocol for each case was 40.4 seconds
(average range by subject, 20.0130.0 seconds; SD, 37.1 seconds), and the average total time to
create and sign each contrast agent order was 27.2 seconds (range, 10.050.0 seconds; SD, 22.4
seconds). Overall, 52.5% (21/40) of survey respondents indicated that radiologist entry of oral
contrast agent orders improved patient safety. A minority of respondents (15% [6/40]) indicated
that contrast agent order entry was either very or extremely disruptive to workflow.
CONCLUSION. Radiologist e-prescribing of CT oral contrast agents using CPOE can
be embedded in a protocol workflow. Integration of health IT tools can help to optimize user
acceptance and adoption.

Keywords: computerized physician order entry, CT,
informatics, oral contrast agent, protocol
DOI:10.2214/AJR.12.9982
Received September 23, 2012; accepted after revision
April 9, 2013.
R. Khorasani receives royalties and has stock or stock
options from Medicalis.
1
Center for Evidence Based Imaging, Brookline, MA.
2 nitis [4, 5]. In addition, caution must be taken
Department of Radiology, Brigham and Womens
Hospital, Boston, MA.
3
Harvard Medical School, Boston, MA.
4
Present address: Cleveland Clinic, 550 Okeechobee Blvd,
West Palm Beach, FL 33401. Address correspondence to
E.J. Wasser (elliot.wasser@gmail.com).
AJR 2013; 201:12981302
0361803X/13/20161298
American Roentgen Ray Society
S
erious complications of CT oral
contrast agent administration are
rare but potentially fatal. Water-
soluble iodine-containing contrast
agents may induce severe anaphylactoid re-
actions, similar to those observed following
IV contrast agent administration, though at
markedly lower frequency [13]. Similar re-
actions to barium-containing agents are even
less common; however, patients with a known
or suspected bowel perforation are at risk of
developing a potentially lethal barium perito-
in prescribing oral contrast agents for patients
with a history of disordered swallowing or
other risks for aspiration, given the possibility
of inducing pneumonitis [6, 7]. For these rea-
sons, the Joint Commission has classified oral
contrast agents as medications, requiring a
physicians prescription before administra-
tion [8]. Depending on the practice setting,
radiologists may rely on referring physicians
to enter oral contrast agent orders or elect to
place such orders themselves.
The advent of computerized physician or-
der entry (CPOE) systems supporting elec-
tronic prescriptions (e-prescribing) can enable
radiology practices to take over the task of or-
dering of oral contrast agent from referring
physicians. As a core measure of Meaningful
Use, e-prescribing is a mandatory component
of certified electronic health care records
(EHRs) [9]. Although the prescribing of oral
and IV contrast agent for CT scans is current-
ly exempted from the requirement for elec-
tronic entry, radiology practices wishing to
qualify for Meaningful Use funds must docu-
ment use of a certified EHR that includes this
capacity [10].
The wide-ranging benefits of CPOE sys-
tems with e-prescribing, including decreased
medication administration errors and improved
compliance with institutional guidelines, have
been previously reported [1113]. As with
any new technology, the implementation of
e-prescribing may also incur direct and indi-
rect costs that impact its adoption [13]. The
absence of CPOE-enabled e-prescribing fea-

1298 AJR:201, December 2013

 Electronic Prescribing of CT Oral Contrast Agent
tures in typical radiology-centric information
technology (IT) systems (e.g., radiology in-
formation systems and PACS) further com-
plicates implementation for radiology be-
cause it requires the use of relevant EHR
capabilities, which are not generally tailored
or optimized for radiologists. Thus, for radi-
Downloaded from www.ajronline.org by 36.82.82.63 on 11/29/17 from IP address 36.82.82.63. Copyright ARRS. For personal use only; all rights reserved
ologists, the primary concerns regarding the
e-prescribing of CT oral contrast media are
the perceived time cost of entering such or-
ders and the potential for workflow disrup-
tion. In a previous investigation by Daven-
port et al. [14], 98% of radiologist survey
respondents did not think that physician entry
of oral contrast agent orders improved patient
safety, and 81% indicated that it was at least
moderately disruptive to workflow.
In our large academic facility, e-prescribing
via CPOE has been available since 1993, with
mandated radiologist prescribing of CT oral
contrast agent for inpatients starting in 2007.
The CPOE system is integrated with the pro-
tocoling software platform, which launches in
context with the patient and radiologist, there-
by forgoing a lengthy login and patient look-
up procedure. The purpose of our study was to
quantify the actual time expenditure for radiol-
ogists ordering oral contrast agent when using
this integrated e-prescribing system. We fur-
ther examined radiologist perceptions regard-
ing workflow disruption when ordering oral
contrast agent and perceptions of the safety
benefits of physician oral contrast agent entry
(as opposed to a nurse or technician).
Subjects and Methods
Institutional review board approval was obtained
for this HIPAA-compliant study. The requirement
for patient informed consent was waived.
Setting
The study was conducted at an urban 793-bed
tertiary academic medical center. Approximately
550,000 imaging examinations are performed an-
nually within the radiology department, including
nearly 3500 inpatient abdominopelvic CT scans
interpreted by the division of abdominal imaging.
Subjects
All radiology residents and fellows rotating
through the division of abdominal imaging (n=
13) during a 2-week study period (March 20, 2012,
through April 2, 2012) who prescribed CT oral
contrast agent were included in the study popu-
lation. Sample size calculations were designed to
detect a difference of 5 seconds in average oral
contrast agent ordering time on a sample, with an
assumed SD of 20 seconds when using the high-
ly integrated protocoling software. On the basis
AJR:201, December 2013 1299
Fig. 1User actions involved in ordering of CT oral
contrast agent. If computerized physician order entry
(CPOE) is launched directly from protocoling portal
software (dashed arrow), login and patient lookup Create
procedures are bypassed (gray boxes). Alternatively, user
Protocol
may manually perform these steps before order entry.
of the power calculation for descriptive statistics,
n= 42(zCRIT)2 / D2, with an assumed SD () of
20 seconds and 95% CI of 5 seconds (D= 10
seconds and zCRIT= 1.96 for the targeted signifi-
CPOE
cance of 95%), a sample size of approximately 62 Login
interactions was deemed necessary [15]. A total of
96 unique oral contrast agent order events were re-
corded during the 14-day study period, entered on
a total of 83 inpatients. Under our current institu-
tional policy, radiologist ordering of oral contrast
Patient
agent is mandated only for inpatients. One subject Lookup
who was not blinded to the data collection because
of their involvement in the project was excluded
from the analyses.
Contrast Material
Menu
Oral contrast agent prescribed for routine ab- Navigation
dominopelvic CT scans at our facility is 2% bar-
ium sulfate suspension (Readi-CAT, Bracco Im-
aging). For patients with known or suspected
perforation of bowel, diatrizoate (Gastrografin,
Bayer Schering) is the preferred agent, unless
Place
there is a known patient history of anaphylactoid Order
reaction to iodinated contrast agents. Standard
dosages of oral contrast agent are 30 mL of diatri-
zoate dissolved in 900 mL of water, or 900 mL of
2% barium sulfate suspension.
Order
Ordering Process Signing
All CT examinations performed in our depart-
ment are protocoled before the scan. As is typi-
cal in an academic teaching practice, almost all
protocoling is done by trainees (residents and fel- patient-identifying information. Alternatively, oral
lows). Any questions by trainees are reviewed by contrast agent orders may be placed directly into
staff radiologists. Also, the protocoling workload the CPOE platform by the radiologist, which incurs
is not required to be equally distributed among all additional steps of user login, patient lookup, and
trainees on a daily basis. A few trainees may take additional menu navigation to arrive at the medica-
on the responsibility for several days at a time in tion-ordering screen. The steps involved in placing
the radiology reading room. Protocols are input an oral contrast agent order are described in Figure
by radiology staff (attending physicians, fellows, 1. Decision support embedded within CPOE pro-
or residents) via an electronic protocoling portal vides an alert (e.g., This patient is allergic to io-
(RadXT, Medicalis). This software allows the ra- dinated contrast) in the case of an attempted order
diologist to view a queue of requested imaging for an oral contrast medication to which the patient
studies along with each patients allergy history, has a known allergy.
renal function laboratory values, and prior radi-
ology reports and relevant imaging. A series of Protocol and Order Entry Timing
drop-down menus are used to select the optimal Data Collection
examination protocol, including contrast agent. To develop a detailed accounting of CT oral
Text fields allow entry of nonstandard orders or contrast agent ordering, specialized software was
special instructions to the technologist. developed to unobtrusively create detailed records
For inpatients, the electronic portal allows sin- of radiologist activity within the CPOE system
gle-click launching into the CPOE platform with and protocoling portal. Protocol-related and or-
automated radiologist login and prepopulation of der entry tasks are performed by the radiologist
 Wasser et al.

on a networked computer (Windows XT, Micro- Fig. 2Four-item

soft) located next to the PACS workstation. The radiologist survey.
software was installed on all relevant computers
during the study period and ran as a hidden back-
ground process with all radiologist participants
blinded to its presence. Detailed data were ac-
quired with selective screen capture and automat-
Downloaded from www.ajronline.org by 36.82.82.63 on 11/29/17 from IP address 36.82.82.63. Copyright ARRS. For personal use only; all rights reserved

ed logging of all activities performed within the

electronic protocoling portal and CPOE system.
Data acquisition was triggered by the presence
of application name identifiers present within the
window header and by detection of application sig-
natures within the operating system process queue.
Screen capture data were delimited to capture only
those windows displaying applications of interest
and only when the window was in active status.
Data were acquired at 5-second intervals with de-
tailed usage statistics compiled into a comma-de-
limited log file for analysis. Each screen capture
contained a time stamp to allow computation of dering, and order signing) by a single observer with protocols and the medications ordered therein, and
timing data. The application was designed to time 6 years of radiology experience and 2 years of ex- their views on ordering oral contrast agent likely
out after 20 seconds without keyboard or mouse perience as an end-user of the CPOE and protocol affect those of their trainees. Two notification e-
activity. All data files were stored in a secure en- portal software. Start and end times for each step mails were sent 3 days apart to recruit survey par-
crypted network-attached storage device. were manually recorded in the usage log spread- ticipants. Multiple choice answer options were dis-
Screen capture data were manually reviewed sheet (Excel 2003, Microsoft) for each patient. played in random order for each user. Training
by one of the authors and were validated against level (resident vs fellow vs attending physician)
the automatically compiled usage statistics. Patient Radiologist Survey was collected for each survey respondent. Answers
medical record numbers were manually collected A voluntary web-based (SurveyMonkey) elec- were otherwise anonymous, and no other specific
from the screen capture information and were used tronic questionnaire (Fig. 2), based on the previ- respondent information was obtained.
to track individual contrast agent orders. ous work of Davenport et al. [14], was distribut-
ed to all radiology residents, fellows, and attending Data Analysis
Observers staff currently or recently active in the division of Descriptive statistics, including means, counts,
Radiologist activity was cataloged into each of abdominal imaging over the prior academic year percentages, and SDs, were computed for the pro-
six steps (protocol specification, CPOE login, pa- (n= 57). Attending radiologists were included be- tocol and contrast timing data. Analysis of variance
tient lookup, menu navigation, contrast agent or- cause they maintain ultimate responsibility for all was used to compare mean timing data across par-

TABLE 1: Detailed Timing Data for Each Subject

User Training Level, Average Time per Average Time per Contrast Agent
No. Postgraduate Year No. of Contrast Agent Orders Protocol (s), Mean (SD) Order Entry (s), Mean (SD) Average Time Total (s)
1 3 9 33.9 (16.0) 24.0 (11.0) 57.9
2 5 17 57.2 (51.3) 23.4 (23.6) 80.6
3 4 3 25.0 (7.1) 30.0 (21.8) 55.0
4 6 21 46.8 (43.9) 40.5 (32.7) 87.3
5 6 4 26.3 (16.5) 26.3 (19.3) 52.6
6 4 1 130.0 (NA) 20.0 (NA) 150.0
7 3 22 20.0 (9.6) 15.5 (8.9) 35.5
8 3 8 41.0 (32.0) 31.4 (22.4) 72.4
9 3 4 51.3 (31.2) 25.0 (9.1) 76.3
10 3 1 15.0 (NA) 35.0 (NA) 50.0
11 6 3 55.0 (40.9) 30.0 (22.9) 85.0
12 3 2 NAa 50.0 (7.1) 50.0
13 6 1 30.0 (NA) 10.0 (NA) 40.0
Overall 96 40.4 (37.1) 27.2 (22.4)
NoteNA= not applicable.
aThe user bypassed the protocol process, launching directly into computerized physician order entry system.

1300 AJR:201, December 2013

 Electronic Prescribing of CT Oral Contrast Agent

Order signing: How much time do you estimate it takes you to

Placing order: 6.1% order an oral contrast agent for a patient?
15.6%
15

No. of Respondents
Menu 10
navigation:
Downloaded from www.ajronline.org by 36.82.82.63 on 11/29/17 from IP address 36.82.82.63. Copyright ARRS. For personal use only; all rights reserved

11.1%
Patient lookup: 5
3.0%
User login:
0.4% 0
< 30 s 30 s1 m 1 m90 s 90 s2 m >2m
Responses

A
Create protocol:
63.8%
Do you think having a physician order an oral
contrast agent improves patient safety?
Fig. 3Mean percentage of radiologist time spent on each component of
protocoling and contrast agent ordering process. 25

No. of Respondents
20

ticipants to assess for individual differences. A 2 15

2 contingency table was constructed to evaluate for
10
mean time differences between orders placed using
the protocol portal integration versus manual entry. 5
Percentages were used to report survey responses.
Pearson chi-square evaluation was performed to 0
Yes No
evaluate the effect of training level on survey feed- Responses
back. Analyses were considered statistically signif-
icant with p< 0.05. Data analysis was performed B
using JMP 9 (SAS Institute) and Microsoft Excel.
How disruptive is ordering oral contrast agents
to your workday?
Results
Thirteen radiologists entered a total of 96 20
inpatient CT oral contrast agent orders into
No. of Respondents

the CPOE system during the 14-day study 15

period. Contrast agent orders were entered
on 84 unique patients because 12 patients
had to undergo two scans each. Of the 96
orders, 94 were launched from the protocol
portal, which automates the process of user
login and patient lookup in the CPOE-en-
abled e-prescribing module of EHR.
Overall, users spent a mean of 40.4 seconds
(SD, 37.1 seconds) to create each individual
protocol and a mean of 27.2 seconds (SD, 22.4
seconds) to initiate and complete each contrast
agent order within the CPOE system (Table
1). The relative time contribution of each in-
dividual step involved in the ordering process
is described in Figure 3. The small proportion
of time spent on user login (0.4%) and patient
lookup (3.0%) tasks can be attributed to the
automation of these elements when launching
the CPOE system directly from the electron-
ic protocoling portal. An evaluation of timing
data across participants was performed using
analysis of variance. No significant differenc-
10
5
0
Minimally
Fig. 4Results of
radiologist survey.
es were detected either between subjects (p=
0.11) or on the basis of the number of years of
training (p= 0.08).
The survey completion rate was 70.2%
(40/57). No significant difference was ob-
served in the response rate by training level
(70.1% of residents, 66.6% of fellows, and
70% of attending physicians; p= 0.97). All
participants who started the survey complet-
ed the questionnaire.
Survey responses are summarized in Fig-
ure 4. Overall, 52.5% (21/40) of respondents
Somewhat Moderately Very Extremely
Responses
C
indicated that having a physician place the
order for oral contrast agent, rather than a
nurse or technician, improved patient safe-
ty. There was no significant effect of training
level on user responses (p= 0.60). Most par-
ticipants indicated that having to place oral
contrast agent orders was either minimally
or somewhat disruptive to workflow (65%
[26/40]). The mean perceived length of time
necessary to enter a single oral contrast order
into CPOE was 56.6 seconds, computed us-
ing the weighted average of user responses.

AJR:201, December 2013 1301


Discussion
We found that CPOE-enabled e-prescrib-
ing of CT oral contrast agent can be embed-
ded in a radiologists protocoling workflow
using integrated health IT tools to mini-
mize duplicate data entry. The survey data
indicate that a slight majority of radiologists
Downloaded from www.ajronline.org by 36.82.82.63 on 11/29/17 from IP address 36.82.82.63. Copyright ARRS. For personal use only; all rights reserved
(52.5% [21/40) think that physician entry of
oral contrast agent orders leads to an im-
provement in patient safety. The mean time
for e-prescribing of each CT oral contrast
agent order was 27.2 seconds. Existing IT
integration standards can enable efficient
implementation of e-prescribing modules
of the electronic medical record in radiolo-
gists protocoling workflow.
In this study, each CT oral contrast order
necessitated an average of 27.2 seconds, ap-
proximately one quarter of the 107 seconds
recorded by Davenport et al. [14]. Though it
is difficult to draw a direct comparison, this
difference in time investment is likely due,
in large part, to the high degree of integration
of the CPOE system with our protocol por-
tal that allows the user to forgo lengthy log-
in and patient lookup steps. Analysis of the
relative time costs of each ordering step in
the Davenport et al. study shows that 72% of
user time was spent logging in, logging out,
or manually searching for patients. At our in-
stitution, these tasks accounted for only 9.4%
of the order entry process (excluding creation
of the protocol, which was not included in
the findings of Davenport et al.). Despite the
shorter times, our participants tended to over-
estimate the amount of time necessary to en-
ter oral contrast orders, estimating that it took
slightly greater than twice the actual recorded
time (56.6 vs 27.2 seconds).
In spite of this overestimation, only a mi-
nority of survey respondents (15% [6/40])
indicated that the ordering of CT oral con-
trast agent was very or extremely dis- Conclusion
ruptive to their workflow. This compares to
a 43% affirmative response in the Daven-
port et al. [14] sample. Although it is im-
possible to definitively attribute these dif-
ferences to a single factor, it seems likely
that the decreased time expenditure per or-
der and reduced need to perform repetitive
logins and patient searches contribute to the
more positive responses in our study.
The survey data indicate that a slight ma-
jority of radiologists (52.5% [21/40]) think
that physician entry of oral contrast agent
orders leads to an improvement in patient
safety. In the Davenport et al. study [14],
only 2% of respondents answered positively
1302 AJR:201, December 2013
Wasser et al.
to the same question. The reasons for this
disparity are unclear. It is possible that re-
spondents are globally more positive about
the benefits of physician ordering when
workflow disruptions are though to be less
pronounced. Alternatively, given that re-
spondents in our sample have had several
years of experience with oral contrast agent
ordering, some may have encountered in-
stances in which physician review averted
potential patient harm. Further research is
necessary to specifically evaluate these dif-
ferences in perception.
Our study has several limitations. The
method of data capture used to observe user
activities within the CPOE system was de-
signed to be as nondisruptive as possible.
However, the method is limited because of
the lack of real-time user observation. For
example, other distractions (e.g., receiving
a page or telephone call) for which we are
unable to account may have impinged on
the user during the ordering process, there-
by prolonging apparent order entry times. In
addition, for reasons of data acquisition and
analysis feasibility, user information was ac-
quired at 5-second intervals. Activities per-
formed in less than 5 seconds may not have
been captured in their entirety and could skew
our results. However, because a large possi-
ble error (e.g., 4.9 seconds) will be equally
applicable to 27.2 or 40.4 seconds, and be-
cause the potential error would affect the
smaller number more (4.9/27.2 is larger than
4.9/40.4), then the 5-second delay would bias
our results toward null (no effect), further
supporting the significance of our findings.
Finally, approximately 30% of eligible par-
ticipants did not complete the survey, and it is
possible that their responses could have sub-
stantially altered our survey results.
CPOE-enabled e-prescribing of CT oral
contrast agents can be performed by radi-
ologists and optimized using existing IT in-
tegration standards. Such an integrated ap-
proach to protocoling enables the use of
EMR e-prescribing features embedded in
the tools radiologists use in their daily work-
flow. Minimizing duplicate data entry and
workflow disruption may improve adoption
and perception of patient safety benefits of
e-prescribing for radiologists.
References
1. Ridley LJ. Allergic reactions to oral iodinated
contrast agents: reactions to oral contrast. Aus-
tralas Radiol 1998; 42:114117
2. Skucas J. Anaphylactoid reactions with gastroin-
testinal contrast media. AJR 1997; 168:962964
3. Miller SH. Anaphylactoid reaction after oral ad-
ministration of diatrizoate meglumine and diatri-
zoate sodium solution. AJR 1997; 168:959961
4. Karanikas ID, Kakoulidis DD, Gouvas ZT,
Hartley JE, Koundourakis SS. Barium peritoni-
tis: a rare complication of upper gastrointestinal
contrast investigation. Postgrad Med J 1997;
73:297298
5. Yamamura M, Masaharu N, Furubayashi H,
Hioki K, Yamamoto M. Barium peritonitis: re-
port of a case and review of the literature. Dis
Colon Rectum 1985; 28:347352
6. Zheng K, Guo MH, Hanauer DA. Using the time
and motion method to study clinical work pro-
cesses and workflow: methodological inconsis-
tencies and a call for standardized research. J
Am Med Inform Assoc 2011; 18:704710
7. Buschmann C, Schulz F, Tsokos M. Fatal aspira-
tion of barium sulfate. Forensic Sci Med Pathol
2011; 7:6364
8. American College of Radiology. Clarifying informa-
tion for MM.4.10 and oral contrast media. American
College of Radiology website. http://www.acr.org/~/
media/ACR/Documents/PDF/QualitySafety/Radia-
tion%20Safety/Clarifying%20MM410%20and%20
Oral%20Contrast%20Media.pdf. Accessed April
26, 2012
9. Center for Medicare and Medicaid Services. Eligi-
ble professionals meaningful use core measures:
measure 4 of 15. Center for Medicare and Medicaid
Services website. http://www.cms.gov/Regula-
tions-and-Guidance/Legislation/EHRIncentivePro-
grams/downloads/4_e-prescribing.pdf. Accessed
May 8, 2012
10. Gibbings T, Konigsbach D, Reicher MA. Mean-
ingful use in radiology. J Am Coll Radiol 2011;
8:657660
11. Bates DW, Leape LL, Cullen DJ, et al. Effect of
computerized physician order entry and a team
intervention on prevention of serious medication
errors. JAMA 1998; 280:13111316
12. McCullough JS, Casey M, Moscovice I, Prasad
S. The effect of health information technology
on quality in U.S. hospitals. Health Aff (Mill-
wood) 2010; 29:647654
13. Kuperman GJ, Gibson RF. Computer physician
order entry: benefits, costs, and issues. Ann In-
tern Med 2003; 139:3139
14. Davenport MS, Gardner CS, Jaffe TA. Mandat-
ed radiologist-performed electronic order entry:
effect on CT oral contrast administration. AJR
2012; 198:616620
15. Eng J. Sample size estimation: how many indi-
viduals should be studied? Radiology 2003;
227:309313