Applied Ergonomics 54 (2016) 218e242

Contents lists available at ScienceDirect

Applied Ergonomics
journal homepage: www.elsevier.com/locate/apergo

Review article

Work system barriers to patient, provider, and caregiver use of

personal health records: A systematic review
Morgan J. Thompson a, Jeremiah D. Reilly b, Rupa S. Valdez b, *
a
Psychology Department, The College of William and Mary, Williamsburg, VA 23187, USA
b
Department of Public Health Sciences, University of Virginia, P.O. Box 800717, Hospital West Complex, Charlottesville, VA 22908, USA

a r t i c l e i n f o a b s t r a c t

Article history: Objectives: This review applied a human factors/ergonomics (HF/E) paradigm to assess individual, work
Received 19 May 2015 system/unit, organization, and external environment factors generating barriers to patient, provider, and
Received in revised form informal caregiver personal health record (PHR) use.
16 October 2015
Methods: The literature search was conducted using five electronic databases for the timeframe January
Accepted 18 October 2015
Available online 15 January 2016
2000 to October 2013, resulting in 4865 citations. Two authors independently coded included articles
(n ¼ 60).
Results: Fifty-five, ten and five articles reported barriers to patient, provider and caregiver PHR use,
Keywords:
Systematic review
respectively. Barriers centered around 20 subfactors. The most frequently noted were needs, biases, be-
Personal health records liefs, and mood (n ¼ 35) and technology functions and features (n ¼ 32).
Human factors and ergonomics Conclusions: The HF/E paradigm was effective in framing the assessment of factors creating barriers to
PHR use. Design efforts should address literacy, interoperability, access to health information, and secure
messaging. A deeper understanding of the interactions between work systems and the role of organi-
zation and external environment factors is required.
© 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.

Contents

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
2. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.1. Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.2. Literature search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.3. Title screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.4. Abstract screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.5. Full-text screening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
2.6. Data extraction and synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
3. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
3.1. Search results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
3.2. Study characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
3.3. HF/E paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
3.4. Patient work system barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
3.5. Provider work system barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
3.6. Caregiver work system barriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
4. Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
4.1. Application of HF/E paradigm to barrier classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
4.2. Application of the HF/E paradigm to caregivers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
4.3. Representation of individuals in the work system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
4.4. Implications for PHR design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

* Corresponding author.
E-mail addresses: mjthompson@email.wm.edu (M.J. Thompson), jdr5bd@
virginia.edu (J.D. Reilly), rsv9d@virginia.edu (R.S. Valdez).

http://dx.doi.org/10.1016/j.apergo.2015.10.010
0003-6870/© 2015 Elsevier Ltd and The Ergonomics Society. All rights reserved.
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 219

4.5. Designing interventions: attending to context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229

4.6. Limitations and future research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
5. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

1. Introduction location (Wynia et al., 2011), and concerns regarding appropriate

reimbursement and liability (Wynia et al., 2011). Low caregiver usage
As clinician and patient roles evolve, responsibility for health rates are associated with caregiving responsibilities for a child who
management is shifting from clinician-governed to patient- has a nonsevere illness (Byczkowski et al., 2014), slow Internet
controlled (Wasson et al., 2012). The continuing development of connection (Tom et al., 2012), insurance type (Tom et al., 2012), and
consumer health information technology (IT), health IT designed for fear of discrimination from insurance companies (Weitzman et al.,
use by lay people, supports this change in roles by enabling patients 2012). Given the multi-dimensionality of these rationales, HF/E
to actively engage in health management both alone and in part- frameworks should be systematically applied to understand barriers
nership with formal healthcare providers (i.e., providers) (Center for to patient, provider, and caregiver consumer health IT use.
Advancing Health, 2010). This engagement through consumer Previous research in HF/E has focused on the impact of work
health IT has been accelerated by market forces such as the prolif- system factors on healthcare providers' clinical health IT use, such as
eration of consumer-facing mobile health applications EHR and computerized provider order entry (CPOE) (Holden et al.,
(PricewaterhouseCoopers LLP, 2012; Greenspun and Coughlin, 2012) 2013; Carayon and 2006; Fuji et al., 2008; Nazi, 2013; Urowitz
and has been promoted by the Meaningful Use initiative within the et al., 2012). These studies demonstrate that health IT use in pro-
United States (The Office of the National Coordinator for Health fessional settings is shaped not only by individual characteristics,
Information Technology (ONC), 2013a; Center for Medicare and but also by a wider range of work system factors. Multiple models of
Medicaid Services, 2015): Stage 2 of this initiative (The Office of the work system (Carayon and 2006) emphasize that system out-
the National Coordinator for Health Information Technology comes are influenced by interactions between users, tasks per-
(ONC), 2013a) incentivizes increased health information exchange, formed, tools used to accomplish these tasks, and the physical,
patient-controlled data, requirements for e-prescribing and incor- social, and organizational environments in which users are
porating lab results, and electronic transmission of patient care embedded (Holden et al., 2013). While work system models tradi-
summaries across multiple settings. The recently proposed Stage 3 tionally characterize the sociotechnical systems of professional
guidelines (Center for Medicare and Medicaid Services, 2015) work, more recent models translate this concept for patients and
extend the Stage 2 requirements for patient engagement by caregivers (Holden et al., 2013; National Research Council, 2011;
increasing the requirements regarding the proportion of a provider's Carayon et al., 2006). These models enable the concept of the
patients who have interacted with their health records (i.e., viewed, work system to be applied to healthcare work conducted by both
downloaded, or transmitted their health records) and used secure healthcare providers and lay people, facilitating the study of work
messaging through the electronic health record (EHR) (Center for system factors in the context of consumer health IT use. In this re-
Medicare and Medicaid Services, 2015). view, barriers are categorized using the HF/E paradigm developed
Patient engagement in health management, however, does not by Karsh and colleagues (Karsh et al., 2006). Although this paradigm
solely rely on actions taken by patients and providers. Rather, health targets patients and providers, we extend and apply it to caregivers.
management in home and community settings often depends upon This review focuses on a specific type of consumer health IT,
the active participation of other individuals (Valdez and Brennan, personal health records (PHRs), because Meaningful Use Stages 2
2015; Skeels, 2010), including informal caregivers (i.e., caregivers). and 3 as well as the Patient Engagement Framework promote
Caregiver as well as patient and provider interest in accessing and increased patient, provider, and caregiver PHR use within the
exchanging health information has been documented (Friction and United States. Though varyingly defined (Archer et al., 2011;
Davies, 2008). The Patient Engagement Framework (National Connecting for Health Personal Health Working Group, 2003; The
eHealth Collaborative and Healthcare Information and Office of the National Coordinator for Health Information
Management Systems Society Foundation, 2014), championed by Technology (ONC), 2013b), PHRs in the present study are
the Healthcare Information Management Systems Society (HIMSS) “Internet-based set[s] of tools that allow people to access and co-
and the National eHealth Collaborative (NeHC), outlines a vision for ordinate their lifelong health information and make appropriate
Meaningful Use Stages 3 and 4 to further support, through con- parts of it available to those who need it” (Connecting for Health
sumer health IT, the full range of individuals (e.g., caregivers, family, Personal Health Working Group, 2003), (p3). Common PHR func-
friends, clergy) involved in patients' health management. The need tionalities include the ability to access lab results, engage in secure
to attend to caregivers' perspectives has also been recently identi- messaging with providers, request prescription renewals, schedule
fied within the HF/E literature (Holden et al., 2013, 2015). appointments, authorize referrals, and view and update medication
Despite widespread development and promotion of consumer and allergy lists (Archer et al., 2011). Studies included here report
health IT, usage rates have remained relatively low (Valdez et al., on PHRs that vary in terms of available features and of organiza-
2015b). Proposed explanations for these low patient usage rates tions through which patients were enrolled (e.g., employer, gov-
include low numeracy and limited technology experience (Taha et al., ernment agency, primary care physician). In contrast to previous
2013), difficult login procedures (McInnes et al., 2013), limited family reviews of PHRs (Archer et al., 2011; Jabour and Jones, 2013;
support for using advanced technologies (Mayberry et al., 2011), and Amante et al., 2014), this review is systematic, restricted to
fears concerning the misuse of stored information (Tjora et al., 2005). empirical peer-reviewed articles, focused on work system barriers
Reasons for low provider usage rates include no prior email to use by patients, providers, and caregivers, and based on an HF/E
communication with patients (Crotty et al., 2013), medical practice paradigm.
220 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

2. Methods
2.1. Scope
Our review was restricted to studies in which users were asked to
engage with PHRs in the context of their everyday lives. Studies
focused solely on lab-based usability testing or system demonstra-
tions were excluded. This review categorized all factors identified as
barriers in quantitative, mixed methods, and qualitative studies.
Traditionally, the term “barrier” is defined as “a law, rule, problem,
etc., that makes something difficult or impossible” (Merriam-
Webster). However, for the purposes of this review the term “bar-
rier” has been extended to allow factors associated with decreased
PHR use such as individual characteristics (e.g., race, age) to be
conceptualized as barriers. The scope did not include conducting a
meta-analysis to assess the degree to which each factor affected PHR
use. Preferred Reporting Items for Systematic Reviews and Meta-
Analyses (PRISMA) (Moher et al., 2009) criteria were adopted to
the extent possible; however, we could not adopt the standards in
full. For example, due to the range of methodologies among included
articles, we were unable to assess risk of bias in a consistent manner.
2.2. Literature search
A health science librarian and a computer science librarian were
consulted in developing the search strategy, including the search
terms. A range synonymous with personal health record and bar-
rier (see Table 1) was used to ensure that all related articles were
included in the initial screening. The search was performed using
five electronic databases during October 2013: PubMed, CINAHL
(excluding MEDLINE records), Engineering Village (Compendex and
INSPEC), IEEE Xplor, and ACM Digital Library. To keep search stra-
tegies consistent across databases, controlled vocabulary was not
used. All searches were limited to articles published between
January 2000 and October 2013.
or telemedicine systems (unless this had been integrated into a PHR
and the article examined both components), (d) were introductions
or prefaces to a special issue, conference proceedings, or book, (e)
were review articles, and/or (f) were not written in English.
Two authors (MJT & RSV) independently coded 100 (2.5%) of the
abstracts for eligibility. The first set of 50 abstracts was used to
finalize inclusion and exclusion criteria. The second set was used
for calculating inter-rater reliability. The resulting value for Cohen's
kappa statistic (k¼.90) was interpreted as strong (McHugh, 2012).
When discrepancies occurred, decisions were discussed and
resolved through consensus-building. Decisions were documented
and subsequently used by the first author to include or exclude the
remaining abstracts.
2.5. Full-text screening
Articles excluded during the initial full-text screening (a) met
abstract exclusion criteria, (b) were opinion articles, (c) were
formative papers, (d) were not full papers, (e) described paper-
based systems, (f) focused on system architecture, and/or (g)
were unavailable after exhausting the University of Virginia's li-
brary resources. Two authors (MJT & RSV) independently screened
10% of the full-text articles (n ¼ 21) before engaging in decision
making similar to that described for abstract screening. The first
author then completed the first round of full-text screening.
A second round of full-text screening ensured that included
articles focused on PHRs implemented in participants' everyday
lives. During this round, articles were excluded if the PHR was (a)
not used, (b) used only during a demonstration, and/or (c) not
implemented in participants' everyday lives. Two authors (MJT &
JDR) individually reviewed all articles during this round. Inter-rater
reliability was moderate, according to Cohen's kappa statistic
(McHugh, 2012) (k¼.67). Discrepancies were resolved through
consensus-building and unresolved differences were referred to
the senior author (RSV).

2.3. Title screening 2.6. Data extraction and synthesis

Title screening consisted of reviewing search results for dupli-
cates and publication in peer-reviewed journals or conference
proceedings. Duplicates were screened for manually by the first
author and the RefWorks duplicate finder tool (ProQuest LLC, 2010).
Ulrichsweb was used to exclude non-peer-reviewed journals and
conference proceedings. If a conference proceeding was not listed
on Ulrichsweb, we determined whether it was peer-reviewed
based on the call for participation.
2.4. Abstract screening
Abstracts considered eligible and included in full-text screening
(a) mentioned PHRs and (b) related to patients', providers', and/or
caregivers' experiences with PHRs. Excluded abstracts (a) focused
solely on provider-facing electronic systems, (b) focused on tech-
nology with functionality limited to symptom tracking without
integration into other PHR functions, (c) focused on self-management
Two authors (MJT&JDR) independently extracted the following:
 sample (patient, provider, or caregiver)
 sample size
 type of PHR (tethered or untethered)
 instrumentation (survey, interview, focus group, advisory panel,
PHR system logs, and EHR patient data)
 study design (quantitative, qualitative, or mixed methods)
 study location (country)
 journal domain (as classified in Ulrichsweb)
 patient characteristics (health condition, race, age)
 provider characteristics (medical practice, age)
 caregiver characteristics (relationship to patient, race, age)
 barriers to PHR use associated with each work system (patients,
providers, and caregivers)
Because the studies reviewed implemented a range of

Table 1
Search terms.

Keywords for PHR Patient facing, personal health record, personal health records, patient internet portal, patient internet portals, patient portal, patient portals, patient
held record, patient held records, patient accessible electronic medical record, personal medical record, personal medical records, electronic patient
portal, electronic patient portals, electronic portal, personally controlled health record, personally controlled health records, electronic patient record,
electronic patient records, web-based personal health record, web-based personal health records, personal electronic health record, personal electronic
health records, web-based patient portal, web-based patient portals, patient web portal, patient web portals, computerized patient record,
computerized patient records, patient accessible electronic health record, and patient accessible electronic health records
Keywords for barrier*, adopt*, satisfaction, implement*, facilitat*, challenge*, accept*, dissem*, use, usefulness, useful, attitude*, belief*, advantage*, and
barrier disadvantage*
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242
qualitative, descriptive, and mixed method approaches, statistical
significance could not be reported for all barriers. However, for
studies reporting statistical significance, only statistically signifi-
cant barriers were extracted for analysis.
Karsh and colleagues' HF/E paradigm specifies four work system
levels (see Table 2) (Karsh et al., 2006). The original model refers to
individual factors as patient and provider factors. However, because
we extended the model to caregivers, we refer to this work system
level as individual factors. Individual factors are characteristics of
the person (e.g., weight, personality). Work system/unit factors are
tasks performed (e.g., searching for health information) and the
tools and technology used to accomplish these tasks (e.g., search
engines). Organization factors are characteristics of the home and
community (patient- and caregiver-focused) or healthcare institu-
tion (provider focused). External environment factors are charac-
teristics of an individual's outside environment (e.g., clinics
[caregiver- and patient-focused] or external clinics [provider-
focused]). As noted, organization and external environment fac-
tors depend on the population being assessed.
Data were entered into Microsoft Excel (Microsoft Office
Professional, 2010) and analyzed using qualitative content anal-
ysis methods (Graneheim and Lundman, 2004; Sandelowski, 2010;
Sandelowski, 2000; Hsieh and Shannon, 2005). Using the HF/E
paradigm as an overarching framework, barriers were coded both
inductively and deductively (see Fig. 1). Deductive coding matched
barriers to existing elements of the framework; however, when a
quote did not fit an existing category, we created a new code (i.e.,
inductive coding). Simultaneous coding was used when content
extracted from an article reflected more than one factor and/or
subfactor. Two authors (MJT & JDR) independently coded all
extracted data. Decisions were discussed and discrepancies
resolved through consensus-building. Unresolved differences were
referred to the senior author (RSV).
3. Results
3.1. Search results
The search, using PubMed, CINAHL, Engineering Village (Com-
pendex and INSPEC), IEEE Xplor, and ACM Digital Library, produced
a total of 4865 records. After title, abstract, and full-text screening,
60 articles remained for data extraction and synthesis (see Fig. 2).
Table 2
HF/E paradigm factors and subfactors (Karsh et al., 2006).
Factor
Individual factors
Work system/unit factors
Organization factors
External environment
221
3.2. Study characteristics
Included studies employed a range of approaches for under-
standing barriers to PHR use (see Appendix A). Forty-five articles
focused exclusively on patient work systems (McInnes et al., 2013;
Tjora et al., 2005; Ancker et al., 2011; Burke et al., 2010; Day and Gu,
2012; Goel et al., 2011; Gu and Day, 2013; Guy et al., 2012; Hess
et al., 2007; Kahn et al., 2010; Kim et al., 2009; Krist et al., 2011;
Lau et al., 2013a, 2013b, 2013c; Lober et al., 2006; Mayberry et al.,
2011; Miller et al., 2007; Nagykaldi et al., 2012; Nielsen et al.,
2012; Osborn et al., 2013; Sarkar et al., 2010, 2011; Schnipper et
al., 2008; Tsai et al., 2012; Tuil et al., 2006; Vodicka et al., 2013;
Wade-Vuturo et al., 2013; Wagner et al., 2012, 2010; Wang et al.,
2004; Weingart et al., 2006; Wen et al., 2010; Wiljer et al., 2010;
Zickmund et al., 2008; Zulman et al., 2011; Emani et al., 2012; Nazi,
2010; Nazi et al., 2013; Wald et al., 2009; Tenforde et al., 2012;
Denton, 2001; Goldner et al., 2013; Gordon et al., 2012; Lin et al.,
2005), three exclusively on provider work systems (Crotty et al.,
2013; Wynia et al., 2011; Fuji et al., 2008), two exclusively on
caregiver work systems (Byczkowski et al., 2014; Britto et al., 2013),
seven on patient and provider work systems (Nazi, 2013; Urowitz
et al., 2012; Wald et al., 2010; Earnest et al., 2004; Jung et al.,
2011; Do et al., 2011; Poon et al., 2007), two on patient and care-
giver work systems (Tom et al., 2012; Weitzman et al., 2012), and
one on patient, provider, and caregiver work systems (Woods et al.,
2013). Sample sizes ranged from 10 to 100,617. A majority reported
barriers to using tethered systems (n ¼ 47), were based in the
United States (n ¼ 50), and were classified in the medical sciences
journal domain (n ¼ 54). Over half (n ¼ 37) used quantitative de-
signs; the remaining studies used mixed methods (n ¼ 15) or
qualitative designs (n ¼ 8). Data collection included a variety of
instruments, with most employing surveys (n ¼ 41).
Participant demographic characteristics varied across the studies
(see Appendix A). Of 28 reporting patient health conditions, diabetes
was the most frequent (n ¼ 14). Twenty-nine studies reported
patients' average age, with only two reporting an average age less
than 40. Thirty-one reported patients' race, most frequently Cauca-
sian (n ¼ 31) or African American (n ¼ 16). Thirteen reported patient
participants of Hispanic/Latino ethnicity. Five studies focusing on
provider work systems reported the type of medical practice, with
two relating to primary care. Three reported provider participants'
average age (ranging from 41 to 51 years). Three studies focusing on
Subfactor
Skills, knowledge, training, education
Size, weight, reach, strength
Age, gender, ethnicity, language
Needs, biases, beliefs, mood
Task demands, complexity, difficulty
Time and sequence demands
Availability of usable technology
Technology functions/features
Noise, temperature, lighting
Physical layout and geography
Organizational policy/priorities
Organizational structure
Financial resources
Rewards structure
Management structure
Training provided
Staffing levels
Social norms and pressures
Social climate/culture
Extra-organizational rules, standards, legislation enforcement
Industry social influence
Industry workforce characteristics
222 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242
Fig. 1. Coding example.
caregiver work systems reported the relationship between the
caregiver and patient, all of which were parents or guardians. Only
one study reported caregivers' race (primarily Caucasian) and only
one reported caregivers' average age (36 years).
3.3. HF/E paradigm
In total, work system barriers were classified under 19 of the 22
subfactors described by Karsh and colleagues and were noted at all
levels of the work system. Barriers to both patient and provider use
were categorized in 16 of the 22 subfactors, while barriers to
caregiver use were categorized in eight. In addition to the sub-
factors identified from the HF/E paradigm, one new subfactor,
behavior, was identified for patient, provider, and caregiver work
system barriers. Thus, the total number of subfactors identified for
each population was 17, 17, and 9, respectively. Fig. 3 lists the
subfactors identified for each population, illustrating subfactors
unique to specific populations and those shared by multiple pop-
ulations. None were identified as unique to caregivers.
3.4. Patient work system barriers
A majority of articles (n ¼ 55) reported patient work system
barriers to using PHRs in at least one of the four HF/E paradigm
levels: individual (n ¼ 46), work system/unit (n ¼ 39), organization
(n ¼ 20), and external environment factors (n ¼ 25). All examples
provided in the Results section are either direct quotes from the
authors of an article or direct quotes from participants in qualitative
studies. Barriers within patient work systems were identified for
four subfactors related to individual factors (see Table 3 and
Appendix A). Patients' skills, employment, knowledge, training, and
education, such as awareness of system features and computer lit-
eracy were recognized as barriers to PHR use. Patients' de-
mographics such as age, gender, ethnicity, language, and marital
status were described as barriers. Barriers related to patients' needs,
biases, beliefs, and mood focused on perceived PHR value, degree of
need or existence of alternative technologies, and communication
preferences. Patients' behavior was acknowledged as a barrier,
including health service utilization, practice of a healthy lifestyle,
and norms for technology use and communication.
Barriers within patient work systems were identified for five
subfactors related to work system/unit factors (see Table 4 and
Appendix A). Barriers related to system task demands, complexity
and difficulty related to issues encountered when using the PHR,
such as unintuitive navigation and password recovery. Barriers to
patient use related to time and sequence demands focused on
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 223

Fig. 2. Flow diagram for search results.

provider and system response times and the PHRs' fit with patients'
everyday routines. Availability of usable technology including
Internet, computer, and smartphone access, was described as a
barrier to patients' use. Technology functions and features was the
most frequently reported work system/unit subfactor and posed
barriers to patients because of challenging content (e.g., difficult
terminology), inadequate privacy and security settings, missing
functionality, and limited technology platform options. Barriers
pertaining to physical layout and geography focused on the location
where the patient lived and whether the patient used the system
while hospitalized or after discharge.
Barriers within patient work systems were identified for five
subfactors related to organization factors (see Table 5 and Appendix
A). Barriers to patient use related to organizational policy/priorities
focused on whether members of their home and community (e.g.,
family, friends) would have access to patients' PHRs. Organizational
structure, or home and community members' presence at doctors'
appointments and in the household, was recognized as a barrier.

Fig. 3. Subfactors identified for each population.

224 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Table 3
Barriers to patient use related to individual factors.

Subfactor Definition Example

Patient: skills, employment, knowledge, Patients' PHR use reflected Compared to those with a college degree, those with a lower educational attainment were more
training, and education (n ¼ 22) individuals' understanding. likely to never have logged on (Sarkar et al., 2011).
Patient: age, gender, ethnicity, language, Patients' PHR use reflected There were marked race/ethnic differences in use, with African American (31%), Latino (34%),
and marital status (n ¼ 20) individuals' demographic groups. and Filipino (32%) participants least likely, and Asian (53%) and White (51%) participants most
likely to both request a password for the internet-based patient portal (a marker for internet
access and intent to use) and log on to the portal after requesting a password was completed
(an early marker for navigability once access is obtained) (Sarkar et al., 2011).
Patient: needs, biases, beliefs, and mood Patients' PHR use reflected There was a significant difference (p < 0.0001) in the number of medications prescribed by
(n ¼ 33) individuals' mindset. multiple sclerosis staff between PatientSite users (M ¼ 4.61, SD ¼ 2.57) and PatientSite non-
users (M ¼ 2.49, SD ¼ 2.10) (Nielsen et al., 2012).
Patient: behavior (n ¼ 24) Patients' PHR use reflected ‘When you use such Net-based systems that have nothing to do with your email account, you
individuals' actions. have to access it separately. And I read so much email for the rest of the day or do so many other
things, that to log on to check if I have had a response todaydI don't bother. Then, it is much
better to use an email account that I use on a daily basis.’ These problems have led many
patients to use ordinary email instead of MedAxess (Tjora et al., 2005).

Available financial resources including household income, insurance
status, and willingness to pay for PHR service, were identified as
barriers. Staffing levels, or the availability of home and community
members to assist patients with using PHRs, was categorized as a
barrier. Barriers related to social climate/culture focused on home
and community members' support, health status, and education.
Barriers within patient work systems were identified for three
subfactors related to external environment factors (see Table 6 and
Appendix A). Extra-organizational rules, standards, and legislation
enforcement, such as third party access to PHRs, PHRs' interopera-
bility with EHRs, and the ability/inability to contact multiple pro-
viders, were recognized as barriers. Patient barriers regarding
industry social influences focused on patienteprovider relationships
and the value of the system to providers. Industry workforce char-
acteristics, such as the degree of involvement from the healthcare
team and issues related to provider work (e.g., difficulty responding
to patient inquiries, inconsistencies in provider notes, and un-
availability of clinical support), also presented barriers to patients.
3.5. Provider work system barriers
Ten articles reported provider-related barriers to the use of PHRs
on at least one of the four HF/E paradigm levels: individual (n ¼ 7),
work system/unit (n ¼ 9), organization (n ¼ 6), and external envi-
ronment factors (n ¼ 4) (see Table 7 and Appendix A). Barriers
within provider work systems were identified for four subfactors
related to individual factors. Providers' skills, employment, knowl-
edge, training, and education, including awareness of PHRs and
provider specialty, were described as barriers to providers' PHR use.
Within the subfactor of age, gender, ethnicity, language, and marital
status only gender was identified as a barrier to PHR use. Providers'
needs, biases, beliefs, and mood, such as perceived PHR value, degree
to which an alternative technology solution was perceived as
feasible or appropriate, and beliefs regarding patient self-care, were
described as barriers. Behavior, including use of EHRs, degree of
engagement with PHRs, and prior use of alternative forms of
technology, was also recognized as a barrier.
Barriers within provider work systems were identified for five
subfactors related to work system/unit factors (see Table 8 and
Appendix A). Barriers related to system task demands, complexity and
difficulty focused on concerns regarding patients' expectations and
issues encountered while using the PHR (e.g., cumbersome system
interface and navigation). Time and sequence demands, such as time
allotted to patient care, workload, timing of information access, and
time required to use and learn the system, were identified as barriers.
The availability of usable technology, specifically the lack of patient-
accessible computers in waiting rooms, posed barriers to pro-
viders. Technology functions and features including accuracy of and
patient access to sensitive information, missing functions, privacy,
awkward security procedures, and general technical issues, were
described as barriers. Barriers related to physical layout and geogra-
phy focused on the clinic setting (e.g., rural or urban settings).

Table 4
Barriers to patient use related to work system/unit factors.

Subfactor Definition Example

Patient: task demands, Patients' PHR use reflected individuals' ability Patients thought the terminology was sometimes difficult to understand and would require
complexity and difficulty to complete related work. help from the healthcare team. ‘[You] have to really read it closely and if you don't know
(n ¼ 22) much about terminology … it could be real confusing’ (Wagner et al., 2010).
Patient: time and sequence Patients' PHR use reflected the temporal Another participant expressed dissatisfaction after he sent a message to his provider about a
demands (n ¼ 14) length and order of completing related work. medication side effect, and he did not get a response within a reasonable time frame. The
consequences of this were threefold: (1) the participant adjusted his medication without
provider input, (2) the participant now relies on more traditional forms of communication
(e.g., a phone call or an office visit); and (3) the participant has been unsatisfied with his care
(Nazi, 2013).
Patient: availability of usable Patients' PHR use reflected individuals' access Some access concerns were identified through patient comments, such as: ‘Internet
technology (n ¼ 9) to electronic resources. Explorer as the only option for access is very restrictive;’ and ‘Not sure I'd be able to do [the
journal] at home [if I didn't have] a job with access to the internet’ (Wald et al., 2009).
Patient: technology functions Patients' PHR use reflected characteristics of More than one half of the respondents (62%) wanted to grant PHR access to their spouse or
and features (n ¼ 28) the electronic resources. partner [but are unable to do so], and a smaller percentage wanted to grant access to a child
(23%), other family member (15%), unrelated caregiver (7%), or friend or neighbor (2%)
(Zulman et al., 2011).
Patient: physical layout and Patients' PHR use reflected individuals' Utilisation patterns showed that users accessed their patient-accessible electronic health
geography (n ¼ 2) location. records significantly more often while the patients were in hospital (median of 6 logins)
than after discharge (median of 4 logins, p < 0.001) (Burke et al., 2010).
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 225

Table 5
Barriers to patient use related to organization factors.

Subfactor Definitions Example

Patient: organizational Patients' PHR use reflected established practices for For example, respondents were more interested in sharing access to medication lists,
policy/priorities home and community members' PHR use. appointment information, and laboratory and test results with their designee than patient-
(n ¼ 1) entered health information or communications with providers. Respondents were similarly
more interested in delegating prescription refill requests and appointment scheduling than
in having the designee communicate with their health care provider … Although
respondents tended to be most interested in sharing information with family members
(especially a spouse or partner), they expressed high levels of interest in allowing unrelated
caregivers to conduct activities in their PHR, such as requesting prescription refills or
scheduling appointments (Zulman et al., 2011).
Patient: organizational Patients' PHR use reflected home and community A common way participants learned about the medical center's PWP [patient web portal]
structure (n ¼ 2) members' structure and participation. was through a knowledgeable family member: ‘My daughter showed [the PWP] to me in my
doctor's office, on the computer in the waiting room. No one in the doctor's office ever
approached me about it. If it wasn't for my daughter, I wouldn't be a [PWP] user’ (Mayberry
et al., 2011).
Patient: financial Patients' PHR use reflected available monetary MHAV users were more likely than nonusers to be Caucasian/white, have higher incomes,
resources (n ¼ 13) resources within the home and community. and be privately insured (Osborn et al., 2013).
Patient: staffing levels Patients' PHR use reflected the availability of home The system was most frequently used on Thursdays (67%, 5387/8008), followed by Fridays
(n ¼ 2) and community members. (14%, 1098/8008), which coincided with the onsite availability of graduate nursing students.
Most (77%, 6174/8008) of the system use happened while assistance from graduate nursing
students or housing staff was available to the residents. On the other hand, 8% (677/8008) of
user activities occurred during off hours when the students or staff were not available (from
5:00 pm to 8:00 am weekdays and weekends) (Kim et al., 2009).
Patient: social climate/ Patients' PHR use reflected home and community Personal health record users were younger, more likely to have commercial insurance,
culture (n ¼ 5) members' values, experiences, and practices. identify as Caucasian, have higher household incomes, and live in a region with higher rates
of high school completion compared to MyChart non-users (Tenforde et al., 2012).

Table 6
Barriers to patient use related to external environment factors.

Subfactor Definition Example

Patient: extra-organizational rules,
standards, legislation enforcement
(n ¼ 15)
Patient: industry social influences
(n ¼ 10)
Patient: industry workforce
characteristics (n ¼ 11)
Patients' PHR use reflected policies
surrounding the healthcare system.
Patients' PHR use reflected
individuals' interactions with
providers.
Patients' PHR use reflected aspects of Although communication seemed to occur primarily with a nurse, dietitian, or other AHP
providers' work.
However, some patients were uncertain about a potential misuse of information
transmitted through MedAxess. Also, the fact that communication is logged and stored in a
database made the situation quite different from that of, for instance, telephone
conversations. If such written communication is stored for a very long time, it is difficult to
foresee who will have access to the information in years to come (Tjora et al., 2005).
Fear of losing relationships: Some participants valued their choices of how to
communicate, and were concerned that the portal might cause them to lose those choices
(Zickmund et al., 2008).
via the portal, patients often wished their physician had taken more of an interest in the
program and had reviewed the information they had entered on the portal during their
clinic visits (this was also largely done by AHPs). Responses revealed that it would have
been beneficial if a health care provider had referred them to information in the Health
Library. The responses typically reflected a widespread notion that physicians were often
busy and may be unable to fulfill this role as much as they would have liked (Urowitz et al.,
2012).

Table 7
Barriers to provider use related to individual factors.

Subfactor Definition Example

Provider: skills, employment,
knowledge, training, education
(n ¼ 4)
Provider: age, gender, ethnicity,
language, marital status (n ¼ 1)
Provider: needs, biases, beliefs, mood Providers' PHR use reflected
(n ¼ 3)
Provider: behavior (n ¼ 5)
Providers' PHR use reflected
individuals' understanding.
Providers' PHR use reflected
individuals' demographic groups. that none of their patients use PHRs (c2 ¼ 13.846, p ¼ 0.000). Yet, more male physicians than
individuals' mindset.
Providers' PHR use reflected
individuals' actions.
Many health care professionals reported general awareness of My HealtheVet but limited
familiarity with its features, with the exception of secure messaging. Health care professionals
note that this lack of knowledge limits their ability to endorse patient use, or to integrate use of
My HealtheVet features within the clinical practice setting (Nazi, 2013).
A greater number of male physicians (32.9 percent of males vs. 20.4 percent of females) perceive
female physicians reported using the patient's PHR information (6 percent vs. 2.8 percent,
c2 ¼ 3.780, p ¼ 0.052), having a member of their staff work with the patients and their PHRs (5.2
percent vs. 1.2 percent, c2 ¼ 7.537, p ¼ 0.006), and being capable of electronically integrating
PHR information into their own EHR (3.5 percent vs. 2.8 percent) (Fuji et al., 2008).
Providers commonly viewed patients' interactions with the portal positively and their own
interaction negatively. Negative comments typically concerned time constraints and technical
barriers. There were instances where providers indicated that they believed the portal may be
more beneficial for patient self-education than for significant provider usage (Urowitz et al.,
2012).
In general, health care professionals reported limited experiences with patient use (and their
own use) of My HealtheVet health education resources, tools to support medication
reconciliation, and tools to support patient self-reported data (with some exceptions), often
using alternative tools and resources instead (Nazi, 2013).
226 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Table 8
Barriers to provider use related to work system/unit factors.

Subfactor Definition Example

Provider: task demands, Providers' PHR use reflected individuals' ability ‘The system is cumbersome and needs an interface that addresses the needs of patients and
complexity difficulty to complete related work. data entry requirements’ (Urowitz et al., 2012).
(n ¼ 2)
Provider: time and sequence Providers' PHR use reflected the temporal Providers believed that accessing patient information was time consuming and sometimes
demands (n ¼ 5) length and order of completing related work. redundant (e.g., due to manual data entry) (Urowitz et al., 2012).
Provider: availability of Providers' PHR use reflected individuals' access
Several factors have inhibited the My HealtheVet PHR adoption, use, and endorsement of
usable technology (n ¼ 2) to electronic resources. patient use: Lack of alignment with structures (e.g., lack of patient-accessible computers in
the clinic setting) (Urowitz et al., 2012).
Provider: technology Providers' PHR use reflected characteristics of Additional topics suggested [functionality to be included in the PHR] were social history
functions/features (n ¼ 8) the electronic resources. update (79%), information about recent appointments with other providers (59%), reason for
visit (34%), office questionnaires (31%), and additional screening questions (24%) (Wald et al.,
2010).
Provider: physical layout Providers' PHR use reflected individuals' The characteristics of users largely tracked those of physicians using electronic records in
and geography (n ¼ 2) location. general; for example, they were more often urban, in group practices or hospital employed,
and in noneprimary care specialties (Wynia et al., 2011).

Barriers within provider work systems were identified for six
subfactors related to organization factors (see Table 9 and Appendix
A). Organizational policy/priorities, such as sensitive information per-
missions and lack of alignment with provider workflow, were
recognized as barriers to providers' use. Barriers related to financial
resources focused on the cost-benefit of using the system. Staffing
levels, or the availability of healthcare institution staff to work with
patients on PHRs, was described as a barrier. Providers had concerns
regarding the rewards structure and whether or not they would be
appropriately reimbursed for time spent reviewing PHRs. Inadequate
training provided to healthcare institution staff and patients posed
barriers to providers. Barriers associated with social norms and pres-
sures included the healthcare institution staffs' technology use norms.
Barriers within provider work systems were identified for two
subfactors related to external environment factors (see Table 10 and
Appendix A). Extra-organizational rules, standards, and legislation
enforcement, such as patient access to provider entered data and
provider liability for all information within a patient's PHR, posed
barriers to providers' use. Barriers related to industry social in-
fluences focused on the amount of time spent with patients and
patients' expectations and assumptions (e.g., providers actively
monitoring patients' health status on the portal).
3.6. Caregiver work system barriers
Five articles reported caregiver-related barriers to the use of
PHRs on at least one of the four HF/E paradigm levels: individual
(n ¼ 4), work system/unit (n ¼ 4), organization (n ¼ 1), and external
environment factors (n ¼ 1) (see Table 11 and Appendix A). Barriers
within caregiver work systems were identified for three subfactors
related to individual factors. Caregivers' skills, employment, knowl-
edge, training, and education, including system awareness and
experience using the Internet, were recognized as barriers to
caregivers' use. Caregivers' needs, biases, beliefs, and mood,
including the perceived system value, degree of need, comfort
sharing medical information on the Internet, and communication
preferences, posed barriers. Caregivers' behavior included the
likelihood of receiving information from other sources and was
described as a barrier.
Barriers within caregiver work systems were identified for four
subfactors related to work system/unit factors (see Table 12 and
Appendix A). Task demands, complexity and difficulty, such as the
ease with which caregivers could access and use the system, were
recognized as barriers. Barriers related to time and sequence de-
mands focused on provider and system response times, Internet

Table 9
Barriers to provider use related to organization factors.

Subfactor Definition Example

Provider: organizational Providers' PHR use reflected established practices Several factors have inhibited the My HealtheVet PHR adoption, use, and endorsement of
policy/priorities (n ¼ 2) for healthcare institutions' PHR use. patient use: Lack of alignment with workflow (e.g., lack of integration with the primary
clinical information system), Lack of alignment with processes (e.g., barriers to information
flow) (Nazi, 2013).
Provider: financial Providers' PHR use reflected available monetary Each [participant] questioned whether the merits of the intervention would warrant the
resources (n ¼ 1) resources within the healthcare institution. resources spent on it (Earnest et al., 2004).
Provider: rewards Providers' PHR use reflected healthcare There was particular concern (greater than 70 percent) about both unintentional and
structure (n ¼ 2) institutions' incentives for PHR use. intentional data inaccuracies in these records. There were also concerns about privacy, lack of
reimbursement for time spent reviewing them, and liability for knowing all of the
information in a patient's personal health record (Wynia et al., 2011).
Provider: training Providers' PHR use reflected healthcare Provider responses revealed that neither they nor the majority of their patients were able to
provided (n ¼ 1) institutions' instruction in PHR use. use the portal easily. More training and improved portal usability testing were said to be
needed for the portal to be used more effectively. Issues with specific features such as the
display of health indicators and with reading weight and exercise values were mentioned
less by respondents (Urowitz et al., 2012).
Provider: staffing levels Providers' PHR use reflected the availability of 6.5 percent of users reported that a member of their staff works with patient PHRs. These
(n ¼ 1) healthcare institution staff. numbers were even lower for planners (3.9 percent) and nonplanners (1.2 percent) (Fuji
et al., 2008).
Provider: social norms Providers' PHR use reflected the healthcare Health care professionals often reported using alternative tools and resources. For example,
and pressures (n ¼ 1) institutions' technology use norms. although My HealtheVet provides a significant library of health education resources, health
care professionals already use alternative resources, such as subscription-based software
that is linked from within the primary clinical workflow system or resources retrieved from
the Internet, with little incentive to change. Health care professionals reported that they
increasingly use Internet resources easily found by search engines, and speculated that
patients do as well. As one health care provider said: ‘Why not just Google?’ (Nazi, 2013).
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 227

Table 10
Barriers to provider use related to external environment factors.

Subfactor Definition Example

Provider: extra-organizational rules,
standards, legislation enforcement
(n ¼ 2)
Provider: industry social influences
(n ¼ 2)
Providers' PHR use reflected the policies
surrounding the larger (beyond own)
healthcare system.
Providers' PHR use reflected individuals'
interactions with patients.
There was particular concern (greater than 70 percent) about both unintentional
and intentional data inaccuracies in these records. There were also concerns about
privacy, lack of reimbursement for time spent reviewing them, and liability for
knowing all of the information in a patient's personal health record (Wynia et al.,
2011).
The messaging system is great yet can be utilized negatively by patients increasing
workload on mydoctor.ca and decreasing time for other patient interactions in
office. The messaging system has also increased expectations from patients for
immediate response (Urowitz et al., 2012).

speed, and general time constraints. The availability of usable
technology, including computer and Internet access, posed barriers
to caregivers. Technology functions and features were described as
barriers due to the lack of specific functions or features and con-
cerns such as accessing frightening information.
Barriers within caregiver work systems were identified for one
subfactor related to organization factors (see Table 13 and
Appendix A). Financial resources, or the type of insurance caregivers
have for their child, were recognized as barriers to caregivers.
Barriers within caregiver work systems were identified for one
subfactor related to external environment factors (see Table 14 and
Appendix A). Extra-organizational rules, standards, and legislation
enforcement, or the sharing of data with external parties (e.g., gov-
ernment agencies, outside providers), posed barriers to caregivers.
4. Discussion
4.1. Application of HF/E paradigm to barrier classification
Overall, the HF/E paradigm was comprehensive of the barriers
identified in this review. Three subfactors included in that paradigm
were not relevant and one additional subfactor was inductively
derived. Size, weight, reach, and strength (individual factor), noise,
temperature, and lighting (work system/unit factor), and management
structure (organization factor) were included in the HF/E paradigm
but not recognized as barriers in this review. The salience of these
work system factors has also been identified in previous literature.
For example, Smith and Sainfort (Smith and Sainfort,1989) described
factors similar to size, weight, reach and strength, noise, temperature
and lighting, and management structure as influential in work system
models focusing on stress reduction. These factors were similarly
specified within the SEIPS 2.0 model (Holden et al., 2013), which
focuses on provider, patient, and collaborative work systems. Addi-
tional work is needed to empirically determine if these theoretically
identified work system factors serve as barriers to PHR use. Addi-
tionally, despite the applicability of the HF/E paradigm, one new
subfactor was identified, behavior, which has been previously
described as a germane work system factor in other domains,
including stress reduction (Smith and Sainfort, 1989), computer se-
curity (Carayon, 2006), and self-care (Holden et al., 2015).

Table 11
Barriers to caregiver use related to individual factors.

Subfactor Definition Example

Caregiver: skills, knowledge,
training, education (n ¼ 2)
Caregiver: needs, biases, beliefs,
mood (n ¼ 4)
Caregiver: behavior (n ¼ 1)
Caregivers' PHR use reflected
individuals' understanding.
Caregivers' PHR use reflected
individuals' mindset.
Caregivers' PHR use reflected
individuals' actions.
Users (versus nonusers) were more likely to have commercial insurance for their child (97% vs. 73%,
P,.001) and have at least a 4-year college degree (74% vs. 50%, P < .001) (Tom et al., 2012).
Only 2 percent of the parents agreed that they sometimes saw information they wish they had not
seen. On the contrary, 12 percent agreed that they sometimes saw information in the portal that
frightened them, and 11 percent reported that they sometimes see information that they would have
preferred to get directly from their provider (Byczkowski et al., 2014).
Parents receive disease-related information through other avenues (n ¼ 3 of 15 nonusers)
(Byczkowski et al., 2014).

Table 12
Barriers to caregiver use related to work system/unit factors.

Subfactor Definition Example

Caregiver: task demands, Caregivers' PHR use reflected individuals' Make it easier to access the web site and log-in (n ¼ 18 of 126 respondents) (Tom et al.,
complexity difficulty ability to complete related work. 2012).
(n ¼ 2)
Caregiver: time and Caregivers' PHR use reflected the temporal Faster information (e.g., quick email responses, updates from clinic visits) (n ¼ 9 of 126
sequence demands (n ¼ 2) length and order of completing related work. respondents) (Tom et al., 2012).
Caregiver: availability of Caregivers' PHR use reflected individuals' ‘I don't have access to the Internet,’ ‘No high-speed Internet,’ ‘I don't have a computer’ (Tom
usable technology (n ¼ 2) access to electronic resources. et al., 2012).
Caregiver: technology Caregivers' PHR use reflected characteristics of Statements illustrated challenges stemming from viewing newly revealed information that
functions/features (n ¼ 3) the electronic resources. had not previously been disclosed to patients. One participant, a wife of a patient, expressed
stress upon seeing an operative report; when asked if reading such notes was harmful, she
denied harm had ensued: ‘I would rather not have known. There was a lot of little things
they wrote, you know, step-by-step what had happened in his operation’ (Woods et al.,
2013).

Table 13
Barriers to caregiver use related to organization factors.

Subfactor Definition Example

Caregiver: financial Caregivers' PHR use reflected available monetary Users (versus nonusers) were more likely to have commercial insurance for their child (97% vs.
resources (n ¼ 1) resources within the home and community. 73%, P < .001) and have at least a 4-year college degree (74% vs. 50%, P < .001) (Tom et al., 2012).
228 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242
Table 14
Barriers to caregiver use related to external environment factors.
Subfactor Definition
Extra-organizational rules, Caregivers' PHR use reflected policies
standards, legislation enforcement surrounding the healthcare system.
(n ¼ 1)
Overall, relatively limited attention to higher work system levels
was demonstrated. A majority of the articles found barriers related
to individual factors (n ¼ 51) and work system/unit factors (n ¼ 45);
however, less than half found barriers related to organization factors
(n ¼ 26) and external environment factors (n ¼ 27). More specif-
ically needs, biases, beliefs, and mood (individual factor) (n ¼ 35) and
technology functions and features (work system/unit factor) (n ¼ 32)
were the most frequently recognized subfactors. This may be partly
due to the familiarity and widespread use of models such as the
Technology Acceptance Model (TAM) (Holden and Karsh, 2010;
Legris et al., 2003) and its derivatives, which emphasize the per-
sonetechnology interaction and focus less on broader contexts.
There is a clear need to apply HF/E models and approaches that
attend to higher levels of the work system. Identifying and un-
derstanding barriers at higher levels will require complementing
traditional lab-based and task-oriented HF/E approaches with field-
based approaches focused on the context of use. This will likely
require HF/E professionals to include qualitative and mixed
methods approaches (Desnoyers, 2004; Hancock and Scalma, 2004;
Hignett and Wilson, 2004; Moray, 2000; Carayon et al., 2015). Using
participatory design methods (Holden et al., 2013; Valdez et al.,
2015b) that engage designers in patients', caregivers', and pro-
viders' naturalistic environments of use and reveal their percep-
tions of roles played by environmental factors (e.g., through
interviews or focus groups) (Holden et al., 2015; Thompson and
Valdez, 2013; Ancker et al., 2015) will be imperative.
4.2. Application of the HF/E paradigm to caregivers
Although Karsh and colleagues' HF/E paradigm was originally
formulated for patients and providers, we found it feasible to apply
it to caregivers. As with patients and providers, it was possible to
map all identified barriers, except individual behaviors, to elements
of the paradigm. However, few studies examined caregiver barriers,
despite growing recognition within the HF/E and medical infor-
matics communities of the critical role these individuals play (Tjora
et al., 2005; Skeels, 2010; Chappell and Reid, 2002). As a likely
consequence, barriers relevant to patients and providers centered
on almost twice as many subfactors as those relevant to caregivers.
As previously mentioned, the review found limited research
focusing on organization and external environment factors. This
general trend also held for caregivers; only two subfactors related
to these phenomena were identified (see Fig. 3), despite evidence of
the organizational and environmental challenges faced by care-
givers (Werner et al., 2012; Chen et al., 2015; Streid et al., 2014). As
encouraged by the Patient Engagement Framework (National
eHealth Collaborative and Healthcare Information and
Management Systems Society Foundation, 2014), future research
should include a focus on caregivers, specifically challenges they
face at the organization and environmental levels.
4.3. Representation of individuals in the work system
Sociotechnical systems, including those specific to healthcare,
comprise multiple individuals (Holden et al., 2013; Vincent, 2003).
This review presents three distinct groups of individuals involved
in patient care (i.e., patients, providers, caregivers). Although all
may benefit from PHR use, this review demonstrates that each
Example
The most common objections to sharing data with an outside provider were relevance to
patient care and the potential for discrimination by insurance companies (Weitzman et al.,
2012).
group has unique roles and is embedded in a different context.
Consequently, work system models that explicitly represent the
roles, contexts, and interactions of all actors jointly involved in the
processes and outcomes of patient care are needed. This recom-
mendation contrasts with existing work system models which,
while theoretically enabling simultaneous representation of mul-
tiple individuals, do not distinguish between nor represent their
overlapping vs. unique individual, work system/unit, organization,
and external environment characteristics (Valdez et al. 2015). The
phenomenon of overlapping work system factors existing at
different levels is clearly illustrated here. For example, the degree of
value providers find in PHRs is an individual factor for provider
work systems (i.e., needs, biases, beliefs, and mood), but an external
environment factor for patient work systems (i.e., industry social
influences). Thus, sophisticated representations of work systems
that account for intersections between what may, with less fidelity,
be represented as multiple, separate work systems, are needed.
4.4. Implications for PHR design
Although we did not conduct a meta-analysis to evaluate the
strength of the evidence, we did observe several repeatedly re-
ported barriers. For example, level of education, degree of health
and computer literacy, and access to technologies were recurring
themes among barriers to patient and caregiver PHR use. According
to an Institute of Medicine (IOM) report (2009) (Institute of
Medicine, 2009), greater disparities exist among underserved
populations, many of which experience these barriers. The National
Library of Medicine (NLM), the National Institute on Aging (NIA),
and the Office of the National Coordinator for Health Information
Technologies (ONC) have prioritized development of consumer
health IT aligned with laypersons' health management skills and
health literacy (Alper et al., 2015; National Institute on Aging,
2013e2016; National Institute on Aging, 2010e2013a, 2013b). To
prevent the growth of health and healthcare disparities caused by
barriers to use, HF/E professionals must engage directly with un-
derserved and elderly populations, ensuring that the next genera-
tion of consumer health IT is accessible and usable across
demographics. Innovative solutions may be needed that focus not
only on access through individually owned devices and text-based
content, but also through community owned devices and voice-
based content (Wells et al., 2015).
Another recurring theme was fear of accessing unwanted or
frightening information (e.g., test results). To our knowledge, no
policies exist that protect patients and caregivers from seeing such
information. The absence of such policies may be partly due to a
focus on management and control of health information access by
others including providers, caregivers, and care partners (Sarkar
and Bates, 2014). However, although the intent of objective 2C of
the Federal Health IT Strategic Plan 2015e2020 (The Office of the
National Coordinator for Health Information Technology (ONC),
2014) is to protect the privacy and security of health information,
the strategies it contains may be relevant to ensuring individual
control over information access. For example, strategy 3 of objec-
tive 2C states, “Support the development of policies, standards,
technology, guidance, and solutions to facilitate individuals' ability
to manage, control, and authorize the disclosure of specific elec-
tronic health information.” This strategy could be extended to
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242
guidelines enabling individuals to manage, control, and authorize
their own access to specific electronic health information. The
ability to tailor personal access preferences will be necessary given
that other patients have expressed the desire for faster release (Do
et al., 2011) and greater accessibility (Krist et al., 2011) of test re-
sults. In other words, effort should be placed on allowing in-
dividuals to determine what information is presented in their PHRs
and when it is made available, although a minimum waiting period
under certain circumstances may be advisable (Do et al., 2011).
Moreover, because elements of the medical record may be difficult
for laypersons to understand, information access requires tools
facilitating understanding and decision-making (Wagner et al.,
2010; Do et al., 2011; Wiljer et al., 2010).
Patients, providers, and caregivers all commented on concerns
regarding secure messaging. Both patients and caregivers worried
that secure messaging could replace more personal modes of
communication (i.e., calling). Similarly, providers expressed con-
cerns that secure messaging would lead to increased workload and
lost face-to-face interaction with patients. Despite all three groups'
concerns, Meaningful Use Stage 2 (The Office of the National
Coordinator for Health Information Technology (ONC), 2013a)
measure 17 calls for patients' ability to electronically message their
providers. In design terms, this requires creating a solution to two
conflicting viewpoints. Patients' and caregivers' fear of losing per-
sonal modes of communication may be partly due to not under-
standing how or when to use secure messaging. Thus, providing
tutorials (Urowitz et al., 2012; Do et al., 2011) through the PHR
either via web link or an embedded video may help patients feel
more comfortable using another means to contact providers.
However, as the use of secure messaging increases (Cronin et al.,
2015), design solutions must also focus on lessening provider
workload. One potential solution may be to implement a triage-
based system that would route secure messages to appropriate
staff members (Katz et al., 2003). Another option is to limit the
number of characters allowed in messages, thus reducing the
amount of text that must be read (Ye et al., 2010) and guiding the
patient to use other forms of communication for in-depth inquiries.
Some barriers encountered in this review are well recognized
within the HF/E and medical informatics communities. One
frequently reported under work system/unit factors was that data
entry was a time consuming and difficult task with which some
patients needed assistance. This barrier may be addressed by tradi-
tional human factors methods (e.g., simplified user interface, ca-
pacity for multiple forms of data). These methods may be
supplemented by training, such as through tutorials on data entry
and retrieval. At a higher level of the work system, barriers related to
system boundaries and lack of interoperability were also recurring
themes. The need to address these barriers not only for PHRs, but also
for EHRs, has been widely recognized within the medical informatics
community and prioritized as a topic of research and intervention
(The Office of the National Coordinator for Health Information
Technology (ONC), 2014; Otte-Trojel et al., 2015; Brennan et al., 2015).
4.5. Designing interventions: attending to context
Identifying barriers should serve as a foundation for devising
design solutions, a key component of HF/E practice (Dul et al., 2012).
This review demonstrates that constructing solutions to improve
the experience of PHR use requires targeting barriers at all levels of
the work system. This means not only targeting the individual, the
task performed, and the technology used, but also the larger envi-
ronment in which these interactions are embedded (Holden et al.,
2013; National Research Council, 2011; Dul et al., 2012). Tradition-
ally, when attending to contextual factors, HF/E professionals have
focused on implications of these work system levels for technology
229
design (i.e., determining how to design technology to align with
contextual factors) (Valdez et al., 2015a; Marquard and Zayas-Caban,
2012; Moen and Brennan, 2005). Yet, as discussed during the recent
HFES annual meeting, we acknowledge that the scope of our pro-
fession should be broadened to determine how to design and
redesign these contextual environments. Current interventions to
increase provider PHR use within the United States include the
Centers for Medicare & Medicaid Services' (CMS) EHR Incentive
Program, which rewards healthcare providers who demonstrate
meaningful use (i.e., improved patient care outcomes through
meeting core objectives of Meaningful Use Stages 1, 2 and 3) with
incentive payments (Center for Medicare and Medicaid Services,
2015). HF/E professionals should collaborate on designing other
interventions at higher work system levels to address barriers to
PHR use for providers, patients, and caregivers. For example, in-
surance companies enact patient wellness programs, which incen-
tivize patients' participation in health-related activities such as
exercising or attending wellness checkups (Department of Health
and Human Services, 2013). Similar programs could address pa-
tient and caregiver barriers to PHR use.
4.6. Limitations and future research
The limitations of this review yield four directions for future
research. First, included articles were limited to studies conducted
in six countries, highlighting the need for work that explicitly ad-
dresses organization and external environment barriers arising in
other national contexts. Second, the coding strategy used in this
review focused on assigning barriers to specific work system factors.
Future research should build on this by studying interactions be-
tween barriers within and across works systems. Third, this review
is systematic in that specific criteria were used to determine article
inclusion; however, the authors did not assess the level of evidence
for the contribution of each work system factor to PHR use. This
latter should be the focus of future inquiry. Lastly, although we
consulted with one health science librarian and one computer sci-
ence librarian to assist in our literature search, we acknowledge that
developing an exhaustive search strategy is challenging. In retro-
spect, additional terms such as “impediment” and “obstacle” should
be incorporated into future systematic reviews on this topic.
5. Conclusion
This review focused on patient, provider, and caregiver work
system barriers to PHR use and is timely given developments
intended to accelerate PHR use such as the recent proposal of
Meaningful Use Stage 3 guidelines within the United States. The
results highlight the need to address how organization and external
environment factors hinder PHR use, as well as how such barriers
can be overcome. Moreover, simultaneous attention to patients,
providers, and caregivers lays the groundwork for developing new
work system models that explicitly account for the unique and
overlapping roles and contexts of distinct individuals within a
sociotechnical system. Future design directions should address low
health literacy, access to unwanted health information, secure
messaging impacts on provider workflow, secure messaging appeal
to patients and caregivers, ease of data entry, and interoperability. A
key next step is to synthesize the level of evidence for each type of
PHR use barrier for each work system to guide intervention design.
Acknowledgments
We would like to thank Kelly Near and Andrea Denton from the
University of Virginia's Library System for guiding and assisting
with our literature search.
230 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Appendix A

Study characteristics of included articles.

Author, year Study name Sample Sample Type of Instrumentation Methodology Study Journal domain
(reference) size PHR location

Ancker Use of an electronic patient portal among Patients 74,368 Tethered EHR patient Quantitative United Medical sciences e Internal
et al., disadvantaged populations data; PHR use States medicine
2011 logs
Britto et al., Parents' perceptions of a patient portal for Caregivers 24 Tethered Interview Qualitative United Medical sciences e Pediatrics
2013 managing their child's chronic illness States
Burke et al., Transforming patient and family access to Patients 252 Tethered EHR patient Quantitative United Medical sciences e
2010 medical information: Utilization patterns of a data; PHR States Cardiovascular diseases
patient-accessible electronic health record system logs
Byczkowski Family perceptions of the usability and value Caregivers 530 Tethered Interview; Mixed United Medical sciences e Computer
et al, of chronic disease web-based patient portals survey methods States applications/Nurses and
2014 nursing
Crotty et al., Preparing residents for future practice: Report Providers 108 Tethered Survey, PHR use Quantitative United Medical sciences
2013 of a curriculum for electronic patient-doctor logs States
communication providers
Day and Gu, Influencing factors for adopting personal Patients 10 Tethered Interview, QualitativeNew Medical sciences e Computer
2012 health record (PHR) observation Zealand applications
Denton, Will patients use electronic personal health Patients 136 Untethered Survey Quantitative United Health facilities and
2001 records? Responses from a real-life experience States administration
Do et al., The military health system's personal health Patients; 250; 10 Untethered Survey, advisory Mixed United Medical sciences e Computer
2011 record pilot with Microsoft HealthVault and providers panels methods States applications
google health
Earnest Use of a patient-accessible electronic medical Patients; 107; 11 Tethered Survey, Mixed United Medical sciences e Computer
et al., record in a practice for congestive heart providers interview, focus methods States applications
2004 failure: Patient and physician experiences group
Emani et al., Patient perceptions of a personal health Patients 760 Tethered Survey Quantitative United Medical sciences e Computer
2012 record: A test of the diffusion of innovation States applicable
model
Fuji et al., Personal health record use by patients as Providers 955 Both Survey Quantitative United Medical sciences e Business
2008 perceived by ambulatory care physicians in States and economics e
Nebraska and South Dakota: A cross-sectional management
study
Goel et al., Disparities in enrollment and use of an Patients 7088 Tethered PHR use logs Quantitative United Medical sciences e Internal
2011 electronic patient portal States medicine
Goldner The intersection of gender and place in online Patients 7674 Not Survey Quantitative United Medical sciences;
et al., health activities Specified States Communications
2013
Gordon Processes and outcomes of developing a Patients 7080 Tethered Focus group, Mixed United Biology e Computer
et al., continuity of care document for use as a survey, PHR use methods States applications; Medical sciences
2012 personal health record by people living with logs e computer applications
HIV/AIDS in New York City
Gu and Day, Propensity of people with long-term Patients 10 Tethered Interview Qualitative New Medical sciences e computer
2013 conditions to use personal health records Zealand applications; Medical sciences
Guy et al., Evaluation of a web-based patient portal for Patients 99 Tethered Survey, focus Mixed Canada n/a
2012 chronic disease management group methods
Hess et al., Exploring challenges and potentials of Patients 39 Tethered Focus group, Mixed United Medical sciences e Computer
2007 personal health records in diabetes self- PHR use logs methods States applications
management: Implementation and initial
assessment
Jung et al., Who are portal users vs. early E-Visit Patients, 10,532 Tethered PHR use logs Quantitative United Medical sciences e Computer
2011 adopters? A preliminary analysis providers States applications
Kahn et al., Personal health records in a public hospital: Patients 136 Tethered PHR use logs, Quantitative United Medical sciences e Computer
2010 experience at the HIV/AIDS clinic at San survey States applications
Francisco General Hospital
Kim et al., Challenges to using an electronic personal Patient 70 Untethered Survey, PHR use Quantitative United Medical sciences e Computer
2009 health record by a low-income elderly logs States applications
population
Krist et al., Designing a patient-centered personal health Patients 7235 Both EHR patient Mixed United Medical sciences e Computer
2011 record to promote preventive care data, focus method States applications
group, survey,
interview
Lau et al., Consumers' online social Network topologies Patients 709 Untethered Survey, PHR use Quantitative Australia Medical sciences e computer
2013a and health behaviors logs applications; Medical sciences
Lau et al., Social and self-reflective use of a web-based Patients 709 Untethered Survey, PHR use Quantitative Australia Medical sciences e Computer
2013b personally controlled health management logs applications
system
Patients 709 Tethered Quantitative Australia
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 231

(continued )

Author, year Study name Sample Sample Type of Instrumentation Methodology Study Journal domain
(reference) size PHR location

Lau et al., Which bundles of features in a web-based Survey, PHR use Medical sciences e Computer
2013c personally controlled health management logs applications
system are associated with consumer help-
seeking behaviors for physical and emotional
well-being?
Lin et al., An internet-based patient-provider Patients 606; 14 Tethered Survey, PHR use Mixed United Medical sciences e Computer
2005 communication system: Randomized logs methods States applications
controlled trial
Lober et al., An internet-based patient-provider Patients 38 Untethered PHR use logs, Quantitative United Medical sciences e Computer
2006 communication system: Randomized observation States applications
controlled trial
Mayberry Bridging the digital divide in diabetes: Family Patients 75 Tethered Focus group, Mixed United Medical sciences e
et al., support and implications for health literacy survey, EHR methods States Endocrinology
2011 data
McInnes Development and evaluation of an internet Patients 14 Tethered Survey; Mixed United Medical sciences; Public
et al., and personal health record training program interview; focus methods States health and safety
2013 for low-income patients with HIV or hepatitis group
C
Miller et al., Determinants of personal health record use Patients 63,295 Tethered PHR use logs Quantitative United Health facilities and
2007 States administration
Nagykaldi Impact of a wellness portal on the delivery of Patients 538 Tethered Survey, PHR use Quantitative United Medical sciences
et al., patient-centered preventive care logs States
2012
Nazi, 2010 Veterans' voices: use of the American Patients 100,617 Tethered Interview, Mixed United Medical sciences e Computer
Customer Satisfaction Index (ACSI) Survey to survey methods States application
identify My HealtheVet personal health record
users' characteristics, needs, and preferences
Nazi, 2013 The personal health record paradox: Patients, 30 Tethered Survey, Mixed United Medical sciences e Computer
Healthcare professionals' perspectives and the providers Interview methods States application
information ecology of personal health record
systems in organizational and clinical settings
Nazi et al., Evaluating patient access to electronic health Patients 668 Tethered Survey Quantitative United Medical sciences; Public
2013 records results from a survey of veterans States health and safety
Nielsen Internet portal use in an academic multiple Patients 240 Tethered EHR patient Quantitative United Medical sciences e Computer
et al., sclerosis center data, PHR use States applications
2012 logs
Osborn Understanding patient portal use: Patients 75 Tethered Focus group, Mixed United Medical sciences e Computer
et al., Implications for medication management survey, EHR methods States applications
2013 patient data
Poon et al., Empowering patients to improve the quality Patients 2779 Tethered Survey Quantitative United Medical sciences e Computer
2007 of their care: design and implementation of a States applications, Medical sciences
shared health maintenance module in a US
integrated healthcare delivery network
Sarkar et al., The literacy divide: Health literacy and the use Patients 14,102 Tethered Survey, PHR use Quantitative United Medical sciences,
2010 of an internet- based patient portal in an logs States Communications
integrated health systemdresults from the
Diabetes Study of Northern California
(DISTANCE)
Sarkar et al., Social disparities in internet patient portal use Patients 14,102 Tethered Survey, PHR use Quantitative United Medical sciences e Computer
2011 in diabetes: evidence that the digital divide logs States applications
extends beyond access
Schnipper Design and implementation of a web-based Patients 5298 Tethered PHR use logs, Quantitative United Medical sciences e Computer
et al., patient portal linked to an electronic health survey States applications
2008 record designed to improve medication safety:
the Patient Gateway medications module
Tenforde The association between personal health Patients 10,746 Tethered EHR patient Quantitative United Medical sciences e Internal
et al., record use and diabetes quality measures data, PHR use States medicine
2012 logs
Tjora et al., Privacy vs. usability: A qualitative exploration Patients 15 Tethered Interview Qualitative Norway Medical sciences e Computer
2005 of patients' experiences with secure internet applications
communication with their general
practitioner
Tom et al., Integrated personal health records use: Patients, 256 Tethered Survey Quantitative United Medical sciences e Pediatrics
2012 association with parent-reported care caregivers States
experiences
Tsai et al., Use of the internet and an online personal Patients 7215 Tethered Survey Quantitative United Medical sciences e Computer
2012 health record system by US veterans: States applications
comparison of Veterans Affairs mental health
service users and other veterans nationally
Tuil et al., Patient-centered care: using online personal Patients 102 Tethered Questionnaire, Mixed The Medical sciences e
2006 medical records in IVF practice PHR use logs, methods Netherlands Endocrinology; Medical
interview sciences e Obstetrics and
(continued on next page)
232 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

(continued )

Author, year Study name Sample Sample Type of Instrumentation Methodology Study Journal domain
(reference) size PHR location

gynecology; Pharmacy and

pharmacology
Urowitz Improving diabetes management with a Patients; 17; 64 Tethered Survey; Qualitative Canada Medical sciences e Computer
et al., patient portal: a qualitative study of diabetes providers interview applications
2012 self-management portal
Vodicka Online access to doctors' notes: patient Patients 3874 Tethered Survey, PHR use Quantitative United Medical sciences e Computer
et al., concerns about privacy logs States applications
2013
Wade- Secure messaging and diabetes management: Patients 54 Tethered Focus group, Mixed United Medical sciences e Computer
Vuturo experiences and perspectives of patient portal survey methods States applications
et al., use
2013
Wagner Personal health records and hypertension Patients 443 Tethered EHR patient Quantitative United Medical sciences e Computer
et al., control: A randomized trial data, survey States applications
2012
Wagner Incorporating patient perspectives into the Patients 16 Untethered Interview Qualitative United Medical sciences; Business
et al., personal health record: implications for care States and Economics e
2010 and caring Management
Wald et al., Implementing practice-linked pre-visit Patients; 3979; Tethered Survey, EHR Quantitative United Medical sciences e Computer
2010 electronic journals in primary care: Patient providers 272 patient data States applications
and physician use and satisfaction
Wald et al., Survey analysis of patient experience using a Patients 126 Tethered Survey, PHR use Quantitative United Medical sciences e Computer
2009 practice-linked PHR for type 2 diabetes logs States applications
mellitus
Wang et al., Personal health information management Patients 61 Untethered PHR use logs Quantitative United Biology e Biotechnology;
2004 system and its application in referral States Medical sciences e Computer
management applications
Weingart Who uses the patient internet portal? The Patients 980 Tethered PHR use logs Quantitative United Medical sciences e Computer
et al., PatientSite experience States applications
2006
Weitzman Willingness to share personal health record Patients, 261 Untethered Survey Quantitative United Medical sciences e Computer
et al., data for care improvement and public health: caregivers States applications
2012 a survey of experienced personal health
record users
Wen et al., Consumers' perceptions about and use of the Patients 7674 Not Survey Quantitative United Medical sciences e Computer
2010 internet for personal health records and health Specified States applications
information exchange: Analysis of the 2007
Health Information National Trends Survey
Wiljer et al., The anxious wait: assessing the impact of Patients 311 Tethered Survey Quantitative Canada Medical sciences e Computer
2010 patient accessible EHRs for breast cancer applications
patients
Woods Patient experiences with full electronic access Patients, 30; 6 Tethered Focus group Qualitative United Medical sciences e Computer
et al., to health records and clinical notes through providers, States applications
2013 the My HealtheVet personal health record caregivers
pilot: Qualitative study
Wynia et al., Many physicians are willing to use patients' Providers 856 Not Survey Quantitative United Insurance; Public health and
2011 electronic personal health records, but doctors Specified States safety
differ by location, gender, and practice
Zickmund Interest in the use of computerized patient Patients 39 Tethered Focus Group Qualitative United Medical sciences e Internal
et al., portals: Role of the providerepatient States medicine
2008 relationship
Zulman Patient interest in sharing personal health Patients 18,471 Tethered Survey Quantitative United Medical sciences e Internal
et al., record information, a web-based survey States medicine
2011

Participant demographics for included studies focusing on patient work systems.

Citation Health condition Race Age

Ancker Hypertension e 16%, Hyperlipidemia 13%, Asthma e 16% Black, 19% Hispanic, 44% White, 6% other, 15% M ¼ 40
et al., 9%, Diabetes e 7%, Depression e 6%, Drug Abuse/ missing/unknown
2011 Dependence e 2.3%, Chronic hepatitis (B, C, or other) -
1.8%, Alcoholism e 1.5%, HIV/AIDS e 1.3%,
Burke Congenial cardiac disease 37.5% Hispanic, 35.2% White, 10.1% Black, 0.7% Asian, n/a
et al., 16.5% Other
2010
Day and One or more long term health condition n/a Range: 35e79
Gu,
2012
Spinal disorders n/a Range: 35e85
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 233

(continued )

Citation Health condition Race Age

Denton,
2001
Do et al., n/a n/a M ¼ 53.14
2011
Earnest Congestive heart failure 90% White, non-Hispanic M ¼ 56
et al.,
2004
Emani Asthma, congestive heart failure (CHF), hypertension, Innovators: 94% Caucasian, Other users: 90% Innovators: M ¼ 55.4, Other users: M ¼ 59.8,
et al., or diabetes Caucasian, Laggards: 94% Caucasian, Rejecters: 86% Laggards: M ¼ 59.0, Rejecters: M ¼ 58.3, Non-
2012 Caucasian, Non-adopters: 76% Caucasian adopters: M ¼ 61.7
Goel et al., n/a 49% White, 15% Black. 4% Latino, 2% Asian, 12% other, Median age ¼ 48
2011 19% missing
Goldner n/a Health seekers: 25.2% Non-White, 74.8% White; Health seekers: 29.6% 18e34, 30.9% 35e49,
et al., Internet users: 24.7% Non-White, 75.3% White 25.7% 50e64, 7.9% 65e74, 5.9% 75þ
2013 Internet users: 36.5% 18e34, 32.2% 35e49,
23.6% 50e64, 5.4% 65e74, 2.3% 75þ
Gordon HIV 38.1% African American/Black, 0.3% Asian, <0.1% Unique members: M ¼ 45.6
et al., Native Hawaiian/Pacific Islander, 0.3% Native
2012 American or Alaskan, 38.3% White/Caucasian, 23.0%
Other/not specified, 38.0% Hispanic/Latino, 39.0%
Non-Hispanic/Latino
Gu and One or more long term conditions including: prostate n/a Range: 35e79
Day, cancer, type 2 diabetes, asthma, heart condition, M ¼ 63.5
2013 essential tremor, benign prostate hypertrophy, high
cholesterol, alcoholism, vertigo, glaucoma, osteo-
arthritis, obesity, chornic abdominal pain
(endometriosis), and depression. They also reported
surgery, including bilateral hip replacement, ‘broken
knee cap’, sinus surgery, removal of gall bladder,
ovarian cystectomy and hysterectomy, thyroidectomy
and ‘heart burn surgery’.
Guy et al., Type 2 diabetes and prostate cancer n/a n/a
2012
Hess et al., Diabetes 28% Non-White M ¼ 54
2007
Jung et al., n/a eVisit users: 76.5% White eVisit: M ¼ 46.88, Range 21e78; Portal Only:
2011 Portal only users: 76.4% White M ¼ 51.73, Range 19e97
Kahn et al., HIV/AIDS 78% Caucasian, 15% Hispanic 37% < 40 years old
2010
Kim et al., n/a n/a PHIIMS users: M ¼ 63.1; Range:21e30 (2.9%),
2009 31e40 (2.9%), 41e50 (11.4%), 51e60 (20.0%), 61
e70 (38.6%), 71e80 (7.1%), 81e90 (10.0%), 91
e100 (7.1%)
Survey respondents: M ¼ 65.5
Krist et al., n/a n/a 56% 18e49, 36% 50e64, 8% 65e75
2011
Lau et al., n/a n/a n/a
2013a
Lau et al., n/a n/a n/a
2013b
Lau et al., n/a n/a M ¼ 25.2
2013c
Lin et al., n/a n/a Portal group M ¼ 52
2005 Control group M ¼ 50
Lober Many had chronic diseases n/a Range: 49e92
et al., M ¼ 69
2006
Mayberry Type 2 diabetes 65% White, 35% Non-white M ¼ 56.8
et al.,
2011
McInnes HIV (n ¼ 8) or hepatitis C (HCV) (n ¼ 6) 14.3% Hispanic, 57.1% White, 21.4% Black, 7.1% Other 14.3% 35e49, 71.4% 50e64, 14.3% 65e74
et al.,
2013
Miller n/a Users: 88% Caucasian, 6% African American, 1% Asian/ Users M ¼ 50.1
et al., Pacific Islander, 1% Hispanic/Latino, 2% Other Never users M ¼ 50.7
2007 Never users: 82% Caucasian, 11% African American, 2%
Asian Pacific Islander, 2% Hispanic/Latino, 2% Other
Nagykaldi n/a 82% White non-Hispanic, 8% Black non-Hispanic, 6% Range: 40e75
et al., Hispanic, 4% American Indian/Alaskan Native Control M ¼ 50.5
2012 Intervention M ¼ 54.6
Nazi et al., n/a n/a 9, 1% < 40, 7% 41e50, 26% 51e60, 45% 61e70,
2013 15% 71e80, 6% 81e90
Nazi, 2010 n/a n/a 1% 19e30, 4% 31e40, 11% 41e50, 34% 51e60,
34% 61e70, 16% 71þ
(continued on next page)
234 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

(continued )

Citation Health condition Race Age

Nielsen Clinically isolated syndrome (n ¼ 12), Relapsing- Users: 4% Non-White Users: 83.3% < 55, 16.7%>¼55
et al., remitting multiple sclerosis (n ¼ 118), Secondary Nonusers: 17.5% Non-White Nonusers: 61.7% < 55, 38.3%>¼55 M¼45.4
2012 progressive multiple sclerosis (n ¼ 17), Primary
progressive multiple sclerosis (n ¼ 7)
Osborn Type 2 diabetes 62.7% White, 33.3% African American/black M ¼ 56.9
et al.,
2013
Poon et al., n/a n/a M ¼ 47.4
2007
Sarkar Diabetes 21% AfricaneAmerican, 28% Non-Hispanic White, 14% 4% 30e39, 15% 40e49, 33% 50e59, 32% 60e69,
et al., Latino, 9% Asian American, 12% Filipino, 17% Other/ 17% 70þ
2010 mixed
Sarkar Diabetes 21% AfricaneAmerican, 28% Non-Hispanic White, 14% 4% 30e39, 15% 40e49, 33% 50e59, 32% 60e69,
et al., Latino, 9% Asian American, 12% Filipino, 17% Other/ 17% 70þ
2011 mixed
Schnipper n/a White 83% M ¼ 48
et al.,
2008
Tenforde Diabetes Mellitus Users: 84.0% Caucasian, 11.0% African American, 3.0% Users: M ¼ 59
et al., Other, 2% unknown, <1% Hispanic Non-Users: M ¼ 62
2012 Nonusers: 67.3% Caucasian, 28.6% African American,
2.0% Other, 2.0% Unknown, <1% Hispanic
Tjora et al., n/a n/a n/a
2005
Tom et al., Asthma 24%, congenital musculoskeletal White 70%, Black 5%, Asian/Pacific Islander 13%, Other n/a
2012 abnormalities 20%, congenital heart disease 18%, 13%; 5% Hispanic, 94% Non-Hispanic
inborn errors of metabolism 9%, cystic fibrosis 7%, and
hereditary and acquired helytic anemias 5%
Tsai et al., n/a Not enrolled: 87.30% White, 8.90% Black, 2.20% Not enrolled: M ¼ 62.23
2012 American Indian/Alaskan Native, 1.60% Asian/Pacific Enrolled but does not use mental health
Islander services: M ¼ 61.75
Enrolled but does not use mental health services: Enrolled and does use mental health services:
81.70% White, 14.56% Black, 2.93% American Indian/ M ¼ 53.45
Alaskan Native, 0.81% Asian/Pacific Islander
Enrolled and using mental health services: 74.60%
White, 18.97% Black, 4.50% American Indian/Alaskan
Native, 1.93% Asian/Pacific Islander
Tuil et al., n/a n/a M ¼ 34.44
2006
Urowitz Type 1 and 2 diabetes n/a n/a
et al.,
2012
Vodicka n/a 92.8% White, 52.6% >¼ 55 years old
et al., 7.2% Non-White 47.4% < 55 years old
2013
Wade- Type 2 diabetes 75.9% Caucasian/White M ¼ 57.1
Vuturo 20.4% African American/Black
et al.,
2013
Wagner Hypertension 50.2% White M ¼ 54.8
et al., 46.1% Black
2012 3.7% Other
Wagner Hypertension n/a n/a
et al.,
2010
Wald et al., n/a 87.1% White M ¼ 48.9
2010
Wald et al., Type 2 diabetes 93% White M ¼ 59.4
2009
Wang n/a n/a Range: 22e75
et al., M ¼ 45.7
2004
Weingart n/a Enrollees: 80% White Enrollees (20e81), Nonuser (21e92)
et al., Nonusers: 54% White Enrollees (M ¼ 42.9), Nonuser (M ¼ 52.9)
2006
Weitzman n/a 91.3% White, non-Hispanic n/a
et al.,
2012
Wen et al., Internet users (cancer survivor ¼ 5.8%) Internet Internet users: 74.5% Non-Hispanic White, 9.4% Internet users: 16.2% 18e24, 19.7% 25e34,
2010 nonusers (cancer survivor ¼ 9.4%) Hispanic, 9.5% Black/African American, 6.7% Other 22.0% 35e44, 19.9% 45e54, 13.7% 55e64%, 8.4%
Internet nonusers: 57.3% Non-Hispanic White, 21.1% 65þ
Hispanic, 15.8% Black/African American7.1%, Other Internet nonusers: 6.1% 18e24, 13.1% 25e34,
13.9% 35e44, 17.6% 45e54, 14.5% 55e64, 34.9%
65þ
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 235

(continued )

Citation Health condition Race Age

Wiljer Breast cancer n/a 75.9% < 60 years

et al., 24.1% > 60 years
2010
Woods n/a n/a Range: 49e82
et al.,
2013
Zickmund Diabetes 28% Non-White M ¼ 54
et al.,
2008
Zulman n/a n/a <50 y (n ¼ 1,836, 10.1%), 50e64 (n ¼ 9,206,
et al., 50.6%), 65 y (n ¼ 7,146, 39.3%)
2011

Note: we list ethnic/racial groups, ages, and health conditions of participants for studies focusing on patient work systems at the level of detail reported in the original
manuscripts.

Participant demographics for included studies focusing on provider work systems.

Citation Medical practice Age

Crotty et al., 2013 n/a n/a

Do et al., 2011 n/a n/a
Earnest et al., 2004 University of Colorado Hospital - cardiology n/a
Fuji et al., 2008 n/a M ¼ 50
Jung et al., 2011 n/a n/a
Nazi, 2013 Primary care: n ¼ 24 n/a
Specialty care: n ¼ 4
Both/other: n ¼ 2
Urowitz et al., 2012 Providers included a mix of general practitioners, nurses, nurse practitioners, dietitians, diabetes educators, and other clinical staff n/a
Wald et al., 2010 n/a M ¼ 41
Wynia et al., 2011 Primary care: 47.3%, Pediatrics: 8.4%, Obstetrics/gynecology: 6.0%, Psychiatry: 7.0%, Surgery: 11.7%, Other: 19.7% M ¼ 51.2

Participant demographics for included studies focusing on caregiver work systems.

Citation Relationship to Race Age

patient

Britto et al., 2013 Parent n/a n/a

Byczkowski et al., Parent White e 91% responders, 89% nonresponders AfricaneAmerican e 1% responders, 1% nonresponders Other e 9% n/a
2014 responders, 10% nonresponders
Tom et al., 2012 Parent/guardian n/a M ¼ 36
Weitzman et al., n/a n/a n/a
2012
Woods et al., 2013 n/a n/a n/a

Summary of articles reporting patient barriers related to individual factors.

Reference Skills, employment, knowledge, training, and Size, weight, reach, and Age, gender, ethnicity, language, Needs, biases, beliefs, Behavior
education strength marital status mood

Ancker et al., 2011 X X X

Burke et al., 2010 X
Denton, 2001 X
Earnest et al., 2004 X X X
Emani et al., 2012 X X X X
Goel et al., 2011 X
Goldner et al., 2013 X X X
Gordon et al., 2012 X X
Gu and Day, 2013 X
(continued on next page)
236 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

(continued )

Reference Skills, employment, knowledge, training, and Size, weight, reach, and Age, gender, ethnicity, language, Needs, biases, beliefs, Behavior
education strength marital status mood

Hess et al., 2007 X

Jung et al., 2011 X X X X
Kahn et al., 2010 X
Kim et al., 2009 X
Krist et al., 2011 X
Lau et al., 2013a X X
Lau et al., 2013b X X
Lau et al., 2013c X
Lin et al., 2005 X X
Lober et al., 2006 X X X
Mayberry et al., 2011 X
Miller et al., 2007 X X X X
Nagykaldi et al., X X X
2012
Nazi, 2010 X
Nazi, 2013 X X
Nielsen et al., 2012 X X X X
Osborn et al., 2013 X X X
Poon et al., 2007 X
Sarkar et al., 2010 X X X
Sarkar et al., 2011 X X X
Schnipper et al., X X
2008
Tenforde et al., 2012 X X X X
Tjora et al., 2005 X X
Tom et al., 2012 X
Tsai et al., 2012 X X
Tuil et al., 2006 X
Urowitz et al., 2012 X X
Vodicka et al., 2013 X X
Wade-Vuturo et al., X X
2013
Wagner et al., 2012 X X X
Wagner et al., 2010 X X
Wald et al., 2009 X
Weingart et al., 2006 X X X
Weitzman et al., X X X
2012
Wen et al., 2010 X X X X
Woods et al., 2013 X
Zickmund et al., X X
2008
Total 22 0 20 33 24

Summary of articles reporting patient barriers related to work system/unit factors.

Reference Task demands, complexity, Time and sequence Availability of usable Technology functions Noise, temperature, and Physical layout and
difficulty demands technology and features lighting geography

Ancker et al., 2011 X

Burke et al., 2010 X
Day and Gu, 2012 X X
Denton, 2001 X X
Do et al., 2011 X X
Earnest et al., 2004 X X
Emani et al., 2012 X X X
Goldner et al., X
2013
Gordon et al., 2012 X
Gu and Day, 2013 X X
Guy et al., 2012 X X X
Hess et al., 2007 X X X
Kahn et al., 2010 X X
Kim et al., 2009 X
Krist et al., 2011 X
Lober et al., 2006 X X
Mayberry et al., X
2011
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 237

(continued )

Reference Task demands, complexity, Time and sequence Availability of usable Technology functions Noise, temperature, and Physical layout and
difficulty demands technology and features lighting geography

McInnes et al., X X X
2013
Nazi, 2010 X X X X
Nazi, 2013
Nazi et al., 2013 X
Osborn et al., 2013 X
Poon et al., 2007 X X
Sarkar et al., 2010 X
Schnipper et al., X
2008
Tjora et al., 2005 X X X
Tuil et al., 2006 X
Urowitz et al., X X X X
2012
Vodicka et al., X
2013
Wade-Vuturo X
et al., 2013
Wagner et al., X X X
2010
Wald et al., 2010 X X
Wald et al., 2009 X X X X
Wang et al., 2004 X X X
Wen et al., 2010 X
Wiljer et al., 2010 X X
Woods et al., 2013 X X X
Zickmund et al., X X X
2008
Zulman et al., X
2011
Total 22 14 9 28 0 2

Summary of articles reporting patient barriers related to organization factors.

Reference Organizational policy/ Organizational Financial Rewards Management Training Staffing Social norms and Social climate/
priorities structure resources structure structure provided levels pressures culture

Ancker et al., 2011 X

Earnest et al., 2004 X
Emani et al., 2012 X
Goldner et al., 2013 X
Guy et al., 2012 X X
Jung et al., 2011 X
Kim et al., 2009 X
Lau et al., 2013a X
Lau et al., 2013b X
Lober et al., 2006 X
Lin et al., 2005 X
Mayberry et al., X X
2011
Miller et al., 2007 X
Nielsen et al., 2012 X
Osborn et al., 2013 X
Schnipper et al., X
2008
Tenforde et al., X X
2012
Tsai et al., 2012 X
Weingart et al., X
2006
Zulman et al., 2011 X
Total 1 2 13 0 0 0 2 0 5
238 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

Summary of articles reporting patient barriers related to external environment factors.

Reference Extra-organizational rules, standards, legislation enforcement Industry social influences Industry workforce characteristics

Do et al., 2011 X
Guy et al., 2012 X X
Hess et al., 2007 X X
Kahn et al., 2010 X
Kim et al., 2009 X
Lober et al., 2006 X
Mayberry et al., 2011 X
Nazi, 2010 X
Nazi, 2013 X X X
Osborn et al., 2013 X
Poon et al., 2007 X
Tjora et al., 2005 X X X
Tuil et al., 2006 X
Urowitz et al., 2012 X
Wade-Vuturo et al., 2013 X X
Wagner et al., 2012 X X
Wagner et al., 2010 X
Wald et al., 2010 X X
Wald et al., 2009 X X
Wang et al., 2004 X
Weitzman et al., 2012 X
Wen et al., 2010 X
Woods et al., 2013 X X
Zickmund et al., 2008 X
Zulman et al., 2011 X
Total 15 10 11

Summary of articles reporting provider barriers related to individual factors.

Reference Skills, employment, knowledge, training, and Size, weight, reach, and Age, gender, ethnicity, language, Needs, biases, beliefs, Behavior
education strength marital status mood

Crotty et al., 2013 X X

Earnest et al., 2004 X
Fuji et al., 2008 X X X
Jung et al., 2011 X
Nazi, 2013 X X X
Urowitz et al., 2012 X
Wynia et al., 2011 X X
Total 4 0 1 3 5

Summary of articles reporting provider barriers related to work system/unit factors.

Reference Task demands, complexity, Time and sequence Availability of usable Technology functions Noise, temperature, and Physical layout and
difficulty demands technology and features lighting geography

Crotty et al., 2013 X

Do et al., 2011 X X
Earnest et al., X X
2004
Fuji et al., 2008 X
Nazi, 2013 X X X X
Urowitz et al., X X X X
2012
Wald et al., 2010 X
Woods et al., 2013 X
Wynia et al., 2011 X X X
Total 2 5 2 8 0 2
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242 239

Summary of articles reporting provider barriers related to organization factors.

Reference Organizational policy/ Organizational Financial Rewards Management Training Staffing Social norms and Social climate/
priorities structure resources structure structure provided levels pressures culture

Do et al., 2011 X
Earnest et al., X X
2004
Fuji et al., 2008 X
Nazi, 2013 X X
Urowitz et al., X
2012
Wynia et al., 2011 X
Total 2 0 1 2 0 1 1 1 0

Summary of articles reporting provider barriers related to external environment factors.

Reference Extra-organizational rules, standards, legislation enforcement Industry social influences Industry workforce characteristics

Crotty et al., 2013 X

Nazi, 2013 X
Urowitz et al., 2012 X
Wynia et al., 2011 X
Total 2 2 0

Summary of articles reporting caregiver barriers related to individual factors.

Reference Skills, employment, knowledge, training, and Size, weight, reach, and Age, gender, ethnicity, language, Needs, biases, beliefs, Behavior
education strength marital status mood

Britto et al., 2013 X

Byczkowski et al., X X
2014
Tom et al., 2012 X X
Weitzman et al., 2012 X X
Total 2 0 0 4 1

Summary of articles reporting caregiver barriers related to work system/unit factors.

Reference Task demands, Time and sequence Availability of usable Technology functions Noise, temperature, Physical layout and
complexity, difficulty demands technology and features and lighting geography

Britto et al., 2013 X

Byczkowski et al., X X X X
2014
Tom et al., 2012 X X X
Woods et al., 2013 X
Total 2 2 2 3 0 0

Summary of articles reporting caregiver barriers related to organizational factors.

Reference Organizational policy/ Organizational Financial Rewards Management Training Staffing Social norms and Social climate/
priorities structure resources structure structure provided levels pressures culture

Tom et al., X
2012
Total 0 0 1 0 0 0 0 0 0

Summary of articles reporting caregiver barriers related to external environment factors.

Reference Extra-organizational rules, standards, legislation enforcement Industry social influences Industry workforce characteristics

Weitzman et al., 2012 X

Total 1 0 0
240 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242

References for human factors/ergonomics: developing the discipline and profession. Er-
gonomics 55 (4), 377e395. http://dx.doi.org/10.1080/00140139.2012.661087.
Earnest, M.A., Ross, S.E., Wittevrongel, L., Moore, L.A., Lin, C.T., 2004 SepeOct. Use of
Alper, J., Rapporteur; Roundtable on Health Literacy; Board on Population Health
a patient-accessible electronic medical record in a practice for congestive heart
and Public Health Practice; Institute of Medicine; The National Academies of
failure: patient and physician experiences. J. Am. Med. Inf. Assoc. 11 (5),
Sciences, Engineering, and Medicine, 2015 Aug. Health Literacy: Past, Present,
410e417. PMID: 15187074.
and Future: Workshop Summary. The National Academies Press, Washington
Emani, S., Yamin, C.K., Peters, E., Karson, A.S., Lipsitz, S.R., Wald, J.S., et al., 2012 Nov
DC. PMID: 26290936.
5. Patient perceptions of a personal health record: a test of the diffusion of
Amante, D.J., Hogan, T.P., Pagoto, S.L., English, T.M., 2014 Nov. A systematic review of
innovation model. J. Med. Internet Res. 14 (6), e150. http://dx.doi.org/10.2196/
electronic portal usage among patients with diabetes. Diabetes Technol. Ther.
jmir.2278.
16 (11), 784e793. http://dx.doi.org/10.1089/dia.2014.0078.
n, Y., Rockoff, M.L., Hauser, D., Pichardo, M., Szerencsy, A., et al., Friction, J.R., Davies, D., 2008. Personal health records to improve health informa-
Ancker, J.S., Barro
tion exchange and patient safety (Internet). In: Kenriksen, K., Battles, J.,
2011 Oct. Use of an electronic patient portal among disadvantaged populations.
Keyes, M., Grady, M. (Eds.), Advances in Patient Safety: New Directions and
J. Gen. Intern. Med. 26 (10), 1117e1123. http://dx.doi.org/10.1007/s11606-011-
Alternative Approaches, vol. 4. Technology and Medication Safety, Rockville,
1749-y.
MD [cited 2015 Oct 16]. Available from: http://www.ahrq.gov/sites/default/files/
Ancker, J.S., Witteman, H.O., Hafeez, B., Provencher, T., de Graaf, M., Wei, E., 2015 Jun
wysiwyg/professionals/quality-patient-safety/patient-safety-resources/
4. The invisible work of personal health information management among
resources/advances-in-patient-safety-2/vol4/Advances-Fricton_21.pdf.
people with multiple chronic conditions: qualitative interview study among
Fuji, K.T., Galt, K.A., Serocca, A.B., 2008. Personal health record use by patients as
patients and providers. J. Med. Internet Res. 17 (6), 137e149. http://dx.doi.org/
perceived by ambulatory care physicians in Nebraska and South Dakota: a
10.2196/jmir.4381.
cross-sectional study. Perspect. Health Inf. Manag. 5, 15. PMID: 18927602.
Archer, N., Fevrier-Thomas, U., Lokker, C., McKibbon, K.A., Straus, S.E., 2011 JuleAug.
Goel, M.S., Brown, T.L., Williams, A., Hasnain-Wynia, R., Thompson, J.A., Baker, D.W.,
Personal health records: a scoping review. J. Am. Med. Inf. Assoc. 18 (4),
2011 Oct. Disparities in enrollment and use of an electronic patient portal. J.
515e522. http://dx.doi.org/10.1136/amiajnl-2011-000105.
Gen. Intern. Med. 26 (10), 1112e1116. http://dx.doi.org/10.1007/s11606-011-
Brennan, P., Valdez, R., Alexander, G., Arora, S., Bernstam, E., Edmunds, M., et al.,
1728-3.
2015 Apr. Patient-centered care, collaboration, communication, and coordina-
Goldner, M., Hale, T.M., Cotten, S.R., Stern, M.J., Drentea, P., 2013. The intersection of
tion: a report from AMIA's 2013 policy meeting. J. Am. Med. Inf. Assoc. 22 (e1),
gender and place in online health activities. J. Health Commun. 18 (10),
e2e6. http://dx.doi.org/10.1136/amiajnl-2014-003176.
1235e1255. http://dx.doi.org/10.1080/10810730.2013.778364.
Britto, M.T., Hesse, E.A., Kamdar, O.J., Munafo, J.K., 2013 Jul. Parents' perceptions of a
Gordon, P., Camhi, E., Hesse, R., Odlum, M., Schnall, R., Rodriquez, M., et al., 2012.
patient portal for managing their child's chronic illness. J. Pediatr. 163 (1),
Processes and outcomes of developing a continuity of care document for use as a
280e281. http://dx.doi.org/10.1016/j.jpeds.2013.02.041 e1e2.
personal health record by people living with HIV/AIDS in New York City. Int. J.
Burke, R.P., Rossi, A.F., Wilner, B.R., Hannan, R.L., Zabinsky, J.A., White, J.A., 2010 Oct.
Med. Inform. 81 (10), e63ee73. http://dx.doi.org/10.1016/j.ijmedinf.2012.06.004.
Transforming patient and family access to medical information: utilisation
Graneheim, U.H., Lundman, B., 2004 Feb. Qualitative content analysis in nursing
patterns of a patient-accessible electronic health record. Cardiol. Young 20 (5),
research: concepts, procedures, and measures to achieve trustworthiness.
477e484. http://dx.doi.org/10.1017/S1047951110000363.
Nurse Educ. Today 24 (2), 105e112. PMID: 14769454.
Byczkowski, T.L., Munafo, J.K., Britto, M.T., 2014 Jun. Family perceptions of the us-
Greenspun, H., Coughlin, S., 2012. mHealth in An MWorld: How Mobile Technology
ability and value of chronic disease web-based patient portals. Heal Inf. J. 20 (2),
Is Transforming Health Care (Internet). Deloitte Center for Health Solutions,
151e162. http://dx.doi.org/10.1177/1460458213489054.
Washington, DC [cited 2015 Oct 16]. Available from: http://www2.deloitte.com/
Carayon, P., 2006 Jul. Human factors of complex sociotechnical systems. Appl.
content/dam/Deloitte/us/Documents/life-sciences-health-care/us-lhsc-
Ergon. 37 (4), 525e535. PMID: 16756937.
mhealth-in-an-mworld-103014.pdf.
Carayon, P., Schoofs Hundt, A., Karsh, B.T., Gurses, A.P., Alvarado, C.J., Smith, M.,
Gu, Y., Day, K., 2013. Propensity of people with long-term conditions to use personal
et al., 2006 Dec. Work system design for patient safety: the SEIPS model. Qual.
health records. Stud. Health Technol. Inf. 188, 46e51. PMID: 23823287.
Saf. Health Care 15 (Suppl. 1), i50e8. PMID: 17142610.
Guy, S., Ratzki-leewing, A., Gwadry-Sridhar, F., 2012. Evaluation of a web-based
Carayon, P., Kianfar, S., Li, Y., Xie, A., Alyousef, B., Wooldridge, A., 2015 Nov.
patient portal for chronic disease management. Elect. Healthc. 91, 114e121.
A systematic review of mixed methods research on human factors and ergo-
http://dx.doi.org/10.1007/978-3-642-29262-0_17. Malaga, Spain: Springer Ber-
nomics in health care. Appl. Ergon. Elsevier Ltd. 51, 291e321. http://dx.doi.org/
lin Heidelberg.
10.1016/j.apergo.2015.06.001.
Hancock, P.A., Scalma, J.L., 2004 NoveDec. On the relevance of qualitative methods
Center for Advancing Health, 2010. A New Definition of Patient Engagement: What
for ergonomics. Theor. Issues Ergon. Sci. 5 (6), 499e506. http://dx.doi.org/
Is Engagement and Why Is it Important?, 2000 Florida Avenue, NW. Suite 210.
10.1080/14639220412331303391.
Washington, DC [cited 2015 Oct 16]. Available from: http://www.cfah.org/pdfs/
Hess, R., Bryce, C.L., Paone, S., Fischer, G., McTigue, K.M., Olshansky, E., et al., 2007
CFAH_Engagement_Behavior_Framework_current.pdf.
Oct. Exploring challenges and potentials of personal health records in diabetes
Center for Medicare and Medicaid Services, 2015 Mar 30. Electronic Health Record
self-management: implementation and initial assessment. Telemed. J. E Heal 13
Incentive Program Stage 3 (Internet) [cited 2015 Oct 16]. Available from:
(5), 509e517. PMID: 17999613.
https://federalregister.gov/a/2015-06685.
Hignett, S., Wilson, J.R., 2004. The role for qualitative methodology in ergonomics: a
Chappell, N.L., Reid, R.C., 2002 Dec. Burden and well-being among caregivers:
case study to explore theoretical issues. Theor. Issues Ergon. Sci. 5 (6), 473e493.
examining the distinction. Gerontologist 42 (6), 772e780. PMID: 12451158.
http://dx.doi.org/10.1080/14639220412331303382.
Chen, S.H., Huang, J.L., Yeh, K.W., Tsai, Y.F., 2015 Apr 14. The stress of caring for
Holden, R.J., Karsh, B., 2010 Feb. The technology acceptance model: its past and its
children with asthma: a qualitative study of primary caregivers. J. Nurs. Res.
future in health care. J. Biomed. Inf. 43 (1), 159e172. http://dx.doi.org/10.1016/
[Epub ahead of print]. PMID: 25875102.
j.jbi.2009.07.002.
Connecting for Health Personal Health Working Group, 2003. The Personal Health
Holden, R.J., Carayon, P., Gurses, A.P., Hoonakker, P., Hundt, A.S., Ozok, A.A., et al.,
Working Group: Final Report (Internet) [cited 2015 Oct 16]. Available from:
2013. SEIPS 2.0. A human factors framework for studying and improving the
http://www.policyarchive.org/collections/markle/index?section¼5&id¼15473.
work of healthcare professionals and patients. Ergonomics 56 (11), 1669e1686.
Cronin, R.M., Davis, S.E., Shenson, J.A., Chen, Q., Rosenbloom, S.T., Jackson, G.P., 2015
http://dx.doi.org/10.1080/00140139.2013.838643.
Apr 29. Growth of secure messaging through a patient portal as a form of
Holden, R.J., Schubert, C.C., Mickelson, R.S., 2015 Mar. The patient work system: an
outpatient interaction across clinical specialties. Appl. Clin. Inf. 6 (2), 288e304.
analysis of self-care performance barriers among elderly health failure patients
http://dx.doi.org/10.4338/ACI-2014-12-RA-0117.
and their informal caregivers. Appl. Ergon. 47, 133e150. http://dx.doi.org/
Crotty, B.H., Mostaghimi, A., Landon, B.E., 2013 Oct. Preparing residents for future
10.1016/j.apergo.2014.09.009.
practice: report of a curriculum for electronic patient-doctor communication.
Hsieh, H.F., Shannon, S.E., 2005 Nov. Three approaches to qualitative content
Postgrad. Med. J. 89 (1056), 554e559. http://dx.doi.org/10.1136/postgradmedj-
analysis. Qual. Health Res. 15 (9), 1277e1288. PMID: 16204405.
2012-131688.
Institute of Medicine, 2009. Health Literacy, eHealth, and Communication: Putting
Day, K., Gu, Y., 2012. Influencing factors for adopting personal health record (PHR).
the Consumer First: Workshop Summary. Institute of Medicine (US) Roundtable
Stud. Health Technol. Inf. 178, 39e44. PMID: 22797017.
on Health Literacy. National Acadamies Press, Washginton, DC. PMID:
Denton, I.C., 2001. Will patients use electronic personal health records? responses
20662120.
from a real-life experience. J. Healthc. Inf. Manag. 15 (3), 251e259.
Jabour, A., Jones, J., 2013. Facilitators and Barriers to Patients' Engagements with
Department of Health and Human Services, 2013 Jun 3. Incentives for nondis-
Personal Health Records: Systematic Review. Universal Access in Human-
criminatory wellness programs in group health plans (Internet). Fed. Regist. 78
computer Interaction Applications and Services for Quality of Life. Springer
(106), 33158e33175. Available from: http://www.gpo.gov/fdsys/pkg/FR-2013-
Berline Heidelberg, Las Vegas, NV, pp. 472e481. http://dx.doi.org/10.1007/978-
06-03/pdf/2013-12916.pdf.
3-642-39194-1_55.
Desnoyers, L., 2004. The role of qualitative methodology in ergonomics: a com-
Jung, C., Padman, R., Shevchik, G., Paone, S., 2011. Who are portal users vs. early e-
mentary. Theor. Issues Ergon. Sci. 5 (6), 495e498. http://dx.doi.org/10.1080/
Visit adopters? A preliminary analysis. AMIA Annu. Symp. Proc. 1070e1079.
14639220412331303373.
PMID: 22195168.
Do, N.V., Barnhill, R., Heermann-Do, K., Salzman, K.L., Gimbel, R.W., 2011. The
Kahn, J.S., Hilton, J.F., Van Nunnery, T., Leasure, S., Bryant, K.M., Hare, C.B., et al., 2010
military health system's personal health record pilot with microsoft Health-
MareApr. Personal health records in a public hospital: experience at the HIV/
Vault and google health. J. Am. Med. Inf. Assoc. 18 (2), 118e124. http://
AIDS clinic at San Francisco General Hospital. J. Am. Med. Inf. Assoc. 17 (2),
dx.doi.org/10.1136/jamia.2010.004671.
224e228. http://dx.doi.org/10.1136/jamia.2009.000315.
Dul, J., Bruder, R., Buckle, P., Carayon, P., Falzon, P., Marras, W., et al., 2012. A strategy
Karsh, B.T., Holden, R.J., Alper, S.J., Or, C.K., 2006 Dec. A human factors engineering
 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242
paradigm for patient safety - designing to support the performnce of the health
care professional. Qual. Saf. Health Care 15 (Suppl. 1), i59e65. PMID: 17142611.
Katz, S.J., Moyer, C.A., Cox, D.T., Stern, D.T., 2003 Sep. Effect of a triage-based E-mail
system on clinic resource use and patient and physician satisfaction in primary
care: a randomized controlled trial. J. Gen. Intern. Med. 18 (9), 736e744. PMID:
12950483.
Kim, E.-H., Stolyar, A., Lober, W.B., Herbaugh, A.L., Shinstrom, S.E., Zierler, B.K., et al.,
2009 Oct 27. Challenges to using an electronic personal health record by a low-
income elderly population. J. Med. Internet Res. 11 (4), e44. http://dx.doi.org/
10.2196/jmir.1256.
Krist, A.H., Peele, E., Woolf, S.H., Rothemich, S.F., Loomis, J.F., Longo, D.R., et al., 2011
Nov 24. Designing a patient-centered personal health record to promote pre-
ventive care. BMC Med. Inf. Decis. Mak. 11, 73. http://dx.doi.org/10.1186/1472-
6947-11-73.
Lau, A.Y.S., Dunn, A.G., Mortimer, N., Gallagher, A., Proudfoot, J., Andrews, A., et al.,
2013a Sep 23. Social and self-reflective use of a web-based personally
controlled health management system. J. Med. Internet Res. 15 (9), e211. http://
dx.doi.org/10.2196/jmir.2682.
Lau, A., Dunn, A., Mortimer, N., Proudfoot, J., Andrews, A., Liaw, S.T., et al., 2013b.
Consumers' online social network topologies and health behaviours. Stud.
Health Technol. Inf. 192, 77e81. PMID: 23920519.
Lau, A.Y., Proudfoot, J., Andrews, A., Liaw, S.T., Crimmins, J., Arguel, A., Coiera, E.,
2013c. Which bundles of features in a web-based personally controlled health
management system are associated with consumer help-seeking behaviors for
physical and emotional well-being? J. Med. Internet Res. 15 (5), e79. http://
dx.doi.org/10.2196/jmir.2414.
Legris, P., Ingham, J., Collerette, P., 2003 Jan. Why do people use information
technology? A critical review of the technology acceptance model. Inf. Manag.
40 (3), 191e204. http://dx.doi.org/10.1016/S0378-7206(01)00143-4.
Lin, C.T., Wittevrongel, L., Moore, L., Beaty, B.L., Ross, S.E., 2005 Aug 5. An internet-
based patient-provider communication system: randomized controlled trial.
J. Med. Internet Res. 7 (4), e47. PMID: 16236699.
Lober, W.B., Zierler, B., Herbaugh, A., Shinstrom, S.E., Stolyar, A., Kim, E.H., et al.,
2006. Barriers to the use of a personal health record by an elderly population.
AMIA Annu. Symp. Proc. 514e518. PMID: 17238394.
Marquard, J.L., Zayas-Caban, T., 2012 JaneFeb. Transforming off-the-shelf consumer
health informatics interventions: recommendations for their design, evaluation
and redesign. J. Am. Med. Inf. Assoc. 19 (1), 137e142. http://dx.doi.org/10.1136/
amiajnl-2011-000338.
Mayberry, L.S., Kripalani, S., Rothman, R.L., Osborn, C.Y., 2011 Oct. Bridging the
digital divide in diabetes: family support and implications for health literacy.
Diabetes Technol. Ther. 13 (10), 1005e1012. http://dx.doi.org/10.1089/
dia.2011.0055.
McHugh, M.L., 2012. Interrater reliability: the kappa statistic. Biochem. Med. 22 (3),
276e282. PMID: 23092060.
McInnes, D.K., Solomon, J.L., Shimada, S.L., Petrakis, B.A., Bokhour, B.G., Asch, S.M.,
et al., 2013 Mar. Development and evaluation of an internet and personal health
record training program for low-income patients with HIV or hepatitis C. Med.
Care 51 (3 Suppl. 1), S62eS66. http://dx.doi.org/10.1097/
MLR.0b013e31827808bf.
Merriam-Webster. Barrier (Internet). An Encyclopӕdia Britannica Company. (cited
2015 Oct 16). Available from: http://www.merriam-webster.com/dictionary/
barrier.
Microsoft Office Professional Plus 2010, 2010. Microsoft Excel, Redmond.
Miller, H., Vandenbosch, B., Ivanov, D., Black, P., 2007 Summer. Determinants of
personal health record use: a large population study at cleveland clinic.
J. Healthc. Inf. Manag. 21 (3), 44e48. PMID: 19195293.
Moen, A., Brennan, P.F., 2005 NoveDec. Health@ Home: the work of health infor-
mation management in the household (HIMH): implications for consumer
health informatics (CHI) innovations. J. Am. Med. Inf. Assoc. 12 (6), 648e656.
PMID: 16049230.
Moher, D., Liberati, A., Tetzlaff, J., Altman, D., 2009 Aug. Preferred reporting items
for systematic reviews and meta-analyses: the PRISMA statement. PRISMA
Group. Ann. Intern. Med. 151 (4), 264e269. W64. PMID: 19622511.
Moray, N., 2000 Jul. Culture, politics and ergonomics. Ergonomics 43 (7), 858e868.
PMID: 10929822.
Nagykaldi, Z., Aspy, C.B., Chou, A., 2012 MareApr. Impact of a wellness portal on the
delivery of patient-centered preventive care. J. Am. Board Fam. Med. 25 (2),
158e167. http://dx.doi.org/10.3122/jabfm.2012.02.110130.
National eHealth Collaborative and Healthcare Information and Management Sys-
tems Society Foundation, 2014 Feb. The Patient Engagement Framework
(Internet) [cited 2015 Oct 16]. Available from: http://www.himss.org/
ResourceLibrary/genResourceDetailPDF.aspx?ItemNumber¼28305.
National Institute on Aging, 2010e2013a. Understanding and Promoting Health
Literacy (RO1). PAR-10e133. (Internet). Available from: https://www.nia.nih.
gov/research/funding/2011/09/understanding-and-promoting-health-literacy-
r01.
National Institute on Aging, 2010e2013b. Understanding and Promoting Health
Literacy (R03) (Internet). PAR-10e134. Available from: https://www.nia.nih.
gov/research/funding/2011/09/understanding-and-promoting-health-literacy-
r03.
National Institute on Aging, 2013e2016. Understanding and Promoting Health
Literacy (R21) (Internet). PAR-13e132. Available from: http://grants.nih.gov/
grants/guide/pa-files/PAR-13-132.html.
National Research Council, 2011. Health Care Comes Home: the Human Factors.
241
Committee on the Role of Human Factors in Home Health Care, Board on
Human-systems Integration, Division of Behavioral and Social Sciences and
Education. The National Academies Press, Washington, DC.
Nazi, K.M., 2010 MareApr. Veterans' voices: use of the American Customer Satis-
faction Index (ACSI) Survey to identify My HealtheVet personal health record
users' characteristics, needs, and preferences. J. Am. Med. Inf. Assoc. 17 (2),
203e211. http://dx.doi.org/10.1136/jamia.2009.000240.
Nazi, K.M., 2013 April 4. The personal health record paradox: health care pro-
fessionals' perspectives and the information ecology of personal health record
systems in organizational and clinical settings. J. Med. Internet Res. 15 (4), e70.
http://dx.doi.org/10.2196/jmir.2443.
Nazi, K.M., Hogan, T.P., Mcinnes, D.K., Woods, S.S., Graham, G., 2013 Mar. Evaluating
patient access to electronic health records: results from a survey of veterans.
Med. Care 51 (3 Suppl. 1), S52eS56. http://dx.doi.org/10.1097/
MLR.0b013e31827808db.
Nielsen, A.S., Halamka, J.D., Kinkel, R.P., Israel, B., 2012 JaneFeb. Internet portal use
in an academic multiple sclerosis center. J. Am. Med. Inf. Assoc. 19 (1), 128e133.
http://dx.doi.org/10.1136/amiajnl-2011-000177.
Osborn, C.Y., Mayberry, L.S., Wallston, K.A., Johnson, K.B., Elasy, T., 2013 Jul 3. Un-
derstanding patient portal use: implications for medication management.
J. Med. Internet Res. 15 (7), e133. http://dx.doi.org/10.2196/jmir.2589.
Otte-Trojel, T., de Bont, A., Rundall, T.G., van de Klundert, J., 2015 Sep 2. What do we
know about developing patient portals? A systematic literature review. J. Am.
Med. Inf. Assoc. http://dx.doi.org/10.1093/jamia/ocv114. Pii: ocv114.
Poon, E.G., Wald, J., Schnipper, J.L., Grant, R., Gandhi, T.K., Volk, L.A., et al., 2007.
Empowering patients to improve the quality of their care: design and imple-
mentation of a shared health maintenance module in a US integrated health-
care delivery network. Stud. Health Technol. Inf. 129 (Pt 2), 1002e1006. PMID:
17911866.
PricewaterhouseCoopers LLP, 2012 Jun. Emerging MHealth: Paths for Growth
(Internet) [cited 2015 Oct 15]. Available from: https://www.pwc.com/gx/en/
healthcare/mhealth/assets/pwc-emerging-mhealth-exec-summary.pdf.
ProQuest LLC. RefWorks. 2010.
Sandelowski, M., 2000 Aug. Whatever happened to qualitative description? Res.
Nurs. Health 23 (4), 334e340. PMID: 10940958.
Sandelowski, M., 2010 Feb. What's in a name? Qualitative description revised. Res.
Nurs. Health 33 (1), 77e84. http://dx.doi.org/10.1002/nur.20362.
Sarkar, U., Karter, A.J., Liu, J.Y., Schillinger, D., 2010. The literacy divide: health lit-
eracy and the use of an internet-based patient portal in an integrated health
system-results from the diabetes study of northern California. J. Health Com-
mun. 15 (Suppl. 2), 183e196. http://dx.doi.org/10.1080/10810730.2010.499988.
Sarkar, U., Karter, A.J., Liu, J.Y., Adler, N.E., Nguyen, R., Lopez, A., et al., 2011 May 1.
Social disparities in internet patient portal use in diabetes: evidence that the
digital divide extends beyond access. J. Am. Med. Inf. Assoc. 18 (3), 318e321.
http://dx.doi.org/10.1136/jamia.2010.006015.
Sarkar, U., Bates, D.W., 2014 Jan 22e29. Care partners and online patient portals.
JAMA 311 (4), 357e358. http://dx.doi.org/10.1001/jama.2013.285825.
Schnipper, J.L., Ghandi, T.K., Wald, J.S., Grant, R.W., Poon, E.G., Volk, L.A., et al., 2008.
Design and implementation of a web-based patient portal linked to an elec-
tronic health record designed to improve medication safety: the patient
gateway medications module. Inf. Prim. Care 16 (2), 147e155. PMID: 18713531.
Skeels, M.M., 2010. Sharing by Design: Understanding and Supporting Personal
Health Information Sharing and Collaboration within Social Networks. Uni-
versity of Washington, ISBN 978-1-1243-0841-8.
Smith, M.J., Sainfort, P.C., 1989 Jul. A balance theory of job design for stress
reduction. Int. J. Ind. Ergon. 4 (1), 67e79. http://dx.doi.org/10.1016/0169-
8141(89)90051-6.
Streid, J., Harding, R., Agupio, G., Dinat, N., Downing, J., Gwyther, L., et al., 2014 Mar.
Stressors and resources of caregivers of patients with incurable progressive
illness in sub-Saharan Africa. Qual. Health Res. 24 (3), 317e328. http://
dx.doi.org/10.1177/1049732314523682.
Taha, J., Czaja, S.J., Sharit, J., Morrow, D.G., 2013 Dec. Factors affecting usage of a
personal health record (PHR) to manage health. Psychol. Aging 28 (4),
1124e1139. http://dx.doi.org/10.1037/a0033911.
Tenforde, M., Nowacki, A., Jain, A., Hickner, J., 2012 Apr. The association between
personal health record use and diabetes quality measures. J. Gen. Intern. Med.
27 (4), 420e424. http://dx.doi.org/10.1007/s11606-011-1889-0.
The Office of the National Coordinator for Health Information Technology (ONC),
2013a. How to Attain Meaningful Use (Internet) [cited 2015 Oct 16]. Available
from: https://www.healthit.gov/providers-professionals/how-attain-
meaningful-use.
The Office of the National Coordinator for Health Information Technology (ONC),
2013b. Personal Health Records: what Health Care Providers Need to Know
(Internet) [cited 2015 Oct 16]. Available from: http://www.healthit.gov/sites/
default/files/about-phrs-for-providers-011311.pdf.
The Office of the National Coordinator for Health Information Technology (ONC),
2014. Federal Health it Strategic Plan 2015e2020 (Internet) [cited 2015 Oct 16].
Available from: https://www.healthit.gov/sites/default/files/federal-healthIT-
strategic-plan-2014.pdf.
Thompson, M., Valdez, R., 2013. Macroergonomic Factors of the Patient Work Sys-
tem: the Case of mHealth Based Asthma Self-management Second Presentation
Format. Summary.
Tjora, A., Tran, T., Faxvaag, A., 2005 May 31. Privacy vs usability: a qualitative
exploration of patients' experiences with secure internet communication with
their general practitioner. J. Med. Internet Res. 7 (2), e15. PMID: 15998606.
242 M.J. Thompson et al. / Applied Ergonomics 54 (2016) 218e242
Tom, J.O., Mangione-Smith, R., Solomon, C., Grossman, D.C., 2012 Jul. Integrated
personal health record use: association with parent-reported care experiences.
Pediatrics 130 (1), 183e190. http://dx.doi.org/10.1542/peds.2011-1786.
Tsai, J., Rosenheck, R.A., Ave, C., 2012 NoveDec. Use of the internet and an online
personal health record system by US veterans: comparison of Veterans Affairs
mental health service users and other veterans nationally. J. Am. Med. Inf.
Assoc. 19 (6), 1089e1094. http://dx.doi.org/10.1136/amiajnl-2012-000971.
Tuil, W.S., Hoopen, A.J., Braat, D.D.M., Robbe , P.F.D.V., Kremer, J.A.M., 2006 Nov.
Patient-centred care: using online personal medical records in IVF practice.
Hum. Reprod. 21 (11), 2955e2959. PMID: 16982658.
Ulrichsweb: Global Serials Directory (Internet). (cited 2015 Oct 16). Available from:
http://ulrichsweb.serialssolutions.com/.
Urowitz, S., Wiljer, D., Dupak, K., Kuehner, Z., Leonard, K., Lovrics, E., et al., 2012 Nov
30. Improving diabetes management with a patient portal: a qualitative study
of diabetes self-management portal. J. Med. Internet Res. 14 (6), e158. http://
dx.doi.org/10.2196/jmir.2265.
Valdez, R.S., Holden, R.J., Novak, L.L., Veinot, T.C., 2015a Apr. Technological infra-
structure implications of the patient work framework. J. Am. Med. Inf. Assoc. 22
(e1), e213e5. http://dx.doi.org/10.1093/jamia/ocu031.
Valdez, R.S., Holden, R.J., Novak, L.L., Veinot, T.C., 2015b Jan. Transforming consumer
health informatics through a patient work framework: connecting patients to
context. J. Am. Med. Inf. Assoc. 22 (1), 1e7. http://dx.doi.org/10.1136/amiajnl-
2014-002826.
Valdez, R.S., Brennan, P.F., 2015 May. Exploring patients' health information
communication practices with social network members as a foundation for
consumer health IT design. Int. J. Med. Inf. 84 (5), 363e374. http://dx.doi.org/
10.1016/j.ijmedinf.2015.01.014.
Valdez, R., Holden, R., Carayon, P., 2015. Expanding representations of individuals
and their interactions within work system models. International Ergonomics
Association, Melbourne, Australia.
Vincent, C., 2003 Mar 13. Understanding and responding to adverse events. N. Engl.
J. Med. 348 (11), 1051e1056. PMID: 12637617.
Vodicka, E., Mejilla, R., Leveille, S.G., Ralston, J.D., Darer, J.D., Delbanco, T., et al., 2013
Sep 26. Online access to doctors' notes: patient concerns about privacy. J. Med.
Internet Res. 15 (9), e208. http://dx.doi.org/10.2196/jmir.2670.
Wade-Vuturo, A., Mayberry, L.S., Osborn, C.Y., 2013 May. Secure messaging and
diabetes management: experiences and perspectives of patient portal users.
J. Am. Med. Inf. Assoc. 20 (3), 519e525. http://dx.doi.org/10.1136/amiajnl-2012-
001253.
Wagner, P.J., Howard, S.M., Bentley, D.R., Seol, Y., 2010 Oct 1. Incorporating patient
perspectives into the personal health record: implications for care and caring.
Perspect. Health Inf. Manag. 7, 1e. PMID: 21063546.
Wagner, P.J., Dias, J., Howard, S., Kintziger, K.W., Hudson, M.F., Seol, Y., et al., 2012
JuleAug. Personal health records and hypertension control: a randomized trial.
J. Am. Med. Inf. Assoc. 19 (4), 626e634. http://dx.doi.org/10.1136/amiajnl-2011-
000349.
Wald, J.S., Grant, R.W., Schnipper, J.L., Gandhi, T.K., Poon, E.G., Alexandra, C., et al.,
2009 Nov 14. Survey analysis of patient experience using a practice-linked PHR
for type 2 diabetes mellitus. AMIA Annu. Symp. Proc. 2009, 678e682. PMID:
20351940.
Wald, J.S., Businger, A., Gandhi, T.K., Grant, R.W., Poon, E.G., Schnipper, J.L., et al.,
2010 SepeOct. Implementing practice-linked pre-visit electronic journals in
primary care: patient and physician use and satisfaction. J. Am. Med. Inf. Assoc.
17 (5), 502e506. http://dx.doi.org/10.1136/jamia.2009.001362.
Wang, M., Lau, C., Matsen, F.A., Kim, Y., 2004 Sep. Personal health information
management system and its application in referral management. IEEE Trans. Inf.
Technol. Biomed. 8 (3), 287e297. PMID: 15484434.
Wasson, J.H., Forsberg, H.H., Lindblad, S., Mazowita, G., McQuillen, K., Nelson, E.C.,
2012 ApreJun. The medium is the (health) measure: patient engagement using
personal technologies. J. Ambul. Care Manag. 35 (2), 109e117. http://dx.doi.org/
10.1097/JAC.0b013e31824a235e.
Weingart, S.N., Rind, D., Tofias, Z., Sands, D.Z., 2006 JaneFeb. Who uses the patient
internet portal? The PatientSite experience. J. Am. Med. Inf. Assoc. 13 (1),
91e95. PMID: 16221943.
Weitzman, E.R., Kelemen, S., Kaci, L., Mandl, K.D., 2012 May 22. Willingness to share
personal health record data for care improvement and public health: a survey
of experienced personal health record users. BMC Med. Inf. Decis. Mak. 12, 39.
http://dx.doi.org/10.1186/1472-6947-12-39.
Wells, K., Vazquez-Otero, C., Rivera, M., Chaet, A., Proctor, S., Arroyo, G., et al., 2015.
Assessing the acceptability of an embodied conversational agent-based com-
puter application for hispanic women. J. Hisp. Health Care Int.
Wen, K.Y., Kreps, G., Zhu, F., Miller, S., 2010 Oct-Dec. Consumers' perceptions about
and use of the internet for personal health records and health information
exchange: analysis of the 2007 Health Information National Trends Survey. J.
Med. Internet. Res. 12 (6), e73. http://dx.doi.org/10.2196/jmir.1668.
Werner, P., Mittelman, M.S., Goldstein, D., 2012 Feb. Family stigma and caregiver
burden in Alzheimer's disease. Gerontologist 52 (1), 89e97. http://dx.doi.org/
10.1093/geront/gnr117.
Wiljer, D., Leonard, K.J., Urowitz, S., Apatu, E., Massey, C., Quartey, N.K., et al., 2010
Sep 1. The anxious wait: assessing the impact of patient accessible EHRs for
breast cancer patients. BMC Med. Inf. Decis. Mak. 10, 46. http://dx.doi.org/
10.1186/1472-6947-10-46.
Woods, S.S., Schwartz, E., Tuepker, A., Press, N.A., Nazi, K.M., Turvey, C.L., et al., 2013
Mar 27. Patient experiences with full electronic access to health records and
clinical notes through the My HealtheVet Personal Health Record Pilot: quali-
tative study. J. Med. Internet Res. 15 (3), e65. http://dx.doi.org/10.2196/
jmir.2356.
Wynia, M.K., Torres, G.W., Lemieux, J., 2011 Feb. Many physicians are willing to use
patients' electronic personal health records, but doctors differ by location,
gender, and practice. Health Aff. 30 (2), 266e273. http://dx.doi.org/10.1377/
hlthaff.2010.0342.
Ye, J., Rust, G., Fry-Johnson, Y., Strothers, H., 2010 Aug. E-mail in patient-provider
communication: a systematic review. Patient Educ. Couns. 80 (2), 266e273.
http://dx.doi.org/10.1016/j.pec.2009.09.038.
Zickmund, S.L., Hess, R., Bryce, C.L., Mctigue, K., Olshansky, E., Fitzgerald, K., et al.,
2008 Jan. Interest in the use of computerized patient portals: role of the
provider-patient relationship. J. Gen. Intern. Med. 23 (Suppl. 1), 20e26. http://
dx.doi.org/10.1007/s11606-007-0273-6.
Zulman, D.M., Nazi, K.M., Turvey, C.L., Wagner, T.H., Woods, S.S., An, L.C., 2011 Dec
20. Patient interest in sharing personal health record information. Ann. Intern.
Med. 155 (12), 805e811. http://dx.doi.org/10.7326/0003-4819-155-12-
201112200-00002.