Anda di halaman 1dari 10

Trust in Health Websites: A Review of an Emerging Field

Laurian Vega Enid Montague, Ph.D. Tom DeHart


Computer Science Industrial and Systems Engineering Computer Science
2202 Kraft Drive 3107 Mechanical Engineering Building 322 BSB, 227 Penn Street
Virginia Tech, Blacksburg, VA University of Wisconsin-Madison, WI Rutgers University, Camden, NJ

Laurian@vt.edu EMontague@wisc.edu tdehart@vt.edu

ABSTRACT has, however, impacted the traditional health care relationships.


As people increasingly turn to health websites for the purposes of The Pew Internet Foundation recently reported that 61% of
self-diagnosis and healthier living, we have an obligation to Americans are currently using the internet for health information,
evaluate the factors that might affect a given user's assessment and and 20% use the internet to look up information about medicine
their willingness to use such sites. Constructs such as quality, and experimental treatments [15]. This rise in use of health
trust, and credibility need to be defined within this space in order websites indicates a need to understand how users evaluate the
for us to truly understand how and why people use health credibility of online information in order to better understand the
websites. In an effort to better understand these constructs we impact it will have on the patient’s trust in different parts of the
conducted a comprehensive analysis of all peer-reviewed health system.
empirical studies on trust in health websites -- this paper is the While there has been prior work modeling user trust of various
result. Work on this topic was provided from eleven fields technologies [18, 30], research has demonstrated that trust in one
including HCI, Informatics, Medicine, and Decision Making. Our system or artifact does not necessarily translate to another system
findings show that authors often value different facets of trust, or artifact [43]. For example, while there may be valid models of
report different outcomes, and rarely cite each other. Without a trust that workers place in factory machinery, those same models
coherence of terms and values, the task of presenting and of user trust may not apply to eCommerce websites. User trust
understanding how users trust health information on the web will models should then be created and evaluated specific to the
be intractable. domain of application in order to understand the nuances of each
particular system. Likewise, this practice should be used even
Categories and Subject Descriptors when attempting to understand different models of trust in online
K.4.3 [Computers and Society]: Organizational Impacts – activities (i.e. looking up medical information v. purchasing a
Computer supported collaborative work; H.1.2 [Information book). For this reason, current models of how a patient trusts their
Systems]: Human Factors. health care provider cannot be merged with current models of how
users trust eCommerce websites. Instead, there exists a need to
General Terms understand what antecedents induce or affect trust, while also
Reliability, Human Factors, and Theory. understanding the facets and components that make up the
Keywords construct of a user’s trust in health websites.
Health, Trust, Credibility, Review, Empirical Evaluation, Human In this paper we evaluate the body of empirical research on user
Factors, Human-Computer Interaction, Websites trust in e-health websites. This research is comprised of 49 papers
collected in Summer 2009 stemming from a multitude of different
1. Introduction backgrounds such as HCI, Medicine, and Informatics, and
Trust, or the lack there of, is a foundational part of any Decision Making. The data and analysis used in this study was
relationship. In particular it is critical to relationships in the realm created in conjunction with a corresponding study that examined
of healthcare. For instance, the patient trusts that their team of differences in trust measurements and varying construct
healthcare professionals is going to manage their wellness definitions as a function of time. It has been submitted as a
appropriately. Health care professionals additionally trust each separate journal publication. For this paper we reviewed 49
other to work cooperatively and to effectively communicate publications in the realm of trust with one particular question in
pertinent patient information. The increase in use of the internet mind -- one that is of particular important to the IHI audience:
how does research from different disciplines impact the
Permission to make digital or hard copies of all or part of this work for comprehensive understanding of trust in health websites? Our
personal or classroom use is granted without fee provided that copies are goal is to provide an example evaluation method for
not made or distributed for profit or commercial advantage and that copies understanding emerging multidisciplinary constructs, like trust
bear this notice and the full citation on the first page. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific
and credibility, for web communities. This framework will
permission and/or a fee. provide a method to analyze the contributions and parlance of
research on social constructs. To meet this goal we 1) used
IHI’10, November 11–12, 2010, Arlington, Virginia, USA.
content analysis to organize constructs presented in the
Copyright 2010 ACM 978-1-4503-0030-8/10/11...$10.00. manuscripts 2) conducted a social network analysis, and 3) linked
facets to empirical findings from each discipline. We address important quality outcomes such as adherence, satisfaction, and
three key concerns: that the construct of trust is overly fragmented improved health outcomes [1]. Models of trust provided by related
by domain; that researchers on trust in health websites lack a clear disciplines, such as eCommerce, are valuable for highlighting
definition and what definitions they have are conflated; and, that domains that might be correlated, but cannot account for
there is distinct lack of collaboration between authors. We intrinsically different relationships and factors that will impact
conclude with a discussion about the future of research on trust trust in health websites. It is for this reason that research trust in
within the larger socio-technical system and an analysis of the health websites is critical in a field that has an increasing impact
user’s assessment of a website’s credibility. on daily health related interactions.

2. Background & Related Work 2.1 Trust and Credibility


Prior work has demonstrated an increase in the use of health Conceptualization of the construct of trust customarily will focus
technology [37, 61, 71]. A major factor in this increase is the push on defining trust’s facets and antecedents. Facets of trust are the
to move to a paperless medical work environment. And, in doing constructs that it may encompass or make up user trust. Examples
so, there is an assumption that there will be an increase in the taken from the body of reviewed literature include 'well-
availability of credible health information online [29]. However, intentioned' [67], 'truthfulness' [52], and 'integrity' [74].
the user’s employment of health websites can vary from user- Antecedents of trust, on the other hand, are the external factors
generated content, such as Wikipedia, to more authoritative that impact or cause variation in a user’s trust. Examples include
websites, such as WebMD or IHI.gov. Understanding the impact 'relevance', 'authority', or 'disposition' [30].
that these web technologies have on the patient’s perceptions of
quality of care and on the patient-provider relationship can impact In the relevant literature, the constructs of credibility and trust are
the design, implementation, and integration of health websites into linked through their facets and antecedents. In forty-nine papers,
the larger health care system and the adoption of electronic only thirteen included definitions of trust. Of these thirteen
medical records. papers, five papers defined trust as a facet of the construct of
credibility [14, 23, 52, 67, 72]. Heavily citing the work of
A health website, also known as a health infomediary, is a Hovland et al. [24], trust was defined as the well-intentioned,
company that supplies health information: “an online provider that unbiased, and truthful nature of the source of the information.
offers information, advice, guidance, and assessment for wellness Along with expertise, which is defined as the quality of the source
and health-related issues, including referrals and wellness-related of information, trust is used to understand the credibility of a
evaluations” [74]. Sillence et al. have also identified different source of information.
types of web technology that can be labeled as health websites
[60]. These include web portal sites, support groups, charity sites, This conceptualization of the relationship of trust and credibility,
government sites, pharmaceutical sites, sales sites, personal sites, however, is only one variation. In our conception of trust in health
medical databases, media sites, and clinician sites. We have websites we argue that credibility is a part of trust. In a socio-
adopted Sillence et al.’s definition of an e-health website as the technical system multiple humans work with technologies and are
focus for our study. influenced by aspects external to the system as well [49].
Interactions with and surrounding health websites are more than
Research on user mental models of trust in technology has been individual patients working with an interface: they are portions of
reviewed from multiple relevant areas [18, 44, 46]. For example, a complex health care work system. Patients have primary
the work on user trust in automation examined how the design of relationships with their care providers and those that provide care
technology impacts trust in the automation technology [46]. While for them. Trust is therefore constructed at different levels of that
this work demonstrates how users evaluate automated system; patients can have trust relationships with care providers
technologies, it does not take into account the particularities of [1, 68], health institutions [21], and health technologies [44]. A
trust in a website and the impact that trust has on related person’s trust in a health website is formed on the system
relationships. In particular, factors such as ‘consistency’ that are surrounding the website and the health system that the patient is a
paramount to user trust in automation do not necessarily translate part of. For example if a patient finds incorrect information on an
into the realm of trust in health websites where there may be electronic medical record, is their trust diminished in the website
dynamic content. Additionally, in the related work of trust in that hosts the record, the company that provides the record
website, few models account for interpersonal relationships that transferring, or the health institution or care provider? Credibility
are impacted by constructs like trust or similar constructs like would be placed in the company that designs the health record
credibility, authority, and expertise within relationships [52] such system, but it is trust that will influence the patient to either use
as the doctor-patient relationship. the system or like systems again and perhaps other aspects of the
system. Given this understanding, we present our work that
A second group of related research explores user trust in web- reviews user evaluations of health websites.
based technologies, such as eCommerce websites [32, 38, 41].
Research in this area examines the relationship between user trust 3. Method
and the resulting behavior (e.g., purchasing, supplying personal Three methods of analysis were used in order to understand how
information) [18]. While this research is similar, it does not trust might impact the design and implementation of health
account for the impact that the resulting behavior then has on websites and, more importantly, how it may affect the use of such
external relationships – such as the patient-provider relationship. sites. After narrowing down the body of work to 49 papers, we
We focus on this relationship in particular given that of people conducted a word frequency analysis, a social network analysis,
who access health information online, 50% of them report that and, finally, a unique analysis of empirical outcomes by
they then talk to their doctor about the found information [72]. discipline; all of which are explained below. While methods like
Additionally, the trust in the health provider has been linked to
Figure 1. The pie graphs depict the use of the factors 'quality' and 'behavior in reference to the research areas that use them. For
example, 30% of the uses of the word 'quality' came from the area of Medicine.

these may lack statistical significance in such small data sets, the meta-constructs (e.g., ‘difficult’ and ‘difficulty’ were combined).
results can still prove insightful as is shown below. Stop words such as 'available', 'care', 'independent', 'positive',
'positively', 'control', and 'certain’ were omitted as these terms
3.1 Data Sources were often used outside of the context of explaining trust (e.g.,
A comprehensive search of thirteen databases (e.g., “Independent sample t-tests were employed…”). Scripts were
ScienceDirect) was completed in the Summer 2009 using search then used to count the terms across all 49 papers, with each term
terms such as *trust*, website*, webpage*, online*, internet*, being counted a maximum of one time per paper. Our goal with
web site*, web page*, health*, and medic*. The initial search this analysis was to gain a richer understanding of what the
produced over 2,000 articles, which were then evaluated research community believes to be related to trust in health
individually by two reviewers based on pre-established criteria of websites. To do this we had to go beyond provided definitions,
being peer-reviewed, empirical, and including trust as a variable. facets, and antecedence, and examine the words that people used
Disagreements were discussed and then agreed upon for the final through discussing the construct of trust. It was our hypothesis
set. Supplementary computer searches were conducted from that research from different domains would utilize different
articles that appeared in the reference sections of the set of articles constructs related to trust and with different frequencies.
and identified as related; 49 relevant articles were identified and
examined by the authors for the purposes of this paper. Included 3.2.2 Social Network Analysis
papers represented both experimental and theoretical peer- Social network analysis maps and measures relationships between
reviewed conference proceedings and journal papers. Excluded entities; in this study, social network analysis was used to analyze
from the set were publications unrelated to the topic, those not ideas about trust in health websites. Our social network analysis
peer-reviewed, and those yet to be published. explored articles and looked for citation patterns between authors
to explore which entities were citing other. The outcome was used
3.2 Analysis Method to generate hypotheses about the flow of ideas and shared
We employed a meta-analysis framework to examine the impact conceptualizations of the definition of trust. The nodes in our
of a research discipline on the concept of trust. This method is network are the authors of the papers while the links show
appropriate given that a review of trust in health is novel and this relationships or flows between the nodes. Social Network analysis
method has been used for similar work (see [12] for example). provides both a visual and a mathematical analysis of
Three strategies were used to analyze the papers for the impact of relationships. More importantly social network analysis can help
research on trust by discipline: (1) content analysis to examine the visualize shared and disparate ideas in a discipline.
frequency of words used to discuss trust research, (2) social
Several analyses were conducted using NodeXL social network
network analysis to examine how researchers from different areas
analysis software:
cited each other, and (3) an analysis of empirically found research
outcomes by research area. 1. Degree: The number of vertices that the node is connected to.
In this study, Degree is the number of author articles that the
3.2.1 Content Analysis of Word Frequency paper is connected to.
To understand how researchers from different backgrounds would
discuss and evaluate the construct of trust we analyzed the content 2. Betweenness centrality: The extent that a node acts as a
of each paper for the frequency of terms (see [10] for an connection to other nodes. In our review this is operationalized to
example). A list of three hundred and ninety-six terms was mean that a paper may function as an interconnection between
compiled from a thesaurus and from articles [28, 45] that different areas of trust research.
employed empirical methods to derive terms related to user trust
and technology. These terms were then combined to form 108
3. Closeness centrality: The measure of the average shortest
distance from each node. A lower centrality score indicates a
more important or central position in the network. 3.2.3 Analysis of Outcomes by Discipline
To evaluate the outcomes from empirical research on trust in
4. Eigenvector centrality: an adjusted measure that reports a health websites a comprehensive list of all outcomes was created.
combination of how many connections a vertex has with the These outcomes were then organized by what trust can and cannot
importance (degree) of the connections. affect, and factors that affect user trust in health websites. These
The areas of research were color coded by clusters in Figure 2. outcomes were then coded according to which realm of research
Clusters are groups of papers that are interconnected. They are the paper stemmed from (e.g., the Medical field; a list is available
determined with a cluster coefficient, which measures how in Figure 1). Papers could contain multiple findings that would
connected a vertex’s neighbors are to one another (i.e. creating span multiple categories. Two members of the research team
"cliques" of how connected authors and articles are to one another negotiated and reached agreement on the outcome categorizations.
based on citation patterns) [64]. Papers that did not cite any other Our goal with this analysis was to determine the full spectrum of
papers in the body of reviewed work were not included in this empirical research outcomes that were found to affect trust in
analysis. health websites. The hypothesis was that the overall research
spectrum was fragmented by the area that the paper stemmed
Table 1. This table depicts the interconnected papers in a from.
social network analysis with the associated degree,
Specifically, outcomes were combined to create more than 60
betweenness, closeness, and eigenvector centralities.
factors that were then combined to create five categories. The five
categories were: Usability, Content of the Webpage (e.g., the
presence of advertisements and the presence of real life
Eigenvector
Betweennes

examples), Information Factors (e.g., quality, adequacy, and


Centrality

Centrality

Centrality
Closeness

personalization of the information), Contextual Factors (e.g.,


Degree
Vertex

access location and consistency with current beliefs), and User


Demographics (e.g., gender, health, and age).
These categories were then further decomposed into sets of
Sillence et al. (2004) 6 0.962 3.063 0.054
empirically determined factors. For example, Gender was an
Dutta-Bergman (2003) 4 0.494 2.875 0.45 empirically derived factor that was within the category of
Rains (2007) 4 0.323 2.938 0.445 Demographics. To assess whether categories and factors were
correlated with particular areas, the outcomes were then coded by
Lemire et al. (2008) 4 1 2.5 0.416 discipline.
Rains &Karmikel (2009) 4 0.105 3.438 0.395
3.3 Data
Sillence, et al. (2005) 2 0 3.938 0.026
All papers included in our review were published between 2001
Bates et al. (2006) 2 0.226 3.688 0.159 and 2009. 2001 is the earliest date that papers on this topic were
Sillence et al. (2007a) 2 0 3.938 0.026 published. The papers were organized into background categories.
These categories were based on the journal or conference in which
Walther et al. (2004) 2 0.005 3.688 0.267 each paper was published (e.g., a paper published at an
Hong (2006) 2 0.005 3.688 0.267 eCommerce conference would be put into the eCommerce area).
The areas are as follows; communication (n=3) [25, 35, 51];
Huntington et al. (2004) 2 0.947 2.563 0.153
decision making (n=2) [53, 65]; human-computer interaction,
Sillence et al. (2007c) 2 0.068 3.188 0.258 computer science, and the internet –called Computing (n=9) [11,
Fisher et al. (2008) 2 0.902 2.75 0.066 13, 52, 58, 59, 60, 61, 62, 67]; health and medical informatics and
health – called Health Informatics (n=11) [3, 9, 16, 20, 27, 33, 37,
Khoo et al. (2008) 2 0.015 1 0 54, 72]; health services and healthcare (n=3) [6, 8, 17];
Menon et al. (2002) 1 0 4.625 0.051 information and library science (n=3) [23, 26, 39]; media and
society (n=1) [14]; medicine (n=14) [2, 3, 5, 7, 19, 22, 31, 34, 42,
Glenton et al. (2006) 1 0 4 0.017 47, 48, 56, 66, 71]; pharmacology (n=1) [69]; and, social sciences,
Newnham et al. (2006) 1 0 1 0 psychology, and sociology – called Social Sciences (n=3) [4, 50,
Newnham et al. (2005) 1 0 1 0 63]. Rosenbaum et al. paper is counted twice because it is
published in BMC Medical Informatics and Decision Making,
Sillenceet al. (2006) 1 0 4 0.017 which spans two areas.
Sillence et al. (2007b) 1 0 4 0.017
3.4 Content Analysis of Word Frequency
Zahedi& Song (2008) 1 0 1 0 The top ten words used in discussing the research on trust in
Song &Zahedi (2007) 1 0 1 0 health websites across all reviewed papers were: quality (47),
understanding (44), reliability (43), communication (41),
Halkias et al. (2008) 1 0 1 0
experience (40), knowledge (39), accuracy (39), credibility (38),
Hesse et al. (2005) 1 0 1 0 concern (38), and behavior (38). The top fifty most frequently
Wainstein et al. (2006) 1 0 1.5 0 used words were found in 37% of all the papers.
Semere et al. (2003) 1 0 1.5 0
To analyze the parlance of trust by field of study, we charted the continues.
frequency of terms that were present in each discipline. With three
areas encompassing 69% of the total group of papers, these 3.6 Analysis of Outcomes by Area
groups had a large impact on the frequency of use. For example, To analyze the outcomes of trust research in health websites, the
the construct of ‘quality’ was used by every paper in every group outcomes were sorted into sets. The first set of empirical
except for the groups of Computing and Health Services & outcomes evaluated how trust in one source would then affect
Healthcare. ‘Quality’ is used 70% of the time in the papers with trust in another. Within these outcomes, the type of health website
the largest base of work. (Figure 1 depicts the factors of quality being assessed varied by the area of research. For example,
and behavior by area and percentage of use.) In contrast to research from the area of communication focused on how trust in
‘quality’, the construct of ‘behavior’ is still used 56% of the time different sources, like the internet, physicians, and entertainment
by the three areas with the largest base of papers. This is a 14% would affect how users then trusted internet information [35, 51].
decrease in use by the three largest groups. Alternatively, research from the area of decision making focused
on how trust in the internet as a whole would enable or disable
A pattern in use and weight can be detected. The areas of people to look up more information about their medical condition
Medicine, Health Informatics, and Computing for most constructs [53, 65]. Another example is from the area of Medicine. Medical
make up the largest portion of their use. This is to be expected research focused on how trust in medical related online
given that they have the largest base of papers. However, as the information, such as prescription drugs information, online health
amount of use decreases among the three largest groups, the records, or general medical websites, all affected trust in other
remaining areas use of those words remains relatively high. This socio-technical systems such as all online prescription
can be seen in use the construct of ‘behavior’ with 31% of the information, the national healthcare service, and the intention to
papers making up 44% of the usage. Other examples include the use health websites, respectively [2, 19, 42].
term ‘negatively,’ which is used in all Communication and
Decision Making papers; whereas the term ‘negatively’ is only Twelve of the reviewed papers mentioned that the internet has an
used 44% of the time in the computing research and 29% of the effect on doctor-patient relationships. Twelve more cited this
time in the research on Medicine. relationship as an important factor for why trust in health websites
should be studied. Twenty-five papers did not cite this
3.5 Social Network Analysis
Twenty-six papers cited at least one other paper
within the body of reviewed literature. Twenty-
three, or 47% of, papers did not include citations
to other work in the body of reviewed literature
instead opting to cite only within their own fields
of research. Social network analysis provided
insight in to the connections between articles and
authors. See Table 1 for values and Figure 2 for
a visual depiction.
1. Degree: The paper published by Sillence et al.
in 2004 [58] had six vertices from the body of
reviewed papers, which was the highest number.
Followed by Dutta-Bergman [9], Lemire et al.
[37], Rains [51], and Rains and Karmikel [52].
All had four degrees.
2. Betweenness centrality: Several papers did not
act as a connection to other nodes, while the
paper by Lemire et al. [37] reported the highest
betweenness centrality score.
3. Closeness centrality: This measurement
conveyed the average shortest distance from
each node; lower centrality scores indicate more
important positions in the network. The lower
centrality measures appear towards the bottom of
Table 1, indicating that there exists a disconnect
between the papers that have more important
positions and the papers that are highly cited, Figure 2. This network is a representation of the social network analysis of all
which are the top of the table. empirical papers on trust in health websites. Boxes represent papers and edges
4. Eigenvecter centrality: This measurement is an represent citations. Colors and location indicate clusters and connectedness. Note
adjusted measure that reports a combination of that colors used in this figure are not related to those used in Figure 1. Colors are
how many connections a vertex has with the used in this figure to clarify different cliques (e.g., Bates [3], Menon [42], and
importance (degree) of the connections. After Dutta-Bergman [9] form a clique as these three papers cite each other).
adjustments with the eigenvector, the same trend
relationship in their analyses. research to determine the ‘quality of care,’ can overwhelmingly
impact the facets of trust studied by the research community.
The second set of empirical outcomes was organized into factors While it may seem remiss to focus so heavily on the separate
and categories of outcomes that affected trust in health websites communities that research trust, the interrelation of trust research
(Table 2). Given that papers could have outcomes spanning and their disciplines proves to be notably significant. For
multiple categories, the breakdown by the number of papers found example, Fruhling's paper [16], one in the realm of health
in each category is as follows: Usability=12, Content of the informatics, investigates the "User Interface usability factors" of
Webpage=18, Informational Factors=19, Contextual Factors=16, eHealth technology while Flanagin's paper [14], in the area of
and User Demographics=13. Several correlations were found Media & Society, instead examines end-user "perceptions of
between the area of research and the types of outcomes. In the sponsor credibility." Different disciplines ask different questions
realm of Decision Making, both papers coded under this area thus making the analysis of the community of trust an important
corresponded to the Contextual Factors of ‘Experience & Family’ topic to discuss.
and ‘Reputation’. In the areas of Health Informatics, 45% of the
papers focused on the Content of the Information, and Our second analysis of the body of research examined the social
Informational Factors. In Computing, 77% of the outcomes were network of how the papers were interrelated. From this analysis
related to usability, with 66% presenting outcomes that related to we derived four measurements of the number of interconnections
the user’s navigation of a webpage. The outcomes from the papers between papers, whether certain papers served as between papers
within Information and Library Science were diverse having in a network and how interconnected the papers were. These
outcomes to all five categories. This finding is similar within the measurements facilitated an analysis of whether work on trust in
papers from the area of Social Science. Social Sciences presented health websites is building off of each other. Perhaps the largest
outcomes from all of the areas except contextual factors. Within and simplest finding from this analysis is how little the research in
the area of Medicine, many of the papers did not present outcomes this area is citing each other. Recall that only half of the dataset
that could be coded with our framework. However, 33% of their was citing each other. While the 23 papers that were not cited and
papers presented outcomes relating to contextual factors. did not cite any other work might be used to build research in
their respective areas, they are not contributing to the research
4. Discussion discussion on trust and health websites. Trust in health websites is
Research on trust stems from a variety of disciplines: psychology still but an emerging field; however, even with only 49 relevant
[55], sociology [57], and business [40], to cite a few. While each papers in this field, the research is still highly fragmented.
of these fields may not discuss the impact of user trust on health
websites, their definitions of user trust can still prove useful. The The work of Sillence et al. [58] had the highest number of
empirical literature on trust and health websites encompasses 49 connections, six papers, and the second highest betweenness
papers from ten areas. The first analysis of this body of research centrality score. However, what should be noted is that the work
analyzed the content of each paper for the frequency of used of Sillence et al. is a body of six papers. This does not diminish
terms. This research demonstrated that the three largest areas of the body of work that Sillence et al. have produced, but explains
research, Medicine, Health Informatics, and Computing had the the large voice that these papers make up when considering the
largest impact on the frequency of term usage. This is to be larger body of work.
expected since they had the largest number of papers to pull from.
This analysis also found that as the frequency of terms being used Table 2. Percentage of papers by area reporting outcomes by
decreased, the percentage of usage became more equalized by category.
areas. While the domains of Medicine, Health Informatics, and
Computing had the largest proportion of papers, it is important for

Demographics
Content of the

Informational
the other voices to not become marginalized as work in this area

Contextual
becomes more prevalent. The construct of trust in health websites
Webpage
Usability

Factors
Factors

is multi-dimensional thus the verbiage used to discuss it should be


as diverse.
To explain further, the analysis of terms used is important when
considering the antecedents and facets of trust in health websites; Medicine 0 0 13% 33% 20%
the choice of some terms over others to define and describe the Health Informatics 18% 45% 45% 18% 36%
construct of trust, its facets, and antecedents, are equally as Computing 77% 66% 66% 18% 36%
important as the focus of research being on trust and health. Our
research indicates that the terms being used by the areas with the Communication 33% 0 0 0 33%
highest number of publications is not an inclusive parlance. Health Services & 0 0 33% 33% 33%
Therefore, researchers should consider terms being used across all Healthcare
areas when defining and using the construct of trust to understand
Information & 33% 66% 66% 66% 33%
the user’s assessment of web credibility.
Library Science
An equally important finding from this analysis is the fact that Social Sciences 33% 33% 33% 0 33%
there was not a cohesive voice in the discussion of user trust in
Decision Making 50% 100% 100% 100% 0
health websites. With different research areas focusing on
different terms the discussion can become fragmented by area – Media & Society 0 100% 0 100% 0
indicating that research from different areas cannot build off of Pharmacology 0 0 0 0 0
each other and further the overall research discussion. For
example, the term ‘quality,’ which is used frequently for medical
findings that spanned many of the categories. This indicates that
while the parlance may not be the same between all the areas, the
The paper with the highest betweenness score was Lemire et al. outcomes that are being derived may be reproducible and valid.
[37]. However, many papers did not function as a node connecting This is a strong and important finding in such a new area of
two pieces of work. For example, in Figure 2, the work of Song research. However, there were some areas that reported outcomes
and Zahedi [65, 74] stands apart from the rest of the work. The within particular categories of research. For example, Computing
lack of papers functioning to interconnect to each other is a was highly focused on the usability of the health webpage. While
further indicator of the fragmentation present in the work on trust the strength of a particular research area is its focus, it is
in health websites. important for research to be able to build off of the findings of
Closeness centrality and eigenvector centrality scores show that others. This means that the body of research from an area should
the more important papers also have lower degrees. This could be examine how their focus could voice an opinion about the
because of disconnected groupings such as Zahedi [65, 74] and findings from different areas.
Newman [47, 48]. Future work in this area should explore citation Unsurprisingly, prior reviews of trust in other domains find that
patterns and groupings with social networking analysis in a larger the use and definition of trust is fragmented [18, 36]. Even within
group of articles. It is important to understand the connections the larger research area of trust in medical/health informatics,
between theoretical papers and empirical findings and for the definitions are in conflict citing trust as being a behavior and a
discipline to adopt a shared or progressive definition of trust. perception [70] or as an interpersonal interdependence [73]. As
In our third form of analysis, we examined the empirical fields of study mature, however, and the context of how a
outcomes by the areas that the papers stemmed from. To particular construct, like trust, is being used allows the research
accomplish this analysis we divided the outcomes into two community congregate towards a shared definition. For instance,
collections. The first collection demonstrated how trust in one the work of Grabner-Krauter and Kaluscha serves as a valuable
would affect trust in another. For example, one set of outcomes example in examining the related constructs in the study of trust
demonstrated that trust in general media would affect trust in and ecommerce [18]. They found that for trust in ecommerce, the
health websites. Research on the transitive nature of trust ‘trust’ construct is multi-layered and reflects institutional
demonstrated that the areas of research would impact what phenomena as well as personal forms of trust. They argue, as we
trustees were to be studied. Given that the spectrum of health do as well, that understanding how the construct of ‘trust’ is being
websites is only growing as technology grows, understanding the used within a particular domain is paramount to understanding the
transitivity of trust in this medium might be intractable at this outcomes and what is really being studied. It is for this reason,
time. For example, there has been little research on studying the that understanding what research areas are contributing to the
impacts on trust of examining online health information on a study of trust and health websites, and how they are defining their
smart phone versus a home PC. With smart phones becoming one constructs is particularly important.
of the primary methods of people accessing online information
across the world, it seems like this would be a fruitful avenue of 5. Conclusion
research. However, it was not one that was readily visible five In this paper we have presented a review of all empirical literature
years ago because of the growth of this emerging technology. Our on trust and health websites. We have demonstrated how the
analysis demonstrates that while outcomes in on this topic will be constructs of trust and credibility are related, and how the
fragmented by the area of research, this is perhaps a constructive construct of trust is varied by domain. We have addressed how the
discourse. current research has affected our collective understanding of trust.
Our findings could indicate that the definition of trust is
It is in this body of research that insight into how trust in health fragmented across the various fields, and patterns of citation
websites can impact the doctor-patient relationship. between authors in this research area.
Approximately 25% of the papers examined this relationship and
found that patient trust in health websites was affecting or was By prioritizing our analysis of the literature on the areas that the
research came from, our results suggest that the research
affected by the doctor-patient relationship. (See the work of [19,
35, 50].) Or, contradictorily, the work of [48] found that this community is divided. We showed that different fields of study
use different language and factors to discuss the construct of trust.
relationship was not impacted. What is of value to take away from
this review is that the research community on trust in health The social network analysis demonstrated that researchers from
different fields are not citing each other furthering the evidence of
websites finds this relationship critical enough to attempt to study
a fragmented field. This fragmentation may impede efforts to
its impact. The fact that there is still disagreement on how and
define how research from different backgrounds might impact the
why it is affected is a clear indicator of a need for future work.
understanding of trust in health websites. Last, our analysis of the
The second collection of outcomes was organized into categories outcomes of empirical trust research demonstrated that
and factors that impacted a user’s trust in a health website. The researchers are finding contradictory and non-reflective outcomes.
outcomes of the papers that made up categories and factors were All of these findings indicate that there is a need for a common
then coded by area. The span of the number of papers by category definition and model of user trust in health websites. While trust
indicates that there is not one category of outcomes that in health websites may be different from, say, trust in
outnumbers the others. This further shows that research in the five eCommerce, it is important to work towards a unification of this
categories of Usability, Content of a Webpage, Informational definition in both fields. When implementing and deploying
Factors, Contextual Factors, and User Demographics might be a content, such as a health infomediary, the understanding of trust
useful framework for examining trust in health websites. When over all related domains is paramount.
examining the outcomes by area there were some, like Social
Future work in the area of trust in health websites should build off
Sciences and Library and Information Science, which did report
of the 49 papers in this review. Additionally, researchers should
evaluate and consider how the constructs of trust and credibility [11] El-Attar, T., J. Gray, S.N. Nair, R. Ownby and S.J. Czaja.
are specific to the domain of health, instead of trying to Older adults and internet health information seeking. in 49th
amalgamate research from related fields. Work could also be done Annual Meeting of the Human Factors and Ergonomics Society,
to apply the methods used in this paper to the research area of HFES 2005, Sep 26-30 2005. 2005. Santa Monica, CA 90406-
trust in health/medical informatics as a broader field. Last, 1369, United States: Human Factors an Ergonomics Society Inc.
research is emerging that examines trust as part of security, which [12] Eysenbach, G., J. Powell, O. Kuss and E.-R. Sa, Empirical
is a growing area of research. Understanding how the field of Studies Assessing the Quality of Health Information for
security might be affecting the use and definition of trust will be Consumers on the World Wide Web: A Systematic Review.
valuable for future work. JAMA, 2002. 287(20): p. 2691-2700.
6. Acknowledgements [13] Fisher, J., F. Burstein, K. Lynch and K. Lazarenko,
This publication was supported by grant 1UL1RR025011 from the "Usability plus usefulness = trust": an exploratory study of
Clinical & Translational Science Award (CTSA) program of the Australian health web sites. Internet Research, 2008. 18(5): p.
National Center for Research Resources National Institutes of 477-498.
Health and by NSF Awards CCF-0830569 and #0851774. The [14] Flanagin, A.J. and M.J. Metzger, The role of site features,
University of Wisconsin-Madison Systems Engineering Initiative user attributes, and information verification behaviors on the
for Patient Safety (SEIPS) provided support on this project perceived credibility of web-based information. New Media &
http://cqpi.engr.wisc.edu. Society, 2007. 9(2): p. 319-342.
7. References [15] Fox, S. and S. Jones (2009) The Social Life of Health
[1] Anderson, L.A. and R.F. Dedrick, Development of the Trust Information, Pew Research Center’s Internet & American Life
in Physician scale: a measure to assess interpersonal trust in Project Web 2.0, Available from: http://bit.ly/WKgoj.
patient-physician relationships. Psychological Reports, 1990. 67(3 [16] Fruhling, A.L. and S.M. Lee, The influence of user interface
Pt 2): p. 1091-1091. usability on rural consumers' trust of e-health services.
[2] Ayantunde, A.A., N.T. Welch and S.L. Parsons, A survey of International Journal of Electronic Healthcare, 2006. 2(4): p. 305-
patient satisfaction and use of the internet for health information. 21.
International Journal of Clinical Practice, 2007. 61(3): p. 458-462. [17] Glenton, C., E.S. Nilsen and B. Carlsen, Lay perceptions of
[3] Bates, B.R., S. Romina, R. Ahmed and D. Hopson, The evidence-based information - a qualitative evaluation of a website
effect of source credibility on consumers' perceptions of the for back pain sufferers. Bmc Health Services Research, 2006. 6.
quality of health information on the Internet. Medical Informatics [18] Grabner-Krauter, S. and E.A. Kaluscha, Empirical research
and the Internet in Medicine, 2006. 31(1): p. 45-52. in on-line trust: a review and critical assessment. International
[4] Benotsch, E.G., S. Kalichman and L.S. Weinhardt, HIV- Journal of Human-Computer Studies, 2003. 58(6): p. 783-812.
AIDS Patients' Evaluation of Health Information on the Internet: [19] Greenhalgh, T., G.W. Wood, T. Bratan, K. Stramer and S.
The Digital Divide and Vulnerability to Fraudulent Claims. Hinder, Patients' attitudes to the summary care record and
Journal of Consulting and Clinical Psychology, 2004. 72(6): p. HealthSpace: Qualitative study. British Medical Journal, 2008.
1004-1011. 336(7656): p. 1290-+.
[5] Borzekowski, D.L.G. and V.I. Rickert, Adolescent [20] Halkias, D., N. Harkiolakis, P. Thurman and S. Caracatsanis,
cybersurfing for health information - A new resource that crosses Internet use for health-related purposes among Greek consumers.
barriers. Archives of Pediatrics & Adolescent Medicine, 2001. Telemedicine Journal and E-Health, 2008. 14(3): p. 255-260.
155(7): p. 813-817.
[21] Hall, M.A., E. Dugan, B. Zheng and A.K. Mishra, Trust in
[6] Chang, H.H. and C.S. Chang, An assessment of technology- physicians and medical institutions: what is it, can it be measured,
based service encounters & network security on the e-health care and does it matter? The Milbank Quarterly, 2001: p. 613-639.
systems of medical centers in Taiwan. Bmc Health Services
[22] Hesse, B.W., D.E. Nelson, G.L. Kreps, R.T. Croyle, N.K.
Research, 2008. 8.
Arora, B.K. Rimer, et al., Trust and sources of health information
[7] Cima, R.R., K.J. Anderson, D.W. Larson, E.J. Dozois, I. - The impact of the Internet and its implications for health care
Hasson, W.J. Sandborn, et al., Internet use by patients in an providers: Findings from the first Health Information National
inflammatory bowel disease specialty clinic. Inflammatory Bowel Trends Survey. Archives of Internal Medicine, 2005. 165(22): p.
Diseases, 2007. 13(10): p. 1266-1270. 2618-2624.
[8] Dart, J., The internet as a source of health information in [23] Hong, T., The influence of structural and message features
three disparate communities. Australian Health Review, 2008. on Web site credibility. Journal of the American Society for
32(3): p. 559-569. Information Science and Technology, 2006. 57(1): p. 114-127.
[9] Dutta-Bergman, M., Trusted online sources of health [24] Hovland, C., I.L. Janis and J.J. Kelley, Communication and
information: differences in demographics, health beliefs, and Persuasion. 1953, New Haven, CT: Yale University Press.
health-information orientation. Journal of Medical Internet
[25] Huh, J., D.E. DeLorme and L.N. Reid, Factors affecting trust
Research, 2003. 5(3).
in on-line prescription drug information and impact of trust on
[10] Ebert, T.A.E., Interdisciplinary trust meta-analysis. Retrieved behavior following exposure to DTC advertising. Journal of
June, 2007. 25: p. 2007-2007. Health Communication, 2005. 10(8): p. 711-731.
[26] Huntington, P., D. Nicholas, B. Gunter, C. Russell, R. and limitations to conducting successful business on the internet,
Withey and P. Polydoratou, Consumer trust in health information 2006. 156: p. 482-491.
on the web. Aslib Proceedings, 2004. 56(6): p. 373-382. [42] Menon, A., A. Ddeshpande, M. PerriIii and G. Mzinkhan,
[27] Ivanitskaya, L., I. O'Boyle and A.M. Casey, Health Trust in online prescription drug information among Internet
information literacy and competencies of information age users: The impact on information search behavior after exposure
students: Results from the interactive online Research Readiness to direct-to-consumer advertising. Health Marketing Quarterly,
Self-Assessment (RRSA). Journal of Medical Internet Research, 2002. 20(1): p. 17-17.
2006. 8(2). [43] Montague, E.N.H., B.M. Kleiner and W.W. Winchester Iii,
[28] Jian, J.-Y., A.M. Bisantz and C.G. Crury, Foundations for an Empirically understanding trust in medical technology.
Empirically Determined Scale of Trust in Automated Systems. International Journal of Industrial Ergonomics, 2009.
International Journal of Cognitive Ergonomics, 2000. 4(1): p. 53- [44] Montague, E.N.H., B.M. Kleiner and W.W. Winchester Iii,
71. Empirically understanding trust in medical technology.
[29] Kaelber, D.C., A.K. Jha, D. Johnston, B. Middleton and International Journal of Industrial Ergonomics, 2009. 39(4): p.
D.W. Bates, A Research Agenda for Personal Health Records 628-634.
(PHRs). J Am Med Inform Assoc, 2008: p. M2547-M2547. [45] Montague, E.N.H., B.M. Kleiner and W.W. Winchester Iii,
[30] Kelton, K., K.R. Fleischmann and W.A. Wallace, Trust in Empirically understanding trust in medical technology.
digital information. Journal of the American Society for International Journal of Industrial Ergonomics, 2009. In Press,
Information Science and Technology, 2008. 59(3): p. 363-374. Corrected Proof.
[31] Khoo, K., P. Bolt, F.E. Babl, S. Jury and R.D. Goldman, [46] Muir, B.M., Trust in automation: Part I. Theoretical issues in
Health information seeking by parents in the Internet age. Journal the study of trust and human intervention in automated systems.
of Paediatrics and Child Health, 2008. 44(7-8): p. 419-423. Ergonomics, 1994. 37(11): p. 1905-1922.
[32] Komiak, S.X. and I. Benbasat, Understanding Customer [47] Newnham, G.M., W.I. Burns, R.D. Snyder, A.J. Dowling,
Trust in Agent-Mediated Electronic Commerce, Web-Mediated N.F. Ranieri, E.L. Gray, et al., Attitudes of oncology health
Electronic Commerce, and Traditional Commerce. Information professionals to information from the Internet and other media.
Technology and Management, 2004. 5(1): p. 181-207. Medical Journal of Australia, 2005. 183(4): p. 197-200.
[33] LaCoursiere, S.P., M.T. Knobf and R. McCorkle, Cancer [48] Newnham, G.M., W.I. Burns, R.D. Snyder, A.J. Dowling,
Patients' Self-Reported Attitudes About the Internet. Journal of N.F. Ranieri, E.L. Gray, et al., Information from the Internet:
Medical Internet Research, 2005. 7(3). attitudes of Australian oncology patients. Internal Medicine
[34] Larsson, M., A descriptive study of the use of the Internet by Journal, 2006. 36(11): p. 718-723.
women seeking pregnancy-related information. Midwifery, 2009. [49] Pasmore, W.A., Designing Effective Organizations: The
25(1): p. 14-20. Sociotechnical Systems Perspective. 1988: John Wiley & Sons.
[35] Lee, C.J. and R.C. Hornik, Physician Trust Moderates the [50] Powell, J. and A. Clarke, Internet information-seeking in
Internet Use and Physician Visit Relationship. Journal of Health mental health - Population survey. British Journal of Psychiatry,
Communication, 2009. 14(1): p. 70-76. 2006. 189: p. 273-277.
[36] Lee, J.D. and K.A. See, Trust in Automation: Designing for [51] Rains, S.A., Perceptions of traditional information sources
Appropriate Reliance. Human Factors, 2004. 46(1): p. 50-80. and use of the world wide web to seek health information:
[37] Lemire, M., G. Paré, C. Sicotte and C. Harvey, Determinants Findings from the Health Information National Trends Survey.
of Internet use as a preferred source of information on personal Journal of Health Communication, 2007. 12(7): p. 667-680.
health. International Journal of Medical Informatics, 2008. 77(11): [52] Rains, S.A. and C.D. Karmikel, Health information-seeking
p. 723-734. and perceptions of website credibility: Examining Web-use
[38] Lumsden, J. and L. MacKay. How Does Personality Affect orientation, message characteristics, and structural features of
Trust in B2C e-Commerce? in Proceedings of the 8th websites. Computers in Human Behavior, 2009. 25(2): p. 544-
International Conference on Electronic Commerce (ICEC’2006). 553.
2006. Fredericton, New Brunswick, Canada. [53] Rosenbaum, S.E., C. Glenton and J. Cracknell, User
[39] Marshall, L.A. and D. Williams, Health information: does experiences of evidence-based online resources for health
quality count for the consumer? How consumers evaluate the professionals: User testing of The Cochrane Library. Bmc
quality of health information materials across a variety of media. Medical Informatics and Decision Making, 2008. 8.
Journal of Librarianship and Information Science, 2006. 38(3): p. [54] Rosenvinge, J.H., S. Laugerud and P. Hjortdahl, Trust in
141-156. health Websites: A surevey among Norwegian Internet users.
[40] Mayer, R.C., J.H. Davis and D. Schoorman, An Integrative Journal of Telemedicine and Telecare, 2003. 9(3): p. 161-166.
Model of Organizational Trust. The Academy of Management [55] Rotter, J.B., Interpersonal trust, trustworthiness, and
Review, 1995. 20(3): p. 709-734. gullibility. American Psychologist, 1980. 35(1): p. 1-7.
[41] McKnight, H.D. and V. Choudhury, Distrust and trust in [56] Semere, W., H.L. Karamanoukian, M. Levitt, T. Edwards,
B2C e-commerce: do they differ? Proceedings of the 8th M. Murero, G. D'Ancona, et al., A pediatric surgery study: Parent
international conference on Electronic commerce: The new e- usage of the Internet for medical information. Journal of Pediatric
commerce: innovations for conquering current barriers, obstacles Surgery, 2003. 38(4): p. 560-564.
[57] Shapiro, S.P., The Social Control of Impersonal Trust. The [66] Taylor, M.R.G., A. Alman and D.K. Manchester, Use of the
American Journal of Sociology, 1987. 93(3): p. 623-658. internet by patients and their families to obtain genetics-related
[58] Sillence, E., P. Briggs, L. Fishwick and P. Harris. Trust and information. Mayo Clinic Proceedings, 2001. 76(8): p. 772-776.
mistrust of online health sites. in 2004 Conference on Human [67] Theng, Y.-L. and E.-S. Soh. An Asian study of healthcare
Factors in Computing Systems - Proceedings, CHI 2004, Apr 24- Web portals: Implications for healthcare digital libraries. in 8th
29 2004. 2004. New York, NY 10036-5701, United States: International Conference on Asian Digital Libraries, ICADL
Association for Computing Machinery. 2005, Dec 12-15 2005. 2005. Heidelberg, D-69121, Germany:
[59] Sillence, E., P. Briggs, L. Fishwick and P. Harris. Guidelines Springer Verlag.
for developing trust in health websites. in Special interest tracks [68] Thom, D.H. and B.B. Campbell, Patient‐physician trust: an
and posters of the 14th international conference on World Wide exploratory study. Journal of Family Practice, 1997. 44(2): p.
Web. 2005. Chiba, Japan: ACM. 169(8).
[60] Sillence, E., P. Briggs, P. Harris and L. Fishwick, A [69] Tio, J., A. LaCaze and W.N. Cottrell, Ascertaining consumer
framework for understanding trust factors in web-based health perspectives of medication information sources using a modified
advice. International Journal of Human-Computer Studies, 2006. repertory grid technique. Pharmacy World & Science, 2007.
64(8): p. 697-713. 29(2): p. 73-80.
[61] Sillence, E., P. Briggs, P. Harris and L. Fishwick, Going [70] Tung, F.-C., S.-c. Chang and C.-M. Chou, An extension of
online for health advice: Changes in usage and trust practices over trust and TAM model with IDT in the adoption of the electronic
the last five years. Interacting with Computers, 2007. 19(3): p. logistics information system in HIS in the medical industry.
397-406. International Journal of Medical Informatics, 2008. 77(5): p. 324-
[62] Sillence, E., P. Briggs, P. Harris and L. Fishwick, Health 335.
Websites that people can trust - the case of hypertension. [71] Wainstein, B.K., K. Sterling-Levis, S.A. Baker, J. Taitz and
Interacting with Computers, 2007. 19(1): p. 32-42. M. Brydon, Use of the Internet by parents of paediatric patients.
[63] Sillence, E., P. Briggs, P.R. Harris and L. Fishwick, How do Journal of Paediatrics and Child Health, 2006. 42(9): p. 528-532.
patients evaluate and make use of online health information? [72] Walther, J.B., Z.M. Wang and T. Loh. The effect of top-level
Social Science & Medicine, 2007. 64(9): p. 1853-1862. domains and advertisements on health Web site credibility. 2004.
[64] Smith, M.A., B. Shneiderman, N. Milic-Frayling, E.M. [73] Winkelman, W.J., K.J. Leonard and P.G. Rossos, Patient-
Rodrigues, V. Barash, C. Dunne, et al. Analyzing (social media) Perceived Usefulness of Online Electronic Medical Records:
networks with NodeXL. in Proceedings of the fourth international Employing Grounded Theory in the Development of Information
conference on Communities and technologies. 2009: ACM New and Communication Technologies for Use by Patients Living with
York, NY, USA. Chronic Illness. Journal of the American Medical Informatics
[65] Song, J. and F.M. Zahedi, Trust in health infomediaries. Association, 2005. 12(3): p. 306-314.
Decis. Support Syst., 2007. 43(2): p. 390-407. [74] Zahedi, F. and J. Song, Dynamics of trust revision: Using
health infomediaries. Journal of Management Information
Systems, 2008. 24(4): p. 225-248.