Anda di halaman 1dari 9


QHRXXX10.1177/1049732316668298Qualitative Health ResearchVandermause et al.

Qualitative Health Research

Qualitative Methods in Patient-Centered

2017, Vol. 27(3) 434­–442
© The Author(s) 2016
Reprints and permissions:
Outcomes Research
DOI: 10.1177/1049732316668298

Roxanne Vandermause1, Frances K. Barg2, Laura Esmail3,

Lauren Edmundson3, Samantha Girard4, and A. Ross Perfetti2

The Patient-Centered Outcomes Research Institute (PCORI), created to fund research guided by patients, caregivers,
and the broader health care community, offers a new research venue. Many (41 of 50) first funded projects involved
qualitative research methods. This study was completed to examine the current state of the science of qualitative
methodologies used in PCORI-funded research. Principal investigators participated in phenomenological interviews to
learn (a) how do researchers using qualitative methods experience seeking funding for, implementing and disseminating
their work; and (b) how may qualitative methods advance the quality and relevance of evidence for patients? Results
showed the experience of doing qualitative research in the current research climate as “Being a bona fide qualitative
researcher: Staying true to research aims while negotiating challenges,” with overlapping patterns: (a) researching
the elemental, (b) expecting surprise, and (c) pushing boundaries. The nature of qualitative work today was explicitly
described and is rendered in this article.

methodology; research evaluation; research design; philosophical hermeneutics; United States

The Patient-Centered Outcomes Research Institute funded by PCORI in 2012. We were particularly focused
(PCORI) is an example of a relatively new federal initia- on the current climate of qualitative research, with the
tive that funds research that may include mixed and quali- entrance of PCORI as an epochal period.
tative methodologies. The PCORI, an independent Decades of discussion in the field of qualitative
nonprofit, nongovernmental organization was authorized research have produced commentary, methods descrip-
by Congress in 2010 “to improve the quality and relevance tions, and debates between “quantitative” and “qualita-
of evidence available to help patients, caregivers, clini- tive” researchers and among qualitative researchers using
cians, employers, insurers, and policy makers make various forms. Many approaches to qualitative research
informed health decisions” (PCORI, 2015). During the have been subjected to critiques centered on lack of clarity
first year, PCORI funded a majority of research proposals for evaluating rigor and concerns about whether qualita-
that included qualitative components in the studies, given tive methods have the capacity to answer important
the apparent utility of mixed method and qualitative meth- research questions. Increasing recognition for the need to
odologies in answering research questions that prioritize address problems relevant to practice and renewed interest
patient needs, voices, and perspectives. This is important in health care science to include patient and stakeholder
because funding for qualitative research has been long voices have emphasized the importance of revisiting past
regarded as a challenge. Expanded approaches in patient- debates and clarifying understandings and current use of
centered research and funding opportunities have qualitative methodologies.
increased (Miller, Crabtree, Harrison, & Fennell, 2013),
yet investigators report continued barriers to funding,
implementing, and disseminating qualitative research. 1
(Willis, 2007). The current opportunity to explore the University of Missouri–St. Louis, Missouri, USA
University of Pennsylvania, Philadelphia, Pennsylvania, USA
qualitative research experience in the context of the avail- 3
Academy Health, Washington, District of Columbia, USA
ability of a new federal funding source offered the poten- 4
Washington State University, Pullman, Washington, USA
tial to better inform the community of researchers about
Corresponding Author:
the evolving field of qualitative research. In this study, we Roxanne Vandermause, University of Missouri–St. Louis, One
set out to examine a specific subset of qualitative research, University Drive, Seton 315, St. Louis, MO 63121-4400, USA.
the proposals and experiences of inaugural researchers Email:
Vandermause et al. 435

The literature on qualitative methodologies is exten- and patient-centered outcomes in their work. This
sive. There are numerous books and peer-reviewed arti- research is intended to broaden understanding and use of
cles that speak to the evolution, types, strengths, and qualitative research paradigms in PCOR and current
challenges of using qualitative methods. Some authors health care research.
take readers through the process of “doing” the research
(Smythe & Giddings, 2007), although such literature also
integrates epistemological discussions as well. Decades
of scholarship attest to the breadth of methodologies in The specific aims of this project were to (a) Generate an
this paradigm and chronological descriptions of epochal interpretation of the state of the science of qualitative
periods show the climate of qualitative research is chang- research methods, specifically to illuminate current
ing (Lincoln, Lynham & Guba, 2011; Munhall, 2012; experiences with funding, implementing and disseminat-
Willis, 2007). Various methodological forms have sharp- ing qualitative findings in PCOR, and (b) Identify the
ened the field (Wertz, McMullen, Josselson, Anderson, & unique contribution that qualitative methodology offers
McSpadden, 2011). We were interested in gaining a to PCOR.
cross-sectional focus on the nature of qualitative research An ontological approach was engaged to conduct this
at a point when the context of such research had shifted. study, which was supported by PCORI Engagement and
Our collaboration began through the PCORI Pilot Implementation funds. It included in-depth interviews,
Project Learning Network (PPPLN), a collaborative ongoing hermeneutic analyses, and presentation to sev-
learning community initiated by PCORI in partnership eral researcher groups, including research faculty at two
with AcademyHealth for pilot project investigators. We participating research institutions at opposite sides of the
proposed, during this association, two studies to examine country and participants at an international qualitative
the current state of the science of qualitative research, one research conference. These presentations were designed
to examine the components of funded studies using quali- to share developing results of the interpretive findings
tative methods (Barg, Vandermause, Edmundson, Esmail, and to elicit responses that could be incorporated into the
& Perfetti, 2015) and the current study to analyze the findings. Such conversations are consistent with philo-
experience of doing qualitative research in today’s sophical hermeneutic approaches which are used to inter-
research climate, with a particular emphasis on what pret underlying meaning through analysis of transcribed
PCORI describes as patient-centered outcomes research interviews and interpretive commentary (Diekelmann &
(PCOR). The long-term goal was to strengthen the under- Ironside, 2006). Dissemination events served as auditing
standing of qualitative research in conventional health mechanisms and added to the interpretive development of
care science and to increase the acceptability of using findings.
multiple methodologies to answer research questions.
In our recent study (Barg et al., 2015), investigators
Recruitment and Data Collection
examined the research plans of PCORI pilot project
investigators (41 of 50 funded projects used qualitative Investigators from the inaugural group of PCORI-funded
methods) and queried them about the resources and chal- pilot projects, who participated in our preliminary study
lenges involved in planning for and using qualitative (Barg et al., 2015), were contacted for willingness to par-
methods in their studies. Investigators described numer- ticipate in interviews. A total of 19 investigators responded
ous challenges (e.g., lack of understanding of qualitative to the invitations and provided in-depth telephone inter-
approaches by the scientific community; lack of time for views, lasting 0.5 to 1.5 hours. Incentives were not
recruitment, stakeholder engagement, and analysis; prob- offered. Interviews were conducted by two investigators
lems locating dissemination venues). Despite these chal- experienced in conducting hermeneutic interviews
lenges, investigators also described benefits in qualitative (Vandermause & Fleming, 2011). Questions were open-
approaches (e.g., understanding the patient perspective; ended, though guided by findings from the preliminary
learning unexpected findings; calling upon community study of qualitative research proposals and investigators’
and patient stakeholders for practical, relevant informa- one-page summaries of the challenges and opportunities
tion). This information helped to discern some of the they faced in enacting their work. Participants were asked
challenges that are experienced in today’s research envi- to describe their experiences generating research find-
ronment and set the conditions for the current study, an ings, applying findings to practice, and disseminating the
examination of the day-to-day experiences of PCORI pilot project work. Interviews were transcribed verbatim,
pilot project qualitative investigators. de-identified and shared with the research team via secure
The current study, using the information obtained servers. This study was deemed exempt by the Institutional
from the descriptive results of our preliminary work, Review Boards at the University of Pennsylvania and
focused on how researchers incorporated patients’ voices Washington State University.
436 Qualitative Health Research 27(3)

Data Analysis are described below with exemplars and interpretation

embedded in the discerned patterns.
The transcribed texts were analyzed via a philosophical
hermeneutic approach to generate an interpretation of
meaning, using a graduated data collection and analysis Pattern: Researching the Elemental
process (Diekelmann & Ironside, 2006). Texts were ana- A frequently voiced idea was that qualitative research is
lyzed in iterative fashion, using in-depth written interpre- done to understand very basic human phenomena, to
tive summaries of each text, followed by discussions know patients and those who care for them better, and to
across texts between two hermeneutic analysts and get “underneath” happenstance to get to the “real.” There
expanding the sessions to include all co-investigators for were numerous references to the manner in which quali-
various select transcripts. A focus on principles of herme- tative research designs are necessary to answer questions
neutic analysis was maintained (Crist & Tanner, 2003; about primal human concerns. Matters essential or intrin-
Vandermause, 2012). That is, texts were read for struc- sic to human nature, constitutive aspects of experience, or
ture, language, content, and inference, resulting in the practices grounded in everyday activities are addressed.
emergence of various common manifestations of experi- For example, one researcher states,
ence, described below. Data were managed via Microsoft
Word files. Initial patterns, named “Being genuine: When I do an interview, I want to get very involved in
Researching as elemental” and “Being safe: Researching hearing the other’s experience. I was able to get underneath
to make a difference” centered the description of the some of the things that were important to people. You know,
emerging phenomena in the study on integrity and risk. get into their source of identity and who they are—who they
Member checking is not used in this form of hermeneu- were as a person in relation to their disease.
tics. Rather, a series of validation steps and consensus
discussion among all members of the research team, For many, these essential discussions were extended, iter-
some with qualitative expertise and some without, led to ative processes involving intensive day-long interviews
the naming of recurrent patterns. After further analysis of and focus group conversations. There was also the desire
the interpretations alongside co-investigator commen- to “drill down” for persons’ reactions, as in the following
tary, reviewed transcript analyses, and elicitations at dis- example.
semination events, the following patterns were refined
and explicated, comprising the study results. And we wanted to do it in a nuanced way. We wanted to hear
what words they used. We wanted to hear what their reactions
were. We wanted to hear if they changed their minds back
Results and forth—as we thought people might, depending on what
other people at the table were saying. And we thought we
The multi-layered text that comprised data for these find- would get a much fuller picture than if we only did a survey.
ings was anchored by the verbatim interview transcripts
of 19 primary investigators of studies funded by PCORI It was important to these researchers to get to the essence
during its inaugural year. These investigators had various of the patient and stakeholder experience to fully under-
levels of expertise, self-describing as having years of stand. Taking the time, circling the data to get to the prime
qualitative experience, very little qualitative experience, or key matters, was required. “We kept asking ourselves,
or some experience with qualitative methods. They repre- how is this helping us understand the patient’s perspec-
sented western, midwestern, eastern, and southern regions tive?” What a patient says, elementally, literally, matters,
of the United States. They were in the final stages of data and this was conveyed simply.
analysis and dissemination of their 2-year projects.
From these texts and the interpretive process that fol- What does hearing the patient’s voice mean? Just like—it’s
lowed, the experience of doing qualitative research in the just hearing their perspective and not making an assumption
current research climate can be understood as “Being a that you know their perspective.
bona fide qualitative researcher: Staying true to research
aims while negotiating challenges.” This overarching The pursuit of the prime or elemental idea was often
pattern (Benner, 1994) subsumed three general, overlap- discussed in terms of the populations of interest, the
ping areas of experience (patterns) that describe the phe- patients and those around them. Thus, these interviews
nomenon of interest (the experience of doing qualitative reflected the inseparability of qualitative research and
research in PCOR): (a) researching the elemental, (b) stakeholder engagement. Many commented on the way
expecting surprise, and (c) pushing boundaries. All of the PCORI is shaping the landscape for qualitative research
accounts signal these patterns, interpret meaning from the by prioritizing stakeholder engagement. An example of
textual data, and guide research implications. Findings this was expressed by a researcher who identified as
Vandermause et al. 437

having a well-established base of community partners research questions and methods on a case by case basis.
brought into the PCOR project. In the project, persons The unsolicited tendency to clarify personal and profes-
with a stake in the issue were “thrilled to have a voice,” to sional associations with methodology was commonly
be heard. This participant told a story about how those demonstrated. This underscored the idea that, for the
gathered in a discussion group expressed themselves, researcher, qualitative research is a serious and genuine
revealing emotions that they wanted to register and endeavor, requiring devotion to principles that reflects on
changing their emotions when they recognized that the researcher as a person, and is substantial in the realm
researchers really wanted to hear what they had to say. of science.
In some accounts, researchers underscored the value
And . . . having the face-to-face discussion provided a format of learning what patients valued, even if it ran counter to
for people to talk about what they were passionate about. providers’ wisdom. In the following example, a researcher
What wasn’t on the survey or just what was most important heard from the patient why he had made a particular
to them. And that really—you could feel the kind of health care decision.
prevailing feeling in the room change as people had a chance
to talk.
. . . we show here’s one perspective and here’s another
perspective. What’s right for you? We used a patient
In one instance, a stakeholder-partner held a negative perspective [that was different than what is usually
opinion about particular survey questions. Nevertheless, recommended]. But hearing his story and digging deeper in
the group continued a respectful, productive conversa- it would probably be a shock to some of those doctors who
tion. This example showed the value in stakeholder got angry at him for not getting a (medical device reference).
engagement because it ultimately led to a practical under-
standing of issues that mattered to the population of inter- In this excerpt, the researcher points out the natural and
est, which the participant distinguished as different from true, elemental, perspective from that of the patient,
traditional data collection methods. emphasizing the genuine nature of the data obtained by
getting to a core issue.
People were really engaged in our topic. You know, everyone In summary, “researching the elemental” involves the
who came was—there was a lot of positive group dynamic. research process and outcome as well as the researcher
Everyone who came finished the survey and stayed and
identity—an investment in quality research. The first
talked a long time.
part, involving the process and outcome, relates to the
many references to getting to the bottom of things, under-
Positive moods surrounding patient and other stakeholder
standing what really happens in everyday life, eliciting
discussions commonly anchored knowledge acquisition
the patient and stakeholder voice. These references are
and the sense that they were getting to essential matters,
found and can be exemplified in nearly every transcript.
generating excitement for the researchers. This idea will
Sometimes researchers talked about ways these methods
be discussed further in the following section.
enhanced or complemented statistical research methods;
In “researching the elemental,” there is also a sense
sometimes they conveyed the way new and surprising
that the work must be done well, thoroughly, enacted with
ideas came forth. Bonus findings occur, those that the
scholarly rigor. Thus, the experience, even identity of the
researcher could not have anticipated, and this makes the
researcher as “bona fide” with regard to qualitative exper-
discovery process genuine. The idea of being genuine is
tise, is basic. Researchers often described themselves by
extended to focusing on things that matter to patients and
their methodologies and years of experience or lack of
those involved with them, illuminating what’s important
experience. Furthermore, there is an accompanying idea
as well as what is elementally learned. The basic human
that such research is “genuine” and those who research in
experience, ontological as well as practical, is recognized
this way must be genuine, attesting to their basic and
using qualitative methods. The second part of being gen-
“elemental” nature.
uine in researching the elemental is related to the
researcher’s ability to adequately process and synthesize
I chose qualitative methods because I feel strongly that it is
a powerful research paradigm that is underutilized, the information. Researchers who have used qualitative
undervalued, and often critiqued . . . it was my career goal to methods over time identify with their approaches.
bring as much rigor and validity to qualitative methods.

Pattern: Expecting Surprise

This excerpt illustrates a researcher with a “career goal”
of elevating the science of qualitative methods, assuming In most accounts, there is a sense that qualitative research
an identity as a qualitative researcher. Others were careful results in findings that could not have been anticipated.
NOT to identify as such, expressing a fidelity to matching This is said literally in many cases and insinuated
438 Qualitative Health Research 27(3)

throughout the data. As noted above, the idea that ele- And the patient did say, “Well, I understand there was a
mental human experiences must be addressed using qual- problem with my heart during surgery, so I had to stay
itative methods is a common understanding. Furthermore, overnight, and I didn’t expect to have to stay overnight.”
researchers who use these methods may be “devoted” to And then he said, “Oh, and the door of my room was really
squeaky, so I didn’t get any sleep all night long.” And we
the approaches they use for this very reason. There were
asked each reporter to rate the severity of the event they’re
frequent accounts of joyful, surprising, and meaningful
reporting, and that patient rated the severity of the squeaky
events in the doing of qualitative research. door much higher than the severity of their cardiac arrest in
the operating room.
And a couple of the folks in that [focus] group said that they
were just so (I hate to use this term, but) grateful—or pleased
This response differed from the clinicians’ expectations.
about being asked their opinions about something that was,
you know, a policy question. And they were just—everybody
Patients and providers had different agendas and, thus,
who heard that on the team was like tearing up. So—I know different perceptions on care.
that happens in qualitative research sometimes, but it was
also I think speaking to—we pulled out all the stops in terms But the biggest take-home is that patients and clinicians . . .
of recruiting from low-income and underserved communities, in the same place and experiencing the same system of care
and that was the emphasis of our project. have two different worldviews of what that’s all about. And
if one of the goals of healthcare is to deliver patient centered
care, clinicians are going to need to learn about this patient
Researchers’ expressions of joyfulness were unexpected
worldview and presumably address it.
and distinct findings in the research, in that the pattern
was conveyed so strongly in many of the transcripts. Part
Researchers heard that complications that occurred in
of this was attributed to the relevance of the findings.
surgery or treatment were less significant to patients than
Also, the possibility of learning something new or com-
a “squeaky door” or getting food on time. “I think what I
pletely unexpected was serendipitous and welcomed by
just said is what I learned, which is that the patient view
was so orthogonal to the clinician view . . .” Other
The pattern, “Expecting surprise” is worded as such
researchers told similar stories. “Nobody had ever told
because the semantics imply a paradoxical element that is
me that when I had them in the exam room as a patient.”
characteristic of the research. There is an attitude of open-
The inclusion of patients and stakeholders into the
ness that is underscored by an expectation that what is
research process meant that expectations were up-ended.
questioned is not yet known. Stories patients tell often
reveal “a bonus finding that happens in qualitative
I can’t think of a single project where I haven’t been
research.” This is different from experimental designs surprised. Bringing stakeholders in-they’re going to look at
where the research question carries with it a hypothesis it in a different way, and you’re going to learn from that. And
that is to be accepted or rejected. Thus, from the onset, a that’s so important . . .
qualitative researcher must assume there is a result she or
he cannot entirely foresee. Ultimately, unanticipated findings in these studies
were revelatory and inspired the teams, both not only in
If I want to get into something that I don’t have any idea terms of promulgating interventions but also in terms of
what I might find—if I want to get into something that’s doing something joyful in research. The research that is
completely new, or if I want to open my mind to the
done is so close to the perspectives of those involved and
possibility that there is a phenomenon, let’s say, or an
experience that I haven’t heard about and learned, I really
often includes asking questions that have not been asked
have to use these kinds of techniques—a technique of previously, making the research findings novel. The
interpretive interviewing, a deeper kind of approach . . . unexpected nature of this form of scholarship brings an
You know, [as a participant] I have to kind of think about aspect of joy even alongside the challenges. For many,
things to tell you about that I might not have thought about this aspect makes the challenges worth continuing quali-
and I wouldn’t necessarily answer in a little survey because tative work.
I’m having to dig a little bit deeper to think about the
issues . . .
Pattern: Pushing Boundaries
This attentiveness to being open necessarily invites The changing research climate is inclusive of qualitative
new and unexpected results. One researcher described research, despite lingering issues related to its acceptance
patient responses that were not what clinicians expected, in the scientific domain. In these accounts, investigators
for example, when asking patients whether they had any voiced challenges specific to qualitative research. Many
outstanding experiences during their hospitalization. challenges related to scientific acceptability; others related
Vandermause et al. 439

to the nature of the work itself. “Story-telling is a weapon demonstrating rigor is a contributing problem. Many
of the weak and not used by the powerful.” Qualitative researchers understood that openness in methods leaves
research can be ambiguous in nature (e.g., changing one open to criticism, and that the value in qualitative
research questions and methods, nonspecific or conflict- methodologies makes this vulnerability tolerable. The
ing evaluation criteria, misconceptions about the proper idea that qualitative methods push boundaries and create
execution of the work) and consequently might meet with risk is strong in today’s qualitative research climate. The
reviewer (of funding or dissemination venues) bias or same concerns that showed themselves decades ago
naiveté, according to experienced qualitative researchers. remain active, even though various qualitative method-
This challenges them as they write proposals. ologies are somewhat more acceptable in the scientific
Sample size and set procedures are the default assumption, Finally, additional challenges of qualitative work
so we go with what is traditional. Yet, there are times in a exacerbate the risk associated with the lingering threat of
qualitative study that we might change our way of asking scientific unacceptability. The issue of time surfaced in
questions or shift the analysis a bit. But we have to present a these interviews, which did not come up in our previous
proper plan upfront to be taken seriously.
work examining research plans. In many of the inter-
views, researchers talked about the extra time needed for
There is, concomitantly, a risk to engaging in qualita-
quality analysis and the challenges of data collection
tive methods. Some researchers did not want to be identi-
methods like interviews.
fied as “qualitative” researchers, implying that there may
be a career disadvantage in identifying as such. For oth- Well, I think one of the challenges, if you’re not experienced
ers, being a “bona fide” or genuine qualitative researcher in the area, you need to understand is that once you’ve
was considered important for the enactment of good actually done a data collection, you’re really maybe a quarter
research. of the way done (Laughter), if you’re really doing a good job
Concerned because reviewers might not understand with your analysis. And that takes much more time than
their methodologies, researchers struggled with how to folks realize. And, it’s really important to dedicate and
present their proposals. devote that time. I think the other thing, you know, as I’ve
said on study sections and grant review panels, I think
As an example, I am used to putting in sample size sometimes folks don’t always recognize both on the
determinations and other measured approaches so that the reviewer’s side and sometimes you’ll see this in proposals
procedure fits into a grant format or provides what the reviewer too, that it’s kind of obvious that somebody just threw in a
expects. In this case, we didn’t know what the reviewers few qualitative—oh, we’re going to do some interviews . . .
might think about procedures so we put in a set sample size but they hadn’t really thought it through.
anyway. The idea is to let them know we are being serious
about the project. The element of time and experience make for quality.
This participant asserted that the quality is not always
Numerous excerpts show that researchers are troubled by there, that the “standards” for qualitative work have not
the ambiguous and sometimes marginalized approaches yet reached those of experimental methods. It is interest-
they must use to answer questions pertaining to patients’ ing that, in the discussion of quality, it is often that other
perceptions or experience. “When you push the boundar- empirical forms of research are brought into the conver-
ies a little bit [beyond] established views and approaches sation. The traditional understanding that use of statistics
to research, you should expect not to be treated with contributes to rigor in approach tends to move the conver-
acclaim.” When the research questions require qualitative sation to the need for quantitative components, mixed
methods, researchers move past the risks, even if reluc- methods that complement one another as a preferred
tant. “I think a lot of researchers worry that when they design. Many of these researchers wanted to illuminate
bring in a patient representative and advocate, that it’s their experience or respect for quantitative methods.
going to mean that they won’t be able to do science any- At the same time, there was a pervasive recognition
more.” Some researchers expressed concern that they that the qualitative components of research are the
could be leaving some stakeholders out of the discussion designs needed to learn what patients and stakeholders
and were hearing from more eloquent and not all voices. really need. All the participants emphasized the value in
This dilemma shows the paradoxes qualitative research- qualitative methods, despite challenges of time, funding,
ers face, adding to the burden related to acceptability of and dissemination. “Where the hell to publish?” was an
their methods. outstanding excerpt because, though pungent, it under-
Participants spoke about legitimacy of qualitative scores some of the frustration experienced by research-
research. There were repeated intimations in most tran- ers wanting to make a difference in their work for
scripts that unacceptability by scientists persists and that improved health care.
440 Qualitative Health Research 27(3)

These practical challenges show the nature of qualita- PCORI funding cycle yielded a majority of studies using
tive work as demanding. At the same time, the results can qualitative methodologies. By explicitly incorporating
transform patient outcomes. Standing outside traditional patient-centered outcomes and stakeholder engagement
boundaries of science while contributing to extant scien- into the research enterprise, PCORI, as a funding source,
tific knowledge, qualitative researchers broaden and has exerted a strong influence over the nature of the work
deepen the research enterprise while they struggle for that is being done (Forsythe et al., 2015). The funding
appropriate funding and dissemination avenues. The focus on patient engagement attracted investigators who
PCORI was recognized as a welcome resource for are skilled in eliciting “insider” perspectives on the expe-
advancing the quality and relevance of evidence for rience of illness. Patient-centeredness, along with stake-
patients. holder engagement in the research enterprise, may inspire
more qualitative and mixed methods, even in compara-
tive effectiveness research, advancing the quality and rel-
evance of evidence in health care research.
This study focused on the experience of doing qualitative Qualitative research that has been shaped by the
research. The literature on the experience of doing quali- PCORI’s focus manifested in this study as addressing the
tative research is often intertwined with discussions of elemental, answering genuine questions and making a
how to do such research. In other words, studies of the difference. Researchers enjoy and care deeply about such
experience itself are not distinct from the guiding intel- work and are willing to push boundaries and take risks in
lectual foundations of qualitative research. Participants in their methodological decisions. The challenges of doing
this study, likewise, not only talked about their experi- qualitative research are offset by the rewards experienced
ence but also spoke at length about how they did their in using such methods. Researchers are willing to commit
work and what their work revealed in the context of extra time and unusual efforts, but they have concerns
PCOR. References to self-reflection, reflexivity, and about acceptability by reviewers, time required, and
identifying or de-identifying with the research focus, ambiguity in procedures. Thus, “Being a bona fide quali-
present in the literature over decades, surfaced in this tative researcher: Staying true to research aims while
study. Recent literature continues to bring the researcher negotiating challenges” reflects a persistent characteriza-
and the research together, personalizing the act of tion, even in a research climate that welcomes attention to
research. For example, Holloway and Biley (2011) dis- patient-centered needs and stakeholder engagement.
cuss the necessity of the involvement of the person in a Good elemental research requires transparency. For
thoughtful way, balancing what is learned from partici- qualitative researchers, it is important to show the process
pants with what is known about the self. Thoreson and of the work to be trusted in the result. As in all research,
Öhlén (2015) link the personal experience to knowledge attention to ethics and scholarly rigor is essential. The
development, resurrecting Merleau-Ponty’s (1962) dis- nature of these methods is dynamic and flexible. Frequent
cussions on embodied experience as an aspect of integrat- protocol changes and shifting views during data collec-
ing the subjective and objective worlds. This goes along tion and analysis are contrary to scientific assumptions.
with van Manen’s (1991) extensive work, carried forward This is part of the experience of risk—claiming and
today. The bona fides manifested in the experiences of the showing expertise while taking a position of being open.
participants in this study underscored the importance of As part of the scientific minority, qualitative researchers
the researchers’ identities as rigorous qualitative research- push boundaries and therefore must defend their pro-
ers. This identification worked for and against them, cesses and their professional identities to be well regarded
depending upon the perceptions of others in the research- in the scientific community.
ing community. Therefore, moving through challenge Most qualitative researchers are aware of guidelines
was personal and necessary and showed itself in the pat- established by national groups, including The National
terns of researching the elemental or direct prime experi- Institute of Health’s (2010) “Best Practices for Mixed
ence, expecting surprise or feeling joyfully open, and Methods Research in the Health Sciences” and
pushing boundaries or doing research despite the chal- “Qualitative Methods in Health Research: Opportunities
lenges. In many ways, these findings are not new. They and Considerations in Application and Review”; The
corroborate what qualitative researchers have been writ- Robert Wood Johnson Foundation’s documents’
ing about over time. “Qualitative Research Guidance project” and “Guidelines
The research climate of today includes a renewed for Reviewing Qualitative Reports and Grant
focus on patient-centered care. The PCORI has created Applications”; and numerous publications in the general
new opportunities for funding research that place the health care literature written for the scientific community
patient and other stakeholders at the center of inquiry. In on qualitative research best practices (Green et al., 2007;
this study we have seen that the response to the first Maggs-Rapport, 2001; Marshall, Cardon, Poddar, &
Vandermause et al. 441

Fontenot, 2013; Nelson, 2008). Many of these documents an added value, especially as the experience of doing
are useful, though various qualitative strategies require qualitative research meshes with the experience of doing
specific guidelines. There are nuances in the variety of a good job, producing revelatory and relevant results that
methodological forms that researchers use. Such nuances can be applied to practice.
are neither widely understood nor consistently referenced
by researchers, making evaluation and critique difficult.
Good research projects may lack credible dissemination
venues. Our study adds to the literature by focusing on This study illuminates the importance of qualitative
ways that the very qualities of qualitative research, when research in PCOR, the challenges in designing, funding,
done well, extend the power of patient and stakeholder implementing, and disseminating qualitative work, and
centered research, although the challenges to funding, the need to recognize and establish appropriate mecha-
enacting, and disseminating the work remain. nisms for enacting the research. The relevance of qualita-
Finally, in recent years, the emphasis on stakeholder tive research in PCOR is evident, and the institution of
engagement has raised new questions about how to con- the PCORI as a mechanism for improving patient out-
duct patient-centered research in the age of PCOR. Early comes is highlighted. The results of this study emphasize
work in Community-based Participatory Research (Israel, the profound role that funders play in the course of health
Eng, Schultz, & Parker, 2005; Minkler, 2000; Reason, care delivery and the generation of new knowledge. The
1994; Small, 1995) has initiated the understanding and experiences of this group of investigators explicated the
acceptance of methods that include patient and stake- practice of current federally funded qualitative research-
holder voices. Various community partners are being ers, updating the literature in this important methodologi-
brought into the research enterprise. The value of stake- cal area.
holder engagement has yet to be thoroughly evaluated in
research (Esmail, Moore, & Rein, 2015), though there is Declaration of Conflicting Interests
a strong perception, evidenced by our findings and the The author(s) declared no potential conflicts of interest with
mere presence of a broad literature on the subject, that it respect to the research, authorship, and/or publication of this
is valuable. Mechanisms for incorporating voices outside article.
the academic community are being refined in today’s
research climate. This study underscores researchers’ Funding
desires to maintain rigor while opening the boundaries of
The author(s) disclosed receipt of the following financial sup-
methodological practices. port for the research, authorship, and/or publication of this arti-
Our study has several limitations. We only queried cle: This work was partially supported through an Engagement
investigators from the first round of PCORI-funded proj- Award (EAIN): Supporting Dissemination and Implementation
ects, and these projects may differ in important ways Activities of the PCORI Pilot Projects Learning Network
from subsequent PCOR studies. We did not query inves- (PPPLN) #1IP2PI000669-01 and AcademyHealth’s Pilot
tigators about their specific backgrounds or the content of Project Monitoring Services.
their studies. Relevant information was expected to show
itself essentially, and the phenomenological approach References
elicited participants’ priorities, but differences related to Barg, F., Vandermause, R., Edmundson, L., Esmail, L., &
investigator background or content could not be ascer- Perfetti, A. (2015). The power of qualitative research:
tained. Two different interviewers conducted the study; Implementing the patient voice. Unpublished manuscript.
realistically, this set differing interview scenarios, even Benner, P. (Ed.). (1994). Interpretive phenomenology:
given similar methodological preparation, so common Embodiment, caring, and ethics in health and illness.
findings should be considered alongside this fact. Thousand Oaks, CA: Sage.
The questions we are led to ask: “What does it mean to Crist, J. D., & Tanner, C. A. (2003). Interpretation/analysis
identify as a qualitative researcher? How is it done well?” methods in hermeneutic interpretive phenomenology.
are issues raised in this study. Most of the researchers Nursing Research, 52, 202–205.
Diekelmann, N. L., & Ironside, P. M. (2006). Hermeneutics.
talked about the challenges of publishing and funding
In J. J. Fitzpatrick & M. W. Kazer (Eds.), Encyclopedia of
qualitative work, even today. These concerns were offset nursing research (pp. 260-262). New York: Springer.
by the conviction that making a difference made the work Esmail, L., Moore, E., & Rein, A. (2015). Evaluating patient
worth doing. The extensive time needed to do qualitative and stakeholder engagement in research: Moving from
work well was a part of this conundrum, as well as the theory to practice. Journal of Comparative Effectiveness
desire for guidelines that would make such work accept- Research, 4, 133–145.
able to a broad audience. The aspect of surprise and joy Forsythe, L. P., Ellis, L. E., Edmundson, L., Sabharwal, R., Rein,
that accompanies these methods for many researchers is A., Konopka, K., & Frank, L. (2015). Patient and stakeholder
442 Qualitative Health Research 27(3)

engagement in the PCORI pilot projects: Description and phenomenological interpretation within an interdisciplin-
lessons learned. Journal of General Internal Medicine, 31, ary, transmethodological study. Qualitative Social Work,
13–21. doi:10.1007/s11606-015-3450-z 11, 299–318.
Green, J., Willis, K., Hughes, E., Small, R., Welch, N., Gibbs, Vandermause, R. K., & Fleming, S. (2011). Philosophical her-
L., & Daly, J. (2007). Generating best evidence from meneutic interviewing. International Journal of Qualitative
qualitative research: the role of data analysis. Australian Methods, 10, 367–377.
and New Zealand Journal of Public Health, 31, 545–550. van Manen, M. (1991). Researching lived experience: Human
doi:10.1111/j.1753-6405.2007.00141.x Science for an action sensitive pedagogy (2nd ed.). London:
Holloway, I. & Biley, F. C. (2011). Being a Qualitative The Althouse Press.
Researcher. Qualitative Health Research, 21(7), 968–975. Wertz, F. J., McMullen, L. M., Josselson, R., Anderson, R.,
Israel, B. A., Eng, E., Schultz, A. J., & Parker, E. A. (Eds.). & McSpadden, E. (2011). Five ways of doing qualitative
(2005). Methods in community-based participatory analysis. New York: Guilford Press.
research for health. San Francisco: Jossey-Bass. Willis, J. (2007). Foundations of qualitative research:
Lincoln, Y. S., Lynham, S. A., & Guba, E. G. (2011). Interpretive and critical approaches. Thousand Oaks, CA:
Paradigmatic controversies, contradictions, and emerging Sage.
confluences, revisited. The Sage Handbook of Qualitative
Research, 4, 97–128. Author Biographies
Maggs-Rapport. (2001). “Best research practice”: In pursuit of Roxanne Vandermause is an associate professor at the
methodological rigour. Journal of Advanced Nursing, 35, University of Missouri-St. Louis College of Nursing. Her role
373–383. as Donald L. Ross Endowed Chair for Advancing Nursing
Marshall, B., Cardon, P., Poddar, A., & Fontenot, R. (2013). Practice focuses her work on academic-clinical partnerships
Does sample size matter in qualitative research? A review and her research involves community based and mental health
of qualitative interviews. The Journal of Computer concerns in vulnerable groups.
Information Systems, 54(1), 11.
Merleau-Ponty, M. (1962). The phenomenology of perception. Frances K. Barg is a medical anthropologist and an associate
London: Routledge & Kegan Paul. professor of family medicine and community health at the
Miller, W. L., Crabtree, B. F., Harrison, M. I., & Fennell, M. L. Perelman School of Medicine at the University of Pennsylvania.
(2013). Integrating mixed methods in health services and Her research centers around using qualitative methods to con-
delivery system research. Health Services Research, 48(6, trast patient and expert views on illnesses and their treatment,
Pt. 2), 2125–2133. including depression, congestive heart failure, diabetes, preg-
Minkler, M. (2000). Using participatory action research to nancy loss, ADHD, and Parkinson’s disease.
build healthy communities. Public Health Reports, 115,
Laura Esmail is a program officer for the Clinical Effectiveness
Research team at the Patient-Centered Outcomes Research
Munhall, P. (2012). Nursing research: A qualitative perspective
Institute (PCORI). Previously, she was a Senior Manager with
(5th ed). Sudbury, MA: Jones & Bartlett Learning.
AcademyHealth where she helped lead and manage a variety of
National Institute of Health Office of Behavioral and Social
PCOR projects, including the PCORI Pilot Projects Learning
Sciences Research (2010). Qualitative methods in health
Network and PCORI’s Research Portfolio Development
research: Opportunities and considerations in application
and review. National Institute of Health.
Nelson, A. M. (2008). Addressing the threat of evidence-based Lauren Edmundson is a quality improvement specialist at
practice to qualitative inquiry through increasing attention Swedish Medical Group in Seattle, WA where she works to
to quality: A discussion paper. International Journal of improve cancer screening care for primary care patients. She
Nursing Studies, 45, 316–322. previously worked at AcademyHealth, a PCORI partner, focus-
Patient-Centered Outcomes Research Institute. (2015). ing on stakeholder engagement in research.
Retrieved from
Samantha Girard is graduate of Washington State University
Reason, P. (1994). Three approaches to participatory inquiry. In
College of Nursing and a Jonas Center for Nursing Excellence
N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of quali-
scholar alum. She lectures at the University of Washington in
tative research (pp. 324–329). Thousand Oaks, CA: Sage.
Tacoma, WA and is a Nursing Education and Practice Specialist
Small, S. A. (1995). Action-oriented research: Models and
at Group Health Cooperative in Seattle, WA. Her research
methods. Journal of Marriage and Family, 57, 941–955.
focuses on academic progression in nursing education, and
Smythe, L., & Giddings, L. S. (2007). From experience to
Global and Population Health.
definition: Addressing the question: “what is qualitative
research?” Nursing praxis in New Zealand, 23, 37–57. A. Ross Perfetti completed her undergraduate training in
Thoresen, L., & Öhlén, J. (2015). Lived observations: Linking Modern Middle Eastern Studies at the University of
the researcher’s personal experiences to knowledge devel- Pennsylvania, and is now a postgraduate student in Medical
opment. Qualitative Health Research, 25, 1589–1598. Anthropology at Durham University. Her interests are in quali-
Vandermause, R. K. (2012). Being wholesome: The paradox of tative health research and clinical medicine, with a focus on
methamphetamine addiction and recovery-A hermeneutical refugee health.