Anda di halaman 1dari 4

VALIDITY AND RELIABILITY IN

QUALITATIVE RESEARCH
H.I.L. Brink (Conference Paper)

Paper delivered at SA Society o f Nurse THE CONCEPTS QUALITATIVE in using the method and scoring or rating its
Researchers Workshop-RAU 19Marchl993 RESEARCH, VALIDITY AND results and that factors related to subjects and
RELIABILITY testing procedures have been managed to
reduce measurement error.
The term qualitative research is really an
INTRODUCTION umbrella term representing a variety of Many qualitative researchers avoid the terms
research approaches which share certain vali^ty and reliability and use terms such as
Validity and reliability are key aspects of all common elements. Qualitative researchers credibility, trustworthliness, truth, value,
research. Meticulous attention to these two are not interested in causal laws but in peoples applicability, consistency and confirmabUity,
aspects can make the difference between good belief, experience and meaning systems from when referring to criteria for evaluating the
research and poor research and can help to the perspective of the people. Methods used scientific merit of qualitative research (Glaser
assure that fellow scientists accept findings as are more subjective than in quantitative and Strauss 1967, Leininger 1991, Lincoln &
credible and trustworthy. This is particulaary research and do not include statistical analysis Guba 1985).
v ita l in q u a lita tiv e w ork, w here the and empirical calculation. Phenomena are
researchers subjectivity can so readily cloud viewed holistically and in their social context RISKS OR THREATS TO VALIDITY
the interpretation of the data, and where Included under this unbrella term are such AND RELIABILITY
research findings are often questioned or methods as grounded theory, phenomenology
viewed with scepticism by the scientific and ethnonursing-the three approaches As stated previously researchers need to be
community. earmarked for discussion today. attuned to the multiple factors that pose risks
to the validity and reliability of their findings
So those of us doing qualitative studies need Validity in research is concerned with the and plan and implement tactics or strategies to
to be especially sensitive to the issues of accuracy and truthfulness of scientific avoid or counter them. One of the key factors
validity and reliability in our projects. We findings (Le Comple and Goetz 1982: 32). A affecting validity and reliability is error. Error
need to be attuned to the multiple factors that valid study should demonstrate what actually is inherent in all investigations and is inversely
pose risks to the validity of our findings; and exists and a valid instrument or measure related to validity and reliability. The greater
plan and im plem ent various tactics or should actually measure what it is supposed to the degree of error the less accurate and
strategies into each stage of the research measure. truthful the results. Researchers thus must be
project to avoid or weaken these threatening especially watchful of the sources of error
factors. We need to be aware that the tactics or There are many types of validity and many when planning and implementing their
strategies used to address validity and names have been used to define the different studies. For convenience sake the major
reliability in qualitative research are not the types of validity. Campbell and Stanley sources of error can be categorised as follows:
same as in quantitative research. (1966) have defined two major forms of
validity that encompass the many types. They (1) the researcher
The very nature o f qualitative research refer to "internal" and "external" validity,
m ethods does not lend to statistical or terms which are today used in most nursing (2) the subjects participating in the
em pirical calculations of validity. The research textbooks. Denzin (1970) used the project
qualitative researcher seeks basically the same distinction between internal and external
ends through different methods which are validity and applied it to qualitative research. (3) the situation or social context
better suited to a human subject matter. A Intem ^ validity is the term used to refer to the
large num ber o f authors focusing on extent to which research findings are a true (4) the methods of data collection and
qualitative research methods have suggested reflection or representation of reality rather analysis
tactics or strategies the researcher can employ than being the effects of extraneous variables.
to enhance the truthfulness or validity of External validity addresses the degree or THE RESEARCHER AS A RISK TO
qualitative findings (Chenitz & Swanson extent to which such representations or VALIDITY AND RELIABILITY
1986, Crabtree & Miller 1992, Field & Morse reflectio n s of reality are legitim ately
1985, Le Comple & Goetz 1982, Morse 1991, applicable across groups. In a qualitative study the data-gathering
Sandelowski 1986 and Corbin & Strauss instrum ent is frequently the researcher
1990). Reliability is concerned with the consistency, himself. Thus questions ofresearcher bias and
stability and repeatability of the informants researcher competency, if unchecked, may
The purpwse of this pap>er is to address the accounts as well as the investigators abUity to in fluence the tru stw o rth in ess of data
m ajor risks and threats to validity and collect and record information accurately considerably. The very presence of the
reliability in qualitative studies and in (Selltiz et al 1976:182). It refers to the ability researcher may affect the validity of the data
particular the tactics and strategies suggested of a research method to yield consistently the provided by subjects. When a new member is
by various qualitative researchers for avoiding same results over repeated testing periods. In introduced into an interaction reactive effects
or weakening the potential risks and threats. other words, it requires that a researcher using can be expected. Participants may behave
However, to refresh your memory and ensure the same or comparable methods obtained the abnormally (Argyris 1952). They may seek to
that we all attach the same meaning to validity, same or comparable results every time he uses reveal themselves in the best possible light or
reliability and qualitative research, attention the methods on the same or comparable withhold or distort certain information; in
will first be given to the definition and subjects. It further requires that the researcher other words the researcher has created social
clarification of these key concepts. has developed consistent responses or habits behaviours in others that would normally not

Curationis, Vol. 16, No. 2, June 1993 35


have occurred. Based on her extensive She recommends the use of the stranger to The researcher can attempt to increase the
fieldwork, Leinger holds that researchers need filed model which she has developed and validity of responses in such a setting
to be trusted before they will be able to obtain validated over many years. The purpose of
any accurate reliable or credible data this model is to serve as an assessment or (1) by making sure that informants are very
(Leininger 1991: 92). reflection guide for the researcher to become clear on the nature of the research eg.
consciously aware of his own behaviours, why the researcher is there, what he
Le Comple & Goetz refer to research findings feelings and responses in relation to the is studying, how he will collect data and
which hold that what the reseacher sees and behaviour and experiences of subjects and as what he wiU do with it
reports is a function of the position he occupies he starts to collect data, for confirmation of
within the participant group, the status truths. (2) by first building a trust-relationship with
accorded to them, and the role behaviour the subjects and staying in that setting
expected of them. The status position of the Once data collection starts this should be done for a long period of time
researcher can be that of an outsider or that of over a long period of time. The researcher as
a participant group member. The status participant observer either "lives" with the (3) by interviewing the same informant on
position can prevent the researcher from subjects as anthropologists do or spends time several occasions and making
obtaining certain information and unless he is visiting the research site regularly over a long observations more than once and over
aware of this, invalid interpretation of the data period of time. When subjects are interviewed time
may result. On the other hand, if the over time, their responses to the same
researcher becomes totally a part of the group questions on the same topic should be (4) by comparing the results obtained with
there is a danger of "going native" or assuming answered with the same information. This is other evidence
the attitudes and behaviours of those under a type of test-retest of the same informant on
study. The researcher may then lose the ability the same material. The threat of "going native" (5) by confirming findings and analysis with
to look objectively at what is happening and or becoming so enmeshed with subjects that informant (the danger with this
may develop bias towards the point of view of researchers lose their own perspective can be technique is that subjects may become
the group. offset by distancing oneself fix)m the subjects sensitised to the researchers inferences
atregular intervals ie spending time away from and provide the answer that support the
Researcher bias may also be introduced by the the site, spread out site visits and discussing researchers point)
tendency of the researchers to observe subjects data with colleagues.
and interpret findings in the light of their own (6) by keeping accurate and detailed
values, the tendency to selectively observe and When field researchers are working alone, fieldnotes to note the variations in^
record certain data at the expense of other data. particularly when they are still unfamiliar with responses over the course of time
The physical appearance of the researcher may the setting, it is advisable that they enlist the
influence the situation as also his dress and aid of an informant who observes the occasion (7) by showing fieldnotes to a second
demeanour and personal attributes. For also. The researcher then records the activity outside researcher. Another researcher is
example, subjects may respond differently to on the spot and then reviews the written record often much quicker to see where or how
males and females or male and female with the informant for completeness and a fieldworker is being misled or coopted.
researchers may treat the subject differently. comprehensiveness of coverage. In some
cases participant informants serve as arbiters, Informant bias may also be introduced by
WHAT CAN BE DONE TO reviewing the days production of field notes to factors within subjects themselves such as
DECREASE OR ELIMINATE co rrect researcher m isperceptions and fatigue, motivation or anxiety, duration of
RESEARCHER EFFECTS? misinterpretations. Commonly the researcher recall, mood, attention span, state of health and
requests reactions to working analysis or whether or not they are in pain.
The first step in decreasing bias is to be aware processed materials from the informants. In
of the possibility of introducing bias at various this confirmation may be sought for various To overcome this bias the researcher conducts
points of the research process. Field & Morse levels of the collection and analysis process repeated interviews at different times and in
recom m end th a t re se a rc h ers undergo (Le Comple & Goetz 1982:42). different settings and then compares results.
extensive and rigorous training as interviewers
and observers before undertaking qualitative THE PARTICIPATING SUBJECTS Another informant bias which is quoted as a
studies. Researchers need to be trained in a AS RISKS TO VALIDITY AND particular problem in qualitative research
manner that encourages an objective view of RELIABILITY (Miles & Huberman 1984: 230, Sandelowski
the phenomena under study. Furthermore, 1986: 32) is the "elite bias" (overweighting
eveiy researcher should examine and declare The truth of responses is a key concern when data from articulate, well-informed, usually
his underlying values and assumptions in light data are obtained through questionnaires and high status informants and under rrepresenting
of the research situation so that they can be interviews. Bias may be introduced because data from intractable, less articulated
considered when reading the research of particular responses or characteristics of the lower-status ones).
informants. Informants may want to make
Several writers recommend that the researcher things seem better or worse than they are. The researcher can build in safeguards against
spends a period of time in the situation before Hospitalised patients who are questioned this bias by good planning of selection of
data collection starts. The researcher will then about the quality of their care may indicate that informants, by looking purposefully for
become sensitised to the situation and at the the care is wonderful because they fear reprisal co n trastin g cases (negative, extrem e,
same time the subjects have the opportunity to of staff. Conversely, they may respond that countervailing and by carefully considering
become used to the presence of the researcher their care is awful. Informants may also contrasting views).
(Field & Morse 1985, Le Comple & Goetz attempt to please the researcher by responding
1992, Miles & Huberman 1984). in the way that they believe he expects. They THE SOCIAL CONTEXT AS A RISK
may also fear that by giving negative TO VALIDITY AND RELIABILITY
Leininger (1991: 11) suggests that the responses, they will be placed in a devalued
researcher should always assess and gauge his position by the researcher. Informants may The social context under which the data are
relationships with the subjects being studied also be unwilling to share certain information gathered is an important consideration in
in order to enter or get close to the people or with the researcher and deliberately withhold establishing validity and reliability of data.
situation under study, or to move from a or distort it Individuals may behave differently under
stranger or distrusted person to a trusted and differing social circumstances, for example,
fnendly person during the research process. when alone with the researcher they may

36 Curationis, Vol. 16, No. 2, June 1993


provide different information than when they researcher should seek a panel of experts to threats to internal reliability with data analysis:-
are in a group, or patients may provide assist with finding appropriate informants.
different information within the health care (1) Low inference descriptors (verbatim
context than what they reveal in their home Thirdly, the researcher should do systematic accounts of information provided by
neighbourhood. The researcher who is "theoretical sampling" or in other words, informants to the researcher). Use of
mindful of this will interview the same continue to select subjects according to the mechanical recording enhances the
inform ants and m ake observations of findings that emerge in the course of the study. accuracy of such transcripts.
behaviour in a variety of settings to make During this phase he should establish
comparisons of similarities and differences typicality or atypicality of observed events, (2) Participant reviews of findings and peer
before attributing meaning. behaviours or responses, extreme views or examination.
contrasting views. He should persist with
He will also specify the physical, social and theoretical sampling until no new information CONCLUSION
interpersonal contexts within which data are is obtained (until saturation or redundancy has
gathered. Particular attention to privacy may been reached). There should be support for In conclusion I would like to recapitulate the
also be of value in certain studies when construction of a core category or several core major critical strategies, suggested by leading
subjects hesitate to answer accurately if they categories which repeatedly occur while less qualitative researchers from various fields as
fear they may be overheard by others in the and less new information emerges. essential for producing trustworthy and
environment believable findings in qualitative research
Another risk to representativeness of data (Benner (1985), Brink & Wood (1988),
RISKS TO VALIDITY AND results from the researchers non-continuous Corbin & Strauss (1990), Field & Morse
RELIABILITY PERTAINING TO presence. The researcher has to infer what is (1985), Glaser & Strauss (1967), Kirk &
DATA COLLECTION AND happening when he is not there and usually MiUer (1986), Kuzel & Like (1991), Lather
ANALYSIS offers plausible reasons rather than evidence. (1986), Le Comple & Goetz (1982), Leininger
To balance this risk once again multiple (1991), Lincoln & Guba (1985), Miles &
Because reliability and validity depend on the so u rces, m u ltip le m ethods, m ultiple H uberm an (1984), M orse (1991),
potential for subsequent researchers to investigators (judge panel) varying and Sandelowski (1981).
reconstruct original strategies, the researcher multiple repetitions of measurement over time
who presents a vague account of his design is are recommended. 1. TRIANGULATION
putting himself at risk of being accused of Triangulation refers to the use of two or more
invalid and unreliable findings. In m o st q u a lita tiv e approaches and data sources, m ethods, in v estigators,
particularly phenomenology, grounded theory theoretical perspectives and approaches to
Researchers should therefore strive to present and ethnomethods data analysis occurs analysis in the study of a single phenomenon
their methods clearly, that is precisely identify simultaneously with data collection. All these and then validating the congruence among
and thoroughly describe all strategies used to methods use a series of similar steps for them. The major goal of triangulation is to
collect data and carefully document their field analysing which begin at the onset of the data circ u m v en t the p e rso n al biases of
notes in the context of what was being collection phase. investigators and overcome the deficiencies
observed to enable fellow researchers to form intrinsic to single-investigator, single-theory,
valid judgement. Many of the risks in data Typical steps are coding for categories and or single-method study thus increasing the
c o lle c tio n p e rta in to th e re search er, themes and making memos about the context validity of the study (Denzil 1989).
respondents and social context and have and variations in the phenomena under study,
already been dealt with. d eveloping nam es for categories and 2. MULTIPLE repetitions of measurement
elaborating classification systems and testing over or long period of time, at different points
Another major risk is sampling bias. The them within the data as they are collected. The of time, in different situations or settings and
subjects under study may be overrepresenting findings at any point in this process wiU by different persons.
or underpresenting the phenomena under provide some direction for further data
study. The researcher may be overreliant on collection and the direction that the analysis 3. EX PER T CON SEN SUAL
accessible and elite informants. He/she may may take. Judgements and inferences are VALIDATION FROM OTHERS familiar
be more likely to see confirming instances of made by the researcher. Major threats to the with the topic under study at various stages of
original beliefs and perceptions than to see validity of data during this phase are firstly the resea rc h p ro cess. This in volves
d isc o n firm in g in sta n c e s even w hen what Miles and Huberman call the "holistic independent analysis of the data by others (eg
disconfirming instances are more fequent fallacy" that tends to make data look more research colleagues, a judge panel or
From one or two concrete vivid instances he patterned or regular or congruent than they are p articip an t inform ants. (For exam ple,
may assume that there are dozens more, but and the tendency of the researcher to following each initial category generation
may fail to verify this (Miles & Huberman selectively observe and record certain data at during analysis - independent fellow
1984: 231). the expense of other data. To avoid such researchers or other experts in the field are
selective inattention, the data analysis invited to generate their own category system.
In qualitative research sample selection is procedures should be exposed to a judge This is them compared with the researchers).
based on the ability of the subject to provide panel. The judge panel is selected on the basis
data relevant to the research question. To of knowledge of content or knowledge of the 4. MEMBER CHECKS
avoid inaccurate or insufficient data, the research project. This refers to recycling of analysis back to
researcher must use his/her judgement based informants. (Conclusions are given to the
up)on the best available evidence to choose O ther means of providing validity and informants and feedback is requested about
subjects who know enough, can recall enough, reliab ility are the use of the constant the accuracy of the content. This ensures that
and are able to respwnde precisely to questions comparative method and the search for the researcher and the informant are viewing
asked. alternative hypothesis or negative cases the data consistently.
(Hutchinson 1986: 116-117), checking that
Secondly, the researcher should choose descriptions, explanations or theories about 5. SEARCHING FOR DISCONFIRMING
subjects who are able to report events not the data contain the typical and atypical EVIDENCE
directly observable or accessible to the elements of the data and obtaining validation In this step there is an active search for
investigator. If the research is carried out in from the subjects themselves (Sandelowski disconfirmation of what is believed to be true.
unfamiliar environment with an unfamiliar 1986: 35). Field and Morse (19895: 120) A proposition deserves some degree of trust
group of people in an unfamiliar culture, the recommend that following strategies to reduce only when it has survived serious attempts to

Curationis, VoL 16, No. 2, June 1993 37


falsify i t (Cronbach in Lather 1986: 67). (6) when and how data were analysed approaches. Beckenham Kent: Croom
Searching for disconfirm ing evidence Hehn.
involves both purposive sam pling and (7) the nature of the setting(s) in which data
prolonged engagement with informants in the were collected Glaser, B. * Strauss A. 1967. The discovery
field. Purposeful sampling allows the o f grounded theory: Strategies fo r
researcher to include informants who may (8) how various elements of the data were qualitative research. Chicago: Aldine
differ from key informants in critical ways. weighted Publishing Co.
This purposeful sampling of individuals and
the inclusion of conflicting as well as (9) the inclusiveness and exclusiveness of Kirk, J. & Miller M. 1986. Reliability,
complementary accounts strengthens the the categories developed to con the data validity and qualitative research. Beverly
description. One varies or contrasts the and Hills CA: Sage.
conditions as methodically as possible in order
to determine what has an impact on the (10) how reliability and validity were Kuzel, A.J. & Like, R.C. 1991. Standards of
phenomenon in question (Corbin & Strauss answered trustworthiness for qualitative studies in
1990). primary care. In P. Norton, M. Stewart, F.
The account also needs to include the D udiver, M. Bass & E D unn (eds).
6. CH ECK ING FOR R E PR E SE N T relationship between field notes and the Primary Care Research: Traditional and
ATIVENESS conclusions based upon them. Memos, data innovative approaches, (p .138-158).
T h is step in c lu d e s ch eck in g for the display charts, indicating coding instructions Newbury park CA: Sage.
representativeness of the data as a whole, of and the actual placem ent of data into
the coding categories and of the examples used categories and the way different elements of Lather, P. 1986. Issues of validity in openly
to analyze and present the data. the data were linked to each other, should be ideological research: Between a rock and
kept available. a soft place. Interchange 17: 63-84.
7. THICK DESCRIPTION
Qualitative research, like its quantitative Corbin and Strauss (1990:20) strongly urge Le Comple, M.D. & Goetz, J.P. 1982.
cousins, can be systematically evaluated only investigators to adhere to the major criteria for Problems of reliability and validity in
if their criteria and procedures are made answ ering credibility unless there are ethnographic research. R e v ie w o f
explicit Hence validity and reliability can exceptional reasons for not doing so. In such Educational Research 52 (no 1): 31-60.
only be judged if a very detailed account of the unusual cases, researchers should know
context or setting within which the study took precisely how and why they depart from the Leininger, M. 1991. Culture care, diversity
place and a thorough description of the criteria, say so in their writing, and submit the and universality: A theory o f nursing. New
procedures from the beginning to the end is credibility of their findings to the reader. If we York: N.LJ'f.
given. Most qualitative researchers refer to want to develop a valid, truthful and believable
this as thick description. However Lincoln account of our qualitative studies it will be Lincoln, Y. & Guba, G. 1985. Naturalistic
and Guba (1985) use the term auditability. wise to take note of and implement the critical inquiry. Beverly Hills: Sage.
Auditability means that any reader or another strateg ies discussed unless otherw ise
researcher can follow the progression of indicated. Miles, M.B. & Huberman, A.M. 1984.
events in the study and understand their logic. Qualitative data analysis: A sourcebook o f
Such an account starts with the researcher LIST OF SOURCES new methods. Beverly HiUs CA: Sage.
re c o g n isin g and d is c lo sin g in itia l
assumptions, suppositions and values thatmay Argyris, C. 1952. Diagnosing differences M orse, j.M . 1991. Qualitative nursing
have in flu e n c e d d ata g ath erin g and against the outsider. Journal o f Social research: A contemporary dialogue.
processing. The researcher furthermore Issues 8 (no 3): 24-34. Rockville, Maryland: Aspen.
describes, explains and justifies.
B enner, P. 1985. Q uality o f life: A Oiler, C. 1982. The phenomenological
(1) how he/she becomes interested in the phenomenological perspective. Advances approach in nursing research. Nursing
subject matter of the study in Nursing Science 8 (no 1): 1-14. Research?)! {no 3Q-. 178-181.

(2) the specific purpose(s) of the study Cam pbell, D.T. & Stanley, J.C. 1963. Reason, P. & Rowan, J. 1981. human
Experimental and quasi-experimental inquiry: A sourcebook o f new paradigm
(3) how subjects or pieces of evidence came designs fo r research. Chicago: Rand research. New York: John Wiley & Sons.
to be included in the study and how they McNally.
were approached, how theoretical Sandelowski, M. 1986. The problem of rigor
sampling was done Chenitz, W.C. & Swanson, J.M. 1986. From in qualitative research. Advances in
practice to grounded theory. Menlo Park Nursing Science 8 (no 3): 27-37.
(4) the impact, the subjects and the CA: Addison Wesley.
researcher/s had on each other Seltiz, C. & Wrightsman, L.C. & Cook, W.S.
Corbin, J. & Strauss A. 1990. Grounded 1976. R esearch m ethods in so cia l
(5) how the data were collected ie theory research: Procedure, canons and relations. 3rd edition. New York: Holt
evaluative criteria. Qualitative Sociology Rinehart & Winston.
the number and types of data collection 13 (no 1): 3-21.
methods and procedures used in the study Strauss, A. & Corbin, J. 1990. Basics o f
Crabtree, B.F. & MiUer, W. 1992. Doing qualitative research. Newbury Park CA:
the number of times data are collected on qualitative research. Newbury Park, CA: Sage.
the same subject using the same method Sage.

how often and over what period were data Denzin, N.K. 1970. The research act: A
collected on tiie same and different informants theoretical introduction to sociological
methods. Chicago: Aldine Publishing Co. H. Brink
the number of investigators collecting the Prcfessor, Department o f Nursing Science
same and different information on the same Field, P.A. & Morse, J.M. 1985. Nursing UNISA
and different subjects; and Research. The application o f qualitative

38 Curationis, Vol. 16, No. 2, June 1993