Anda di halaman 1dari 14

Mixed Methods Research: A Research Paradigm Whose Time Has Come

Author(s): R. Burke Johnson and Anthony J. Onwuegbuzie


Source: Educational Researcher, Vol. 33, No. 7 (Oct., 2004), pp. 14-26
Published by: American Educational Research Association
Stable URL: http://www.jstor.org/stable/3700093 .
Accessed: 30/10/2013 13:45

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

American Educational Research Association is collaborating with JSTOR to digitize, preserve and extend
access to Educational Researcher.

http://www.jstor.org

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
I I

Mixed Methods Research:


A Research Paradigm Whose Time Has Come
by R. BurkeJohnson and AnthonyJ. Onwuegbuzie

The purposes of this articleare to position mixed methods research writing style using the impersonal passivevoice and technical ter-
(mixedresearchis a synonym) as the natura!complement to tradi- minology, in which establishing and describing social laws is the
tional qualitativeand quantitativeresearch, to present pragmatism major focus (Tashakkori & Teddlie, 1998).
as offeringan attractivephilosophicalpartnerfor mixed methods re- Qualitative purists (also called constructivistsand interpretivists)
rejectwhat they call positivism. They argue for the superiorityof
search, and to provide a framework for designingand conducting
constructivism, idealism, relativism, humanism, hermeneutics,
mixed methods research. In doing this, we brieflyreview the para- and, sometimes, postmodernism (Guba & Lincoln, 1989; Lincoln
digm "wars"and incompatibilitythesis, we show some commonali- & Guba, 2000; Schwandt, 2000; Smith, 1983, 1984). These
ties between quantitativeand qualitativeresearch, we explain the purists contend that multiple-constructed realities abound, that
tenets of pragmatism,we explainthe fundamentalprincipleof mixed time- and context-free generalizations are neither desirable nor
researchand how to applyit, we provide specific sets of designsfor possible, that researchis value-bound, that it is impossible to dif-
ferentiate fully causes and effects, that logic flows from specific
the two major types of mixed methods research (mixed-modelde-
to general (e.g., explanations are generated inductively from the
signsandmixed-method designs),and,finally,we explainmixed meth-
data), and that knower and known cannot be separated because
ods research as following (recursively)an eight-step process. A key the subjective knower is the only source of reality (Guba, 1990).
feature of mixed methods research is its methodologicalpluralism Qualitative purists also are characterized by a dislike of a de-
or eclecticism,which frequentlyresults in superior research (com- tached and passive style of writing, preferring, instead, detailed,
pared to monomethod research). Mixed methods research will be rich, and thick (empathic) description, written directly and some-
successfulas more investigatorsstudy and help advanceits concepts what informally.
Both sets of purists view their paradigms as the ideal for re-
and as they regularlypractice it.
search, and, implicitly if not explicitly, they advocate the in-
compatibility thesis (Howe, 1988), which posits that qualitative
and quantitative research paradigms, including their associated
or more than a century, the advocates of quantitative and methods, cannot and should not be mixed. The quantitative
qualitative researchparadigmshave engaged in ardent dis- versus qualitative debate has been so divisive that some gradu-
pute.1 From these debates, purists have emerged on both ate students who graduate from educational institutions with an
sides (cf. Campbell & Stanley, 1963; Lincoln & Guba, 1985).2 aspiration to gain employment in the world of academia or re-
Quantitative purists (Ayer, 1959; Maxwell & Delaney, 2004; search are left with the impression that they have to pledge alle-
Popper, 1959; Schrag, 1992) articulateassumptions that are con- giance to one research school of thought or the other. Guba (a
sistent with what is commonly called a positivist philosophy.3 4 leading qualitative purist) clearly represented the purist position
That is, quantitative purists believe that social observations when he contended that "accommodation between paradigms
should be treated as entities in much the same way that physical is impossible ... we are led to vastly diverse, disparate, and to-
scientists treat physical phenomena. Further, they contend that tally antithetical ends" (Guba, 1990, p. 81). A disturbing fea-
the observer is separate from the entities that are subject to ob- ture of the paradigm wars has been the relentless focus on the
servation. Quantitative purists maintain that social science differences between the two orientations. Indeed, the two dom-
inquiry should be objective. That is, time- and context-free gen- inant researchparadigms have resulted in two researchcultures,
eralizations (Nagel, 1986) are desirable and possible, and real "one professing the superiority of'deep, rich observational data'
causes of social scientific outcomes can be determined reliably and the other the virtues of 'hard, generalizable' . . . data"
and validly. According to this school of thought, educational re- (Sieber, 1973, p. 1335).
searchers should eliminate their biases, remain emotionally de- Our purpose in writing this article is to present mixed meth-
tached and uninvolved with the objects of study, and test or ods research as the third research paradigm in educational re-
empiricallyjustify their stated hypotheses. These researchershave search.5We hope the field will move beyond quantitative versus
traditionally called for rhetorical neutrality, involving a formal qualitative research arguments because, as recognized by mixed
methods research, both quantitative and qualitative researchare

EducationalResearcher,Vol. 33, No. 7, pp. 14-26


important and useful. The goal of mixed methods researchis not
to replace either of these approaches but rather to draw from the

1| EDUCATIONALRESEARCHER

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
strengthsand minimizethe weaknessesof both in singleresearch is mosthelpfulandwhen andhow theyshouldbe mixedor com-
studies and acrossstudies. If you visualizea continuum with bined in theirresearchstudies.
qualitativeresearchanchoredat one pole and quantitativere- We contendthatepistemological andmethodological pluralism
searchanchoredat the other,mixedmethodsresearchcoversthe shouldbe promotedin educationalresearchso thatresearchers are
largeset of points in the middlearea.If one prefersto thinkcat- informedaboutepistemologicaland methodologicalpossibilities
egorically,mixedmethodsresearchsitsin a new thirdchair,with and, ultimately,so thatwe areableto conductmoreeffectivere-
qualitativeresearchsitting on the left side and quantitativere- search.Today's researchworld is becomingincreasinglyinter-
searchsittingon the rightside. disciplinary,complex,anddynamic;therefore,manyresearchers
Mixed methods researchoffersgreatpromisefor practicing needto complementone methodwithanother,andallresearchers
researchers who would like to see methodologistsdescribeand need a solid understandingof multiplemethodsused by other
developtechniquesthat are closerto what researchersactually scholarsto facilitatecommunication,to promotecollaboration,
use in practice.Mixed methods researchas the third research and to providesuperiorresearch.Takinga non-puristor com-
paradigmcan also help bridgethe schismbetweenquantitative patibilistor mixedpositionallowsresearchers to mix and match
and qualitativeresearch(Onwuegbuzie& Leech,2004a). Meth- designcomponents that offer the best chanceof answeringtheir
odologicalwork on the mixed methodsresearchparadigmcan specificresearch questions.Althoughmany research procedures
be seen in several recent books (Brewer & Hunter, 1989; or methodstypicallyhavebeen linkedto certainparadigms,this
Creswell,2003; Greene,Caracelli,& Graham,1989; Johnson linkagebetweenresearchparadigmand researchmethodsis nei-
& Christensen,2004; Newman & Benz, 1998; Reichardt& thersacrosanctnor necessary(Howe, 1988, 1992). Forexample,
Rallis, 1994; Tashakkori& Teddlie, 1998, 2003). Much work qualitativeresearchersshould be free to use quantitativemeth-
remainsto be undertakenin the areaof mixedmethodsresearch ods, andquantitativeresearchers shouldbe freeto usequalitative
regardingits philosophicalpositions,designs,data analysis,va- methods.Also, researchin a content domainthat is dominated
lidity strategies,mixing and integrationprocedures,and ratio- by one method often can be betterinformedby the use of mul-
nales, among other things. We will try to clarify the most tiple methods(e.g., to give a readon methods-inducedbias,for
importantissuesin the remainderof this article. corroboration, forcomplimentarity, forexpansion;see Greeneet
al., 1989). We contend that epistemologicaland paradigmatic
Commonalities Among the Traditional Paradigms ecumenicalismis withinreachin the researchparadigmof mixed
methodsresearch.
Althoughtherearemanyimportantparadigmatic differencesbe-
tweenqualitativeandquantitativeresearch(whichhavebeenfre-
Philosophical Issues Debates
quentlywritten about in the EducationalResearcher and other
As notedby OnwuegbuzieandTeddlie(2003), someindividuals
places),there aresome similarities
between the variousapproaches who engagein the qualitativeversusquantitativeparadigmdebate
that aresometimesoverlooked.For example,both quantitative
and qualitativeresearchers use empiricalobservationsto address appear to confuse the logic ofjustificationwith researchmethods.
That is, there is a tendency among some researchersto treat
researchquestions.Sechrestand Sidani(1995, p. 78) point out
thatboth methodologies"describetheirdata,constructexplana- epistemologyand methodas beingsynonymous(Bryman,1984;
Howe, 1992). This is farfrombeingthe casebecausethe logic of
tory argumentsfrom their data, and speculateabout why the
outcomes they observedhappenedas they did." Additionally, justification(an importantaspectof epistemology)does not dic-
tatewhat specificdatacollectionand dataanalyticalmethodsre-
bothsetsof researchers incorporatesafeguards into theirinquiries searchersmustuse.Thereis rarelyentailmentfromepistemology
in orderto minimizeconfirmationbiasand othersourcesof in- to methodology(ohnson, Meeker,Loomis, & Onwuegbuzie,
validity (or lack of trustworthiness)that have the potentialto 2004; Phillips,2004). Forexample,differencesin epistemologi-
existin everyresearchstudy(Sandelowski,1986). cal beliefs(such as a differencein beliefsabout the appropriate
Regardlessof paradigmaticorientation,all researchin the so- logic of justification)shouldnot preventa qualitativeresearcher
cialsciencesrepresentsan attemptto providewarrantedassertions fromutilizingdatacollectionmethodsmoretypicallyassociated
abouthumanbeings(orspecificgroupsof humanbeings)andthe with quantitativeresearch,and vice versa.
environmentsin which they live and evolve(Biesta& Burbules, There areseveralinterestingmythsthat appearto be held by
2003). In the socialand behavioralsciences,this goal of under- some purists.For example,on the "positivist"side of the fence,
standingleadsto the examinationof manydifferentphenomena, the barriersthat quantitativeeducationalresearchers have built
includingholisticphenomenasuchasintentions,experiences,at- arisefrom a narrowdefinitionof the conceptof "science."6 As
titudes,andculture,aswellas morereductivephenomenasuchas noted by Onwuegbuzie(2002), modernday "positivists"claim
macromolecules,nerve cells, micro-levelhomunculi, and bio- that science involves confirmationand falsification,and that
chemicalcomputationalsystems(deJong,2003). Thereis room these methodsand proceduresareto be carriedout objectively.
in ontologyfor mentalandsocialrealityaswell as the moremicro However,they disregardthe factthat manyhuman(i.e., subjec-
and more clearlymaterialreality.Althoughcertainmethodolo- tive)decisionsaremadethroughoutthe researchprocessandthat
gies tend to be associatedwith one particularresearchtradition, researchers aremembersof varioussocialgroups.A few examples
DzurecandAbraham(1993, p. 75) suggestthat "theobjectives, of subjectivismandintersubjectivism in quantitativeresearchin-
scope, and natureof inquiryareconsistentacrossmethodsand cludedecidingwhat to study (i.e., what arethe importantprob-
acrossparadigms."We contend that researchersand research lems?),developinginstrumentsthatarebelievedto measurewhat
methodologistsneed to be askingwhen each researchapproach the researcherviews as being the targetconstruct,choosingthe

OCTOBER2004 |5

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
specifictestsand itemsfor measurement,makingscoreinterpre- Fortunately,many(ormost?)qualitativeresearchers andquan-
tations,selectingalphalevels(e.g., .05), drawingconclusionsand titativeresearchers (i.e., postpositivists)have now reached basic
interpretationsbasedon the collecteddata, decidingwhat ele- agreement on several majorpoints of earlierphilosophicaldis-
ments of the data to emphasizeor publish, and decidingwhat &
agreement(e.g., Phillips Burbules,2000; Reichardt & Cook,
findings are practicallysignificant.Obviously,the conduct of 1979; Reichardt& Rallis, 1994). Basic agreement has been
fullyobjectiveandvalue-freeresearchis a myth,eventhoughthe reachedon each of the followingissues:(a) the relativityof the
regulatoryidealof objectivitycan be a usefulone. "lightof reason"(i.e.,whatappearsreasonablecanvaryacrossper-
Qualitativeresearchers alsoarenot immunefromconstructive sons);(b)theory-ladenperceptionor the theory-ladenness of facts
criticism. Some qualitativepurists (e.g., Guba, 1990) openly (i.e., what we notice and observeis affected by our background
admitthattheyadoptan unqualifiedor strongrelativism,which knowledge,theories,andexperiences; in short,observationis not
is logicallyself-refutingand (in its strongform)hindersthe de- a perfect and direct window into "reality");(c) underdeter-
velopmentand use of systematicstandardsfor judgingresearch minationof theoryby evidence(i.e., it is possiblefor morethan
quality(whenit comes to researchquality,it is not the casethat one theoryto fit a singleset of empiricaldata);(d) the Duhem-
anyone'sopinion about qualityis just as good as the next per- Quine thesisor idea of auxiliaryassumptions(i.e., a hypothesis
son's,becausesome peoplehaveno trainingor expertiseor even cannot be fully tested in isolationbecauseto makethe test we
interestin research).We suspectthatmost researchers aresoftrel- alsomustmakevariousassumptions;the hypothesisis embedded
ativists(e.g., respectingthe opinionsand viewsof differentpeo- in a holisticnetworkof beliefs;and alternativeexplanationswill
ple and differentgroups).When dealingwith human research, continueto exist);(e) the problemof induction(i.e., the recogni-
soft relativismsimply refersto a respectand interestin under- tion thatwe only obtainprobabilisticevidence,not finalproofin
standingand depictingindividualand social groupdifferences empiricalresearch;in short,we agreethat the futuremaynot re-
(i.e., theirdifferentperspectives)anda respectfordemocraticap- semblethe past);(f) the socialnatureof the researchenterprise
proachesto groupopinion and valueselection.Again,however, areembeddedin communitiesand they clearly
(i.e., researchers
a strongrelativismor strongconstructivismrunsinto problems; are
haveand affectedby theirattitudes,values,and beliefs);and
for example,it is not a matterof opinion (or individualreality) (g) the value-ladenness of inquiry(thisis similarto the lastpoint
that one shouldor can driveon the left-handside of the roadin but specificallypointsout thathumanbeingscan neverbe com-
GreatBritain-if one choosesto driveon the rightside,he or she pletelyvalue free, and that valuesaffectwhat we choose to in-
will likelyhavea head-oncollision,at some point, and end up in vestigate,whatwe see, and how we interpretwhatwe see).
the hospitalintensivecareunit, orworse(thisis a casewheresub-
jectiveandobjectiverealitiesdirectlymeetandclash).The strong Pragmatism as the Philosophical Partner
for Mixed Methods Research
ontologicalrelativisticor constructivistclaim in qualitativere-
searchthatmultiple,contradictory,but equallyvalidaccountsof We do not aim to solvethe metaphysical,epistemological,axio-
the samephenomenonaremultiplerealitiesalsoposessome po- logical(e.g.,ethical,normative),and methodologicaldifferences
tentialproblems.Generallyspeaking,subjectivestates(i.e., cre- betweenthe puristpositions.And we do not believethat mixed
ated and experiencedrealities)that varyfrom personto person methodsresearchis currentlyin a positionto provideperfectso-
and that are sometimes called "realities"should probablybe lutions. Mixed methodsresearchshould, instead(at this time),
called(forthe purposesof clarityandgreaterprecision)multiple use a methodandphilosophythatattemptto fit togetherthe in-
perspectives or opinionsor beliefi(dependingon the specificphe- sights providedby qualitativeand quantitativeresearchinto a
nomenonbeingdescribed)ratherthanmultiplerealities(Phillips workablesolution.Along theselines,we advocateconsideration
& Burbules,2000). If a qualitativeresearcher insistson usingthe of the pragmaticmethod of the classical pragmatists(e.g.,
word realityfor subjectivestates,then for claritywe would rec- CharlesSandersPeirce,WilliamJames,and John Dewey) as a
ommendthatthe wordsubjectivebe placedin frontof the word way for researchers to think about the traditionaldualismsthat
reality(i.e., as in subjectiverealityor in manycasesintersubjec- have been debatedby the purists.Taking a pragmaticand bal-
tive reality)to directthe readerto the focusof the statement.We anced or pluralistposition will help improvecommunication
agreewith qualitativeresearchersthat value stancesare often among researchers from differentparadigmsas they attemptto
neededin research;however,it also is importantthat researchis advanceknowledge(Maxcy,2003; Watson, 1990). Pragmatism
morethansimplyone researcher's highlyidiosyncraticopinions alsohelpsto shedlighton how researchapproachescanbe mixed
writteninto a report.Fortunately,manystrategiesarerecognized fruitfully(Hoshmand,2003); the bottomline is thatresearchap-
andregularlyusedin qualitativeresearch(suchasmembercheck- proachesshouldbe mixedin waysthat offerthe bestopportuni-
ing, triangulation,negativecasesampling,patternmatching,ex- ties for answeringimportantresearchquestions.
ternal audits) to help overcome this potential problem and Thepragmaticruleor maximor methodstatesthatthe current
producehigh-qualityand rigorousqualitativeresearch.Finally, meaningor instrumentalor provisionaltruthvalue(whichJames
qualitativeresearcherssometimesdo not pay due attention to [1995, 1907 original]would term"cashvalue")of an expression
providingan adequaterationalefor interpretations of their data (e.g., "allrealityhasa materialbase"or "qualitativeresearchis su-
(Onwuegbuzie, 2000), and qualitative methods of analysestoo for
perior uncovering humanistic research findings")is to be de-
"often remain privateand unavailablefor public inspection" terminedby the experiencesor practicalconsequencesof belief
(Constas, 1992, p. 254). Without public inspection and ade- in or use of the expression in the world (Murphy, 1990). One
quatestandards,how is one to decidewhetherwhatis claimedis canapplythissensibleeffects-or outcome-orientedrulethrough
trustworthyor defensible? thinking(thinkingaboutwhatwill happenif you do X), practi-

16I EDUCATIONALRESEARCHER

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
calexperiences(observingwhathappensin yourexperiencewhen some agreementabout the importanceof many (culturallyde-
you do X), or experiments(formallyor informallytryinga rule rived)valuesand desiredends, such as, for example,preventing
and observingthe consequencesor outcomes). the droppingout of schoolby adolescents,reducingthe useof il-
In the wordsof CharlesSandersPeirce(1878), the pragmatic licit drugsby childrenandadolescents,findingeffectiveteaching
method or maxim (which is used to determinethe meaningof techniquesfor differentkinds of students, educatingchildren
words,concepts,statements,ideas,beliefs)impliesthatwe should and adults(i.e., increasingtheir knowledge),helping to reduce
"considerwhat effects, that might conceivablyhave practical discriminationin society,and attemptingto eliminateor reduce
bearings,we conceivethe objectof ourconceptionto have.Then mental,learning,and otherdisabilities.In otherwords,pragma-
our conceptionof theseeffectsis the whole of our conceptionof tism takesan explicitlyvalue-orientedapproachto research.
the object"(this quote is found at the end of SectionII in How We rejectan incompatibilist,either/orapproachto paradigm
toMakeOurIdeasClear).Buildingon Peirce'slead,James(1995, selectionand we recommenda morepluralisticor compatibilist
1907 original)arguedthat "Thepragmaticmethod is primarily approach.Beyond the basicpragmaticmethod or maxim (i.e.,
a methodof settlingmetaphysicaldisputesthat otherwisemight translatedin mixed methodsresearchas "choosethe combina-
be interminable.... The pragmaticmethod in such casesis to tion or mixtureof methodsand proceduresthat worksbest for
tryto interpreteachnotionby tracingits respectivepracticalcon- answeringyour researchquestions")there also is a full philo-
sequences"(p. 18). Extendingthe works of Peirceand James, sophicalsystemof pragmatismwhich was systematicallydevel-
Dewey spent his careerapplyingpragmaticprinciplesin devel- oped by the classicalpragmatists(Peirce,James,Dewey)and has
oping his philosophyand in the practiceof educatingchildren been refinedin newerdirectionsby latter-dayneo-pragmatists
(e.g., the ExperimentalSchoolof Chicago).Dewey (1948, 1920 (e.g., Davidson, Rescher,Rorty,Putnam) (see Menand, 1997;
original)statedthat"inorderto discoverthe meaningof the idea Murphy, 1990; Rescher,2000; Rorty, 2000). To providethe
[we must] ask for its consequences"(p. 132). In short, when readerwith a betterunderstanding of the fullphilosophyof prag-
judging ideaswe should considertheir empiricaland practical matism(for consideration),we haveoutlined,in Table 1, what
consequences.Peirce,James,andDeweywereallinterestedin ex- we believeareclassicalpragmatism's mostgeneralandimportant
aminingpracticalconsequencesandempiricalfindingsto helpin characteristics.
understandingthe import of philosophicalpositions and, im- Although we endorsepragmatismas a philosophy that can
portantly,to help in decidingwhich action to take next as one help to build bridgesbetweenconflictingphilosophies,pragma-
attemptsto betterunderstandreal-worldphenomena(including tism, like all currentphilosophies,has some shortcomings.In
psychological,social,and educationalphenomena). Table2 we presentsomeof these.Researchers who areinterested
If two ontologicalpositions about the mind/body problem in applyingpragmatismin theirworksshouldconsiderthe short-
(e.g., monism versusdualism),for example,do not makea dif- comings,which alsoneed to be addressedby philosophicallyin-
ferencein how we conductour researchthen the distinctionis, clinedmethodologistsas theyworkon the projectof developing
for practicalpurposes,not very meaningful.We suspect that a fullyworkingphilosophyfor mixedmethodsresearch.Practic-
some philosophicaldifferencesmay lead to importantpractical ing researchers should be reflexiveand strategicin avoidingthe
consequenceswhile many othersmay not.7The full sets of be- potentialconsequencesof theseweaknessesin theirworks.
liefscharacterizingthe qualitativeandquantitativeapproachesor
paradigms haveresultedin differentpractices,and, basedon our Comparing Qualitative, Quantitative,
observationand study,we believeit is clearthat both qualitative and Mixed Methods Research
and quantitativeresearchhavemanybenefitsand manycosts.In Mixedmethodsresearch is formallydefinedhereas the classof re-
somesituationsthe qualitativeapproachwill be moreappropriate; searchwheretheresearcher mixesorcombines quantitativeandqual-
in othersituationsthe quantitativeapproachwill be moreappro- itativeresearch methods,
techniques, approaches, or
concepts language
priate.In many situations,researchers can put togetherinsights intoa singlestudy.Philosophically,it is the "thirdwave"or third
andproceduresfrombothapproaches to producea superiorprod- researchmovement,a movementthat movespast the paradigm
uct (i.e., oftenmixedmethodsresearchprovidesa moreworkable warsby offeringa logicalandpracticalalternative. Philosophically,
solutionand producesa superiorproduct).We areadvocatinga mixedresearchmakesuse of the pragmaticmethod and system
needs-basedor contingencyapproachto researchmethod and of philosophy.Its logic of inquiryincludesthe use of induction
conceptselection. (or discoveryof patterns),deduction (testing of theories and
Philosophicaldebateswill not end as a resultof pragmatism, hypotheses), and abduction (uncoveringand relying on the
and certainlythey shouldnot end. Nonetheless,we agreewith bestof a set of explanationsforunderstanding one'sresults)(e.g.,
othersin the mixedmethodsresearchmovementthat consider- de Waal, 2001).
ation and discussionof pragmatismby researchmethodologists Mixed methods research also is an attempt to legitimate the
and empiricalresearchers will be productivebecauseit offersan use of multiple approachesin answeringresearchquestions, rather
immediateandusefulmiddlepositionphilosophicallyandmeth- than restricting or constraining researchers'choices (i.e., it rejects
odologically;it offersa practicaland outcome-orientedmethod dogmatism). It is an expansive and creative form of research, not
of inquirythatis basedon actionandleads,iteratively,to further a limiting form of research. It is inclusive, pluralistic, and com-
action and the eliminationof doubt;and it offersa method for plementary, and it suggests that researcherstake an eclectic ap-
selecting methodological mixes that can help researchersbetter proachto methodselectionand the thinkingaboutand conduct
answermanyof theirresearchquestions.Pragmatically
inclined of research.Whatis most fundamentalis the researchquestion-
and
philosophers researchers
alsowould that
suggest we can reach researchmethodsshouldfollowresearchquestionsin a way that

OCTOBER2004 |I

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
Table 1
General Characteristics of Pragmatism

* The project of pragmatism has been to find a middle ground see what works, what solves problems, and what helps us to
between philosophical dogmatisms and skepticism and to find survive. We obtain warranted evidence that provides us with
a workable solution (sometimes including outright rejection) answers that are ultimately tentative (i.e., inquiry provides the
to many longstanding philosophical dualisms about which best answers we can currently muster), but, in the long run,
agreement has not been historically forthcoming. use of this "scientific" or evolutionary or practical epistemol-
* Rejects traditional dualisms (e.g., rationalism vs. empiricism, ogy moves us toward larger Truths.
realism vs. antirealism, free will vs. determinism, Platonic ap- * Endorses a strong and practical empiricism as the path to de-
pearance vs. reality, facts vs. values, subjectivism vs. objec- termine what works.
tivism) and generally prefersmore moderate and commonsense * Views current truth, meaning, and knowledge as tentative and
versions of philosophical dualisms based on how well they as changing over time. What we obtain on a daily basis in re-
work in solving problems. search should be viewed as provisional truths.
* Recognizes the existence and importance of the natural or * Capital "T"Truth(i.e., absolute Truth)is what will be the "final
physical world as well as the emergent social and psycholog- opinion" perhaps at the end of history. Lowercase "t"truths
ical world that includes language, culture, human institutions, (i.e., the instrumental and provisional truths that we obtain
and subjective thoughts. and live by in the meantime) are given through experience and
* Places high regard for the reality of and influence of the inner
experimenting.
world of human experience in action. * Instrumentaltruthsare a matterof degree (i.e., some estimates
* Knowledge is viewed as being both constructed and based on
are more true than others). Instrumentaltruth is not "stagnant,"
the reality of the world we experience and live in.
and, therefore, James (1995: 1907) states that we must "be
* Replaces the historicallypopular epistemic distinction between
ready tomorrow to call it falsehood."
subject and external object with the naturalistic and process- * Prefersaction to philosophizing (pragmatism is, in a sense, an
oriented organism-environment transaction.
* Endorses fallibilism (current beliefs and research conclusions anti-philosophy).
* Takes an explicitly value-oriented approach to research that is
are rarely, if ever, viewed as perfect, certain, or absolute).
* Justification comes in the form of what Dewey called "war- derived from cultural values; specifically endorses shared val-
ues such as democracy, freedom, equality, and progress.
ranted assertability."
* According to Peirce, "reasoningshould not form a chain which * Endorses practical theory (theory that informs effective prac-
is no strongerthan its weakest link, but a cable whose fibers may tice; praxis).
be ever so slender, provided they are sufficiently numerous and * Organisms are constantly adapting to new situations and en-
vironments. Our thinking follows a dynamic homeostatic
intimatelyconnected" (1868, in Menand, 1997, pp. 5-6).
* Theories are viewed instrumentally (they become true and process of belief, doubt, inquiry, modified belief, new doubt,
new inquiry, . . ,in an infinite loop, where the person or re-
they are true to different degrees based on how well they cur-
rently work; workability is judged especially on the criteria of searcher (and research community) constantly tries to improve
predictability and applicability). upon past understandings in a way that fits and works in the
* Endorses eclecticism and pluralism (e.g., different, even con- world in which he or she operates. The present is always a
flicting, theories and perspectives can be useful; observation, new starting point.
experience, and experiments are all useful ways to gain an un- * Generally rejects reductionism(e.g., reducing culture,thoughts,
derstanding of people and the world). and beliefs to nothing more than neurobiological processes).
* Human inquiry (i.e., what we do in our day-to-day lives as we * Offers the "pragmatic method" for solving traditional philo-
interact with our environments) is viewed as being analogous sophical dualisms as well as for making methodological
to experimental and scientific inquiry. We all try out things to choices.

offers the best chance to obtain useful answers. Many research Gaining an understanding of the strengths and weaknesses of
questions and combinations of questions are best and most fully quantitative and qualitative researchputs a researcherin a posi-
answered through mixed researchsolutions. tion to mix or combine strategies and to use what Johnson and
In order to mix researchin an effective manner, researchersfirst Turner (2003) call the fundamental principle of mixed research.
need to consider all of the relevant characteristicsof quantitative According to this principle, researchersshould collect multiple
and qualitative research.For example, the major characteristicsof data using different strategies, approaches, and methods in such
traditional quantitative researchare a focus on deduction, confir- a way that the resulting mixture or combination is likely to re-
mation, theory/hypothesis testing, explanation, prediction, stan- sult in complementary strengths and nonoverlapping weaknesses
dardized data collection, and statistical analysis (see Table 3 for (also see Brewer & Hunter, 1989). Effective use of this principle
a more complete list). The major characteristics of traditional is a major source of justification for mixed methods researchbe-
qualitative researchare induction, discovery, exploration, theory/ cause the product will be superior to monomethod studies. For
hypothesis generation, the researcheras the primary"instrument" example, adding qualitative interviews to experiments as a ma-
of data collection, and qualitative analysis (see Table 4 for a more nipulation check and perhaps as a way to discuss directly the is-
complete list). sues under investigation and tap into participants' perspectives

I| EDUCATIONALRESEARCHER

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
Table 2 Table 3
Some Weaknesses of Pragmatism Strengths and Weaknesses of Quantitative Research

* Basic research may receive less attention than applied research Strengths
because applied research may appear to produce more im- * Testing and validating already constructed theories about
mediate and practical results. how (and to a lesser degree, why) phenomena occur.
* Pragmatismmay promote incremental change ratherthan more * Testing hypotheses that are constructed before the data are
fundamental, structural,or revolutionary change in society. collected. Can generalize research findings when the data
* Researchers from a transformative-emancipatory framework are based on random samples of sufficient size.
have suggested that pragmatic researchers sometimes fail to * Can generalize a research finding when it has been repli-
provide a satisfactory answer to the question "Forwhom is a cated on many different populations and subpopulations.
pragmatic solution useful?"(Mertens, 2003). * Useful for obtaining data that allow quantitative predictions
* What is meant by usefulness or workability can be vague un- to be made.
less explicitly addressed by a researcher. * The researcher may construct a situation that eliminates the
* Pragmatic theories of truth have difficulty dealing with the
confounding influence of many variables, allowing one to
cases of useful but non-true beliefs or propositions and non- more credibly assess cause-and-effect relationships.
useful but true beliefs or propositions. * Data collection using some quantitative methods is rela-
* Many come to pragmatism looking for a way to get around
tively quick (e.g., telephone interviews).
many traditional philosophical and ethical disputes (this in- * Provides precise, quantitative, numerical data.
cludes the developers of pragmatism). Although pragmatism * Data analysis is relatively less time consuming (using statis-
has worked moderately well, when put under the microscope, tical software).
many current philosophers have rejected pragmatism because * The research results are relatively independent of the re-
of its logical (as contrasted with practical) failing as a solution searcher (e.g., effect size, statistical significance).
to many philosophical disputes. * It may have higher credibility with many people in power
* Some neo-pragmatistssuch as Rorty(and postmodernists)com-
(e.g., administrators,politicians, people who fund programs).
pletely reject correspondence truthin any form, which troubles * It is useful for studying large numbers of people.
many philosophers.
Weaknesses
* The researcher's categories that are used may not reflect
local constituencies' understandings.
and meanings will help avoid some potential problems with the * The researcher's theories that are used may not reflect local
experimental method. As another example, in a qualitative re- constituencies' understandings.
search study the researchermight want to qualitatively observe * The researcher may miss out on phenomena occurring be-
and interview, but supplement this with a closed-ended instru- cause of the focus on theory or hypothesis testing rather
ment to systematically measure certain factors considered im- than on theory or hypothesis generation (called the confir-
portant in the relevant researchliterature. Both of these examples mation bias).
could be improved (if the researchquestions can be studied this * Knowledge produced may be too abstract and general for
direct application to specific local situations, contexts, and
way) by adding a component that surveys a randomly selected
individuals.
sample from the population of interest to improve generalizabil-
ity. If findings are corroborated across different approaches then
greater confidence can be held in the singular conclusion; if the
findings conflict then the researcherhas greaterknowledge and can many other typologies (especiallyCreswell, 1994; Morgan, 1998;
modify interpretationsand conclusions accordingly.In many cases Morse, 1991; Patton, 1990; and Tashakkori & Teddlie, 1998), as
the goal of mixing is not to search for corroboration but ratherto well as severaldimensions which one should consider when plan-
expand one's understanding (Onwuegbuzie & Leech, 2004b). ning to conduct a mixed researchstudy. For example, it has been
Tables 3 and 4 are specifically designed to aid in the con- noted that one can construct mixed-model designs by mixing
struction of a combination of qualitative and quantitative re- qualitative and quantitative approaches within and across the
search. After determining one's research question(s), one can stages of research (in a simplified view, one can consider a single
decide whether mixed researchoffers the best potential for an an- study as having three stages:stating the research objective, col-
swer; if this is the case, then one can use the tables as an aid to lecting the data, and analyzing/interpreting the data; see mixed-
model designs in Johnson & Christensen, 2004; Tashakkori &
help in deciding on the combination of complementary strengths
and nonoverlapping weaknesses that is appropriate for a partic- Teddlie, 1998). According to Morgan (1998) and Morse (1991),
ular study. Table 5 shows some of the strengths and weaknesses one also may consider the dimension of paradigm emphasis (de-
of mixed methods research, which should aid in the decision to ciding whether to give the quantitative and qualitative compo-
use or not use a mixed methods researchapproach for a given re- nents of a mixed study equal status or to give one paradigm the
search study. dominant status). Time ordering of the qualitative and quanti-
tative phases is another important dimension, and the phases can
Development of a Mixed Methods be carriedout sequentially or concurrently. Our mixed-method
Research Typology
designs (discussed below) are based on the crossing of paradigm
Our mixed methods research typologies (mixed-model designs emphasis and time ordering of the quantitative and qualitative
and mixed-method designs) resulted from our consideration of phases. Another dimension for viewing mixed methods re-

OCTOBER2004 I|E

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
Table 4
Strengths and Weaknesses of Qualitative Research

Strengths * Qualitative researchersare responsive to changes that occur


* The data are based on the participants' own categories of during the conduct of a study (especially during extended
meaning. fieldwork) and may shift the focus of their studies as a result.
* It is useful for studying a limited number of cases in depth. * Qualitative data in the words and categories of participants
* It is useful for describing complex phenomena. lend themselves to exploring how and why phenomena
* Provides individual case information. occur.
* Can conduct cross-case comparisons and analysis. * One can use an important case to demonstrate vividly a
* Provides understanding and description of people's per- phenomenon to the readers of a report.
sonal experiences of phenomena (i.e., the "emic"or insider's * Determine idiographic causation (i.e., determination of
causes of a particularevent).
viewpoint).
* Can describe, in rich detail, phenomena as they are situated Weaknesses
and embedded in local contexts.
* Knowledge produced may not generalize to other people or
* The researcher identifies contextual and setting factors as
other settings (i.e., findings may be unique to the relatively
they relate to the phenomenon of interest. few people included in the research study).
* The researcher can study dynamic processes (i.e., docu-
* It is difficult to make quantitative predictions.
menting sequential patterns and change). * It is more difficult to test hypotheses and theories.
* The researcher can use the primarilyqualitative method of
* It may have lower credibility with some administratorsand
"grounded theory" to generate inductively a tentative but commissioners of programs.
explanatory theory about a phenomenon. * It generally takes more time to collect the data when com-
* Can determine how participants interpret"constructs"(e.g.,
pared to quantitative research.
self-esteem, IQ). * Data analysis is often time consuming.
* Data are usually collected in naturalistic settings in qualita-
* The results are more easily influenced by the researcher's
tive research.
* Qualitative approaches are responsive to local situations, personal biases and idiosyncrasies.
conditions, and stakeholders' needs.

search is the degree of mixture, which would form a continuum the table. To construct a mixed-method design, the researcher
from monomethod to fully mixed methods. Another dimen- must make two primary decisions: (a) whether one wants to
sion pertains to where mixing should occur (e.g., in the objec- operate largely within one dominant paradigm or not, and
tive[s], methods of data collection, research methods, during (b) whether one wants to conduct the phases concurrently or
data analysis, data interpretation). Yet another important sequentially. In contrast to mixed-model designs, mixed-method
dimension is whether one wants to take a critical theory/
designs are similar to conducting a quantitative mini-study and
transformative-emancipatory (Mertens, 2003) approach or a less a qualitative mini-study in one overall researchstudy. Nonethe-
explicitly ideological approach to a study. Ultimately, the possi- less, to be considered a mixed-method design, the findings must
ble number of ways that studies can involve mixing is very large be mixed or integrated at some point (e.g., a qualitative phase
because of the many potential classification dimensions. It is a
might be conducted to inform a quantitative phase, sequentially,
key point that mixed methods research truly opens up an excit- or if the quantitative and qualitative phases are undertaken con-
ing and almost unlimited potential for future research.
currently the findings must, at a minimum, be integrated during
Toward a Parsimonious Typology the interpretation of the findings).
of Mixed Research Methods It is important to understand that one can easily createmore
The majority of mixed methods research designs can be devel- userspecificand morecomplexdesignsthan the ones shown in Fig-
ures 1 and 2. For example, one can develop a mixed-method de-
oped from the two major types of mixed methods research:
mixed-model (mixing qualitative and quantitative approaches sign that has more stages (e.g., Qual -> QUAN -> Qual); one
within or across the stages of the research process) and mixed- also can design a study that includes both mixed-model and
method (the inclusion of a quantitative phase and a qualitative mixed-method design features. The point is for the researcherto
phase in an overall research study). Six mixed-model designs are be creative and not be limited by the designs listed in this article.
shown in Figure 1 (see Designs 2 through 7). These six designs Furthermore, sometimes a design may emerge during a study in
are called across-stagemixed-model designs because the mixing new ways, depending on the conditions and information that is
takes place across the stages of the researchprocess. An example obtained. A tenet of mixed methods research is that researchers
of a within-stage mixed-modeldesignwould be the use of a ques- should mindfully create designs that effectively answer their re-
tionnaire that includes a summated rating scale (quantitative searchquestions; this stands in contrast to the common approach
data collection) and one or more open-ended questions (qualita- in traditional quantitative research where students are given a
tive data collection). menu of designs from which to select.8'9 It also stands in stark
Nine mixed-method designs are provided in Figure 2. The no- contrast to the approach where one completely follows either the
tation used (based on Morse, 1991) is explained at the bottom of qualitative paradigm or the quantitative paradigm.

i2| EDUCATIONALRESEARCHER

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
Table 5
Strengths and Weaknesses of Mixed Research

Strengths * Can add insights and understanding that might be missed


* Words, pictures, and narrativecan be used to add meaning when only a single method is used.
to numbers. * Can be used to increase the generalizability of the results.
* Numbers can be used to add precision to words, pictures, * Qualitative and quantitative research used together produce
and narrative. more complete knowledge necessary to inform theory and
* Can provide quantitative and qualitative research strengths practice.
(i.e., see strengths listed in Tables 3 and 4). Weaknesses
* Researcher can generate and test a grounded theory.
* Can be difficult for a single researcher to carry out both
* Can answer a broader and more complete range of research
qualitative and quantitative research, especially if two or
questions because the researcher is not confined to a single more approaches are expected to be used concurrently; it
method or approach.
may require a research team.
* The specific mixed research designs discussed in this article * Researcher has to learn about multiple methods and ap-
have specific strengths and weaknesses that should be con-
proaches and understand how to mix them appropriately.
sidered (e.g., in a two-stage sequential design, the Stage 1 * Methodological puristscontend that one should always work
results can be used to develop and inform the purpose and within either a qualitative or a quantitative paradigm.
design of the Stage 2 component). * More expensive.
* A researcher can use the strengths of an additional method * More time consuming.
to overcome the weaknesses in another method by using * Some of the details of mixed research remain to be worked
both in a research study. out fully by research methodologists (e.g., problems of par-
* Can provide stronger evidence for a conclusion through adigm mixing, how to qualitativelyanalyze quantitativedata,
convergence and corroboration of findings. how to interpretconflicting results).

A Mixed Methods Research Process Model pose can be revised when needed. Figure 3 shows several arrows
Our mixed methods researchprocess model comprises eight dis- leading from later steps to earlier steps indicating that mixed re-
tinct steps: (1) determine the research question; (2) determine search involves a cyclical, recursive,and interactional process. Re-
whether a mixed design is appropriate; (3) select the mixed- cursion can take place within a single study (especiallyan extended
method or mixed-model research design; (4) collect the data; study); recursion can also take place across related studies by in-
(5) analyze the data; (6) interpret the data; (7) legitimate the forming future research and leading to new or reformulated re-
data; and (8) draw conclusions (if warranted) and write the final search purposes and questions.
report. These steps are displayed in Figure 3. Although mixed re- Three steps in the mixed methods research process warrant
search starts with a purpose and one or more researchquestions, some further discussion, especially purpose (Step 2), data analy-
the rest of the steps can vary in order (i.e., they are not necessar- sis (Step 5), and legitimation (Step 7). As noted by Greene et al.
ily linear or unidirectional), and even the question and/or pur- (1989), there are five major purposes or rationalesfor conducting

QualitativeResearch QuantitativeResearch
Objective(s) Objective(s)

/
Collect Collect Collect Collect
qualitative quantitative qualitative quantitative
data data data data

Perform Perform Perform Perform Perform Perform Perform Perform


qualitative quantitative qualitative quantitative qualitative quantitative qualitative quantitative
analysis analysis analysis analysis analysis analysis analysis analysis
1 2 3 4 5 6 7 8
Designs
Note. Designs 1 and 8 on the outeredgesarethe monomethoddesigns.The mixed-modeldesignsareDesigns2, 3, 4, 5, 6, and 7.10

FIGURE 1. Monomethod and mixed-modeldesigns.

OCTOBER2004 |I

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
Time Order
Decision
Concurrent Sequential

QUAL + QUAN QUAL -- QUAN


Equal
Status QUAN - QUAL

Paradigm
Emphasis QUAL + quan QUAL - quan
Decision qual - QUAN
Dominant
Status QUAN -) qual
QUAN + qual quan -) QUAL

Note. "qual" stands for qualitative, "quan" stands for quantitative, "+" stands for concurrent, "->" stands
for sequential, capital letters denote high priority or weight, and lower case letters denote lower priority or
weight."

designmatrixwith mixed-method
FIGURE2. Mixed-method designsshown
research
in thefourcells.

mixed methodsresearch:(a) triangulation(i.e., seekingconver- titativedatasources.Data integration characterizes


the finalstage,
gence and corroborationof resultsfrom differentmethodsand whereby both quantitative and qualitativedata are integrated
designsstudyingthe same phenomenon);(b) complementarity into eithera coherentwhole or two separatesets (i.e., qualitative
(i.e., seekingelaboration,enhancement,illustration,and clarifi- and quantitative)of coherentwholes.
cationof the resultsfromone methodwith resultsfromthe other The legitimationstep involvesassessingthe trustworthiness of
method);(c) initiation (i.e., discoveringparadoxesand contra- both the qualitativeand quantitativedataand subsequentinter-
dictionsthatleadto a re-framingof the researchquestion);(d) de- pretations.Frameworkssuch as the QuantitativeLegitimation
velopment (i.e., using the findings from one method to help Model (Onwuegbuzie,2003; whichcontains50 sourcesof inva-
informthe othermethod);and (e) expansion(i.e., seekingto ex- lidityfor the quantitativecomponentof the mixedmethodsre-
pandthe breadthand rangeof researchby usingdifferentmeth- searchat the datacollection,dataanalysis,anddatainterpretation
ods for differentinquirycomponents). stages of the study) and the QualitativeLegitimationModel
The mixed methods researchprocess model incorporates (Onwuegbuzie,2000; Onwuegbuzie,Jiao, & Bostick, 2004;
Onwuegbuzieand Teddlie's(2003) seven-stageconceptualiza- which contains29 elementsof legitimationfor the qualitative
tion of the mixed methods data analysisprocess.Accordingto componentof the mixedmethodsresearchat the datacollection,
theseauthors,the sevendataanalysisstagesareasfollows:(a)data dataanalysis,and datainterpretationstagesof the study)can be
reduction,(b) datadisplay,(c) datatransformation, (d) datacor- used to assessthe legitimacyof the qualitativeand quantitative
relation,(e) dataconsolidation,(f) datacomparison,and (g) data phasesof the study, respectively.We havebegunworkingon a
integration.Data reductioninvolvesreducingthe dimensionality validityor legitimationtypologyspecificallyfor mixedresearch
of the qualitativedata (e.g., via exploratorythematicanalysis, in Onwuegbuzieand Johnson (2004). It is importantto note
memoing)and quantitativedata (e.g., via descriptivestatistics, that the legitimationprocessmight includeadditionaldatacol-
exploratoryfactor analysis,clusteranalysis).Data display,in- lection, data analysis,and/or data interpretationuntil as many
volvesdescribingpictoriallythe qualitativedata (e.g., matrices, rivalexplanationsas possiblehavebeen reducedor eliminated.
charts,graphs,networks,lists, rubrics,and Venn diagrams)and
quantitativedata(e.g., tables,graphs).This is followed(option- The Future of Mixed Methods
ally) by the data transformation stage,whereinquantitativedata Research in Education
areconvertedinto narrative datathatcanbe analyzedqualitatively Mixedresearchactuallyhasa longhistoryin researchpracticebe-
(i.e., qualitized;Tashakkori& Teddlie, 1998) and/orqualitative causepracticingresearchers frequentlyignorewhat is writtenby
dataareconvertedinto numericalcodes that can be represented methodologists when they feel a mixedapproachwill best help
Tashakkori& Teddlie,1998). Data
statistically(i.e., quantitized; them to answertheirresearchquestions.It is time thatmethod-
correlation involvesthe quantitativedatabeingcorrelatedwith the ologistscatchup with practicingresearchers!
It is now time that
qualitizeddataor the qualitativedatabeing correlatedwith the allresearchersandresearchmethodologistsformallyrecognizethe
quantitizeddata.This is followedby dataconsolidation, wherein thirdresearchparadigmand beginsystematically writingaboutit
bothquantitativeandqualitativedataarecombinedto createnew and using it. In generalwe recommendcontingencytheoryfor
or consolidatedvariablesor datasets. The next stage,data com- researchapproachselection, which accepts that quantitative,
parison involves comparing data from the qualitative and quan- qualitative,andmixedresearchareall superiorunderdifferentcir-

2|1 EDUCATIONALRESEARCHER

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
IF

Note. Circlesrepresentsteps(1-8) in the mixedresearchprocess;rectanglesrepresentstepsin the mixeddataanalysisprocess;diamondsrepresentcomponents.

FIGURE 3. Mixed researchprocessmodel.

cumstances taskto examinethe specific


and it is the researcher's well as in our sisterdisciplinesin the social and behavioralsci-
contingencies and make the decisionaboutwhich researchap- ences,as the thirdmajorresearchparadigm.
proach, or which combination of approaches,shouldbe used in As noted by Sechrestand Sidana(1995), growthin the mixed
a specificstudy.In thisarticlewe haveoutlinedthe philosophyof methods(i.e., pragmatist)movementhasthe potentialto reduce
pragmatism, we havedescribedmixedresearchandprovidedspe- some of the problemsassociatedwith singularmethods.By uti-
cific mixed-modeland mixed-methoddesigns,and we havedis- lizing quantitativeand qualitativetechniqueswithin the same
cussedthe fundamentalprincipleof mixedresearchandprovided framework, mixed methods research can incorporate the
tablesof quantitativeandqualitativeresearchstrengthsandweak- strengthsof both methodologies.Most importantly,investiga-
nessesto helpapplythe principle.Also,we haveprovideda mixed torswho conductmixedmethodsresearcharemorelikelyto se-
methodsprocessmodelto help readersvisualizethe process.We lect methods and approacheswith respectto their underlying
hopewe havemadethe casethatmixedmethodsresearchis here researchquestions,ratherthanwith regardto somepreconceived
to stayand that it shouldbe widely recognizedin education,as biasesaboutwhich researchparadigmshouldhavehegemonyin

OCTOBER2004 |

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
social science research. By narrowing the divide between quanti- 9 For additional mixed-method
designs, see Creswell, Plano, Clark,
tative and qualitative researchers,mixed methods researchhas a Guttmann, and Hanson, 2003; Maxwell and Loomis, 2003.
10Here is the etiology of Figure 1: As far as we know, Patton (1990)
great potential to promote a shared responsibility in the quest for
first listed 6 of the mixed model designs (Designs 1, 2, 3, 5, 6, and 8).
attaining accountability for educational quality. The time has
Then Tashakkori and Teddlie (1998) built on this by adding two de-
come for mixed methods research.
signs (Designs 4 and 7) that were left out by Patton and they changed
NOTES some labels to better fit their thinking (e.g., they introduced the term
1 Thomas Kuhn mixed model). Finally, in its present form, we first used (in an AERA
(1962) popularized the idea of a paradigm. Later,
conference paper) the full set of eight designs identified by Tashakkori
when he was asked to explain more precisely what he meant by the term,
and Teddlie (1998) while changing some labels to better fit our concep-
he pointed out that it was a general concept and that it included a group
tualization. The term monomethods probably originated in Campbell
of researchershaving a common education and an agreement on "exem-
and Fiske (1959).
plars"of high quality research or thinking (Kuhn, 1977). In this article, 11In
developing Figure 2, we were probably most influenced by
by researchparadigm we mean a set of beliefs, values, and assumptions
that a community of researchers has in common regarding the nature Morgan (1998), Morse (1991), and Tashakkori and Teddlie (1998). Sev-
eral of the designs shown in the figure were introduced by Morse (1991).
and conduct of research. The beliefs include, but are not limited to, on-
tological beliefs, epistemological beliefs, axiological beliefs, aesthetic be- REFERENCES
liefs, and methodological beliefs. In short, as we use the term, a research
Ayer, A. J. (1959). Logicalpositivism. New York: The Free Press.
paradigm refers to a research culture. We will be arguing that there is Biesta, G. J. J., & Burbules, N. C. (2003). Pragmatismand educational
now a trilogy of major research paradigms: qualitative research, quanti-
research.Lanham, MD: Rowman and Littlefield.
tative research, and mixed methods research.
Brewer, J., & Hunter, A. (1989). Multimethod research:A synthesisof
2 Campbell modified his view of
qualitative research over time. For styles.Newbury Park, CA: Sage.
example, based on criticisms by qualitative and case study researchers Bryman, A. (1984). The debate about quantitative and qualitative re-
of his term "one-shot case study" (which, unfortunately, is still used in
search:A question of method or epistemology? BritishJournal of So-
several educational research books), Campbell changed this design
ciology,35, 78-92.
name to the one-group posttest-only design; he made this change as
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant
part of his endorsement of case study research as an important research validation by the multitrait-multimethod matrix. PsychologicalBul-
approach (e.g., see Campbell's introduction to Yin's case study research letin, 56, 81-105.
book: Yin, 1984).
Campbell, D. T., & Stanley, J. C. (1963). Experimental and quasi-
3 We do not mean to
imply that there is anything inherently wrong experimentaldesignsfor research.Chicago, IL: Rand McNally.
with taking an extreme intellectual position. Most of the great thinkers
Constas, M. A. (1992). Qualitative data analysis as a public event: The
in the history of philosophy and science (including social and behavioral documentation of category development procedures. American Edu-
science) were "extreme"for their times. Also, both qualitative and quan- cational ResearchJournal, 29, 253-266.
titative philosophies continue to be highly useful (i.e., both have many Creswell, J. W. (1994). Researchdesign:Qualitative and quantitative ap-
advantages when used in their pure forms). proaches.Thousand Oaks, CA: Sage.
4 Positivism is a
poor choice for labeling quantitative researcherstoday Creswell, J.W. (2003). Researchdesign: Qualitative, quantitative, and
because positivism has long been replaced by newer philosophies of sci- mixed approaches.Thousand Oaks, CA: Sage.
ence (Yu, 2003). The term is more of a straw man (easily knocked down) Creswell, J. W, Plano Clark, V. L., Guttmann, M. L., & Hanson, E. E.
for attack than standing for any actual practicing researchers. A term (2003). Advanced mixed methods research design. In A.Tashakkori
that better represents today's practicing quantitative researchersis post- and C. Teddlie (Eds.), Handbook of mixed methods in social and be-
positivism (Phillips & Burbules, 2000). havioral research(pp. 209-240). Thousand Oaks, CA: Sage.
s Both of the authors of the current article prefer the label mixed re- de Jong, H.L. (2003). Causal and functional explanations. Theoryand
searchor integrativeresearchratherthan mixed methodsresearch.The alter-
Psychology,13(3), 291-317.
native labels are broader, more inclusive, and more clearly paradigmatic. de Waal, C. (2001). On Peirce. Belmont, CA: Wadsworth.
We chose to use the term mixed methodsin this article because of its cur-
Dewey, J. (1948, 1920). Reconstructionin philosophy.Boston, MA: Bea-
rent popularity. con Press.
6 Here is a
practical definition of science from an educational research Dzurec, L. C., & Abraham, J. L. (1993). The nature of inquiry: Link-
textbook (Johnson & Christensen, 2004) that should be inclusive of
ing quantitative and qualitative research.Advancesin Nursing Science,
quantitative and qualitative research: ". . . the root of the word science 16, 73-79.
is the Latin scientia, which simply means 'knowledge.' We define science Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a con-
in this book in a way that is inclusive of the different approachesto educa-
ceptual framework for mixed-method evaluation designs. Educa-
tional research.We define science as an approach for the generation of tional Evaluation and Policy Analysis, 11, 255-274.
knowledge that places high regard for empirical data and follows certain Guba, E. G. (1990). The alternative paradigm dialog. In E. G. Guba
norms and practices that develop over time because of their usefulness. (Ed.), Theparadigm dialog (pp. 17-27). Newbury Park, CA: Sage.
... The ultimate objective of most social, behavioral, and educational Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation.
research is improvement of the world or social betterment." Newbury Park, CA: Sage.
7 This is a very interesting empirical question that deserves more at- Hoshmand, L.T. (2003). Can lessons of history and logical analysis en-
tention in the literature. sure progress in psychological science? Theory and Psychology, 13,
8 Note that Shadish, Cook, and
Campbell (2002) have attempted to 39-44.
move quantitative researchaway from this traditional "menu" approach. Howe, K. R. (1988). Against the quantitative-qualitative incompatabil-
In this latest edition of Campbell and Stanley (1963), there is increased ity thesis, or, Dogmas die hard. Educational Researcher,17, 10-16.
focuson understanding how to constructor createa researchdesignthat Howe, K. R. (1992). Getting over the quantitative-qualitative debate.
fits a particularsituation. AmericanJournal ofEducation, 100, 236-256.

2|[ EDUCATIONALRESEARCHER

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
James, W. (1995, 1907). Pragmatism. New York: Dover. Onwuegbuzie, A. J., & Johnson, R. B. (2004, April). Validity issuesin
Johnson, R. B., & Christensen, L. B. (2004). Educational research: mixed methodsresearch.Paper presented at the annual meeting of the
Quantitative, qualitative, and mixed approaches.Boston, MA: Allyn American Educational Research Association, San Diego, CA.
and Bacon. Onwuegbuzie, A. J., & Leech, N. L. (2004a). On becominga pragmatic
Johnson, R. B., Meeker, K. M., Loomis, E. J., & Onwuegbuzie, A. J. researcher:The importance of combining quantitative and qualitative
(2004, April). Development of thephilosophicaland methodologicalbe- researchmethodologies.Manuscript submitted for publication.
liefi inventory. Paper presented at the annual meeting of the Ameri- Onwuegbuzie, A. J., & Leech, N. L. (2004b, February). Enhancing the
can Educational Research Association, San Diego, CA. interpretationofsignificantfindings: The role of mixed methodsresearch.
Johnson, R. B., & Turner, L. A. (2003). Data collection strategies in Paper presented at the annual meeting of the Eastern Educational Re-
mixed methods research. In A.Tashakkori, and C. Teddlie (Eds.), search Association, Clearwater, FL.
Handbook of mixed methods in social and behavioral research (pp. Onwuegbuzie, A. J., & Leech, N. L. (in press). Taking the "Q" out of
297-319). Thousand Oaks, CA: Sage. research:Teaching research methodology courses without the divide
Kuhn, T. S. (1962). The structureofscientific revolutions. Chicago, IL: between quantitative and qualitative paradigms. Quality e&Quantity:
University of Chicago Press. InternationalJournal ofMethodology.
Kuhn, T. S. (1977). The essentialtension:Selectedstudies in scientifictra- Onwuegbuzie, A. J., & Teddlie, C. (2003). A framework for analyzing
dition and change. Chicago, IL: University of Chicago Press. data in mixed methods research. In A. Tashakkori & C. Teddlie
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly (Eds.), Handbook of mixed methods in social and behavioral research
Hills, CA: Sage. (pp. 351-383). Thousand Oaks, CA: Sage.
Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, con- Patton, M. Q. (1990). Qualitative evaluation and researchmethods (2nd
tradictions, and emerging confluences. In N. K. Denzin, and Y. S. ed.). Newbury Park, CA: Sage.
Lincoln (Eds.), Handbookofqualitative research(pp. 163-188). Thou- Peirce, C. S. (1878). How to make our ideas clear. Popular Science
sand Oaks, CA: Sage. Monthly, 12, 286-302.
Maxcy, S. J. (2003). Pragmatic threads in mixed methods research in Phillips, D. C. (2004, April). Is mixed methodsresearchan epistemologi-
the social sciences: The search for multiple modes of inquiry and the cal oxymoron?Paper presented at the annual meeting of the American
end of the philosophy of formalism. In A.Tashakkori & C. Teddlie Educational Research Association, San Diego, CA.

(Eds.), Handbook of mixed methods in social and behavioral research Phillips, D. C., & Burbules, N. C. (2000). Postpositivism and educa-
tional research.New York: Rowman & Littlefield.
(pp. 51-89). Thousand Oaks, CA: Sage.
Maxwell, J. A., & Loomis, D. M. (2003). Mixed methods design: An Popper, K. R. (1959). Thelogicofscientificdiscovery.New York: Routledge.
alternative approach. In A.Tashakkori, and C. Teddlie (Eds.), Hand- Reichardt, C. S., & Cook, T. D. (1979). Beyond qualitative versus
bookofmixed methodsin social and behavioralresearch(pp. 241-272). quantitative methods. In T. D. Cook & C. S. Reichardt (Eds.),
Thousand Oaks, CA: Sage. Qualitative and quantitative methodsin evaluation research(pp. 7-32).
Maxwell, S. E., & Delaney, H. D. (2004). Designing experimentsand Newbury Park, CA: Sage.
Reichardt, S. S., & Rallis, S. F. (1994). Qualitative and quantitative in-
analyzing data. Mahwah, NJ: Lawrence Erlbaum.
Menand, L. (1997). Pragmatism:A reader.New York: Vintage. quiries are not incompatible: A call for a new partnership. In C. S.
Reichardt & S. F. Rallis (Eds.), The qualitative-quantitative debate:
Mertens, D. M. (2003). Mixed methods and the politics of human re-
search:The transformative-emancipatoryperspective.In A. Tashakkori Newperspectives (pp. 85-91). San Francisco, CA: Jossey-Bass.
Rescher, N. (2000). Realistic pragmatism:An introduction to pragmatic
& C. Teddlie (Eds.), Handbook of mixed methods in social and be-
havioral research(pp. 135-164). Thousand Oaks, CA: Sage. philosophy. Albany, NY: State University of New York.
Rorty, R. (2000). Pragmatism. InternationalJournal of Psychoanalysis,
Morgan, D. L. (1998). Practical strategies for combining qualitative and 81,819-823.
quantitative methods: Applications to health research. Qualitative Sandelowski, M. (1986). The problem of rigor in qualitative research.
Health Research,3, 362-376.
Advances in Nursing Science, 8(3), 27-37.
Morse, J. M. (1991). Approaches to qualitative-quantitative method-
Schrag, F. (1992). In defense of positivist research paradigms, Educa-
ological triangulation. Nursing Research,40, 120-123. tionalResearcher,21(5), 5-8.
Murphy, J. P. (1990). Pragmatism: From Peirce to Davidson. Boulder, Schwandt, T. A. (2000). Three epistemological stances for qualitative
CO: Westview.
inquiry: Interpretivism, hermeneutics, and social constructionism. In
Nagel, T. (1986). The view fom nowhere. New York: Oxford Univer- N. K. Denzin & Y. S. Lincoln, Handbook of qualitative research(pp.
sity Press. 189-213). Thousand Oaks, CA: Sage.
Newman, I., & Benz, C. R. (1998). Qualitative-quantitative research Sechrest, L., & Sidana, S. (1995). Quantitative and qualitative methods:
methodology:Exploring the interactive continuum. Carbondale, IL: Is there an alternative?Evaluation and ProgramPlanning, 18, 77-87.
Southern Illinois University Press. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimen-
Onwuegbuzie, A. J. (2000, November). Validity and qualitative re- tal and quasi-experimental designs for generalized causal inference.
search:An Oxymoron?Paper presented at the annual meeting of the Boston, MA: Houghton Mifflin.
Association for the Advancement of Educational Research (AAER), Sieber, S. D. (1973). The integration of fieldwork and survey methods.
Ponte Vedra, Florida. AmericanJournal of Sociology,73, 1335-1359.
Onwuegbuzie, A. J. (2002). Positivists,post-positivists, post-structuralists, Smith, J. K. (1983). Quantitative versus qualitative research: An at-
and post-modernists:Why can't we all get along?Towards a framework
tempt to clarify the issue. Educational Researcher,12, 6-13.
for unifying researchparadigms. Education, 122(3), 518-530. Smith, J. K. (1984). The problem of criteria for judging interpretive in-
Onwuegbuzie, A. J. (2003). Expanding the framework of internal and quiry. Educational Evaluation and Policy Analysis, 6, 379-391.
external validity in quantitative research. Researchin the Schools, 10, Tashakkori, A., & Teddlie, C. (1998). Mixed methodology:Combining
71-90. qualitativeand quantitative approaches.Applied Social ResearchMeth-
Onwuegbuzie, A. J., Jiao, Q. G., & Bostick, S. . (2004). Library anx- ods Series (Vol. 46). Thousand Oaks, CA: Sage.
iety: Theory, research, and applications. Lanham, MD: Scarecrow Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook ofmixed meth-
Press. ods in social and behavioral research.Thousand Oaks, CA: Sage.

OCTOBER2004 |[

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions
Watson, W. (1990). Types of pluralism. The Monist, 73(3), 350-367. ANTHONY J. ONWUEGBUZIEis an AssociateProfessor,Depart-
Yin, R. K. (1984). Case study research:Design and methods. Thousand ment of Educational Measurement and Research, University of South
Oaks, CA: Sage. Florida, 4202 East Fowler Avenue, EDU 162, Tampa, FL 33620-7750;
Yu, C. H. (2003). Misconceived relationshipsbetween logicalpositivism tonyonwuegbuzie@aol.com. His areasof specialization are disadvantaged
and quantitative research.Research Methods Forum [On-line]. Re- and under-served populations (e.g., minorities and juvenile delinquents)
trieved September 2, 2004 from http://www.aom.pace.edu/rmd/ and methodological topics in the areas of quantitative, qualitative, and
2002forum.html. mixed methods.
AUTHORS
R. BURKEJOHNSON is a Professor,University of South Alabama, Col- Manuscriptreceived October 14, 2003
lege of Education, BSET, 3700 UCOM, Mobile, AL 36688; bjohnson@ Revisionsreceived March I and April30, 2004
usouthal.edu. His area of specialization is research methodology. Accepted May 12, 2004

Justoffthepress!

"TeachersMatter:Evidence From
Value-AddedAssessments"
Points,Summer2004)
(Research

Readit at
www.aera.net

2| EDUCATIONALRESEARCHER

This content downloaded from 128.232.230.2 on Wed, 30 Oct 2013 13:45:36 PM


All use subject to JSTOR Terms and Conditions