ComparingClassicalTestTheorywithCFA

and
HowToUseTestScoresin
SecondaryAnalyses
LatentTraitMeasurementand
StructuralEquationModels
Lecture#8
March6,2013

PSYC948:Lecture#8

TodaysClass

ComparingclassicaltesttheorytoCFA

Theuseandmisuseofsumscores
ReliabilityforsumscoresunderCFA

HowtouseCFAtotestassumptionsinCTT

WhattodowhenSEMisntanoption

Secondaryanalyses

PSYC948:Lecture#8

DataforTodaysClass

Datawerecollectedfromtwosources:

144experiencedgamblers

1192collegestudentsfromarectangularmidwestern state

Manynevergambledbefore

Today,wewillcombinebothsamplesandtreatthemas
homogenous onesampleof1346subjects

Manyfromanactualcasino

Laterwewilltestthisassumption measurementinvariance(calleddifferentialitem
functioninginitemresponsetheoryliterature)

Wewillbuildascaleofgamblingtendenciesusingthefirst24items
oftheGRI

Focusedonlongtermgamblingtendencies

PSYC948:Lecture#8

PathologicalGambling:DSMDefinition

1.
2.

3.
4.

5.

6.

Tobediagnosedasapathologicalgambler,anindividualmustmeet5of10definedcriteria:

Ispreoccupiedwithgambling
Needstogamblewithincreasing
amountsofmoneyinorderto
achievethedesiredexcitement
Hasrepeatedunsuccessfuleffortsto
control,cutback,orstopgambling
Isrestlessorirritablewhen
attemptingtocutdownorstop
gambling
Gamblesasawayofescapingfrom
problemsorrelievingadysphoric
mood
Afterlosingmoneygambling,often
returnsanotherdaytogeteven

PSYC948:Lecture#8

7.

8.

9.

10.

Liestofamilymembers,therapist,or
otherstoconcealtheextentof
involvementwithgambling
Hascommittedillegalactssuchas
forgery,fraud,theft,or
embezzlementtofinancegambling
Hasjeopardizedorlostasignificant
relationship,job,educational,or
careeropportunitybecauseof
gambling
Reliesonotherstoprovidemoney
torelieveadesperatefinancial
situationcausedbygambling

Final12ItemsontheScale
Item

Criterion

Question

GRI1

GRI3

Iwouldliketocutbackonmygambling.
IfIlostalotofmoneygamblingoneday,Iwouldbemorelikelytowanttoplay
againthefollowingday.

GRI5

Ifinditnecessarytogamblewithlargeramountsofmoney(thanwhenIfirst
gambled)forgamblingtobeexciting.

GRI6

Ihave gonetogreatlengthstoobtainmoneyforgambling.

GRI9

IfeelrestlesswhenItrytocutdownorstopgambling.

GRI10

ItbothersmewhenIhavenomoneytogamble.

GRI11

Igambletotakemymindoffmyworries.

GRI13

Ifinditdifficulttostopgambling.

GRI14

IamdrawnmorebythethrillofgamblingthanbythemoneyIcouldwin.

GRI15

Iamprivateaboutmygamblingexperiences.

GRI21

Itishardtogetmymindoffgambling.

GRI23

Igambletoimprovemymood.

PSYC948:Lecture#7

GRI12ItemAnalysis

The12itemanalysisgavethismodelfitinformation:
Themodel indicatedthemodeldidnotfit
betterthanthesaturatedmodel butthis
statisticcanbeoverlysensitive
ThemodelRMSEAindicatedgoodmodelfit
(wantthistobe<.05)

ThemodelCFIandTLIindicatedthemodelfit
well(wantthesetobe>.95)
TheSRMRindicatedthefitwell(wantthisto
be<.08)

PSYC948:Lecture#7

CLASSICALTESTTHEORY

PSYC948:Lecture#8

Classical TestTheory(CTT)

Whatyouhavelearnedaboutmeasurementsofarlikelyfallsunder
thecategoryofCTT:

Writingitemsandbuildingscales
Itemanalysis
Scoreinterpretation
Evaluatingreliabilityandconstructvalidity

Bigpicture:WewillviewCTTasmodelwitharestrictivesetof
assumptionswithinamoregeneralfamilyoflatenttrait
measurementmodels

ConfirmatoryFactorAnalysisisameasurementmodel

PSYC948:Lecture#8

DifferencesAmongMeasurementModels

Whatisthenameofthelatenttrait measuredbyatest?

ClassicalTestTheory(CTT)=TrueScore(T)
ConfirmatoryFactorAnalysis(CFA)=FactorScore(F)
ItemResponseTheory(IRT)=Theta()

Fundamentaldifferenceinapproach:

CTT unitofanalysisistheWHOLETEST (itemsumormean)

Sum=latenttrait,andthesumdoesntcarehowitwascreated
Onlyusingthesumrequiresrestrictiveassumptionsabouttheitems

CFA,IRT,andbeyond unitofanalysisistheITEM
Modelofhowitemresponserelatestoanestimatedlatenttrait
Differentmodelsfordifferingitemresponseformats
Providesaframeworkfortestingadequacyofmeasurementmodels

PSYC948:Lecture#8

ClassicalTestTheory(CTT)

InCTT,theTEST istheunitofanalysis:

TruescoreT:

Observedvariance=truevariance+errorvariance

Goalistoquantifyreliability

Expectedvalue(mean)of0,expectedtobeuncorrelatedwithT
esaresupposedtowashoutoverrepeatedobservations

SotheexpectedvalueofTisYtotal
Intermsofobservedvarianceofthetestscores:

Bestestimateoflatenttrait:Meanoverinfinitereplications

Errore:

Reliability=truevariance/(truevariance+errorvariance)

True
Score
?

Ytotal
?

error

BecausetheCTTmodeldoesnotincludeindividualitems,
itemsmustbeassumedexchangeable (andmoreitemsisbetter)

PSYC948:Lecture#8

10

ClassicalTestTheory,continued

CTTunitofanalysisistheWHOLETEST(sumofitems)

Wanttoascertainhowmuchofobservedtestscorevariance
isduetotruescorevarianceversuserrorvariance
Quantifyerrorvarianceinvariousways
ErrorisaunitaryconstructinCTT(anderrorisbad)
Goalisthentoreduceerrorvarianceasmuchaspossible
Standardizationoftestingconditions(makeconfoundsconstants)
Aggregation=moreitemsarebetter(errorsshouldcancelout)

Itemsareexchangeable;propertiesarenottakenintoaccount

Followedbygeneralizabilitytheory todecomposeerror

e.g.,ratervariance,personvariance,timevariance

PSYC948:Lecture#8

11

AdvantagesofCFAoverCTT

Morereasonableassumptionsaboutitems

CTTassumestauequivalentitems

CFAallowsatestofwhethereachitemrelatestothefactor,aswellaswhether
differentfactorloadingsacrossitemsareneeded

Wouldindicatesomeitemsarebetterthanothers

Comparabilityacrosssamples,groups,andtime

CTT:Noseparationofobserveditemresponsesfromtruescore

Sumacrossitems=truescore;itempropertiesarefor thatsampleonly

CFA:Latenttraitisestimatedseparatelyfromitemresponses

Tau equivalentitems:equalfactorloadings

Separatespersontraitsfromspecificitemsgiven
Separatesitempropertiesfromspecificpersonsinsample

Advantagesapplytoanylatenttraitmodel

PSYC948:Lecture#8

12

ReliabilityMeasuredbyAlpha

Forquantitativeitems(itemswithascale althoughusedon
categoricalitems),thisisCronbachs Alpha

OrGuttmanCronbach alpha(Guttman 1945>Cronbach 1951)

Anotherreducedformofalphaforbinaryitems:KR20

Alphaisdescribedinmultipleways:

Isthemeanofallpossiblesplithalfcorrelations
Isexpectedcorrelationwithhypotheticalalternativeformofthe
samelength
Islowerboundestimateofreliabilityunderassumptionthatallitemsaretau
equivalent(moreaboutthatlater)
Asanindexofinternalconsistency

PSYC948:Lecture#8

Althoughnothingabouttheindexindicatesconsistency!

13

WhereAlphaComesFrom

Thesumoftheitemvariances isgivenby:

Var(I1)+Var(I2)+Var(I3).+Var(Ik)(justtheitemvariances)

Thevarianceofthesumoftheitems isgivenbythesumofALLthe
itemvariancesandcovariances:

Var(I1 +I2 +I3)=Var(I1)+Var(I2)+Var(I3)

+2Cov(I1,I2)+2Cov(I1,I3)+2Cov(I2,I3)
Wheredoesthe2comefrom?
Covariancematrixissymmetric
Sumthewholethingtogettothe
varianceofthesumoftheitems

PSYC948:Lecture#8

I1

I2

I3

I1

12

12

13

I2

21

22

23

I3

31

32

32

14

GuttmanCronbach AlphaforReliability

Covariance
Version:
k=#items

Numeratorreducestojustthecovarianceamongitems

Sumoftheitemvariances

VarianceoftotalY(thesumoftheitems)

Var(X)+Var(Y)=Var(X)+Var(Y) justtheitemvariances
Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y) PLUScovariances

So,iftheitemsarerelatedtoeachother,thevarianceofthetotalYitemsum
shouldbebiggerthanthesumoftheitemvariances

PSYC948:Lecture#8

Howmuchbiggerdependsonhowmuchcovarianceamongtheitems the
primaryindexofrelationship

15

AssessingReliabilityinOur12ItemGamblingScale

TogettheGuttmanChronbach Alphaofour12itemscale,weneed
thecovariancematrix

ThiscanbefoundbytheSAMPSTAToptionunderthe
OUTPUTstatement

Sumofitemvariances=11.834
Sumofitemcovariances=21.575
VarianceofTotalY=11.834+2*21.575=54.984

Alphareliability:

PSYC948:Lecture#8

.
.

.852

16

Reliabilityforwhat?

Thealphareliabilityisthereliabilityfor:

Thetotaltestscore
Undertheassumptionthattheitemsaretauequivalent
Tauequivalentmeanseachitemcontributesequally
Inafewslides,wewillseehowthistranslatestoCFA

Whatalphaisnot:

Anindexofmodelfit(unidimensionality)

PSYC948:Lecture#8

17

MeasurementLanguage:DontSayThese

Often,peoplerefertoitemsastappingsomelatenttrait

Ithinkthismakestheprocesslesstransparent itemsmeasurethetrait

Whenalphaisused,youcansometimeshearpeoplesaysomething
abouthowwelltheitemshangtogether

Thisiscertainlynottrue

PSYC948:Lecture#8

18

HowtoGetAlphaUP

PSYC948:Lecture#8

19

AlphaasReliabilityWhatcouldgowrong?

Alphadoesnotindexdimensionality itdoesnotindextheextenttowhichitems
measurethesameconstruct

Thevariabilityacrosstheinteritemcorrelationsmatters,too!
Weuseitembasedmodels(CFA)toexaminedimensionality

PSYC948:Lecture#8

20

CaseInPoint:All24Items

Lastclassweshowedthatthe24itemsoftheGRIdidnotfitaone
factormodel whatwouldhappenifweneglectedtocheckmodel
fitandusedthetotalscoreasourestimateofgamblingtendency?

Thereliabilityestimate fromthecovariancematrixofalltheitems
(thesaturatedmodelH1)was
.861

Wewouldhaveconcludedwehadagoodscaleforgambling

But,fromCFAlastweek,wefoundthatonefactordidntdescribeall
theitems

Anysubsequentanalysiswillhavethemisfitbiastheresults

PSYC948:Lecture#8

21

TestingCTTAssumptionsinCFA

Alphaisreliabilityassumingtwothings:

Wecantesttheassumptionoftauequivalencetoovianestedmodel
comparisonsinwhichtheloadingsareconstrainedtobeequal
doesmodelfitgetworse?

Allfactorloadings(discriminations)areequal,orthattheitemsare
truescoreequivalentortauequivalent
Localindependence(dimensionalitynowtestedwithinfactormodels)

Ifso,dontusealpha usemodelbasedreliability(omega)instead.Omegaassumes
unidimensionality,butnottauequivalence
Researchhasshownalphacanbeanoverestimateoranunderestimatedependingon
particulardatacharacteristics

TheassumptionofParallelitemsisthentestablebyconstraining
itemerrorvariancestobeequal,too doesmodelfitgetworse?

Parallelitemswillhardlyeverholdinrealdata
Notethatiftauequivalencedoesnthold,thenneitherdoesparallel

PSYC948:Lecture#8

22

AnotherBlastfromthePast:ParallelItems

AnotherCTTmodelthatexiststhatofparallelitems

Undertheparallelitemsmodel,thealphareliabilityforthetotaltestscoreiscalledthe
SpearmanBrownreliability

Allitemshavethesamecovarianceandvariance
Goesonestepfurtherthantauequivalence(equalcovariancesbutunequalvariances)

Usedtoprophesythenumberofitemsneededtoincreasereliabilitytoadesiredlevel

SpearmanBrownProphesyFormula
ReliabilityNEW

=ratio*relold /[(ratio1)*relold +1]

Ratio=ratioofnew#itemstoold#items
Forexample:

Oldreliability=.40
Ratio=5timesasmanyitems(had10,whatifwehad50)
Newreliability=.77

PSYC948:Lecture#8

23

Reliabilityvs.ValidityParadox

GiventheassumptionsofCTT,itcanbeshownthatthecorrelationbetweenatest
andanoutsidecriterioncannotexceedthereliabilityofthetest(seeLord&
Novick1968)

Reliabilityof.81?Noobservedcorrelationspossible>.9,
becausethatsallthetruevariancetheretoberelatable!
Inpractice,thismaybefalsebecauseitassumesthattheerrorsareuncorrelatedwith
thecriterion(andtheycouldbe)

Selectingitemswiththestrongestdiscriminations(orthestrongestinter
correlations)canhelptopurifyorhomogenizeatest,butpotentiallyatthe
expenseofconstructvalidity

Canendupwithabloatedspecific
Itemsthatareleastinterrelatedmaybemostusefulinkeepingtheconstructwell
definedandthusrelatabletootherthings

PSYC948:Lecture#8

24

UsingCTTReliabilityCoefficients:BacktotheScoreEstimates

Reliabilitycoefficientsareusefulfordescribingthebehaviorofthetestinthe
overallsampleVar(Y)=Var(T)+Var(e)
Butreliabilityisameanstoanendininterpretingascoreforagivenindividual
weuseittogettheerrorvariance

Var(T)=Var(Y)*reliability;soVar(e)=Var(Y) Var(T)
95%CIforindividualscore=Y 1.96*SD(e)
Givesanindicationofhowprecisethetruescoreestimateisonthemetricofthe
originalvariable
Example:Y=100,Var(e)=9 95%CI 94to106
Y=100,Var(e)=25 95%CI 90to110
Notethisassumesasymmetricdistribution,andthuswillgooutofboundsofthescale
forextremescores
NotethisassumestheSD(e)ortheSEforeachpersonisthesame
CuemindblowingGREexample

PSYC948:Lecture#8

25

95%ConfidenceIntervals:Quantitative

SEMrangesfrom9to55

PSYC948:Lecture#8

26

REVISITINGCTTFROMA
CFAPERSPECTIVE

PSYC948:Lecture#8

27

ClassicalTestTheoryfromaCFAPerspective

InCTTtheunitofanalysisisthetestscore:
,

InCFAtheunitofanalysisistheitem:

TomapCFAontoCTT,wemustputthesetogether:
,

PSYC948:Lecture#8

28

FurtherUnpackingoftheTotalScoreForumla

BecauseCFAisanitembasedmodel,wecanthensubstituteeach
itemsmodelintothesum:
,

MappingthisontotruescoreanderrorfromCTT:
and

PSYC948:Lecture#8

29

FamiliarTerms

Thetauequivalentmodelassumes:

0,

Theparallelitemsmodelsassumes:

Allitemsmeasurethefactorthesame:
Eachitemhasitsownuniquevariance:

Allitemsmeasurethefactorthesame:
Allitemshavethesameuniquevariance:

0,

Assuch,eachofthesemodelscanbetestedbyusingtheCFA
approach eacharenestedwithinthefullCFAmodel

PSYC948:Lecture#8

30

TauEquivalence:ModelImpliedCovarianceMatrix

TheCFAmodelimpliesaveryspecificformforthecovariancematrix
oftheobserveditems:

Thevarianceofanitem was:

Thecovarianceofapairofitems and was:

UnderTauEquivalence,allloadingsarethesame,meaning:

Theitemvariancescanbedifferent(becauseof
Allitemcovariancesarethesame(
)

Thisiscalledthecompoundsymmetryheterogeneousmodel

Wecanactuallyachievethesamemodelwithoutthefactor

PSYC948:Lecture#8

31

TauEquivalenceModelfor12ItemGamblingScale

ThefollowingtwopiecesofMplussyntaxresultinthesameequivalentmodel:
TauEquivalenceasaFactorModel:

TauEquivalenceasaCompundSymmetryHeterogeneousVariancesModel:

PSYC948:Lecture#8

32

ModelImpliedCovarianceMatrix

Allcovariancesequal/allvariancesdifferent

PSYC948:Lecture#8

33

TestingforTauEquivalence

TheTauEquivalencemodel(assumedwhenyousumitems)canbetestedagainst
thefullCFAmodel

LoglikelihoodfromCFAmodel:18,988.425;SCF=2.4309

Themodelsarenested,sowecanusealikelihoodratiotest

36parameters(12itemintercepts,11factorloadings,1factorvariance,12uniquevariances)

LoglikelihoodfromTEmodel:19,051.350;SCF=2.5172

25parameters(12itemintercepts,1factorloading,12uniquevariances)

56.315,

.001

MLRLikelihoodratiotest:

Therefore,werejectthetauequivalentmodelinfavoroftheCFAmodel this
meansthesimplesumoftheitemsisnotsufficient

WeshouldusetheCFAmodelfactorscoreinsteadofasumscore

PSYC948:Lecture#8

34

ParallelItems:ModelImpliedCovarianceMatrix

TheCFAmodelimpliesaveryspecificformforthecovariancematrixofthe
observeditems:

Thevarianceofanitem was:

Thecovarianceofapairofitems and was:

UnderParallelItems,allloadingsanduniquevariancesarethesame:

)
)

Thisiscalledthecompoundsymmetrymodel

Allitemvariancesarethesame(
Allitemcovariancesarethesame(

Wecanactuallyachievethesamemodelwithoutthefactor

Becauseparallelitemsarenestedwithintauequivalentitems,wedonothaveto
testthismodelasweknowitwillnotfitwhencomparedtotheCFAmodel

PSYC948:Lecture#8

35

TestScoreReliabilityUndertheCFAModel

Coefficientalphagavereliabilityforthetotaltestscoreunderthe
TauEquivalentItemsModel

WerejectedthatmodelinfavoroftheCFAmodel
Therefore,coefficientalphawillnotbecorrectforourtotaltestscore(ifwewereto
stillsumuptheitems)

ThenotionsoftestscorereliabilityundertheCFAmodelnowinvolve
thefactorloadings

Butstillcomebacktoclassicalnotionofreliabilitybeingtheproportionofvariancedue
totruescore:

PSYC948:Lecture#8

36

DerivingReliabilityForSumScoresUndertheCFAModel

ToshowwheretotalscorereliabilityundertheCFAmodelcomes
from,recallourCFAmodelforthetotalscore:
,

MappingthisontotruescoreanderrorfromCTT:
and

WenowmustderivethevarianceforTandE

PSYC948:Lecture#8

37

TrueScoreVarianceUndertheCFAModel

Thevarianceforthetruescore:

PSYC948:Lecture#8

38

ErrorVarianceUndertheCFAModel

BecausetheCFAmodelallowsfortheestimationoferror
covariances(althoughyoushouldntdothat),theerror
varianceundertheCFAmodelbecomes:

Whenerrorcovariancesarenotestimated,thelasttermis
zero,leaving

PSYC948:Lecture#8

39

ReliabilityforTotalScoreUnderCFA

ThereliabilityofthetotalscorefromCFA,isthen:

ThisreliabilitycoefficientiscalledcoefficientOmega( )

Ifthetauequivalentmodeldoesnothold
totaltestscore(sumscore)

isthereliabilityofa

TypicallyishigherthanAlpha
Ifunidimensionalmodelholds,coefficientswillbeclose

PSYC948:Lecture#8

40

CalculatingOmegaforOurTest

WecanuseMplustocalculateOmegaforourtest:

Here,Omegais.855
PSYC948:Lecture#8

41

OmegaUnderTauEquivalentItems

OmegaequaltoAlphawhenyouusethetauequivalentitemsmodel

OmegaistheSpearmanBrownreliabilityunderparallelitems

Here,Omegais.852
whichisequaltothe
Alphawecalculatedusing
thecovariancematrix
PSYC948:Lecture#8

42

Recapping:CTTusingCFA

Classicaltesttheory andmorespecifically,totaltestscores,isthe
dominantwaytoassesssubjects

ThisistrueevenunderCFA

Thekeyistobesuretocheckifaonefactormodelfitsthedata
beforeusinganytypeofreliabilitycoefficient

Ifnot,donotuseatestscore

Iftheonefactormodelfits thenasinglescorecanrepresent
thetest

Thenextworryisaboutrepresentingtheerrorinthetestscore
(relatedtoreliability)

Ifreliabilityishigh(?Howhigh,standardof.8),thenusingthetestscoreina
subsequentanalysisisacceptedpractice

PSYC948:Lecture#8

43

SecondaryAnalyseswithFactorScores

Ifyouwanttouseresultsfromasurveyinanewanalysis

Best: UseSEM errorinfactorscoresisalreadypartitionedvariance

Similarlygood: Useplausiblevalues(repeateddrawsfromposterior
distributionofeachpersonsfactorscore) essentiallywhatSEMdoes but
withfactorscoresthatvarywithinaperson

CanbedoneinMplus notdescribed

SlightlyLessGood: UseSEMwithsingleindicatorfactorsusingsumscores
Thefocusofthenextsection
Makeerrorvariance=(1reliability)*Variance(Sumscore);factorloading=1

Okay(butwidespread):forscalesthatareunidimensional(andverifiedin
CFA),usesumscores

Assumesunidimensionalityandhighreliability

NotCool: Usefactorscoresonly

PSYC948:Lecture#8

44

WhataboutUsingFactorScores?

AlthoughCFAfactorscoreshavefewerproblemsthanEFAfactor
scores(becausethereisnorotationinCFA),theystillhaveissues:

Theywillbeshrunken (i.e.,pushedtowardsthemean,suchthattheobservedvariance
ofthefactorscoreswillbelessthantheoriginalfactorvariance)

Cangetestimatesoffactordeterminacy howcorrelatedestimatedfactorscores
arewithtruefactorscores(basicallyhowmucherrorisintroducedbyestimatingthe
factorscoresasobservedvariables)

Theyarejustestimatesofcentraltendency fromadistributionforeachperson,not
knownvalues andusingestimatesasknownvaluesinanothermodelmakesthe
relationshipswithinthatmodellookmoreprecisethantheyare(likeSE=0)

YouCANNOTcreatefactorscoresbyusingtheloadingsassuch:
F=11y1 +21y2 +21y3 ThisisaCOMPONENTmodel,notaFACTORmodel.

PSYC948:Lecture#8

45

SINGLEINDICATORMODELS

PSYC948:Lecture#8

46

SingleIndicatorModels

SingleindicatormodelsareCFAlikemodelswhereafactorismeasuredbya
singleindicator:

Shownhereforthegamblingfactor

Gambling
Tendencies

1
Sumof10
GRIItems
,

1
PSYC948:Lecture#8

47

IdentificationinSingleIndicatorModels

Howisthispossible?Isntasingleindicatorfactormodelunidentified?

Singleindicatormodelparameters:

Wefixthefactorvariance,factorloading,anduniquevariance
Factorvariancerepresentsreliableportion

factorvariance; factorloading;
zeroanditeminterceptissettoitmean)

itemuniquevariance(assumefactormeanfixedto

Ourconstraintsare:

PSYC948:Lecture#8

(theportionofYthatisreliable)

(theportionofYthatisleftover)

48

AssumptionsinaSingleIndicatorAnalysis

Touseasingleindicatoryoumustassume:

Theindicatorisunidimensional(onlyonefactor)
ThisistestableinCFA(butifyouhaveasmallsampleishardtodo)
Ifnotpossibletotest,youmustassumeyouhaveonefactor

Thisisanassumptionthatthetestis*as*dimensionalinyoursample/population

Thereliabilityoftheindicatorisknown
AlsoobtainablefromCFA
Ifnotpossibletoobtain,thenyoumustuseapreviouslyreportedreliability
coefficient

Thisisanassumptionthatthetestis*as*reliableinyoursample/population

PSYC948:Lecture#8

49

SingleIndicatorExampleAnalysis

Todemonstrateasingleindicatorexampleanalysis,wewillusethe
12itemGRItopredicttheSOGSscore

SOGS=SouthOakGamblingScreen(wecollectedthis)

The1factorCFAmodelfitthe12itemGRI

Step#2:getthesingleindicatorreliability

The12itemGRIisthesingleindicatorofthegamblingfactor

Step#1:determinethatthesingleindicatorisunidimensional

Note:weassumethishasreliabilityof1.0

FromtheCFAanalysiswefoundthatthereliabilityofthe12itemGRIwas.855

Step#3:estimatethevarianceofthe12itemGRItotalscore

WecandothisinMplus foundthevariancetobe55.050

PSYC948:Lecture#8

50

SingleIndicatorAnalysis

NowthatwehaveourreliabilityoftheGRIandthevarianceoftheGRI,wecanputthese
intothesingleindicatormodel:

PSYC948:Lecture#8

51

SingleIndicatorModelResults:

Note:47.067+7.982=55.05(thevarianceoftheGRI)

.167(.011)
2.440(.213)

SOGSScore

Gambling
Tendencies

47.067
(3.459)

1
Sumof10
GRIItems
,

7.982

PSYC948:Lecture#8

52

SingleIndicatorModelInterpretation

Thestandardizedregressionslopeforthegamblingfactor,predictingtheSOGSwas.591
asgamblingwentup1SD,theSOGSscorewentup.591SD

Correlationbetween
gamblingandSOGS

Thegamblingfactor
accountedfor35.0%ofthe
varianceintheSOGSscore

PSYC948:Lecture#8

53

ComparisonwithNonSingleIndicator

Withoutusingthesingleindicator:

PSYC948:Lecture#8

54

ITEMPARCELING

PSYC948:Lecture#8

55

ItemParceling

Frequently,sumscoresareusedinSEMunderadifferentlabel:as
itemparcels

Evidentlyparcelsoundsmorepolitestuffthatdidntfit

ItemparcelsaresumsofsetsofitemsthatareinsertedintoaSEM
withoutanyfurtherinspection

Frequently,itemparcelswillhidebadfitofmodel

Blindparceling=cheating

Asparcelsaresums andtodaysclassisaboutusingsums,wecan
nowdiscussparcelsunderCTTwithCFA

PSYC948:Lecture#8

56

ApplyingOurUnderstandingofTotal/SumScorestoParcels

Aswehaveseentoday,atotalscoreisastatisticalmodel

Tauequivalentitems

Aswithanystatisticalmodel,ifthemodeldoesnotfit(adequately
representthedata),misleadingresultsoccur

Parcelingitemsmakesanimplicitassumptionabouttheirstructure
thattheytooaretauequivalent

Ifthatassumptionisnotvalid,resultscannotbebelieved

Mostusesofparcelingmakenoattempttodetermineifthetau
equivalenceassumptioniscorrect

PSYC948:Lecture#8

57

Revisitingour24ItemGRI

Asyouwillrecall,our24itemGRIdidnotfittheonefactormodel
Todemonstrateparceling,wewilltakethe12misfittingitemsand
createaparcel(sumtheirscorestogether)

WewillthenaddtheparceltoaCFAmodelwiththeother12items
Themodel indicatedthemodeldidnotfit
betterthanthesaturatedmodel butthis
statisticcanbeoverlysensitive
ThemodelRMSEAindicatedthemodeldidnot
fitwell(wantthistobe<.05)

ThemodelCFIandTLIindicatedthemodeldid
notfitwell(wantthesetobe>.95)
TheSRMRindicatedthemodeldidnotfitwell
(wantthistobe<.08)
PSYC948:Lecture#8

58

The12ItemGRIPlustheParcelofBadItems

Thesyntax:

Themodelfitstatistics(adequatemodelfit):

Note:12itemGRIhadRMSEAof.045

PSYC948:Lecture#8

59

HowParcelingHidesPoorModelFit

Theitemparcelhidespoormodelfitbyusinganumbersgame
toitsadvantage

Modelwith24itemshad300elementsinsaturatedcovariance
matrix(but48parametersforthatmatrix)

Modelwith12itemsplusparcel(12items)had91elementsin
saturatedcovariancematrix(and26parameters)

Therelativeratioofparameterstosaturatedcovariancesmakesthe
parcelhidethefitissues

Especiallywhentheremainderoftheitemsfitwellalready

PSYC948:Lecture#8

60

ParcelingDoneRight

Toaddaparcelyoumustfirstexaminethefitofaonefactormodeltotheitems
oftheparcel:

Themodelfitsuggestsaonefactormodeldoesntfit

PSYC948:Lecture#8

61

WhyParcelingisCheatingandWhyYouShouldntDoIt

Ifyoudidntchecktheparcelbeforeaddingittothe12itemGRI
modelyouwouldconclude1factormodelfitthedatawell

Thesumscorefromthe12good+1bad(parcel)modelisjustthe
sumscorefromthe24itemGRI whichdidntfitaonefactormodel

Ifaonefactormodelfits,thenwhatcomesnextistypicallytheuseofitssumscore

Thecaveat:the12+1modelsumscorehadanomegareliabilityof.516!
Mostofthelackofreliabilitycomesfromtheestimateduniquevarianceoftheparcel

Youcannotmakeagoodfactorbycheatingwithaparcel!

PSYC948:Lecture#8

62

CONCLUDINGREMARKS

PSYC948:Lecture#8

63

WrappingUp

Todaywasspentoncomparingclassicaltesttheory(synonymous
withsumscores)toCFA

UnderstandinghowCTTandCFAarerelatedisimportant

ManypeoplebelievethatsumscoresareAOK

Manypeopledontthinkparcelinginvolvessumscores

Theyonlyareiftheyfita1factormodeland haveahighreliability
Thelabelmustbetheproblem

Singleindicatormodelscanbeagoodwaytousesumscoresif:

The1factormodelfits
Thereisahighdegreeofreliability
WewillreturntothisoncewediscussSEMmorethoroughly

PSYC948:Lecture#8

64

ComingUp

Nextweekslecture:multidimensionalCFAmodels
Morethanonefactor
Reliabilityforatotaltestscorenolongerapplies(eachfactoris

wherereliabilityisimportant)
Timepermitting:anintroductiontoExploratoryFactorAnalysis
FollowedbyacomparisonofCFAandEFA
AndwhyyoualsoshouldntbedoingEFA
ButcouldexplorethedatabetterusingCFAtechniques

PSYC948:Lecture#8

65