A
LookatCorrelationsBetweenCorpsmemberExperienceandSuccess
ProposalsubmittedtotheLosAngelesConservationCorps
August14th,2015
SubmittedbyOscarJasklowski
AcademicTutor,Analyst
CivicorpsSchools,Oakland,CA
oscarjasklowski@gmail.com
|206.512.5031
ProjectMotivation
WorkingasvolunteeracademictutoratCivicorpsallowedmetogettoknowafewstudents
personally.Throughourconversations,itbecameclearthatsuccessinjobtrainingtranslatedto
successandmotivationinschool,andviceversa.IcametorealizethattheCorpsstrengthlies
intheprovisionofallofitsservices:education,jobtraining,andCorpsmembersupport.Inother
words,IbelievetheCorpsisbettersuitedtoachieveitsmissionthanitscomponentpartswould
beindividually.
Thismotivatedmetoexploretherelationshipsbetweeneducation,jobtraining,andsupport
servicesattheCorpsmoreclosely.Below,Iproposeananalysisinordertounderstandwhich
experiencesandservicesattheCorpsreinforcestudentsdesiretosucceed,andtowhat
degree.
Objective
Theobjectiveofthisprojectistoidentifyfactorsthatcontributetosuccessfuloutcomesfor
Corpsmembers,whichincludegraduation,employment,andcollegeenrollment.Similarly,
factorsthathinderCorpsmembersuccesswillalsobeidentified.Knowledgeofthesefactorscan
beappliedtoimprovingCorpsprogramsandincreasingtherateofsuccessfuloutcomes.
Thisanalysiswillutilizehistoricalinformationfromtheeducation,jobtraining,andsupport
servicesprograms.
Approach
BygroupingCorpsmembersthatsharecommonfactors(i.e.cohorts)andcomparingtheir
outcomestoacontrol,wecaninferwhichindividualfactorscontributetosuccessintheCorps.
Forexample,wemightlookatwhetherstudentswhoparticipateinextracurricularactivitiestend
tocompletetheirhighschooleducationatahigherratethanstudentswhodont(see
Examplessectionbelow).Thisanalysiswouldtelluswhetheracorrelationbetween
extracurricularsandhighschoolcompletionexists.Byitself,however,thisanalysiswouldnottell
us
why
itexistsor
how
totakeadvantageofthatinsight.
Significance
Lookingathistoricalprogramdataisalowcostwayofidentifyingleadingindicatorsforstudent
success.ThisanalysiswillsupplementLACCstaffsintuitionaboutwhycertainstudentsare
successfuland,ideally,willuncoverunexpectedindicatorsforsuccess.
Learningsfromthisanalysiscanbeappliedinthreeways:
1. ProgramModification
ImprovetheexperienceforCorpsmembersbyfurther
incorporatingfactorsthatcorrelatetosuccessandbymitigatingnegativefactors.
2. GrantWritingandReporting
Havingindicatorsforsuccessatvariousstagesofthe
programisanassetforwritingandreportingongrants,particularlywhenthosegrants
targetaspecificcomponentoftheCorps.
3. InformDataCollectionPractices
Thisanalysiswillprovideanotherperspectiveonwhat
additionalinformationcanandshouldbecollectedinthefuture.Thiscanbethoughtof
asbrainstormingbydoing.
Examples
(foramorecomprehensivelist,see#4intheStepstoCompletionsectionbelow)
DoesextracurricularorcivicinvolvementinfluencecompletionoftheSaitoSchoolprogram?
Figure1
ThesampleplotshowsCorpsmemberdropoutoverthecourseoftheprogramandat1yearafter
programcompletion.Twogroupsofstudentsarerepresentedhere:thosewhoparticipateinCorpsrelated
extracurricularorcivicactivities(yellow)andthosewhodonot(blue).
Wemightexpectstudentswhoparticipateinextracurricularstobemoreengagedduringthe
SaitoSchoolprogramandthusmorelikelytocompletethatpartoftheprogram.Bycomparing
thesestudentsagainstacontrolgroupofnonparticipants,wecansupportoropposethat
assumptionquantitativelyatdifferentlevelsofcertainty.
Doescommutetimetoworksitesinfluencecompletionofthejobtrainingprogram?
Figure2
SimilartoFigure1,twogroupsofstudentsarerepresentedhere:thosewhohaveanaverage
commuteofover30minutestojobtrainingsites,andthosewhohaveashortercommute.Inthisexample,
thereisa22%differencebetweenthetwogroupswithrespecttoCorpsmemberdropout.Whetherornot
thisresultissignificantwoulddependonthesizeofthesampleofCorpsmembers.
Anecdotally,itmaybewellknownthatCorpsmemberswithlongercommutesstrugglewithwork
attendance.Thisanalysiswouldsupportthatintuition,butimportantly,itwouldquantitatively
illustratethestrengthofthatfactor.
FactorsandProjectScope
Inthiscontext,factorsarestudentcharacteristicsorexperiencesthatmayimpacttheirsuccess
intheCorps.Theyfallintotwomaincategories:
1. Programmaticfactors
TheseareexperiencesattheCorps(e.g.thetypeofjobtraining
aCorpsmemberattends),andtheycanbefurthersubdividedintotwocategories:
a. Educational
TheseareexperiencesstudentshavethroughtheSaitoSchooland
throughotherprofessionaleducationwhileattheCorps.
b. Careerrelated
ThesedescribetheCorpsmemberscareerinterestsand
aptitudes(supportservicesinformation),aswellasworkexperiences(jobtraining
information).
2. Demographic
ThesearecharacteristicsthatdescribetheCorpsmembersthemselves
(e.g.ageuponentry),andmayinfluencesuccessatanypointduringtheprogram.
Figure3
Corpsmemberdropoffbetweenmilestonesisshownabove.Thetypesoffactorsthatpresumably
shapethecompletiongrapharealsoshown.
Becausethisprojectisexploratory,itwillseektoanalyzeasmanyfactorsaspossibleto
maximizethechanceidentifyingfactorsthatcanleadtoprogrammaticimprovement.Themain
constraintonthenumberoffactorstobeanalyzedwillbetheavailabilityofinformation.Seethe
StepstoCompletionsection,Step2,belowformoreonthisissue.
StepstoCompletion
BecausethisanalysiswillrelyexclusivelyonexistingCorpsinformation,itcanbegin
immediately.Thestepsthatneedtobetakentocompletetheanalysisfollow:
1. Compileanexhaustivelistoffactorsandoutcomes
WorkwithLACCprogrammanagersineducation,jobtraining,andsupportservicesto
comeupwith:
a. AllfactorsthatmaycontributetoCorpsmembersuccess.
b. Outcomesatallstagesoftheprogram.
Theseoutcomesarethedependentvariablesagainstwhichfactorswillbecompared.
2. Narrowdownandprioritizefactorlist
Removeanyfactorsoroutcomesfromthelistforwhichinformationisnotavailable(or
cannotbeeasilyinferredfromexistinginformation).Fortheremainingfactorsand
outcomes,prioritizethembasedonwhethertheconclusionisactionable.Inotherwords,
factorsthat,ifsignificant,presentaclearopportunityforamodificationinprogramming
shouldbeprioritized.
3. Codestudentsintocohorts/groups
Thisisrequiredsothatstudentscanbequicklyfilteredandaggregatedbyfactors.
Figure4
Studentsinformationisanonymizedandtransformedintoaformatthatcaneasilybe
filteredandaggregatedbasedonfactors.Thehighlightedrowrepresentsamalestudent,aged
1820,whoenteredLACCin2012.
4. RunbasicCorpsmembersuccessanalyses
Compareeachfactoracrosseachsuccessoutcome(e.g.DoesFactorAaffect
OutcomeX?).Asshowninthefigurebelow,factorsandoutcomescanbeorganizedin
amatrix,witheachcellrepresentingafactoroutcomerelationship.
Figure5
Inthematrixabove,eachcell(intersectionbetweenfactorandoutcome)representsa
possiblerelationship.Forexample,theresultfromthehighlightedcellwouldtellusiftheresa
relationship,eitherpositiveornegative,betweenapriorworkexperienceandSaitoSchool
completion.PerhapsstudentswithpriorworkexperiencecompletethefirstSchoolcomponentata
lowerratebecausetheyareaccustomedtoreceivingahigherwagethanthedailyschoolstipend,or
becausetheyfailtoseethebenefitofacademicsinsecuringafuturelivingwagejob.Establishinga
relationshipherewouldpromptadeeperinvestigationintothecauseandpotentialsolutions.
Somefactorswillbebinary,meaningthereareonlytwopossibilities(e.g.astudentcan
eitherhavepreviousworkexperienceornot).Somefactorsproducemorethantwo
cohorts(e.g.Ageuponentrymighthave3cohortsifstudentsaregroupsbyage1820,
2123,and2426).
5. Factorfactorrelationshipsandtimeinprogramanalysis
Inadditiontostudyingrelationshipsbetweenfactorsandoutcomes,wecanlookatthe
relationshipsbetweenthefactorsthemselves.Thiswillbroadenthescopeoftheanalysis
significantly.
Figure6
Inthefigureabove,threefactorshavebeentreatedasoutcomes.Iftherearesignificant
differencesinageorgenderbetweenintakeperiods,thenextstepwouldbetolookatdifferencesin
recruitment/intakeprocessduringthosetimes.Inthisway,wecanquantifyeffectsofcertain
programmaticchangesthathavealreadyhappenedinordertobetterevaluatethem.
Also,becausetheamountoftimeastudentspendsintheCorpsisvariable,
timeinprogrammayprovetobeadimensionthatinfluencesgraduation.FortheSaito
School,jobtraining,andthefullprogram,wecancomparethetimetakenbygraduates
andnongraduates.
Figure7aThehypotheticaldistributionshowstherelativenumberofstudentsthatexittheprogram
atagiventimepoint.Inthisexample,moststudentsexitorgraduateataround12months.
Figure7bWhenthishypotheticaldistributionisdeconstructedforgraduatesandnongraduates,it
becomesapparentthatmostnongraduatesdropoutearlyintheprogram(thisisintuitive).
However,alargenumberofstudentsmaybefoundtodropoutafter21months,wherethereisa
similarlikelihoodthatastudentwillgraduateordropout.Thiscouldbeexplainedbythefactthat,
lateintheprogram,mostofaCorpsmembersfellowentrantswillhavemovedonandmotivationto
graduatemaybelow.Ifthiswerethecase,stepscouldbetakenatorbefore21monthstomake
sureCorpsmembersareontrack.
6. Presentationandfuturedirections
Alongwithapresentationoffindings,allanalyticalworkwillbefullydocumentedand
scriptedsominimaleffortisrequiredtherefreshtheanalysis.Aprotocolforadding
factorsandoutcomeswillalsobedocumentedsoindividualanalysescanbeadded
easilybyLACCstaff.
Conclusion
WhilethiskindofanalysismightidentifyfactorsthatcontributeorhinderCorpsmembersuccess,
itonlyrepresentsastartingpointforexploringrelationshipsbetweenfactorandoutcome.
Ideally,thistypeofanalysiswillbeusedalongsideaqualitativeinvestigationastowhythese
relationshipsexistandhowtobesttakeadvantageofthem.