Effect of Failing A High School Exit Exam On Course Achievement, Persistence and Graduation

Educational Evaluation and Policy Analysis
http://eepa.aera.net Effects of Failing a High School Exit Exam on Course Taking, Achievement, Persistence, and Graduation
Sean F. Reardon, Nicole Arshan, Allison Atteberry and Michal Kurlaender EDUCATIONAL EVALUATION AND POLICY ANALYSIS 2010 32: 498 DOI: 10.3102/0162373710382655 The online version of this article can be found at: http://epa.sagepub.com/content/32/4/498
Published on behalf of
American Educational Research Association
and
http://www.sagepublications.com
Additional services and information for Educational Evaluation and Policy Analysis can be found at: Email Alerts: http://eepa.aera.net/alerts Subscriptions: http://eepa.aera.net/subscriptions Reprints: http://www.aera.net/reprints Permissions: http://www.aera.net/permissions
Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011
Educational Evaluation and Policy Analysis Educational Evaluation and Policy Analysis December 2010, Vol. XX, No. 4, pp. 498520 Fall XXXX, Vol. 32, X, pp. 215229 DOI: 10.3102/0162373710382655 2010 AERA. http://eepa.aera.net
Effects of Failing a High School Exit Exam on Course Taking, Achievement, Persistence, and Graduation
Sean F. Reardon Nicole Arshan Allison Atteberry Stanford University Michal Kurlaender University of California, Davis
The increasing use of state-mandated public high school exit exams is one manifestation of the current movement in U.S. public schooling toward more explicit standards of instruction and accountability. Exit exam requirements implicitly argue that raising the bar for graduation creates incentives both for students to work harder in school and for schools to increase their efforts for low-achieving students. Such incentives should most strongly affect the motivation of students who fail an exit exam the first time they take the test because failing provides a clear signal of students need to improve their academic skills. Others argue that failing an exit exam discourages low-achieving students from staying in school. In this article, the authors use a regression discontinuity design and studentlevel longitudinal data from four large California public school districts to estimate the effect of failing a high school exit exam in 10th grade on subsequent student achievement, course taking, persistence in high school, and graduation. The analyses show no evidence of any significant or sizeable effect of failing the exam on high school course-taking, achievement, persistence, or graduation for students with test scores near the exit exam passing score. In each case, the estimates are precise enough to rule out modest to large effects. This implies that the negative impacts of high school exit exam policies on graduation rates found in other studies are more likely a result of reduced graduation rates of very low-achieving students than of discouragement of marginally lowachieving students. Keywords: accountability, exit exams, regression discontinuity
The increasing use of state-mandated public high school exit examstests each student must pass before becoming eligible to receive a high school
diplomais one manifestation of the current movement in U.S. public schooling toward more explicit standards of instruction and accountability.
The research reported here was supported by a grant from the James Irvine Foundation and by the Hewlett Foundation through a grant to the Institute for Research on Educational Policy and Practice at Stanford University. We are indebted to the staff of the four school districts (in particular, James Gulek, Dave Calhoun, Robert Maass, and Peter Bell) for sharing their data and expertise with us; without their generosity and commitment to quality research this work would not have been possible. We also benefited from the excellent research assistance of Noli Brazil and Demetra Kalogrides. All errors, of course, are our own.
High School Exit Exam
The number of states requiring students to pass an exam to graduate has increased from 18 in 2002 to 24 in school year 20082009, with an additional two states intending to implement exit exams by 2012. Soon, more than 70% of U.S. students will be subject to such exam requirements (see, e.g., Center on Education Policy, 2004, 2005, 2009; Dee & Jacob, 2006; Warren, Jenkins, & Kulick, 2006). The effects of exit exam policies remain somewhat unclear, despite a number of recent studies. Competing notions of how such exams might influence student and school behaviors lead to divergent predictions of how these policies will affect students. Some argue that a high school exit exam requirement creates incentives both for schools to provide better instruction to struggling students and for these students to work harder to learn more before graduation. Others argue that creating additional barriers to graduation discourages students particularly academically and socially disadvantaged studentsfrom persisting in school and hence leads to increased dropout rates and greater inequality (for discussion, see Dee & Jacob, 2006; Reardon & Galindo, 2002; Warren et al., 2006). In this article, we use longitudinal data from four large California school districts to estimate the effect of failing a high school exit exam in 10th grade on subsequent student achievement, persistence in high school, and graduation. We begin in Section I with a discussion of the mechanisms through which exit exams might influence student outcomes. In Section II, we review relevant research on these topics with an emphasis on distinguishing between two strands of research: research on the effects of exit exam requirements per se and research on the effects of failing (relative to passing) an exit exam, given the existence of the requirement. This article falls under the second type of research, which may be particularly useful for providing insight about the mechanisms through which exit exams operate. In Section III, we describe the California High School Exit Exam (CAHSEE) policy and its history to provide a context for the analyses that follow. Section IV briefly describes the data, sample, and measures used in our analyses. In Section V we describe our regression discontinuity design, followed by a discussion of the results of these analyses in Section VI.
I. Potential Effects of Exit Exams By adding the requirement that students must pass a test to graduate from high school, exit exams may affect the motivation, effort, and behavior of both students and schools. According to the implicit logic of exit exam policies, schools and teachers may change their curricula in response to the requirement, focusing instruction on subject areas and material covered by the exam. Likewise, low-achieving students may alter their study habits, attendance, and attention to ensure they pass the exam. This presumptionthat there is a link between exit exam policies and student or school motivationis common in both policy discourse and academic literature regarding exit exams (see, e.g., Bishop & Mane, 2001, p. 205; Carnoy & Loeb, 2002, p. 306; Catterall, 1986, p. 4; Dee & Jacob, 2006, pp. 56; Jacob, 2001, p. 100; Madaus & Clarke, 2001, p. 9; Martorell, 2005, pp. 78; Ou, 2009, p. 3; Roderick & Engel, 2001, pp. 198199; Warren et al., 2006, pp. 132133).1 Nonetheless, the nature of the relationship between the two is far from straightforward. Exit exams may affect students in two ways. First, the policy itself may have an effect if students and schools alter their behavior prior to the grade when students first take the exam. Second, after students take the exit exam for the first time (typically in 10th grade), their scores serve as a signal of their ability to ultimately pass the exam. In particular, students who fail the exam may feel motivated to work harder to pass or may drop out because of discouragement arising from their failure. This article focuses on this latter issue how failing an exit exam affects student motivation, behavior, and outcomes. Students initial performance on an exit exam may provide key information to students and their teachers that clarifies areas of weakness, stimulates student effort, and triggers the creation of a plan for acquiring missing skills before the end of high school. Bishop (2006) posits that curriculum-based, external exit exams may motivate students because they produce real, visible signals about their likelihood of graduation and hence ultimate potential to reach college or succeed in the labor market. Higher extrinsic rewards for learning, he argues, are associated 499
Reardon et al.
[with] the taking of more rigorous courses, teachers setting higher standards and more time devoted to homework (p. 910). Schools may also respond in ways that positively affect student outcomes by targeting needed instructional services to students who fail. For instance, many schools delineate intervention plans for all students who fail on their first attempt. When these remediation efforts are effective, students receive the targeted assistance they need to progress, and their chances of subsequent success may increase (for more on this argument, see Dee & Jacob, 2006). There is relatively little empirical evidence to confirm these hypothesized motivational or curricular responses exit exam failure. One study in Chicago found that high-stakes tests appear to induce greater effort and motivation among low-achieving middle school students (Roderick & Engel, 2001). Roderick and Engel (2001) reported that the majority of the 120 lowachieving middle school students they surveyed described increased work effort under a grade retention policyincluding greater attention to class work, increased academic press and support from teachers, and more time spent studying outside school. Teachers reports corroborated these claims. The increased effort reported in this study, however, appears to be a result of the test-based retention policy itself, not a result of failing the test. Moreover, the motivational effects of the policy appear to be heterogeneous. One third of the respondents showed little work effort despite an expressed desire not to be retained. These students tended to be lower-achieving students than those with high work effort, suggesting that perhaps the links among testing policies, test failure, motivation, effort, and subsequent achievement operate differently depending on students skill level relative to the passing threshold on the test. Despite claims that exit exam failure serves as a useful signal to schools and students that triggers increased effort and motivation, many scholars have argued the opposite. According to this alternative view, exit exam failure does not lead to genuine student achievement gains and may, in fact, discourage students from persisting in school. Some argue that reliance on a single standardized test may have unintended consequences (Huebert & Hauser, 1998). For instance, 500
it is possible that students who experience failure are not motivated to try hardersome may instead feel discouraged about their prospects of eventual success (Madaus & Clarke, 2001). This concern about the unintended discouragement of low-achieving students has been a common theme in educational and sociological literature for many years (for early examples, see Blau, 1980; Glass, 1978). Psychological research on general student motivation suggests that students responses to an exit exam depend largely on students perceptions of the reward. Goal theory suggests that passing an exit exam may represent a performance goala goal based on attaining some extrinsic standard, as opposed to a goal based on attaining mastery of some particular domain (Ames, 1984; Covington, 2000). Research on student motivation shows that performance goals generally do not lead students to improve their substantive mastery of the material but rather lead students to focus on attaining a performance standard that may be irrelevant to their mastery. That is, students perceiving the exit exam as a performance goal will focus on passing the test rather than mastering the substantive material tested. To the extent that an exit exam validly assesses students knowledge of the material these two goals will coincide, and any motivational impact of test failure will lead to improved mastery of the material. Such motivational impacts, however, rely on the students perception that the goal is obtainable. For low-achieving students, particularly those who score far below the passing score on an exit exam, however, the effects of failing may not lead to increased motivation. Students motivated by external standards link performance to innate ability instead of hard work and effort and are therefore more likely to interpret low performance as implying low self-worth. As a result, students who fail or who are at risk of failing an exam may implement defensive psychological strategies to protect themselves from such negative self-assessments (Ames, 1984; Covington, 2000), much as Roderick and Engel (2001) observed among the low-achieving students in their Chicago middle school retention study. At best, according to this argument, performance goals lead to superficial, ineffective study techniques such as memorization and rehearsal, which may
help on the exam but are unlikely to boost subsequent achievement. At worst, students looking to avoid failure (which they link to their own selfworth) will avoid challenging tasks (Ames, 1984). Only a few studies have attempted to capture the psychological and intermediate behavioral responses specific to exit exams. Richman, Brown, and Clark (1987) found that students who failed minimum competency tests exhibited a significant increase in apprehension alongside a corresponding decrease in general self-esteem. Because neither the students with little risk of failing nor the students who had a high risk of failing but passed had any such changes along these dimensions, the authors attribute the psychological changes they observed to the experience of failing. These findings corroborate the hypothesis that students who fail will feel discouraged rather than encouragedand will not necessarily believe that increased effort will lead to later success. Teachers and schools may also respond to a students failure on an exit exam in ways that undermine subsequent student outcomes.2 Indeed, some evidence suggests that high-stakes exams trigger unintended teacher and school behaviors. For example, Booher-Jennings (2005) studied one Texas public school where teachers confronted a new test-based grade promotion policy. Booher-Jennings describes an educational triage system whereby teachers used the test scores of initial failers to classify them into three groupssafe cases, hopeless cases, and cases suitable for treatmentand diverted resources away from the first two groups toward the latter. Teachers understood that students just below the proficiency cut score required modest educational interventionperhaps only improved test-taking skillsto pass the exam. Neal and Schanzenbach (2007) find evidence of a similar strategy of focusing on the bubble kidsthose who score very near the proficiency cut pointin response to a Chicago high-stakes testing policy. They found that although overall proficiency rates improved, there were little to no gains among the least academically advantaged students. These studies suggest that if schools feel pressure to respond to high-stakes tests by targeting instructional efforts on students just below the margin of initially passing the test, we might see no improvement
(or declines) in the outcomes of the very lowest achieving students, although we may see improvements in the outcomes of students who score just below the passing score. In sum, there are cogent arguments and some empirical evidence supporting competing hypotheses about the effects of exit exams, with evidence that the effects may differ between those who just fail the exam and those who are far from passing. Evidence from the Chicago grade-retention policy suggests that students may increase their effort once they learn they are in danger of being retained, unless the students are very low achieving. It is also possible exit exam failure leads instead to low self-assessment, disengagement, and no increase in student effort. In terms of school responses, the early administration of the exam several years prior to graduation could provide schools the chance to intervene on the students behalf. Yet because the stakes are high and the amount of time in which to produce results is quite short, teachers may feel forced to adopt coping strategies that are not productive in the long run. To complicate the question of the motivational and curricular effects of failing an exit exam, we note that these effects may be heterogeneous, affecting students just below the cut score differently from those who initially score far below. Our goal in this article is to provide empirical evidence of the effects of initially failing a high school exit exam on subsequent persistence in school, course-taking, achievement, and graduation on the population of students who fall just below the passing threshold. To the extent that our results indicate negative or positive impacts of initially failing the exam, they may provide evidence that is more or less consistent with each of the arguments described above. II. Prior Research on the Effects of Exit Exams Prior research on the effects of exit exams varies in the treatment of interest (effects of the presence of exit exam policies or the effects of failing the exam), the outcome of interest (student achievement versus dropout, persistence, or graduation), and the type of data used (individual longitudinal data or aggregate cohort data). Most
501
Reardon et al.
prior research has estimated the effect of a high school exit exam requirement on high school dropout or completion rates. Several studies using individual-level data from nationally representative samples (mostly from cohorts of students graduating high school in the early 1990s) found that state high school exit exams increase high school dropout rates among low-achieving students (Bishop & Mane, 2001; Jacob, 2001) or Black males (Dee & Jacob, 2006), although one similar study found no such effects (Warren & Edwards, 2005). In contrast, a set of studies examining the relationship between state exit exam policies and state-level graduation rates generally finds no effect of exit exams on dropout rates (Carnoy & Loeb, 2002; Greene & Winters, 2004; Grodsky, Warren, & Kalogrides, 2009; Warren & Jenkins, 2005), although at least two such studies find a different result (Amerin & Berliner, 2002; Marchant & Paulson, 2005). Some of these studies have important methodological shortcomings, however, that may bias their estimated effects of exit exam policies on dropout rates (discussed at length in Dee & Jacob, 2006; Warren et al., 2006). Two newer studies that correct many of the methodological shortcomings of these studies find that high school dropout rates tend to increase by roughly 1 to 2 percentage points, on average, when states implement rigorous exit exams (Dee & Jacob, 2006; Warren et al., 2006).3 Dee and Jacob (2006) find that these effects are concentrated among Black students and students in high-poverty schools. The studies described above estimate the effect of exit exam requirements on student outcomes. Because they estimate the effect of exit exam policies on student outcomes, they provide little information about the mechanisms through which exams have their effects or about whether the effects are concentrated among very lowachieving students or students at the margin of passing. The finding that graduation rates decline under exit exam policies could result from discouragement effects but could also be simply because of the fact that some students never ultimately pass the exams. Exit exams could increase student motivation but still reduce graduation rates, if some students never pass the exam despite increased motivation. Moreover, exit exams heterogeneous effects may reduce graduation rates
of very low-performing students while having no effect or a positive effect on the graduation rates of other students. One way to investigate the mechanisms through which exit exams operate is to examine the effects of failing a high-stakes exit exam on subsequent student academic outcomes. Moreover, investigating the effects of initially failing the exam for students at the margin of passing may help us understand what type of students the exam requirement affects. Several existing studies provide evidence regarding the effects of failing an exit exam on the likelihood of high school graduation. Catterall (1989) finds that students who initially failed an exit exam expressed more doubt about their chances of finishing high school than those who passed, although the authors do not observe whether these students were in fact less likely to finish school nor whether test failure caused the desire to drop out. Griffin and Heidorn (1996), using longitudinal student data, find that exit exam failure is associated with increased dropout rates, but only for students with moderately high GPAs, a pattern the authors interpret as evidence of a discouragement or stigma effect. Neither of these two articles is fully persuasive regarding the effects of failing an exit exam because both rely on regression adjustment to estimate the effects of failing. Better evidence comes from a set of more recent research that relies on regression discontinuity designs. Because passing the exam is determined by a sharp threshold on an observable variable (the exit exam score), regression discontinuity estimators can provide unbiased estimates of the effects of failing an exit exam for students with skill levels near the passing threshold (Imbens & Lemieux, 2008). Using a regression discontinuity analysis, Martorell (2005) finds that Texas students who barely fail the state exit exam on their first attempt are no more likely to drop out of school by 12th grade than those who just pass on their first attempt. Martorell does not investigate whether these effects differ among student subgroups defined by race/ethnicity, poverty, or language status. In a similar analysis using Massachusetts data, Papay, Murnane, and Willett (2010) find that failing the exam reduces the probability of ontime graduation by 8 percentage points among low-income urban students at the margin of passing
502
but failure has no effect on the graduation rates of other students. One possible explanation for these patterns is that schools attended by lowincome urban students may not provide adequate remedial instruction for students who initially fail the exam, but Papay et al. cannot test this possibility with their data. Likewise, Ou (2009) finds that failing a math exit exam in 10th grade lowers students probability of staying in school through 12th grade and graduating, although this effect is confined to low-income and minority students. Each of these recent studies provides significant evidence regarding the effects of exit exams on student persistence and graduation. Two of the three studies suggest that failing an exit exam leads to lower graduation rates, particularly for disadvantaged students, a finding more consistent with the discouragement hypothesis than the motivational hypothesis. Nonetheless, none of these three studies provide estimates of the effect of failing an exit exam on student achievement, an important element in understanding the effects of exit exams. In fact, there is little evidence on the effects of exit exam requirements on achievement.4 In this article, we estimate the effects of failing an exit exam using a regression discontinuity design similar to that used by Martorell (2005), Papay et al. (2010), and Ou (2009). In addition to examining the effects of initial exit exam failure on persistence and graduation, we also estimate the effects of 10th grade exit exam failure on subsequent achievement. Our estimates on achievement are particularly useful for assessing the extent to which exit exams may motivate students who fail to work harder or may motivate schools to target instructional resources toward such students. We rely on data from four large school districts in California, where an exit exam requirement was implemented for students in the graduating class of 2006. III. The California High School Exit Exam The California State Legislature passed Senate Bill 2X in March 1999, requiring all local school districts to administer the CAHSEE and provide supplemental instruction to those students who do not demonstrate sufficient progress toward
passing the exam. The bill claimed that the CAHSEE would significantly improve pupil achievement in high school and to ensure that pupils who graduate from high school can demonstrate grade level competency in reading, writing, and mathematics, the state must set higher standards for high school graduation (Senate Bill 2X, sec. 1(b)).5 The CAHSEE is a two-part exam of mathematics and English language arts (ELA) skills. The math section assesses students mastery of the California math content standards for 6th and 7th grade and their Algebra 1 skills using a multiple-choice format. The ELA section is aligned with state content standards through 10th grade and utilizes a multiple-choice format along with one essay. Both tests are administered in English, regardless of a students primary language.6 Students must pass both parts (with a minimum score of 350) to earn a high school diploma. The test is first administered to students in the spring of 10th grade, and students have at least five subsequent opportunities to retake the sections they have not yet passed (twice in 11th grade and 12th grade and at least once following the end of the 12th grade school year).7 Districts notify students and their parents of their CAHSEE performance about 7 weeks after the exam is administered. Because students are told their exact score, not simply whether they passed or failed, students who fail have some sense of how close they came to scoring the requisite 350 they need to meet the CAHSEE requirement. IV. Data We use longitudinal student-level data from four large California school districts to investigate the effects of failing the CAHSEE. The districts are 4 of the 10 largest school districts in California, collectively enrolling more than 110,000 high school students (about 5.5% of high school students in the state) annually. For our primary analyses, we use data from five cohorts of studentsstudents scheduled to graduate in 2006 through 2010for whom the CAHSEE requirement was binding and for whom we have outcome data.8 Specifically, our analyses include students who took the CAHSEE for the first time in 10th grade in spring 2004 through spring 2008. We 503
Reardon et al.
exclude from our analyses students classified as special education students (roughly 10% of students) because these students were not subject to the CAHSEE requirement in most of the years covered by our analyses. Outcome Measures We estimate the effect of failing the CAHSEE exam on four primary outcomesacademic achievement, subsequent course taking, persistence to 12th grade, and graduation. We measure academic achievement in two ways. As a measure of academic achievement, we use the spring 11th grade ELA California Standards Test (CST) score. All students in California take the same ELA CST test in 11th grade. We have no outcome measure of mathematics achievement, however, because there is no common math test taken by all students in 11th grade in California. Instead, students take one of a number of different subject or content math CST tests (e.g., Geometry, Algebra 2), depending on what 11th grade math course they are enrolled in. As a result, 11th grade math CST scores are not comparable across students. Moreover, because students GPAs are partly a function of the courses they takewhich may be affected by performance on the 10th grade CAHSEEwe do not use GPA as a measure of students academic achievement. Although we have no common measure of mathematics achievement, the math test taken by a student indicates the level of mathematics curriculum to which he or she was exposed in 11th grade. We use this information as an additional outcome measure. Specifically, we construct two math course indicator variables using the information on which specific math CST test a student took in 11th gradeone indicating whether a student was enrolled in a class higher than Algebra I in the 11th grade, the other indicating whether a student was enrolled in a class higher than Geometry in the 11th grade.9 Although we cannot directly determine whether students have dropped out of high school because students who leave a given district prior to graduation may be dropouts or may have left and enrolled elsewherewe can identify whether students are present in the district 2 years after first taking the CAHSEE (in the spring of 12th grade). We construct a binary variable indicating 504
whether students are present in the district in the spring semester 2 years after they first took the CAHSEE in 10th grade.10 We use the indicator of presence in spring of the scheduled 12th grade year as an indicator of persistence in schooling. Of course, some students may not be present in the district because they have transferred to another district. Nonetheless, if we observe that failing the CAHSEE affects the probability that a student is present in the district in 12th grade, we can assume that this is because failing the CAHSEE affects persistence or dropout rates. It is unlikely that CAHSEE failure affects the probability of transferring to another district within the state because students will be subject to the CAHSEE requirement in any district within the state. It is, however, possible that CAHSEE failure may affect the probability of transferring into private schools (where the CAHSEE is not required) or even out of the state, but such effects would likely be extremely small. Thus, we argue that any effects of CAHSEE failure on persistence are likely because of dropout effects. Finally, we estimate the effect of failing the CAHSEE on the probability of graduating from the district using a binary indicator of graduation status provided by the districts. We have access to 11th grade outcome dataELA CST scores and math course enrollmentfor the five cohorts of students who were in 10th grade in 2004, 2005, 2006, 2007, and 2008; 12th grade outcome data (persistence and graduation) are available only for the first four of these cohorts.11 Descriptive Statistics The first column (All Students) of Table 1 displays basic descriptive statistics and mean outcomes for the students included in this analysis (we discuss columns 27 of Table 1 in the following section on the analytic strategy). Students in our sample have slightly higher ELA CST test scores in 8th and 10th grade than the state average. The graduation rate is about 69% (though recall that this understates the true graduation rate, as some students have likely transferred and graduated from other districts). The persistence rate, however, is 8 percentage points higher than the graduation rate, indicating that a large number of students remain in school and in the same district through 12th grade but do not graduate
TABLE 1 Selected Student Characteristics and Outcomes, for Various Student Samples Students whose lower score is 345355 0.103 0.482 0.180 0.214 0.532 0.544 0.225 0.432 0.438 0.471 0.762 0.664 0.785 0.331 9,744 4,463 0.406 0.781 0.687 0.823 0.749 0.647 0.757 0.270 5,342 0.595 0.361 0.800 0.686 0.500 0.679 0.191 6,277 0.084 0.467 0.139 0.287 0.422 0.573 0.328 0.586 0.584 0.119 0.491 0.214 0.154 0.623 0.513 0.136 0.297 0.314 0.064 0.563 0.185 0.171 0.514 0.596 0.401 0.789 0.809 0.067 0.568 0.218 0.133 0.579 0.588 0.311 0.724 0.749 0.765 0.682 0.496 0.648 0.161 3,264 Students who passed math and scored 345355 on ELA Students who passed ELA and scored 345355 on math Students whose higher score is 345355 Students who failed math and scored 345355 on ELA Students who failed ELA and scored 345355 on math 0.059 0.565 0.147 0.213 0.444 0.617 0.511 0.884 0.885 0.852 0.685 0.498 0.703 0.216 2,952
All students 0.194 0.388 0.134 0.259 0.514 0.455 0.171 0.086 0.068 0.090 0.772 0.686 0.833 0.539 106,454
Covariates Percentage White Percentage Hispanic Percentage Black Percentage Asian Percentage Female Percentage free-lunch eligible Percentage ELL 8th grade ELA CST score (standardized) 10th grade ELA CST score (standardized) Outcomes Mean 11th grade ELA CST score (standardized) Percentage present in spring 12th grade Graduation rate (by spring of 12th grade) Took 11th grade math course higher than Algebra 1 Took 11th grade math course higher than Geometry n
Note. ELA = English language arts; ELL = English language learner; CST = California Standards Test. Column 1 contains descriptive statistics of student covariates and outcomes for the entire analytic sample. The sample is limited to non-special-education-status students who have complete covariate data, California High School Exit Exam (CAHSEE) test score data, and nonmissing 11th grade ELA CST scores. Columns 27 are restricted to students within five points of the specified CAHSEE cut score. These samples represent the approximate populations to whom the estimates from each of our models apply (the models are described below). Students are designated as ELL if they were designated as an ELL in spring of 10th grade. A student was categorized as a special education student if he or she had ever received special education services or had been designated special education status at any point after 8th grade. All ELA CST scores are reported in z scores, standardized by test year within the state.
505
Table 2 Initial Attempt Passing Rates and Average Scores on ELA and Math CAHSEE, by Student Characteristics Proportion Who Pass. .ELA on First Attempt Overall Race/ Ethnicity White Hispanic Black Asian Gender Male Female Socio-Economic Status Not Free-Lunch Eligible Free-Lunch Eligible ELL Status Non-Ell ELL 8th Grade ELA Performance Level Far Below Basic Below Basic Basic Proficient Advanced 0.789 0.944 0.693 0.742 0.828 0.759 0.817 0.850 0.716 0.886 0.308 0.210 0.518 0.895 0.993 0.999 Math on First Attempt 0.777 0.916 0.670 0.651 0.885 0.788 0.767 0.826 0.718 0.840 0.465 0.326 0.527 0.827 0.973 0.998 Both on First Attempt 0.681 0.873 0.556 0.560 0.774 0.672 0.690 0.751 0.598 0.777 0.214 0.125 0.345 0.757 0.956 0.990 Mean CAHSEE Scores ELA 379.0 401.9 366.2 369.8 384.3 375.1 382.7 387.5 368.8 387.7 335.8 331.4 349.8 375.5 403.7 427.0 Math 381.4 399.3 366.9 364.2 396.5 383.1 379.8 388.1 373.4 387.5 351.2 341.9 353.6 375.9 404.1 428.2
Note: Columns 1-3 cell contents represent the percentage of students in each subcategory who passed either the ela, math, or both sections of the spring 10th grade CaHSee administration. Columns 4-5 represent mean CaHSee scores of each group. The sample is limited to non-special education status students who have complete covariate data, CaHSee test score data, and non-missing eleventh grade ela CST scores.
from the district. It is not clear from these figures, however, to what extent, if at all, this failure to graduate is because of the CAHSEE. Certainly some students do not graduate because of a failure to accumulate sufficient credits, regardless of whether they have passed the CAHSEE or not. Table 2 summarizes initial CAHSEE passing rates and mean ELA and math CAHSEE scores, broken down by race/ethnicity, gender, socioeconomic status, English language learner (ELL) status, and eighth grade ELA proficiency level (as defined by state standards on the state eighth grade ELA test). Overall, about two thirds of students pass both of the CAHSEE exams on their first attempts. White students have the highest initial passing rates on the CAHSEE, followed by Asian students. Hispanic and African American students have passing rates more than 506
30 percentage points lower than those of their white counterparts. Students designated as ELLs have by far the lowest passing rates of any demographic group, with an average pass rate of only 21.4%. Furthermore, eighth grade CST performance level is a very strong predictor of subsequent CAHSEE pass rates, which is not surprising given that the CAHSEE is intended to measure skills largely in line with Californias middle school performance standards (also see Kurlaender, Reardon, & Jackson, 2008; Zau & Betts, 2008). Evidence from Table 2 suggests that initial passing status is generally related to many factors that may independently influence student outcomes. This highlights the importance of adopting a research design that can disentangle exit exam failure from these other factors.
V. Analytic Strategy We rely on a regression discontinuity strategy to obtain unbiased estimates of the effect of failing the CAHSEE for students near the cut score. Regression discontinuity relies on the plausible assumption that, in the limit, students who score arbitrarily close to, and below, the passing score are similar on average in every way to students who score arbitrarily close to, but above, the passing score. As a result, the estimated average outcomes of those who score just above the passing score can stand as valid counterfactual estimates for what the average outcomes of those who score just below the passing score would have been had they passed the test (Imbens & Lemieux, 2008). Because the CAHSEE consists of two separate testsmath and ELAa student may fail the CAHSEE in 10th grade in one of three different ways (failing the ELA portion, failing the math portion, or failing both). As a result, there are several possible estimands of interest (for a discussion of the multiple estimands available when using multiple rating scores in regression discontinuity, see Reardon & Robinson, 2010). For example, one may estimate the effect of failing at least one of the exams (relative to passing both); in addition, we estimate the effect of failing both of the exams (relative to passing at least one). These two estimates each may provide some evidence of the effect of failing an exit exam in 10th grade. Moreover, differences among the two estimates may be suggestive of motivational and discouragement effects. For example, if students who perceive themselves as unlikely to ever pass the test are more likely to be discouraged and students who perceive themselves as close to passing are more likely to be motivated, we might expect more positive effects of failing one versus no exams than of failing both versus one exam. The effects also may depend on which test a student failed. This possibility is particularly likely with subject-specific outcomes. For example, failing the ELA exam may have positive effects on subsequent ELA achievement if it motivates students or causes schools to provide remedial ELA instruction for them. But failing the math exam may have no effects or negative effects on ELA achievement because it likely
induces little motivation to improve ones ELA skills or causes students (and schools) to focus on math skills at the expense of ELA. Thus, taking into account the specific subject failed may be important to understanding the effects on the subject-specific outcomes investigated herein, including 11th grade ELA CST scores and 11th grade math course taking. To estimate the effects of failing different numbers and subjects of the exit exams, we fit multiple regression discontinuity models. Each of the models has the form
yicd = f (Ricd) + (Ricd < 350) + Xicd B + cs+ icd, (1)
where yicd is the outcome (CST score in 11th grade, math course type in 11th grade, presence in spring 12th grade, graduation) for student i in cohort c in district d, Ricd is the 10th grade math or ELA CAHSEE score for student i in district d and cohort c (or some monotonic combination of the math or ELA scores), f is a continuous function of R, X is a vector of student covariates, is a vector of cohort-by-school fixed effects, and is an error term. The parameter of interest here is , which indicates the average effect of failing the CAHSEE on the outcome for a student with a score right at the margin of passing. We estimate these models using ordinary least squares (OLS); although two of our outcomes are binary, our OLS linear probability models provide results nearly identical to those from logistic regression models, but the OLS estimates are much more readily interpretable. We compute standard errors accounting for the clustering by CAHSEE scores, as recommended by Lee and Card (2008) because of the discrete nature of the scores. Fitting regression discontinuity models requires the choice of a functional form f and a choice of bandwidth around the cutoff score. Based on visual inspection of the data, we use a quadratic function for f, allowing the quadratic to have a different slope and different curvature on either side of the passing score. We use the cross-validation method suggested by Ludwig and Miller to choose the optimal bandwidth, given this functional form (Imbens & Lemieux, 2008; Ludwig & Miller, 2007). Depending on the specific outcome and/or model, this procedure indicates optimal bandwidths ranging from 30 to 50 points on the CAHSEE scale (roughly 1 SD), 507
Reardon et al.
although the cross-validation criterion is largely insensitive to differences in the bandwidth above bandwidths of 20. We present results in the article using a bandwidth of 50 for all analyses. Most of the results are substantively unchanged if we use larger or smaller bandwidths (see Table A2 in the appendix). To estimate the effect of failing both tests (relative to passing at least one), we construct a new rating score that is the maximum of a students scores on the math and ELA tests. If this maximum score is less than 350 (i.e., the passfail cut score for both tests), the student has failed both tests; if the maximum is 350 or greater, the student has passed at least one of the two. Using this maximum score as the rating score in the regression discontinuity model allows us to estimate the effect of failing both versus passing at least one of the tests. Likewise, using the minimum of the math and ELA scores as the rating score allows us to estimate the effect of failing at least one test versus passing both. To estimate the effects of failing a specific subject test, we restrict the sample to include only students who have either passed or failed one of the tests and then use the regression discontinuity model to estimate the effect of failing the other test. For example, we limit the sample to those who passed the math test and estimate the effect of failing the ELA test, using the ELA test score as the rating score in Equation 1. This yields four additional estimands: the effect of failing the math CAHSEE for those students who passed the ELA CAHSEE and for those students who failed the ELA CAHSEE and the effect of failing the ELA CAHSEE for those students who passed the math CAHSEE and for those who failed the math CAHSEE. In summary, we estimate the effect of six different types of exit exam failure: (a) the effect of failing at least one of the two CAHSEE sections, (b) the effect of failing the ELA section, when passing the math section, (c) the effect of failing the math section, when passing the ELA section, (d) the effect of failing both sections, (e) the effect of failing the ELA section, when also failing the math section, and (f) the effect of failing the ELA section, when also failing the math section. Each of our regression discontinuity models yields an estimate of the average effect of failing (vs. passing) the exam for students with scores at 508
the margin of passing, as defined by the specific type of failure. As with all regression discontinuity designs, these estimated effects may not generalize to students far from passing, as we discuss in the conclusion. To this end, columns 2 to 7 of Table 1 provide descriptive statistics for students near (i.e., within 5 pointsroughly one tenth of a standard deviationon either side) each passing margin. These descriptive data indicate that students on the margin of passing the CAHSEE (those to whom our estimates apply) are more likely to be Black or Hispanic, low-income, ELL, and low-achieving students than the population of all students in the four districts. Prior to estimating the effects of CAHSEE failure, we conduct a series of checks to assess the validity of the key regression discontinuity assumptionsthe exogeneity of the cutoff score and the rating scores and the continuity of the potential outcomes (Imbens & Lemieux, 2008). The data show no evidence of manipulation of the rating scores; the density of the CAHSEE scores is similar on each side of the 350 passing score (results not shown). Moreover, we find no evidence of any systematic discontinuity on any pretreatment student covariates (see Table A1 in the appendix). Finally, in addition to estimating the effect of failing the CAHSEE on academic achievement, math course taking, persistence, and graduation for the total sample, we also estimate the models separately by race/ethnicity, gender, ELL status, and free or reduced-price lunch eligibility status to investigate whether disadvantaged populations experience different average effects than their more advantaged counterparts (as found in Ou, 2009; Papay et al., 2010). VI. Results We begin by examining the estimated effects of failing the CAHSEE ELA or math exam on subsequent achievement, course enrollment, persistence, and graduation. Figure 1 graphically illustrates the relationship between students minimum CAHSEE scores (the minimum of their ELA and math 10th grade CAHSEE scores) and four of the outcomes of interest. The graphs show little or no discontinuity in average outcomes across the passing score of 350. Figures using the maximum score, the ELA score, or the
1 .5 0 .5 1 300
Outcomes, by 10th Grade Minimum CAHSEE Score Mean 11th Grade ELA CST Score Proportion Still Present in Spring 12 (Standardized) .9 .8 .7 .6 .5 320 340 360 380 400 10th Grade Minimum CAHSEE Score 300 320 340 360 380 400 10th Grade Minimum CAHSEE Score
Proportion Who Graduate in Spring 12 1 .8 .6 .4 .2 300 320 340 360 380 400 10th Grade Minimum CAHSEE Score .8 .6 .4 .2
Proportion Who Took 11th Math > Geometry
300
320 340 360 380 400 10th Grade Minimum CAHSEE Score
FIGURE 1.
Outcomes, by 10th grade minimum CAHSEE score.
CAHSEE = California High School Exit Exam; ELA = English language arts; CST = California Standards Test.
math score as the rating score likewise display little or no evidence of discontinuities in outcomes (additional figures not shown, in the interest of space). Table 3 reports the estimated overall effects of the six ways in which students can fail the CAHSEE in 10th grade, based on regression discontinuity models of the form shown in Equation 1. In general, there is no evidence of any systematic effect of failing one or both parts of the CAHSEE on students 11th grade ELA achievement or their persistence in school through 12th grade. With respect to graduation, the estimated effect of failing at least one part of the CAHSEE is positive (+2.7 percentage points) and statistically significant. However, this estimate is very sensitive to the bandwidth used (see Table A2 in the appendix); the point estimate ranges from 2.8 to +2.8 percentage points, depending on the bandwidth (and is not statistically significant in most cases). The instability of this estimate leads us to discount its reliability.
Failing the math CAHSEE exam appears to lower the probability of taking a math course higher than Geometry in 11th grade by 5.1 percentage points (against a base rate of 33% for students who barely passed the math CAHSEE and who passed the ELA CAHSEE). No such effect is evident for failing the ELA exam, suggesting that the reduction in upper-level math course taking is a result of students being placed in remedial or lower-level math courses as a result of failing specifically the math portion of the CAHSEE. Failing the math CAHSEE does not appear to significantly reduce students likelihood of taking upper-level math courses if they have also failed the ELA CAHSEE, although the point estimate is negative. Nor does failing an exam affect students likelihood of enrolling in a lowlevel math course (Algebra 1 or below) in 11th grade, likely because the remedial math courses are at a higher level than Algebra 1. The general lack of statistically significant estimated effects of failing the CAHSEE here is
509
TABLE 3 Estimated Effects of Failing the CAHSEE (First Attempt) on Achievement, Persistence, Graduation, and Math Course Taken Effect of failing . . . At least one section 11th grade ELA CST score (standardized) n Present in spring 12th grade n Graduation (by spring of 12th grade) n 11th grade math course higher than Algebra 1 n 11th Grade math course higher than Geometry n 0.005 (0.012) 59,998 0.006 (0.009) 55,418 0.027* (0.011) 43,855 0.017 (0.009) 63,408 0.027** (0.009) 63,337 ELA section, when passing math section 0.003 (0.018) 38,751 0.004 (0.014) 33,746 0.007 (0.018) 26,901 0.003 (0.013) 40,394 0.007 (0.015) 40,350 Math section, when passing ELA section 0.027 (0.017) 38,104 0.013 (0.014) 34,372 0.019 (0.017) 27,310 0.024 (0.014) 40,025 Both sections 0.005 (0.014) 44,551 0.009 (0.014) 43,036 0.012 (0.016) 33,600 0.008 (0.013) 47,612 ELA section when also failing math 0.020 (0.020) 13,924 0.012 (0.019) 15,270 0.026 (0.022) 11,691 0.003 (0.018) 15,407 0.016 (0.013) 15,395 Math section, when also failing ELA 0.008 (0.021) 13,770 0.001 (0.020) 15,066 0.002 (0.023) 11,553 0.017 (0.018) 15,194 0.030 (0.016) 15,190
0.051*** 0.016 (0.013) (0.011) 39,960 47,558
Note. CAHSEE = California High School Exit Exam; ELA = English language arts; CST = California Standards Test. Clustercorrected standard errors (clustered on the rating score) are shown in parentheses. The table reports the point estimate and standard error on the coefficients estimating the effect of failing the CAHSEE exam on each of the outcome variables for students near the passing score. Models use students from all available data from the four districts (see text for details). All models use school-by-cohort fixed effects. The estimated models regress the outcome variable on the math and ELA scale scores, the math and ELA scale scores squared, and interaction terms that allow both the linear and quadratic terms to vary on either side of the passing score. Models also include covariates to control for ELL status, gender, free-lunch eligibility status, and race/ ethnicity. All models restrict the sample to students scoring within a bandwidth of 50 points from the passing score. Columns correspond to various conceptualizations of the treatment, that is, different combinations of CAHSEE failing status (see text for a complete explanation). All models exclude students who were classified as special education at any point since 8th grade. *p < .05. **p < .01. ***p < .001.
not the result of low statistical power. The standard errors of these estimated effects are small. The estimates of the effects on 11th grade CST scores are sufficiently precise to rule out effects as small as 0.03 to 0.05 standard deviations; the estimates of the effects on persistence, graduation, and math course taking are sufficiently small to rule out effects as small as 3 to 5 percentage points. Thus, if there are undetected systematic effects of failing an exit exam, they are very modest in size.
Our final analysis investigates the effects of CAHSEE failure among student subgroups. Table 4 reports the estimated effects of failing one or both parts of the CAHSEE separately by race/ethnicity, gender, socioeconomic status, and ELL status. In the interest of space, we report only the estimates from the models using the minimum or maximum of the CAHSEE scores as the rating score. Each set of estimates (e.g., the race/ ethnicity-specific effects) comes from a model that allows the shape of the fitted model to vary
510
TABLE 4 Estimated Effect of Failing CAHSEE (First Attempt) on Achievement, Persistence, Graduation, and Subsequent Math Test Taking, by Student Characteristics Percentage present in spring 12th grade Effect of failing at least one section Effect of failing both sections Effect of failing at least one section Effect of failing both sections Effect of failing at least one section Effect of failing both sections Effect of failing at least one section Graduation rate (by spring of 12th grade) Took 11th grade math higher than Algebra 1 Took 11th grade math higher than Geometry Effect of failing both sections
Mean 11th grade ELA CST score (standardized) Effect of failing both sections
Effect of failing at least one section
Race/ethnicity White
Hispanic
Black
Asian
Pacific Islander
American Indian
F statistic p value on F test
0.055 (0.032) 0.024 (0.017) 0.025 (0.029) 0.003 (0.023) 0.114 (0.096) 0.209 (0.197) 1.666 0.139
0.008 (0.048) 0.002 (0.018) 0.012 (0.033) 0.026 (0.034) 0.033 (0.118) 0.138 (0.208) 0.242 0.944
0.027 (0.023) 0.005 (0.014) 0.042 (0.024) 0.025 (0.019) 0.075 (0.084) 0.084 (0.164) 1.496 0.187
0.043 (0.042) 0.009 (0.019) 0.023 (0.032) 0.037 (0.033) 0.083 (0.119) 0.244 (0.191) 1.132 0.341
0.025 (0.026) 0.021 (0.017) 0.055 (0.029) 0.008 (0.023) 0.099 (0.112) 0.253 (0.185) 1.386 0.226
0.001 (0.041) 0.007 (0.022) 0.002 (0.037) 0.016 (0.038) 0.163 (0.148) 0.357 (0.197) 0.950 0.447
0.073** (0.023) 0.006 (0.013) 0.030 (0.023) 0.014 (0.017) 0.040 (0.072) 0.193 (0.131) 1.958 0.081
0.042 (0.040) 0.013 (0.018) 0.001 (0.030) 0.035 (0.030) 0.071 (0.109) 0.148 (0.160) 0.542 0.745
0.023 (0.021) 0.021 (0.013) 0.032 (0.022) 0.035 (0.022) 0.048 (0.074) 0.044 (0.131) 0.155 0.979
0.048 (0.030) 0.023 (0.013) 0.028 (0.023) 0.007 (0.030) 0.040 (0.068) 0.029 (0.107) 1.052 0.385
Gender Male 0.012 (0.021) 0.004 (0.019) 0.335 0.563 0.001 (0.014) 0.010 (0.013) 0.200 0.655 0.019 (0.020) 0.002 (0.019) 0.641 0.423
Female
0.015 (0.018) 0.021 (0.015) 2.285 0.131
0.012 (0.017) 0.037* (0.016) 1.155 0.283
0.020 (0.024) 0.006 (0.022) 0.172 0.678
0.016 (0.013) 0.018 (0.012) 0.017 0.897
0.013 (0.019) 0.003 (0.018) 0.120 0.729
0.002 (0.013) 0.050*** (0.012) 7.169 0.007**
0.007 (0.015) 0.038** (0.015) 5.007 0.025* (continued)
511
512
Percentage present in spring 12th grade Effect of failing at least one section Effect of failing both sections Effect of failing at least one section Effect of failing both sections Effect of failing at least one section Effect of failing both sections Effect of failing at least one section Graduation rate (by spring of 12th grade) Took 11th grade math higher than Algebra 1 Took 11th grade math higher than Geometry Effect of failing both sections Effect of failing both sections 0.020 (0.024) 0.003 (0.017) 0.595 0.441 0.005 (0.015) 0.005 (0.012) 0.001 0.975 0.001 (0.024) 0.013 (0.017) 0.170 0.680 0.001 (0.017) 0.050** (0.015) 5.221 0.022* 0.004 (0.024) 0.015 (0.021) 0.121 0.728 0.014 (0.013) 0.020 (0.012) 0.121 0.728 0.014 (0.020) 0.004 (0.017) 0.159 0.690 0.032* (0.014) 0.026* (0.012) 0.105 0.746 0.000 (0.020) 0.026* (0.012) 1.281 0.258 0.007 (0.022) 0.009 (0.024) 0.009 (0.036) 0.098 (0.074) 0.657 0.579 0.017 (0.020) 0.039* (0.016) 0.035* (0.018) 0.004 (0.045) 3.424 0.016* 0.016 (0.021) 0.027 (0.024) 0.055 (0.033) 0.074 (0.073) 1.726 0.159 0.021 (0.024) 0.046* (0.019) 0.006 (0.023) 0.016 (0.046) 1.212 0.304 0.004 (0.025) 0.010 (0.026) 0.084* (0.041) 0.067 (0.077) 1.626 0.181 0.003 (0.018) 0.022 (0.015) 0.026 (0.018) 0.018 (0.036) 0.643 0.587 0.000 (0.020) 0.008 (0.022) 0.015 (0.034) 0.069 (0.069) 0.421 0.738 0.032 (0.020) 0.020 (0.014) 0.036* (0.017) 0.006 (0.042) 0.385 0.764 0.020 (0.018) 0.018 (0.017) 0.008 (0.026) 0.025 (0.058) 0.307 0.820
Reardon et al.
TABLE 4
(continued)
Mean 11th grade ELA CST score (standardized)
Socioeconomic status (SES) Not free-lunch eligible
Free-lunch eligible
0.011 (0.019) 0.001 (0.014) 0.177 0.674
ELL status ELL
Native English speaker
Redesignated fluent
Initially fluent
0.039 (0.024) 0.010 (0.019) 0.000 (0.022) 0.087 (0.052) 1.668 0.172
Note. CAHSEE = California High School Exit Exam; ELA = English language arts; CST = California Standards Test; ELL = English language learner. Cluster-corrected standard errors (clustered on the rating score) are shown in parentheses. The table reports the point estimate and standard error on the coefficients estimating the effect of failing the CAHSEE exam on each of the outcome variables, by selected student characteristics, using the same models and samples used to estimate the effects shown in Table 3, albeit including interactions of the scale scores, the scale scores squared, and scale scorepassing interactions with the relevant set of demographic group dummies. Separate models were run to test differences for each of the four demographic attributesrace/ethnicity, gender, SES, and ELL statusthough models include controls for the other demographic attributes. We conducted post-estimation joint F tests on the Group Treatment interaction variables. *p < .05. **p < .01. ***p < .001.
one demographic characteristic (e.g., race/ ethnicity) and allows the effect of failing to vary by that same demographic characteristic. Separate models are used to test for interactions by race/ethnicity, by gender, by socioeconomic status, and by ELL status. For each model, we conduct an F test of the null hypothesis that the effects are the same for each group of students. Overall, 4 of 40 of the F tests in Table 4 are significant at = .05, double what we would expect by chance. However, the significant effects are not obviously clustered in any one subgroup. The one notable finding among these subgroups is that the effect of failing the CAHSEE on math course taking varies by gender. More specifically, the differential course taking effect observed in Table 3 appears to be driven entirely by the effect among girls. In the last two columns of Table 4, the estimated effects of the CAHSEE on the likelihood of taking a math class higher than geometry in 11th grade differ significantly by gender. In the model testing the effect of failing at least one section, the estimated effect of failing for girls is 0.050, compared to an estimated effect of 0.002 for boys (note that the average effect failing at least one test among all students was 0.027; see column 1 in Table 3). Likewise, the estimated effect of failing both sections of the CAHSEE on higher level math course taking is 0.038 (p < .01) for girls and +0.007 (p > .05) for boys. VII. Discussion and Conclusion In this article, we used a regression discontinuity design to estimate the effects of failing a high school exit exam in 10th grade for the population of students with scores near the passing score. We find little evidence of any systematic positive or negative effects of failing an exit exam. CAHSEE exit exam failure does appear to have some effect on subsequent course-taking patterns in the following year. When students near the cut score fail at least one section specifically the math sectionin spring of 10th grade, they appear less likely to take higher-level math courses the following year, a pattern that suggests that these students are placed in remedial or lower-level math courses to improve their likelihood of passing the exam in the future. We
are not able to estimate whether these effects on course-taking patterns are beneficial, however, because we have no common measure of subsequent math achievement. Moreover, this pattern seems to be driven exclusively by changes in female students curriculum, although we do not know whether this is because schools respond differently to girls failure of the math exam than they do to boys failure or whether it is because girls choose lower courses in response to failing the exam than do boys (or choose higher courses in response to passing than do boys). We find no effects of exit exam failure on subsequent ELA achievement, persistence, or graduation. However, we caution against overgeneralization of this finding of no effects. Although the regression discontinuity design provides unbiased estimation of these effects, the estimates are strictly generalizable only to a particular subset of the population of students who fail the exit examthose who score near the passing score. These estimates do not, for example, indicate the effect of failing the exam for students with very low skills, whose scores are far from the passing threshold. For these students, it is quite plausible that the effects may differ (positively or negatively) from those estimated here.12 Nonetheless, this subsample of students is of interest in its own right. Unlike in Massachusetts, where Papay et al. (2010) report that only 13% of students fail (implying that the density of students near the passing score is relatively low), in California there are many student whose initial scores are very close to the passing score (on each exam, roughly 25% of students fail, and roughly 15% of students have scores within a quarter of a standard deviation of the passing score, implying a high density of students near the passing score). Hence our findings pertain to a nontrivial number of students. Furthermore, students who score near the cut score are a substantively interesting group of students from the perspective of understanding the mechanisms through which exit exams operate. For these students, there is likely considerable uncertainty about whether they will pass the exam the first time they take it. As a result, their initial score on the exam provides a strong signal of their likelihood of ultimately passing the test. For this group of students, then, motivational or
513
Reardon et al.
discouragement effects of failing the exam may be particularly strong. Likewise, students scoring just below the passing score may be those who are most likely to receive additional instructional resources and support from their schools. The fact that we find little evidence of any difference in outcomes for those just above and just below the passing score, therefore, may suggest that exit exam failure has little motivational or discouragement on students and yields little in the way of effective intervention by schools and teachers. Our results are consistent with some prior worknamely Martorells (2005) analysis of the effects of failing the Texas exit exam. They are less consistent with Papay et al.s (2010) finding, using Massachusetts data, that failing an exit exam appears to decrease graduation rates by 8 percentage points for low-income urban students. Because most students in Massachusetts who are on the margin of passing the math exam in 10th grade have passed the ELA exam, Papay et al.s estimates are most comparable to our estimates of the effect of failing the math exam when passing the ELA exam. Our estimates of the effect of failing at least one exam for students of low socioeconomic status, however, suggest that failing appears to increase graduation rates for these students by five percentage points (compared to no effect for higher-SES students, although we cannot reject the null hypothesis that the effects are the same for the lowand high-SES students). Although we do not think it prudent to make too much of the apparent positive effect of failing the math exam for lowincome students, given the fact that the estimates for low- and high-SES students are not significantly different, and given the large number of hypothesis tests in Table 4, it is clear that our results do not correspond well to the Massachusetts findings. Likewise, our results do not correspond well to those in New Jersey, where students who fail the exam have lower persistence rates and lower graduation rates than those who just barely pass the exam (Ou, 2009). The preponderance of estimates from our models suggest that failing an exit exam in 10th grade has little or no effect on students persistence, achievement, or graduation. Why might exit exam failure have so little effect on
subsequent educational outcomes? One reason may be that the passing threshold is set relatively low. A low passing threshold means that students at the margin of passing (who are the students most likely to have an incentive to be motivated to work harder if they fall the 10th grade test) are relatively low-achieving students, who have relatively low graduation rates in the absence of the exit exam policy. For such students, the exit exam is therefore often not the binding constraint on their graduation. A closer look at our data supports this argument: Among the students who score just above the passing score on the math and ELA CAHSEE exam in 10th grade, 34% leave their district by 12th grade and 24% remain in their district but do not graduate (Table 1, column 2). This suggests that for students near the margin, passing the CAHSEE is not sufficient to ensure graduation. It is unclear what the effects on those near the margin of passing would be if the passing threshold were higher. A higher threshold would mean the CAHSEE would be more of a constraint to graduation; this might motivate students and schools to work harder to improve the outcomes of students near the passing score, and so have positive effects. On the other hand, because the CAHSEE would be more of a binding constraint for those near the margin of passing, passing or failing the exam would be more determinative of graduation for students near the higher passing threshold, and so students who failed would be more likely not to graduate than those who passed. Our data cannot tell us which of these effects would dominate if the passing threshold were higher. Another possible explanation for why exit exam failure might have no effect on subsequent educational outcomes is that although the threat of failing causes increased student motivation and effort as well as intervention by their schools that increase students cognitive skills sufficiently to pass the test, those responses are not enough to affect student achievement on state standardized CST tests. In general, when a student fails on his or her first attempt at the CAHSEE in 10th grade, California districts are expected to intervene in some way to ensure that the student has the opportunity to acquire missing skills before he or she graduate. However, a great deal of variation exists in how districts ensure that these
514
students receive support; indeed, a great deal of variation also seems to exist across schools within the same district, based on our conversations with district officials in the four districts studied. Each of the four districts reports making use of some sort of school-level coordinator who is responsible for monitoring student intervention plans and communicating those plans to the district. In addition, schools in each district have created CAHSEE-targeted remedial courses into which first-time failers are placed in 11th grade, although these courses may vary considerably across schools. Nonetheless, in some additional analyses (not shown here), we find little variation among schools and districts in students initial and subsequent CAHSEE passing rates, conditional on students prior test scores (which are very
strong predictors of passing). This analysis suggests that variation among schools in CAHSEE preparation and remediation strategies yields little variation in student outcomes, although it is worth investigating this further to determine if there are strategies used by some schools that lead to better outcomes for students who initially fail one or both of the tests. Finally, it is important to note that our estimates here reflect the impact of failing the CAHSEE exam in 10th grade, given that the exit exam policy is in place. These should not be compared to estimates of the effect of the policy itself, which prior work suggests may lead to higher dropout rates (Dee & Jacob, 2006; Warren et al., 2006). We investigate this possibility in a companion article to the present one.
TABLE A1 Effect of Failing CAHSEE (First Attempt) on Student Covariates Effect of failing . . . At least one section White Hispanic Black Asian Female Free-lunch eligible ELL 9th grade ELA CST score 10th grade ELA CST score n 0.005 (0.007) 0.012 (0.011) 0.012 (0.008) 0.012 (0.009) 0.012 (0.011) 0.008 (0.008) 0.012 (0.009) 0.226 (0.649) 0.491 (0.635) 59,025 ELA section, when passing math section 0.011 (0.009) 0.029 (0.016) 0.012 (0.011) 0.018 (0.015) 0.014 (0.018) 0.003 (0.012) 0.004 (0.017) 0.129 (1.038) 1.053 (0.949) 38,259 Math section, when passing ELA section 0.001 (0.011) 0.005 (0.016) 0.018 (0.013) 0.007 (0.012) 0.001 (0.016) 0.015 (0.011) 0.004 (0.011) 1.296 (0.893) 0.252 (0.926) 37,514 Both sections 0.001 (0.007) 0.002 (0.014) 0.002 (0.011) 0.003 (0.011) 0.001 (0.015) 0.005 (0.010) 0.012 (0.014) 0.274 (0.824) 0.713 (0.732) 43,673 ELA section when also failing math 0.012 (0.010) 0.018 (0.020) 0.003 (0.017) 0.023 (0.014) 0.007 (0.021) 0.012 (0.014) 0.013 (0.019) 0.574 (1.134) 1.454 (1.041) 13,527 Math section, when also failing ELA 0.002 (0.009) 0.042* (0.020) 0.013 (0.015) 0.023 (0.017) 0.001 (0.022) 0.014 (0.013) 0.034 (0.020) 1.935 (1.190) 0.209 (1.086) 13,398
Note. CAHSEE = California High School Exit Exam; ELA = English language arts; ELL = English language learner; CST = California Standards Test. Cluster-corrected standard errors (clustered on the rating score) are shown in parentheses. The table reports the point estimate and standard error on the coefficients estimating the effect of failing the CAHSEE exam on student covariates, using the same models and samples used to estimate the effects shown in Table 3. Columns correspond to various conceptualizations of the treatment, that is, different combinations of CAHSEE failing status (see text for a complete explanation). All models exclude students who were classified as special education at any point since 8th grade. *p < .05.
515
516
Bandwidth = 20 Effect of failing at least one section Effect of failing both sections Effect of Effect of failing at failing least one both section sections Effect of failing at least one section Effect of failing both sections Effect of failing at least one section 0.005 (0.012) 59,998 0.006 (0.009) 55,418 0.021 (0.018) 28,172 0.018 (0.013) 35,237 0.007 (0.009) 50,068 0.006 (0.013) 35,237 0.027* (0.011) 43,855 0.024* (0.009) 58,388 0.008 (0.009) 58,388 0.005 (0.014) 44,551 0.009 (0.014) 43,036 0.012 (0.016) 33,600 0.013 (0.012) 43,160 0.004 (0.012) 43,160 0.005 (0.015) 36,475 0.015 (0.015) 36,131 0.006 (0.013) 51,527 0.006 (0.010) 48,025 0.020 (0.013) 37,949 0.023* (0.010) 50,068 0.010 (0.015) 41,104 0.003 (0.017) 27,400 0.015 (0.017) 27,503 0.019 (0.020) 21,379 0.030* (0.014) 26,434 0.016 (0.014) 26,434 0.005 (0.011) 38,422 0.006 (0.014) 30,350 0.022 (0.012) 39,908 0.003 (0.010) 39,908 0.005 (0.020) 18,281 0.014 (0.021) 18,611 0.015 (0.024) 14,417 0.036* (0.017) 17,577 0.006 (0.017) 17,577 0.027 (0.018) 28,742 0.018 (0.014) 26,879 0.012 (0.017) 21,271 0.018 (0.014) 27,859 0.003 (0.012) 27,859 Effect of failing both sections Bandwidth = 30 Bandwidth = 40 Bandwidth = 50 Bandwidth = 60 Effect of failing at least one section 0.004 (0.011) 66,728 0.003 (0.009) 61,116 0.025* (0.011) 48,533 0.021* (0.009) 65,015 0.006 (0.008) 65,015 Effect of failing both sections 0.003 (0.013) 52,116 0.007 (0.013) 49,531 0.006 (0.015) 38,908 0.005 (0.011) 50,578 0.009 (0.012) 50,578 0.029 (0.028) 9,024 0.006 (0.036) 7,242 0.047 (0.024) 8,677
TABLE A2 Estimated Effects of Failing CAHSEE (First Attempt), Using Alternative Bandwidths Around PassFail Cut Score
Bandwidth = 10
Effect of failing both sections
11th grade ELA CST score (standardized) n
0.032 (0.026) 14,924
Present in spring 12th grade n
0.013 (0.020) 13,994
0.036 (0.029) 9,397
Graduation (by spring of 12th grade) n
0.028 (0.025) 11,115
11th grade CST higher than Algebra 1 n
0.052* (0.020) 14,430
0.026 (0.024) 8,677
11th grade CST higher than Geometry n
0.026 (0.017) 14,430
Note. CAHSEE = California High School Exit Exam; ELA = English language arts; CST = California Standards Test. Columns correspond to varying bandwidths around the cut score of 350 (a bandwidth of 50 was used throughout the rest of the article). Results are shown only for minimum and maximum scores as treatments. The above table reports the point estimate and standard error on the coefficients estimating the effect of failing the CAHSEE exam on each of the outcome variables for students near the cut score. Models use students from all four districts in the 2004, 2005, 2006, 2007, and 2008 cohorts provided by the districts (some data not available for all cohorts in some districtssee Footnote 11). All models use school-bycohort fixed effects. The estimated models regress the outcome variable on the centered math and ELA scale scores, the centered math and ELA scale scores squared, and interaction terms that allow both the linear and quadratic forms to vary on either side of the cut score. Models also include covariates to control for ell status, gender, free-lunch eligibility status, and race/ ethnicity. Standard errors are clustered at the rating score. All models exclude students who were classified as special education at any point since 8h grade.
Notes
1 Californiathe source of the data used in this studyprovides examples of policy rhetoric presuming a link between exit exam policies and student and school motivation. California state superintendent of public instruction Jack OConnell claims that schools hold students to higher expectations as a result of the exam (News Release 09-103, July 8, 2009, Schools Chief Jack OConnell Announces Latest High School Exit Exam Results Show Steady Progress in Passage Rate, http://www.cde.ca.gov/nr/ne/yr09/yr09rel103.asp) and that the exit exam also motivates students: As a result of the exit exam, students are working harder, learning more and persevering in school (State of Education Address, February 6, 2007, annual address from Superintendent OConnell on the status of education in California, http://www.cde.ca.gov/eo/in/se/ yr07stateofed.asp). Likewise, Mary Bergen, the president of the California Teachers Association, has claimed that having students take an exit exam early in high school gave students an idea of the work expected of them and allows schools to steer resources to where they are most needed. 2 Numerous studies have discussed the motivational effects of high-stakes testing and accountability systems more generally on teachers and schools, focusing largely on how these actors respond to incentives and sanctions. See Booher-Jennings (2005) for a list of such studies. When stakes for a single exam are high, teachers and schools may feel compelled to dedicate more time to preparing for its administration. If teachers opt to teach to the test in place of more authentic teaching, students may learn only the material to pass that particular exam but miss out on the broader set of skills they may need in their future endeavors (Pedulla et al., 2003; Popham, 2001). If schools respond in this way to students who fail the initial exit exam, then we might expect to see these students subsequent scores on the exit exam improve, but such improvements might be small (i.e., just large enough to reach the tests passing score) or might not be mirrored in achievement as measured by a different standardized assessment of achievement. 3 Warren, Jenkins, and Kulick (2006) also address why their results are inconsistent with prior analyses based on National Education Longitudinal Study (NELS) data that tend to show no significant relationships between exit exams and high school completion measures. When they confine their sample to state-years prior to 1992 (i.e., when the final NELS-88 cohort should have graduated from high school), Warren et al. also find no significant associations. They suggest that many states have moved to more rigorous exit exams since that time and generally conclude
that the consequences of state HSEEs have changed in important ways since the NELS-88 cohort moved through the secondary school system (p. 146). 4 One exception is Jacob (2001), who uses NELS data to examine whether exit exam policies lead to increased student achievement. He does not find any evidence that exit exam policies affect student achievement. Marchant and Paulson (2005), in contrast, report that states with exit exam requirements have lower average SAT scores, although this association is not clearly causal. Moreover, neither of these studies investigates the effect of failing an exam on student achievement. 5 See http://www.leginfo.ca.gov/pub/99-00/bill/ sen/sb_0001-0050/sbx1_2_bill_19990329_chaptered .html. 6 However, English learners must be permitted to take the California High School Exit Exam (CAHSEE) with certain test variations if used regularly in the classroom. For example, if regularly used in the classroom, English learners must be permitted to hear the test directions in their primary language or use a translation glossary. 7 Testing dates are centrally scheduled by individual districts, and the exam is administered over the course of 2 days (1 day for each portion). The test is untimed, although students typically complete each section in 3 to 4 hours. 8 The first class for whom the CAHSEE requirement was binding was the graduating class of 2006, who first took the test in 10th grade in spring 2004, at which time they knew it to be a graduation requirement. A minor complication in the CAHSEE timeline arose for the class of 2006the first class to ultimately be subject to the CAHSEE exam requirement. In February of their senior year (2006), a lawsuit (Valenzuela v. OConnell) was filed on behalf of high school students who had not yet passed the CAHSEE exam. As a result, for 12 days of their final semester, students in the class of 2006 were relieved by an Alameda Superior Court Judge from their requirement to pass the CAHSEE. This decision was quickly overturned, however, by the California Supreme Court. Nonetheless, one may worry that the debate surrounding the legality of the CAHSEE in the spring of 2006 may have led to some ambiguity for students about whether the CAHSEE would be enforced. However, seniors in the class of 2006 had already completed their final administration of the CAHSEE before the 12 days when the CAHSEE requirement was temporarily suspended. The results of our analyses are substantively unchanged if we exclude the class of 2006 from the analyses. 9 Each of these outcomes is conditional on whether a student took any math course in 11th grade. Supplemental analyses (not shown), however, indicate
517
Reardon et al. no effect of failing the exit exam on whether a student took a math course in 11th grade. 10 We construct this variable using data on students GPA, California Standards Test (CST) scores, and CAHSEE scores. Students with any evidence that they were enrolled and attended school in a given semesterspecifically, a nonzero GPA, a nonmissing CST score, or a nonmissing CAHSEE scoreare coded as present in the district in that semester, and as not present otherwise. For students who leave the district and then return (are coded as present in some later term), we retroactively code them as present for all terms prior to the final one in which they are observed to be present. In addition, students who received a diploma from the district in an earlier semester are coded as present so that they not be counted among leavers or dropouts in our persistence models (i.e., our present indicator is coded 0 for anyone who left the district without returning prior to receiving a diploma and is coded 1 for those who have graduated or are still enrolled at a given semester). 11 We have five cohorts with outcome data in two of the four districts. In one of the remaining districts, we have graduation data for only two cohorts but other outcome data for five cohorts. The fourth district provided outcomes for only three cohorts. 12 In fact, in preliminary analyses comparing outcomes of low-skill students in cohorts who were and were not required to pass the CAHSEE, we find a substantial negative effect of the CAHSEE policy on graduation rates for low-skill students. These analyses are the subject of a companion article to this one. early labor market success. Labour Economics, 8, 203222. Blau, T. H. (1980). Minimum competency testing: Psychological implications for students. In R. Jaeger & C. Tittle (Eds.), Minimum competency achievement testing (pp. 172181). Berkeley, CA: McCutchan. Booher-Jennings, J. (2005). Below the bubble: Educational triage and the Texas accountability system. American Educational Research Journal, 42, 231268. Carnoy, M., & Loeb, S. (2002). Does external accountability affect student outcomes? A cross-state analysis. Educational Evaluation and Policy Analysis, 24, 305331. Catterall, J. S. (1986). Dropping out of schools as a process: Implications for assessing the effects of competency test required for graduation. Effects of testing reforms and standards (Working paper). Los Angeles: University of California, Los Angeles. Catterall, J. S. (1989). Standards and school dropouts: A national study of tests required for high school graduation. American Journal of Education, 98, 134. Center on Education Policy. (2004). State high school exit exams: States try harder but gaps persist. Washington, DC: Author. Center on Education Policy. (2005). How have high school exit exams changed our schools? Some perspectives from Virginia and Maryland. Washington, DC: Author. Center on Education Policy. (2009). State high school exit exams: Trends in test programs, alternate pathways, and pass rates. Washington, DC: Author. Covington, M. V. (2000). Goal theory, motivation, and school achievement: An integrative review. Annual Review of Psychology, 51, 171200. Dee, T. S., & Jacob, B. A. (2006). Do high school exit exams influence educational attainment or labor market performance? (Working paper). NBER Working Paper Series Working Paper 12199 http://www.nber .org/papers/w12199 National Bureau of Economic Research 1050 Massachusetts Avenue Cambridge, MA 02138, April 2006. Glass, G. V. (1978). Minimum competence and incompetence in Florida. Phi Delta Kappan, 59, 602605. Greene, J. P., & Winters, M. A. (2004). Pushed out or pulled up? Exit exams and dropout rates in public high schools. New York, NY: Manhattan Institute. Griffin, B. W., & Heidorn, M. H. (1996). An examination of the relationship between minimum competency test performance and dropping out of high school. Educational Evaluation and Policy Analysis, 18, 243252. Grodsky, E., Warren, J. R., & Kalogrides, D. (2009). State high school exit examinations and NAEP
References
Amerin, A. L., & Berliner, D. C. (2002). An analysis of some unintended and negative consequences of high-stakes testing (ESPL-0211-125-EPRU). Tempe: Arizona State University, Educational Policy Research Unit, Education Policy Studies Laboratory. Ames, C. (1984). Competitive, cooperative, and individualistic goal structures: A cognitive-motivational analysis. In C. Ames & R. Ames (Eds.), Research on motivation in education (Vol. 1, pp. 177207). San Diego, CA: Academic Press. Bishop, J. H. (2006). Drinking from the fountain of knowledge: Student incentive to study and learn Externalities, information problems and peer pressure. In E. A. Hanushek & F. Welch (Eds.), Handbook of the economics of education (Vol. 2, pp. 909944). Amsterdam, Netherlands: Elsevier. Bishop, J. H., & Mane, F. (2001). The impacts of minimum competency exam graduation requirements on high school graduation, college attendance and
518
High School Exit Exam long-term trends in reading and mathematics, 19712004. Educational Policy, 23, 589614. Huebert, J. P., & Hauser, R. M. (1998). High stakes testing for tracking, promotion and graduation. Washington, DC: National Academy Press. Imbens, G. M., & Lemieux, T. (2008). Regression discontinuity designs: A guide to practice. Journal of Econometrics, 142, 615635. Jacob, B. A. (2001). Getting tough? The impact of high school graduation exams. Educational Evaluation and Policy Analysis, 23, 99121. Kurlaender, M., Reardon, S. F., & Jackson, J. (2008). Middle school predictors of high school achievement in three California school districts. http:// www.cdrp.ucsb.edu/dropouts/pubs_reports.htm California Dropout Research Project. Lee, D. S., & Card, D. (2008). Regression discontinuity inference with specification error. Journal of Econometrics, 142, 655674. Ludwig, J., & Miller, D. (2007). Does Head Start improve childrens life chances? Evidence from a regression discontinuity design. Quarterly Journal of Econometrics, 122, 159208. Madaus, G. F., & Clarke, M. (2001). The adverse impact of high-stakes testing on minority students: Evidence from one hundred years of test data. In G. Orfield & M. L. Kornhaber (Eds.), Raising standards or raising barriers? Inequality and highstakes testing in public education (pp. 85106). New York, NY: Century Foundation Press. Marchant, G. J., & Paulson, S. E. (2005). The relationship of high school graduation exams to graduation rates and SAT scores. Education Policy Analysis Archives, 13. Retrieved from Citation: Marchant, G. J. & Paulson, S. E. (2005, January 21). The relationship of high school graduation exams to graduation rates and SAT scores. Education Policy Analysis Archives, 13(6). Retrieved from http:// epaa.asu.edu/epaa/v13n6/. Martorell, P. (2005). Do high school exit exams matter? Evaluating the effects of high school exit exam performance on student outcomes. Berkeley: University of California, Berkeley. Neal, D., & Schanzenbach, D. (2007). Left behind by design: Proficiency counts and test-based accountability. Cambridge, MA: National Bureau of Economic Research. Ou, D. (2009). To leave or not to leave? A regression discontinuity analysis of the impact of failing the high school exit exam. London: Centre for Economic Performance, London School of Economics. Papay, J. P., Murnane, R. J., & Willett, J. B. (2010). The consequences of high school exit examinations for low-performing urban students: Evidence from Massachusetts. Educational Evaluation and Policy Analysis, 32, 523. Pedulla, J. J., Abrams, L. M., Madaus, G. F., Russell, M. K., Ramos, M. A., & Miao, J. (2003). Perceived effects of state mandated testing programs on teaching and learning: Findings from a national survey of teachers. Boston, MA: Boston College, National Board of Educational Testing and Public Policy. Popham, W. J. (2001). The truth about testing: An educators call to action. Alexandria, VA: Association for Supervision and Curriculum Development. Reardon, S. F., & Galindo, C. (2002, April). Do highstakes tests affect students decisions to drop out of school? Evidence from NELS. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. Reardon, S. F., & Robinson, J. (2010). Regression discontinuity designs with multiple rating score variables (Working paper). Stanford, CA: Stanford University, Institute for Research on Education Policy and Practice. Richman, C. L., Brown, K., & Clark, M. (1987). Personality changes as a function of minimum competency test success or failure. Contemporary Educational Psychology, 12, 716. Roderick, M., & Engel, M. (2001). The grasshopper and the ant: Motivational responses of lowachieving students to high-stakes testing. Educational Evaluation and Policy Analysis, 2, 197227. Warren, J. R., & Edwards, M. R. (2005). High school exit examinations and high school completion: Evidence from the early 1990s. Educational Evaluation and Policy Analysis, 27, 5374. Warren, J. R., & Jenkins, K. N. (2005). High school exit examinations and high school dropout in Texas and Florida, 19712000. Sociology of Education, 78, 122143. Warren, J. R., Jenkins, K. N., & Kulick, R. B. (2006). High school exit examinations and state-level completion and GED Rates, 19722002. Educational Evaluation and Policy Analysis, 28, 131152. Zau, A. C., & Betts, J. R. (2008). Predicting success, preventing failure: An investigation of the California High School Exit Exam. San Francisco: Public Policy Institute of California.
Authors
SEAN F. REARDON is Associate Professor of Education and (by courtesy) Sociology at Stanford University, 520 Galvez Mall, #526, Stanford, CA 94305; sean.reardon@stanford.edu. His research
519
Reardon et al. focuses on the causes, patterns, and consequences of educational and social inequality. NICOLE ARSHAN is a PhD candidate in Administration and Policy Analysis at Stanford University, 520 Galvez Mall, Stanford, CA 94305; narshan@stanford.edu. Her research investigates how education policy reproduces or alleviates inequality. ALLISON ATTEBERRY is a PhD candidate in Education at Stanford University, 520 Galvez Mall, #509, Stanford, CA 94305; acma@stanford.edu. Her research focuses on education policy analysis and value-added modeling. MICHAL KURLAENDER is Associate Professor of Education Policy at the University of California, Davis, One Shields Avenue, Davis, CA 95616; mkurlaender@ucdavis.edu. Her research focuses on policies and interventions at various stages of the educational attainment process; in particular, dimensions of racial/ethnic and socioeconomic inequality in achievement, and postsecondary access and completion.
Manuscript received January 29, 2009 Final revision received July 22, 2009 Accepted July 22, 2010
520

Effect of Failing A High School Exit Exam On Course Achievement, Persistence and Graduation

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Judul Asli

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Effect of Failing A High School Exit Exam On Course Achievement, Persistence and Graduation

Diunggah oleh

Hak Cipta:

Format Tersedia

Educational Evaluation and Policy Analysis

American Educational Research Association

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

High School Exit Exam

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

High School Exit Exam

High School Exit Exam

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

High School Exit Exam

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

Proportion Who Took 11th Math > Geometry

Outcomes, by 10th grade minimum CAHSEE score.

0.051*** 0.016 (0.013) (0.011) 39,960 47,558

Effect of failing at least one section

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

F statistic p value on F test

F statistic p value on F test

0.015 (0.018) 0.021 (0.015) 2.285 0.131

0.012 (0.017) 0.037* (0.016) 1.155 0.283

0.020 (0.024) 0.006 (0.022) 0.172 0.678

0.016 (0.013) 0.018 (0.012) 0.017 0.897

0.013 (0.019) 0.003 (0.018) 0.120 0.729

0.002 (0.013) 0.050*** (0.012) 7.169 0.007**

0.007 (0.015) 0.038** (0.015) 5.007 0.025* (continued)

Mean 11th grade ELA CST score (standardized)

Effect of failing at least one section

Socioeconomic status (SES) Not free-lunch eligible

F statistic p value on F test

0.011 (0.019) 0.001 (0.014) 0.177 0.674

ELL status ELL

Native English speaker

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

F statistic p value on F test

High School Exit Exam

High School Exit Exam

Effect of failing at least one section

Effect of failing both sections

11th grade ELA CST score (standardized) n

0.032 (0.026) 14,924

Present in spring 12th grade n

0.013 (0.020) 13,994

0.036 (0.029) 9,397

Graduation (by spring of 12th grade) n

0.028 (0.025) 11,115

Downloaded from http://eepa.aera.net at Stanford University Libraries on February 15, 2011

11th grade CST higher than Algebra 1 n

0.052* (0.020) 14,430

0.026 (0.024) 8,677

11th grade CST higher than Geometry n

0.026 (0.017) 14,430

High School Exit Exam

Anda mungkin juga menyukai

0.002 (0.013) 0.050* (0.012) 7.169 0.007