Anda di halaman 1dari 8

Natural (quasi) experiments in education research

Sofia 12.05.2009

Why natural experiments


Education is a complex issue, many
overlaping factors have influence on its outcome

Difficult to distinguish between effects


Reverse causality problem High social cost of real experiments So what is natural experiment? More idea than technique...

What issues can be addressed by NE - examples


Equity of education funding Education production function what matters for outcome? Competition and school choice

Cheating in school
Returns to education

Typical ideas of natural experiment


To assess whether school matters for achievements independently on inherited human capital .... do research on siblings (twins)
Public versus private schools. Relligious schools as instrument Month of birth as instrument in returns to education research Reform implementing sequence (differences in time) considered as natural experiment

Estimating returns to education using different natural experiment techniques Andrew Leigh, Chris Ryan, Economics of Education Review 27 (2008) 149160
Authors suspect that traditionally measured returns to education are overestimated because people with higher SES choose to stay longer at school (reverse causality problem)

They apply two instruments: month of birth and change in compulsory school law (instrumental variables technique)
They found real reate of return to be close to 10%
Abstact: How much do returns to education differ across different natural experiment methods? To test this, we estimate the rate of return to schooling in Australia using two different instruments for schooling: month of birth and changes in compulsory schooling laws. With annual pre-tax income as our measure of income, we find that the na ve ordinary least squares (OLS) returns to an additional year of schooling is 13%. The month of birth IV approach gives an 8% rate of return to schooling, while using changes in compulsory schooling laws as an IV produces a 12% rate of return. We then compare our results with a third natural experiment: studies of Australian twins that have been conducted by other researchers. While these studies have tended to estimate a lower return to education than ours, we believe that this is primarily due to the better measurement of income and schooling in our data set. Australian twins studies are consistent with our findings insofar as they find little evidence of ability bias in the OLS rate of return to schooling. Together, the estimates suggest that between one-tenth and two-fifths of the OLS return to schooling is due to ability bias. The rate of return to education in Australia, corrected for ability bias, is around 10%, which is similar to the rate in Britain, Canada,the Netherlands, Norway and the United States.

Schooling and test scores: A mother-natural experiment Dave E. Marcotte, Economics of Education Review 26 (2007) 62964
Do school inputs matter for the outcomes? Usually hard to assess because of endogeneity of school reseources (reverse causality) Winter weather (average snowfall) used as proxy for instruction time (differences between years and between districts) Negative impact of snow (i.e. Instruction time) on the test scores in all fields of education
Abstract In the economics of education, no task has been more important or more difficult than identifying the relationship between school inputs and student performance. The literature on this topic has reached little resolution, largely owing to the endogeneity of school resources. In this paper I examine the effect of a vital but little studied component of the education production function: instructional time. To identify the impact of schooling on test scores I make use of the fact that variation in winter weather made non-trivial differences in the number of school days students received prior to taking the Maryland School Performance Assessment Program (MSPAP) exams. I find evidence that students who took exams in years with heavy snowfall performed significantly worse than their peers in the same school who took MSPAP exams in other years. I also find that performance in a subject with relatively inflexible curricula (mathematics) and students in earlier grades were most affected by snow. Both of these findings are consistent with the interpretation that education inputs in the form of instructional days improve students test scores.

Chicago experiment as described in Levitt,Dubner, Freakonomics.


Do teachers cheat on standardized tests? Experts re-analysed all tests performed by the students of 3rd to 7th grade (1993-2000) in Chicago school district They built an algorithm seeking for
Blocks of identical responses in the groupings of difficult questions Students giving good answers for difficult questions and bad answers for easy questions Classes that score significantly better one year than before and after

Evident and not-subtle cheating was found im 5% of classes As a result of report publication in 2002 the test was repeated in 120 schools, in most cases confirming the existence of cheating. 12 teachers have been fired, several received a warning.

Quasi-experimental estimates of class size effect in primary schools in Poland Maciej Jakubowski,Pawel Sakowski International Journal of Educational Research, 45,3 2006, Pages 202-215 Does class size matter for student achievements? But large classes are in cities. How to distinguish the city effect form classsize effect? But better school (having pupils with high SES) usually have small classes? How to eliminate this effect? Applying IV technique we obtain negative impact of class size (elasticity coeff. -0,6)
Abstract In this paper we analyze class size effects in the case of primary schools in Poland. We use two empirical strategies to avoid endogeneity bias. First, we use average class size in a grade as an instrumental variable for actual class size. This allows us to control for within school selection of pupils with different abilities to classes of different sizes. Additionally, we estimate fixed effects for schools to control for differences between them. Second, we exploit the fact that there is an informal maximum class size rule. We estimate class size effect only for those enrollment levels where some schools decide to add a new class and thus dramatically lower class sizes. For such enrollment levels variance of class size is mainly exogenous and we argue that this allows estimation of quasi-experimental class size effects. In this case we again use average class size as an instrument with enrollment as a key control variable. Using both strategies we obtain similar findings. We found that the positive effects observed with OLS regression disappear when we use instrumental variables. If we avoid endogeneity bias, then class size negatively affects student achievement. However, this effect is rather small. We discuss methodology, possible bias of results and the importance of our findings to current policy issues in Poland.

Anda mungkin juga menyukai