NYC ACHIEVEMENT GAINS COMPARED TO OTHER LARGE CITIES SINCE 2003

Changes in NAEP scores 2003 -2011

Leonie Haimson & Elli Marcus Class Size Matters January 2012 www.classsizematters.org

NAEP Scores: Why are they important?

The National Assessment of Educational Progress (NAEP) is largest

continuing assessment of the knowledge and abilities of American students.

NAEP assessments are given by the federal govt. every two years to

statistical samples of students, change little over time & are low-stakes, and so can be used as a reliable metric to compare achievement trends among states and urban districts.

The Trial Urban District Assessment (TUDA) has been given in 10 large cities

incl. NYC since 2003 in four categories: reading and math in 4th and 8th grades.

What follows is an analysis of the changes in NYC NAEP scores since 2003,

when Bloombergs educational policies were first implemented, compared to changes in scores in the 9 other cities, plus large cities in general (w/ at least 250,000 inhabitants).

How did we compare trends among the large urban districts?

Since overall scores can change depending on changes in

student population, we compared changes in scores since 2003 for six major NYC subgroups (white, black, Hispanic, Asian, free lunch and non-free lunch students) compared to their peers in other large cities.
Only major subgroups whose results we did not compare were

students with disabilities and English language learners, since rates of identification and exclusion from testing differ widely among the ten cities.
Our comparisons give insights into where NYC stands

nationally, and allows us to assess the reality of DOEs claims of great improvement.
These comparisons give insight into where NYC stands nationally and provides a robust examination of the DOEs claims o

When 2011 NAEP scores were released this fall, NYC DOE claimed great progress *
Claim: NYC students have improved significantly on

three of the four math and reading tests between 2003 and 2011.
Reality: This is true in nearly every city tested since

2003.
Claim: .since 2003, the gap between black and white

students in New York City has narrowed on all four exams, and on all four since 2009.
Reality: There has been no statistically significant

narrowing of the achievement gap between any of the racial/ethnic groups in NYC in any subject tested since 2003.
*Source: NYC DOE Press release , December 7, 2011

DOEs other unfounded claims of progress

Claim: On all four tests, low-income students in NYC now outperform their

peers across the nation, and thats a reason to be proud, said Chief Academic Officer Shael Polakow-Suransky.
Reality: In 2003, NYC low-income students already outperformed their

peers nationwide in all four categories tested, and since then have made fewer gains than peers in several other cities.
Claim: By the gold standard for measuring academic progress, our students

have made impressive gains since 2003especially compared to their peers across New York State and the nation, said Chancellor Walcott.
Reality: When measured across subgroups, NYC students have made

less academic progress since 2003, compared to their peers, in every other city except one.
*Source: NYC DOE Press release , December 7, 2011

NYC comes in 2nd to last among all 10 cities + large city category when NAEP score gains are averaged across 6 subgroups*
18 16 14 12 10 8 6 4 2 0 Cleveland NYC Charlotte large city Chicago SD Houston DC LA Boston Atlanta

15.3 10.3 10.4 10.9 12.4 12.9

7.9 4.3 1

8.8

8.9

*Subgroups include white, Hispanic, Black, Asian, free-lunch & non-free lunch Test score gains since 2003, averaged across all four categories: reading & math in 4th & 8th grades

Scores by subgroup: In NYC, Black students scores

rose less than their peers in most other cities
In 4th grade reading, NYC black students dropped from

tied for 3rd to 4th place among all cities since 2003.
In 8th grade reading, NYC blacks were tied for 2nd and

dropped to 3rd.
In 4th grade math, NYC blacks dropped from 3rd to 4th

place.
in 8th grade math, NYC blacks went from 3rd to tied for

4th place.

NYC scores by subgroup: Black Students

4th and 8th grade reading and math gains in average scale scores since 2003

change in 4th grade reading scores 2003-2011

23 20

Change in 4th grade math scores 2003-2011

14 6 7 7 8 8 10 10 10 3

10

-4

change in 8th grade reading scores 2003-2011

12 9 1 2 2 3 3 4 6 0

change in 8th grade math scores 2003-2011

21 9 4 9 10 12 12 15 15 21

-5

-4

Subgroup: White students fell sharply behind their peers in other large cities since 2003, especially in 8th grade reading & math
In 4th grade reading, NYC white student scores dropped

from 5th to 7th place.

In 4th grade math, NYC white students dropped from 5th

place to 8th place.

In 8th grade reading, NYC white students dropped from

tied for 2nd to 7th place, and came in last in score gains. from 4th to 8th place and came in last in score gains.

In 8th grade math, NYC white student scores dropped

NYC scores by subgroup: White Students

change in 4th grade reading scores 2003-2011
16 4 5 6 7 8 8 9 5 7 8 10 11 11 15 4 1 1 1

change in 4th grade math scores 2003-2011

21

2 -1

Change in 8th gr reading scores 2003-2011 white students

13 5 1 5 6 6 7 8 10

change in 8th gr math scores 2003-2011 white students

3 8 10 10 11 14 16 16 18 20

Subgroup: Hispanic Students fell sharply behind

peers since 2003
In 4th grade reading, NYC Hispanic students dropped from

1st place among large cities to tied for 4th.

In 4th grade math, NYC Hispanic students dropped from

third place to sixth place among other large cities.

In 8th grade reading, NYC Hispanic students dropped from

2nd to 5th place, with a net negative change in scores.

In 8th grade math, NYC Hispanic students came in last

place in score gains, falling from third place to 7th place.

NYC scores by subgroup: Hispanic Students

Change in 4th grade reading scores 2003-2011
28 10 13 17 6 -2 7

Change in 4th grade math scores 2003-2011

7 7 9 9 10 13 18 19

-5

change in 8th grade reading scores 2003-11

12 6 -1 0 1 -8 7 7 8 13

change in 8th grade math scores 2003-2011

15 7 9 10 11 12 15 17 19

Asian Students were the only NYC group to make substantial gains compared to peers in other cities.
4th grade reading, NYC Asian student scores dropped

Subgroup:

from first place to second place, and placed fourth in overall score improvement among large cities.
In 4th grade math, Asian student scores dropped from

second place to third place among large cities.

In 8th grade reading, NYC Asian student scores moved

up from third place to second place among large cities.

In 8th grade math, NYC Asian student scored moved up

from third place to second place

Subgroup: Asian Students

Change in 4th grade reading scores 2003-2011 Asian students
15 7 1 large city 2 SD 3 3 3 LA Charlotte large city 4 10 6 10

change in 4th grade math scores 2003-2011 Asian students

16

NYC

Boston

NYC

Charlotte

LA

SD

Boston

change in 8th grade reading scores 2003-2011 Asian students

12 9 6 7 10 10

change in 8th grade math scores 2003-2011 Asian students

18 15 11 15 19 20

Chicago -4

Boston

SD

NYC

large city

LA Chicago Charlotte SD large city NYC Boston LA

Changes in demographics: Asian student pop rising faster in NYC than elsewhere; otherwise progress on NAEPS would have
been even smaller
Asians as % of total students tested 4th grade reading
20 15 10 5 0 0 Atlanta Boston Chicago DC Houston LA large city NYC 1 0 8 5 3 1 2 3 3 6 6 5 8 8 2002 2011 19

Asians as % of total students tested 4th grade math

20 18 16 14 12 10 8 6 4 2 0 19 12 8 8 4 5 0 1 6 3 1 2 2 3 6 6 7 8 2003 2011 18 15

NYC scores by subgroup: Free Lunch students had only middling gains
In 4th grade reading, NYC free lunch student scores remained

in 1st place but placed behind five other large cities in gains since 2003.
In 4th grade math, NYC free lunch student scores dropped from

second place to third place, and placed fifth in score gains among large cities.
In 8th grade reading, NYC free lunch student scores remained

in 1st place but placed behind three other large cities in score gains.
In 8th grade math, NYC free lunch student scores dropped from

1st place to 3rd place.

Subgroup: free lunch

change in 4th grade reading scores 2003-2011
11 6 6 7 7 7 8 8 8 6 1 -2 7 9 9 9 12 10 10 11 12

change in 4th grade math scores 2003-2011

16

change in 8th grade reading scores 2003-2011

11 7 8 6

change in 8th grade math scores 2003-2011

9 3 11 11 13 14 15 17 17 19 21

5 2 0 3

-4

NYC non-free lunch students made the smallest gains of any city in every category; and dropped sharply at 8th grade
In 4th grade reading, NYC non-free lunch students fell from 1st place to 2nd

place.
In 4th grade math, NYC non-free lunch students fell from 2nd place to 3rd

place.
In 8th grade reading, NYC non-free lunch student scores dropped 11 points

the only city where scores dropped and fell from 1st place to 8th place.
In 8th grade math, NYC non-free lunch students dropped seven points the

only city where scores dropped -- and fell sharply from 1st to 8th place
In 8th grade reading and math, basic and proficient levels of non-free lunch

also dropped sharply.

Subgroup: non-free lunch

change in 4th reading scores 2003-2011
28 14 1 3 6 9 15 16 16 18 5 8 10 14 16 16 18 19

change in 4th grade math scores 2003-2011

29

change in 8th grade reading scores 2003-2011

24 19 5 6 8 10 10 11 11

change in 8th grade math scores 2003-2011

38 20 21 24 27

10

14

14

15

-7 -11

NYC is ONLY city where proficiency levels in 8th grade reading and math have dropped for non-free lunch students
8th grade math for nonfree lunch students
90 82 80 70 60 50 40 30 20 10 0 49 41 75 90 80 70 60 87 79 100

8th grade reading for nonfree lunch students

2003 2011

50 40 30 20 10 0

48 34

2003 2011

at or above basic

at or above proficient

at or above basic

at or above proficient

All other cities made gains in 8th grade proficiency in reading & math for non-free lunch students, while in NYC they dropped
change in % non-free lunch students at or above basic & proficient in 8th grade math 2003-2011
40 30 20 10 0 -10 -20 NYC -7 -8 Chicago Charlotte large city Boston Houston SD DC Atlanta LA diff proficient 7
10 15

37 29
22 24 30 25

24 23

11

14

14

15 16

16 diff basic

change in % non-free lunch students at or above basic & proficient in 8th grade reading 2003-2011
25 20 15 10 5 0 -5 -10 -15 -20 13 2 NYC -8 -14 5 8 6 13 7 9 15 10 10 11 17 12 diff basic diff proficient Chicago Charlotte Boston large city SD Houston DC Atlanta LA 19 19 21 22

Summary of findings:
When analyzing subgroup performance, NYCs relative progress since

2003 compared to other large cities has been mediocre to poor.

NYC came in 2nd to last in NAEP gains among 10 cities and large city

category tested since 2003 when averaged across six subgroups.

All NYC subgroups fell in ranking, compared to peers in other large cities,

with White, Hispanic and non-free lunch students dropping most sharply.
White students made the smallest gains compared to their peers in other

cities in both 8th grade reading and math; Hispanics in 8th grade math. subject or grade;

Asian students were only NYC subgroup to advance in ranking in any

NYC was only city in which non-free lunch students scored lower in 2011

than in 2003, in both 8th grade reading and math, and their proficiency levels also dropped sharply.

What about mayoral control?

Two districts under mayoral control made least progress & on average, cities with elected school boards have done better
18 16 14 12 10 8 6 4 2 0 Cleveland NYC Charlotte large city Chicago SD Houston DC LA Boston Atlanta

15.3 12.4 10.3 7.9 4.3 1 8.8 8.9 10.4 10.9 12.9

Cities with mayoral control since 2003 or earlier in red; DC has had mayoral control since 2007.

What else do these results suggest?

The administrations aggressive free-market strategies of

high-stakes accountability, school report cards, fair student funding, principal empowerment, and the closing of more than one hundred schools & the opening of more than 400 new schools & charters, while allowing class sizes to increase sharply, have not worked to increase achievement compared to cities elsewhere.
In fact, the relative positions of white, Hispanic and non-

free lunch students in NYC have all dropped substantially, with the declines especially sharp at the 8th grade level.