Numbers Don't Always Equal Evidence: Revisiting IPI's Argument for School Choice in Rockford

On February 23^rd, Joshua Dwyer wrote an article for the Illinois Policy Institute in which he argued that policymakers should focus on increasing school choice in responding to Rockford's failing school system. First, I want to make clear that I am agnostic about school choice policy as lever for improving the education system, as I feel that neither "side" of the argument has adequately demonstrated the effectiveness (or lack thereof) of school choice as a lever for system-wide improvement. However, I am not agnostic about the appropriate use of evidence and argument. This response is not about school choice, but is about misusing evidence to build a poor argument; which is fitting, because Mr. Dwyer's argument wasn't really about school choice either. 

Dwyer’s argument that Rockford is second highest in number of low-performing schools, while being the third highest in population (152,871 pop.) blatantly exploits the fact that Aurora, the city second highest in population (197,899 pop.), is split into two districts while Chicago and Rockford are only single districts. Most cities in Illinois have a single public school district, but Elgin is split into an east and a west district. So, while Dwyer is correct that Rockford is the third largest city with the second most low-performing schools, it is also true that Rockford is the second largest district with the second most low-performing schools.

Dwyer’s misuse of the numbers continues as he laments the low test scores in Rockford. In 2013, the percentage of students in Rockford meeting and exceeding standards dropped dramatically, as did the same percentage for most of the state. This is because the “cut scores” for determining whether a student taking the ISAT met or exceeded standards changed in 2013. Prior to the change in cut score, according to the Illinois Interactive Report Card, 60% of Rockford’s 3^rd graders met or exceeded standards in reading, and 75% of Rockford’s 3^rd graders met or exceeded standards in math.
The biggest, and most unforgivable, transgression in the article is the assertion that the lack of school choice is the only explanation for Rockford’s crisis (ignoring the fact that Chicago is home to some of the most aggressive charter expansion in state, perhaps the country; but, as Dwyer points out, Chicago has 45% of the lowest performing schools in the state). 78.8% of Rockford’s students are identified as “low-income,” up 26.2% since 2000. In fact, while only 36% of Rockford’s low-income 3^rd grade students met or exceeded standards in reading, 70% of Rockford’s 3^rd grade non-low-income students met or exceeded standards in reading. In mathematics, the split was 32% to 65%; and in science, 41% to 64%.  According to the 2013 data from the Illinois State Board of Education, 23,094 of Rockford’s 28,663 students are identified as eligible for free or reduced-price lunch; and, only 277 of Rockford’s students attend a school where less than 50% of the students are eligible for free or reduced-price lunch. While, as Dwyer points out, the majority of Rockford’s students are not performing at grade level, he does not acknowledge that the majority of Rockford’s students are also low-income, or that the students that are not low-income are, for the most part, performing at or above grade level.

Mr. Dwyer presents all of his evidence of Rockford’s failure to educate students in order to arrive at the predetermined political argument that Rockford needs more school choice — although none of the evidence he provides supports the expansion of choice in any way, and none of his evidence is in any way related to school choice. Unfortunately for Mr. Dwyer, the evidence that school choice improves outcomes for students in choice schools is inconclusive (Rouse & Barrow, 2008; Usher & Kober, 2011). The evidence that school choice improves outcomes for students not in choice schools is also inconsistent (Linick, in press). However, there is a voluminous amount of evidence that demonstrates that a student’s socioeconomic status or family’s financial situation matters a great deal (Berliner & Biddle, 1995; Coleman et al., 1966; Newman & Chin, 2003; Sacks, 2007).

The existing base of evidence suggests that it is possible that increasing school choice in Rockford may improve district-wide academic achievement, but it is also suggests that school choice may hinder academic achievement, or have no effect on academic achievement at all (Linick, in press). If the Illinois Policy Institute is truly invested in improving “real-life” outcomes like high school and college graduation, employment, incarceration, and income for Rockford’s students, the evidence suggests that the better bet is focusing their attention on improving the financial stability of Rockford’s families, not promoting school choice. Of course, the Illinois Policy Institute is an “organization that advocates for the free market ideas developed by the Illinois Policy Institute,” apparently, regardless of whether those ideas align with the evidence or not.
by Matt Linick


References

Berliner, D. C., & Biddle, B. J. (1995). The manufactured crisis: Myths, fraud, and the attack on America's public schools. In C. Kridel (Ed.), Classic edition sources, education (Fourth ed., pp. 190-194). New York: McGraw Hill.

Coleman, J. S., Campbell, E. Q., Hobson, C. J., McPartland, J., Mood, A. M., Weinfeld, F. D., & York, R. L. (1966). Equality of educational opportunity. Washington, DC: U.S. Department of Health, Education, and Welfare.

Linick, M. A. (in press). Measuring Competition: Inconsistent definitions, inconsistent results. Education Policy Analysis Archives, 22 (33).

Newman, K. S., & Chin, M. M. (2003). High stakes: Time poverty, testing, and the children of the working poor. Qualitative Sociology, 26(1), 3-34.

Rouse, C. E., & Barrow, L. (2008). School vouchers and student achievement: Recent evidence, remaining questions. New York: National Center for the Study of Privatization in Education.

Sacks, P. (2007). Tearing down the gates: Confronting the class divide in American education. Los Angeles, CA: University of California Press.

Usher, A., & Kober, N. (2011). Keeping Informed about School Vouchers: A Review of Major Developments and Research. Washington, DC: Center on Education Policy.