Multiple issues in multiple measures

In his July 30 statement at the National Press Club, House Education and Labor Committee Chair George Miller said that his plans for reauthorizing the No Child Left Behind Act included the addition of multiple measures, an incantation that has provoked more Sturmunddrang in national education politics than if Rep. Miller had stood at the podium and revealed he was a Visitor from space. While Congress is in recess this month, the politics of reauthorization continue. I'll parse the debate over multiple measures or multiple sources of evidence, and then I'll foolishly predict NCLB politics over the next month or so.

The different issues

Calculating AYP

At one level, the discussion appears entirely to focus on the determination of adequate yearly progress. Add measures and you "let schools off the hook," according to Education Trust (with similar noises from the Chamber of Commerce's Arthur Rothkopf [RealAudio file-hat tip]. No escape hatch, promised Miller when asked. Maybe if you add measures, there are more ways to fail AYP, as one reporter noted at the press conference; not so, said Miller, for we'll figure out some way so that the extra measures only get you over the hump if you're almost there. Since AYP is the largest chunk of NCLB politics, all of the talking points are familiar. In the end, this piece of the debate will get bundled into the most likely package that includes growth measures.

Teaching to the test

As the Forum on Educational Accountability has argued, as well as last week's letter by civil rights groups, narrow measures of learning tend to distort how schools behave in several ways, from narrowing the taught curriculum to teaching test-taking skills and engaging in various forms of triage. One argument in favor of multiple sources of evidence is Lauren Resnick's old one, that a better test is likely to encourage better behavior by schools, both in terms of better assessments and school indicators that penalize schools for triage. To the extent that more input dilutes the incentive for systems to attend to single indicators, that may be true. On the other hand, multiple sources of evidence by themselves will not eliminate the corrupting effect of brain-dead accountability formulas, and to some extent the resolution of the debate over AYP can blunt the effect of multiple sources of evidence. On the third hand, I suspect most of those who support multiple sources of evidence are adults and prefer some improvement over none. Including multiple sources of evidence will not eliminate the deleterious side-effects of high-stakes testing, but they should ameliorate them.

Improving the quality of exams and their cost

Connecticut's NCLB lawsuit is based on the claim that the federal government has not provided enough support for the state to develop its performance-heavy exam for all the required grades. The feds allegedly told Connecticut that it doesn't need to use the performance-heavy exams, claiming that an off-the-shelf commercial test system would work just fine. After investing state money and political capital in the performance exams, Connecticut officials were rather peeved. The Title I Monitor nailed this issue in May, noting that the argument over multiple measures is in part a matter of the quality of assessments and cost. The Monitor also noted a level of denial in the US Department of Education that should be familiar to Bush-watchers:

[A] senior ED staffer acknowledged the benefits of states using varying assessment formats compared to a single test, but challenged the idea that costs and timelines are a barrier to states developing tests with multiple formats.

And the escalation in Iraq is currently providing an environment conducive to the reconciliation of factions. Right. Officials from a variety of states and a number of players in Washington agree that NCLB has essentially stressed if not broken the testing industry's credibility and infrastructure, and the inclusion of multiple measures is part of the negotiations over how much Washington will pay for better assessments.

Reframing accountability

One doesn't have to agree with George Lakoff's version of framing to recognize that the politics of accountability are driven by assumptions about the need for centralization and authoritarian/bureaucratic discipline. These themes are obvious in the dominant inside-the-Beltway narrative about NCLB: We can't trust the states. The best argument for this position is Jennifer Hochschild's thesis in The New American Dilemma (1984), a claim that sometimes we need a non-pluralistic tool to advance democratic aims, a contradiction she saw in desegregation. But we don't have an open debate about this dilemma. We didn't have it about desegregation, and we certainly don't have it about accountability.

Instead of reflecting some honesty about policy dilemmas, the arguments defending No Child Left Behind today are generally at the soundbite level. A common metaphor used by many supporters of NCLB relies on time, such as the Education Trust's organizing an administrators' letter several years ago warning against a thinly veiled attempt to turn back the clock. A step forward is another phrase that the same letter uses to describe NCLB, and Education Trust's response to the Forum on Educational Accountability proposals describes them as a giant step backward. This is an ad hominem metaphor: It says, "Our opponents are Luddites. They are not to be trusted to defend anything except their own narrow and short-sighted interests."

The other language commonly used by NCLB supporters is a simple assertion that they own accountability. Anyone who disagrees with them is against accountability. Together, these bits of accountability language imply that there is one true accountability and that NCLB skeptics like me are apostates or blasphemers. Pardon me, but I don't believe in an accountability millennium.

To shift the debate away from accountability millennialism, critics of NCLB have to provide a counter-narrative. Both the August 7 civil rights-group letter and the August 13 researchers' letter (or the letter signed by mostly researchers) describe the current NCLB implementation with words such as discourage, narrowed, and fail. In its August 2 recommendations for reauthorization, the Forum on Educational Accountability uses the words build, support, and strengthen. The Forum and August 7 letter also use a single word to describe the best use of assessment: tool. In their recommendations, the Forum and its allies use an architectural metaphor: we need to strengthen the system while keeping it mostly intact. The criticisms directed against multiple-choice statistics aren't part of that story, though I suppose a purist would insist on that, some how described as undermining foundations, eroding under the foundation, blowing out a window, or somesuch.

I don't know to what extent the debate over multiple measures will shift debate, but it is potentially the most far-reaching of the consequences of the letter.

Where we're headed in the short term

My guess is that Miller's September draft will bless consortia of states that develop assessments with more performance, authorize funding for more (but not all) of that test development if small states work in consortia, and promise to pay for almost all of the infrastructure needed to track student data.

We will also see the true character of education advocates in Education Trust and the Chamber of Commerce. The Education Trust is now under the greatest pressure of its existence over both growth measures and the issue of multiple measures. In Washington, almost no one gets their way all the time. How people negotiate and handle compromise reveals their true character.

Cross-posted from Sherman Dorn