Sunday, December 10, 2006

Folk positivism

(or So there, Michael Bérubé! and other pseudo- philosophical rants)

With Michael Bérubé's challenge to his fellow nominees in the 2006 Weblogs Award Best Educational Blog category (hey, see that icon in the left margin? go vote for us now!), I'm going to ponder one of the problems I'm considering in Accountability Frankenstein. To wit, how the heck can we create accountability systems rooted in distrust of school officials and teachers (and students!), while simultaneously crafting systems that have a fairly blind trust of tests. That trust in test instruments undermines one of the main arguments for accountability, transparency. This is not a claim that psychometricians are absorbed in secrecy—well, not generally, though I understand Bill Sanders still keeps his value-added algorithms secret as a proprietary secret and plenty of state departments of ed fail to publish the psychometric data on tests, lest someone like Greg Camilli and Sadako Vargas discover the fudging that frequently happens in state testing systems—but an argument about the limits of test scores to tell us what children can do. In contrast to most other areas of expertise, the core purposes of school accountability are transparency and equity. Yet the routine of testing makes such knowledge virtually impossible, creating one or more layers of abstraction between what children do and what state governments say children do. That conflict has its roots in the development of professional expertise in testing early in the twentieth century. The end result of a century of the standardized testing industry is in tension with transparency. It raises a serious question whether testing technocracies are consistent with the democratic demands of accountability.

Others have written about technocracy in education. Art Wise's 1979 book Legislated Learning has the dangers of technocracy as one of its main themes, and Harry Wolcott's Teachers and Technocrats (1977) proposes that education is a moiety system divided into teachers and technocrats, if one looks at the creation and disintegration of efforts to apply the planning-programming-budgeting system (PPBS). And others have looked at technocracy more broadly, from Walter Lippmann's proposals that society be run by a small group of activists enlightened by neutral expert analysis to John Dewey's response (that we need the experts educating and informing the population, which can then become a Great Community) to the modern debates among folks such as Steven Brint, Frank Fischer, and Stephen Turner. There is a small but substantive literature on the growth of professions and the role of expertise-linked occupations in setting and implementing social policy. Based on it and the history of the testing industry, one might be tempted to attribute the role of tests to the growth of psychometrics as an occupation. And if we stopped in 1930, that supposition might be true.

Yet there is something a little different about the worshipping of test scores; it is less the power and authority given to psychometricians than the social assumptions made about tests. As Haney, Madaus, and Lyons wrote in The Fractured Marketplace for Standardized Testing (1993), a variety of market and political pressures has led to the corrupt use of tests (e.g., one NYC principal who tried to decide on valedictorians via SAT scores), so much that trained assessment experts casually acknowledge that the uses of standardized tests commonly fall outside their validated purposes. Some trained in assessment are the harshest critics of much modern standardized test use (Bob Linn is probably the shining star in that constellation), and yet their voices are largely ignored. Why?

Martin, Overholt, and Urban's Accountability in American education: A critique (1976) provides an alternative explanation: it is not so much in the professionalization (which they didn't discuss) as the intellectual history of positivism that leads accountability advocates to believe that test scores are concrete and consequences will drive improved instruction in a behaviorist manner. In referring to Auguste Comte and 19th century positivist arguments, the authors assert that there is a direct line from Comtean positivism to the uses of testing in the 1970s and Leon Lessinger's 1970 arguments for accountability.

While I understand the appeal of this argument, and I've seen similar arguments in other contexts, I am less than convinced, for both theoretical and evidentiary reasons. Theoretically, it is perhaps a mild inconsistency for postmodernist critiques of positivism to assert a linear intellectual path for anything, let alone a genealogy of contemporary beliefs in the utility of statistics. I'm not saying that Martin, Overholt, and Urban were being all postmodernist 30 years ago, but since they're using post-Romantic continental philosophers, there is this guy named Nietzsche who talked about the problems with asserting linear intellectual legacies for anything. Second, it's hard to discern where Comtean positivism would be more influential than American pragmatists, who (Dewey among them) were confident that this new thing called science could be used to create beneficial social policies.

But theory isn't the real reason why I'm skeptical of this linealogy. Fundamentally, it requires considerable faith to believe that politicians and others who advocate high-stakes accountability are well-read in 19th century philosophy, know that much of behaviorist literature, or even can put forth anything more than bluster to argue the utility of test scores as a direct reflection of reality. What we see today is a folk version of positivism, the rough-hewn confidence that test scores must "mean something." Arguments about the construction of social problems don't really help us here, because this is the classic case of ideas diffusing in different directions without any Powers That Be giving them direction. In short, I'm not sure we have a way to wrap our heads around folk positivism. But I think we need to.

7 comments:

Dan W. Butin said...

Sherman,

I think this folk positivism can be easily linked to how we as a culture view ourselves and our world as stable, coherent, and unitary. To bring this back to schools, I always return to the educational sociologists Carl Simpson and Susan Rosenholtz (e.g., Simpson, Carl and Susan J. Rosenholtz. 1986. Classroom Structure and the Social Construction of Ability. Pp. 113-138 in J.G. Richardson. (ed.) Handbook of Theory and Research for the Sociology of Education. New York: Greenwood Press.) who showed (very nicely, in my opinion) that classrooms that had “uni-dimensional” characteristics (versus classrooms that had “multi-dimensional” characteristics) created students who more often viewed ability as inherent and stable. And ability, as they point out in their work, is just a proxy for IQ; which brings us right back to your argument. My dissertation looked at two private elementary schools without grades and I found an immense difference in how these children assumed “success” to be. They certainly saw “winners” and “losers” (going completely against the assumptions I walked in with); e.g., they could immediately tell me who was the fastest runner, the best painter, the one with the cleanest desk, etc. But they had no clue when I asked them who was the smartest child in the room. It did not compute, if you will.

Sherman Dorn said...

Dan,

Thanks for the suggestion. I'm trying to figure out how unitary characteristics necessarily imply quantification. I could imagine two counterexamples: where we think there is a unitary construct—let's call it "ethics" to get it out of the cognitive realm—but where we can't quantify it. I suspect that many folks would say that someone's ethical character is unitary and unquantifiable. (We'll skip all this nasty teleological-deontological stuff for now.)

On the other hand, I can draw from history to find multidimensional constructs that people think are quantifiable, and we can turn either to phrenology or to contemporary test scores (math, reading, science), where many people have no problems seeing characteristics as multdimensional but still quantifiable.

Am I missing something obvious?

Dan W. Butin said...

Ah, Sherman, I see the confusion.

A uni-dimensional classroom (for Simpson and Rosenholtz) had the teacher in control of curriculum, organization, and timing, had public and oft-repeated grades, and had whole group instruction. The point was that ability was thus defined along a single, unitary, public, and seemingly consensus-decided criterion. So it’s not between being quantifiable and non-quantifiable. It’s between a singular and multiple basis for quantification. To put it grossly, it’s between saying “you’re either with us or against us” or saying “it depends.” To be more formal, a uni-dimensional classroom-with its undifferentiated task structure and formal grading-forces students to rely on adult and external authority in making decisions about their sense of self-competence and ability. A multi-dimensional classroom, alternatively, allows students to form their sense of identity by using across-domain comparisons and peer group comparisons. Multiple intelligences, if you will. I think about it by thinking about how most normal people look at the IQ. If I tell you my IQ is 100 and yours is 120, not many people will say that I am smarter. Sure, they may use some folk sentiment of “street smarts”, but IQ is IQ is IQ; most people think of it as objective, stable, unitary, and internal. Just like, I should note, all psychometricians tell us it is.

Does this help??

Anonymous said...

A uni-dimensional classroom (for Simpson and Rosenholtz) had the teacher in control of curriculum, organization, and timing, had public and oft-repeated grades, and had whole group instruction. The point was that ability was thus defined along a single, unitary, public, and seemingly consensus-decided criterion. So it’s not between being quantifiable and non-quantifiable. It’s between a singular and multiple basis for quantification. To put it grossly, it’s between saying “you’re either with us or against us” or saying “it depends.”

Dan,

Thanks for following up. As my entry yesterday indicates, I'm heading in a direction more rooted in political models than in personal experiences and perspectives.

It may be my background as an historian rather than contemporary qualitative observer, but I think you may be putting too much weight on what happens inside classrooms to assert that the dynamic inside the class are more powerful than other factors in shaping how we think about society, including things such as achievement. I know there's the literature on this stretching back to Dreeben (for structural-functionalists) or Bowles and Gintis (for reproduction), let alone McLaren and others. But I suspect it's more complicated, and not just because most adults have been through a few dozen classrooms and seen a variation in styles and modes. Even the flexibility of our mental scripts (see Mary Metz's famous "Real School" book chapter) suggests that there is slippage between schooling and our mental models.

Anonymous said...

Sherman said,
It may be my background as an historian rather than contemporary qualitative observer, but I think you may be putting too much weight on what happens inside classrooms to assert that the dynamic inside the class are more powerful than other factors in shaping how we think about society...

But this is just what systems theorists have been saying for the past 50 years! You don't have to be a historian to notice it.

Robert Thompson said...

Though, a lot of folk suggestion but still a gap...we need to cover up.

Muazzam Mehmood
Study Stuff Downloads for Brain Hunters
Phd & MPhill Discussions over Study problems, new Ideas & Expertise
Tutorial: Cryptography & Linux Security

Unknown said...

This could be something very new to everybody but everyone believes that the folk has it's own values. Starting from the Online Blackjack Learning to any kind of other Education.