Thursday, November 01, 2012

Won't Step Up: Idealism, Evidence, and Entertainment

The most interesting thing about the recent box-office bomb of Won’t Back Down wasn't just how big-money ideologues backed such a dog of a movie, but that audiences handed a resounding “two thumbs down” to this attempt to alter the time-tested teacher-as-hero formula that has been so profitable to Hollywood over the years.

In the 1980s and 90s, movies about troubled schools had a common theme:  get tough on students.  Stand and Deliver, Lean on Me and Dangerous Minds, for instance, reflected the common wisdom that charismatic educators could force those unruly urban (and usually minority) kids to shape up, allowing the US to compete with Japan in the global economy.

But the heroic teacher myth has recently been flipped to the teacher-as-villain, as indicated by a new crop of films that pounce on teachers for not improving the inner cities.  Recent Hollywood movies such as Won’t Back Down and Bad Teacher are buttressed by documentaries like Waiting for Superman, The Lottery and The Cartel in promoting the notion that education failure is due primarily to bad schools and, more specifically, to those who teach in them.  In this narrative, without finding ways of identifying, sanctioning, and firing ineffective teachers (who all happen to work at schools with lots of disadvantaged kids), the US will be unable to compete with India and China in the new global economy.  

This embrace of teacher effectiveness and rejection of student disadvantage as the primary factor influencing student outcomes has become something of a theme with the self-described “reform” crowd, as when Indiana Governor Mitch Daniels made the amazing claim that “Teacher quality has been found to be twenty times more important than any other factor, including poverty, in determining which kids succeed.”  The “Rhee-formers” believe that good teaching trumps the “demographic determinism” of bad background factors.

Improving teacher quality is, of course, a laudable goal.  But what does the evidence actually tell us? 

Going back at least to the Coleman Report of the mid-1960s, researchers have consistently found that non-school factors are the primary — if not overwhelming — consideration in predicting student outcomes (see, e.g. here, here, here, here, here).  Decades of research suggest that school-factors may explain perhaps one-third of the variance in student achievement, while teaching itself may influence one-fifth, at best.  And it’s not at all clear that exceptional teaching has the sustained impact over time that we might wish.  Great teaching can make a difference, and it does for many students.  But for many more, the environment soon re-asserts itself.

Certainly, it’s much easier for policymakers to mandate better teaching than it is to mandate better parenting.  We can’t legislate that all parents care about their child’s education, take prenatal vitamins, or limit exposure to lead… and that wouldn’t make for a very good movie.

But what we see in Won’t Back Down is a willful disregard of evidence in favor of Rhee-formist idealism — the appeal of simplistic solutions to complex problems.  This is not to say that we shouldn’t try to improve the educational experiences of poor kids.  But idealism and ideological desire cannot counter data.  Without a foundation in facts, any reforms are likely to go the way of the many other efforts that are based on good intentions and ideological assumptions, but which ignore evidence.  (Recall the Gates Foundation’s expensive and ineffective foray into smaller schools.)  There will be little overall impact on academic outcomes from reforms that neglect the primary problems… outside of schools.

Divorced from evidence, idealism may make for a good movie, but it is not the best strategy for improving educational outcomes. 


Anonymous said...

One of the hardest elements to deal with, in terms of public perception of this issue, is the overwhelming weight of valid stories involving that one inspirational teacher who was instrumental in turning around a troubled kid or mentoring a future scientist.

These happen far to infrequently to make them the basis for educational reform, in truth, but they happen too often for them to be ignored, either in the reform efforts or in the public debate around those efforts.

Anonymous said...

Agreed. And it doesn't help that Hollywood (funded by agenda-driven ideologues) exaggerates the significance of these exceptions.

Anonymous said...

To be fair, for all that I grant Hollywood glorifies the exceptions, I think it is a minor contributor.

We tend to look at attainment and go backwards: See these people who accomplished X, Y, and Z? Each of them could point to an inspiring teacher who changed their lives.

Therefore, to change people's lives in such positive ways, we need to make sure that every kid has an inspiring teacher of this ilk in their life!

We don't notice, as we make this analysis, that that same inspiring teacher had 30 kids (elementary) or 120 kids (secondary) every year for a decade, and had that huge impact on two or three of the kids, but not all or even many of them.

Yes, it is still more of an impact than most of the other teachers had, but it is not enough for us to build curriculum or educational design or teacher training around.

We need to cherish and encourage those inspirational teachers, but we need to know that we are not going to have one in every school, let alone every classroom.

So, given that truth, we need a plan which takes advantage of them when we have them, while not dependent upon them or expectant of teachers that they will fill that role.

Michael Culbertson said...

The anonymous comment about looking backward from success to effective teachers reminded me of an article I read recently (Reichard, "Evaluating Methods for Estimating Program Effects," American Journal of Evaluation, 2011) that distinguishes between two causal questions: What is the effect of a given cause? And, what is the cause of a given effect?

Linking successful students with inspiring teachers answers the second question, but when thinking about education reform, we should probably be more interested in the first. It then becomes important to clarify which causes we're interested in.

Education is a complex system, so fixating on just one component (teachers) won't get us very far. As Anonymous says, how many other students did a successful student's teacher inspire? (Perhaps many, in which case we should hand out an award.) If I were a teacher, I'm sure I would feel very satisfied to learn I had inspired one of my students, and proud of the student's successful hard work. But, getting through to one (or a handful) of students isn't much of a plan for sweeping education reform. What, on the other hand, about entire schools in which most of the students, who would otherwise be predicted for failure, graduate with strong mastery of the curriculum and enter good jobs or go on to college? Perhaps in these schools, they've somehow figured out a way to maneuver the system into their favor.

The Education Trust has been collecting stories of such schools, published in two volumes by Karin Chenoweth (It's Being Done: Academic Success in Unexpected Schools and How It's Being Done: Urgent Lessons from Unexpected Schools). I haven't read them yet, so I can't comment on how, exactly, these schools have "beat the odds."

Perhaps it was just the happy confluence of a number of exceptional people. Perhaps their methods won't scale well. That would be disappointing. But, I wouldn't be in the field of education if I believed that there was nothing we could do (i.e. the only important causes are out of our control) to improve people's lives through better education—but it's certainly going to take a larger view than looking at curriculum, or class size, or parental engagement, or teacher quality, or any other single factor alone.

Anonymous said...

Does it make sense to look at life-long smokers who haven't died of lung cancer yet? They "beat the odds."

Michael Culbertson said...

Hmm, I'm not sure I see your point about smokers. If there are systematic differences between lifetime smokers who haven't died of lung cancer yet and those that have, it would be helpful to know and understand them—an understanding that could help craft better health messages or reduce lung cancer mortality in those who insist on smoking.

Ah, but, lifetime smokers were supposed to be an analogy for schools with disadvantaged children. So, what could the point be?

Perhaps that disadvantaged children, like smoking, are damaging to schools' health? That doesn't sound quite right.

Let's try: just as smokers inevitably will get lung cancer, so disadvantaged children inevitably will fail. I choose not to adopt such a defeatist attitude.

Most likely, the point is something like this: Some smokers, by chance and factors outside their control, don't die of lung cancer. Similarly, some schools with disadvantaged children will perform well, simply due to random variation.

But this assumes that our rudimentary model for "the odds" explains all the meaningful sources of variation in school success—that zipcode (income, race, language, neighborhood crime, etc.) is all there is to say about which schools do well and which schools don't. I find that rather hard to believe. If there are other meaningful factors associated with high performing schools, a good start for identifying them is to take a close look at schools that don't fit the prevailing pattern.