Standards vs. Criteria....a distinction worth maintaining?

This is a comment on Ken's post about Alfie Kohn's article on national standards, and Barb's comment. This got to be too long to "fit" into the comments, so I decided to make it a post on its on.

I agree wholeheartedly with Ken's post (and Alfie's thoughts, mostly). And Barb's distinction between standards and standardization is useful, although in a practical sense, I'm not quite sure how national standards would NOT lead to national standardized tests, especially if "accountability" remains the mantra of federal "reform" efforts.

I'd like to introduce another distinction that I haven't seen mentioned in this debate, and that is the distinction between "criteria" and "standards." What's the difference?

The terms are confused by the concept of "criterion-referenced tests," which stipulate that students should be able to DO a particular thing (the "criterion") without regard to the population as a whole. In other words, "criterion-referenced tests" do not use "norms" to determine was is acceptable: they simply set a minimum acceptability based on some predetermined criterion. This makes some limited sense for tests such as those used for employment in certain jobs (able to type at 70 WPM, for example) that require certain specific performance levels for successful performance of the job. (The fact that some people may be great for that job without having that minimal performance level is treated as, well, an accident that can't be avoided if we are to have efficiencies of selection.) Thus, a "criterion" for acceptable performance becomes a "performance standard." This is made even more confusing by the way that percentile measures (i.e. "normed scores") are used as cut-off points for standardized tests, for example in Chicago where 3rd graders must achieve a normed score of 18th percentile (or whatever it is currently) in order to be passed out of third grade. Superficially, this assumes that a certain percentage of students will inevitably FAIL that performance standard (since they are determined by percentiles), but the fact that these percentile scores aren't REALLY determined by the population actually taking the test (but by a previously selected "normative sample) means this simplistic idea that a certain percentage must fail is, in fact, not true.

(Gene Glass helpfully addressed this blurring of the distinction between criteria and standards in his 1977 occasional paper "Standards and Criteria," available here: http://www.wmich.edu/evalctr/pubs/ops/ops10.html, although his primary purpose was to argue that ALL criteria used to determine "passing" scores on a test are arbitrary, rather than the point I wish to make here.)

The distinction I wish to make is between criteria vs. standards as a way of evaluating the quality of a piece of academic work. Elliot Eisner refers to John Dewey's Art as Experience in describing this distinction in his 1995 Phi Delta Kappan article "Standards for American Schools: Help or Hindrance?" (vol 76, no 10; excerpted here and available here). Eisner wrote:

"Standards fix expectations; criteria are guidelines that enable one to search more efficiently for the qualities that might matter in any individual work...To say that by the end of a course students will be able to write a convincing essay on the competing interests of environmentalists and industrialists that marshalls good arguments supported by relevant facts is to identify criteria that can be used to appraise the essay; it is not to provide a standard for measuring it."

The problem with standards (especially as instantiated in standardized tests...which, I've suggested, they inevitably are) is that they lead to "arbitrary" (cp. Glass) numerical determinations of what constitutes acceptable vs. unacceptable work. Such standards, while useful, cannot possibly be applied in such a simple way to complex academic works such as the essay described by Eisner or any other product that wraps multiple skills together. Eisner continues:

"The qualities that define inventive works of any kind are qualities that by definition have both unique and useful features. The particular form those features take and what it is that makes them useful are not necessarily predictable, but sensitive critics--and sensitive teachers--are able to discover such properties in the work." Einser goes on to discuss the variability of human development, and the obvious lack of fit between "cut-offs" for minimal performance on standardized tests and the huge range of competencies visible in any classroom full of children.

While I don't have the time or space here to explore this distinction in greater detail (I refer you to Eisner's excellent article for more), I wanted to say it on the table as one way to gain some perspective on the issue of "standards" and "standardization" and their effects on teachers and schools.