Tuesday, March 07, 2006

Chronicle article: "The Grading Factory"

An article in this week's CHE describes Fred Kemp's TOPIC program of teaching composition at Texas Tech. I've read about this for some years but had not seen it fully described before. To minimize the effects of bad teachers and to have students write more (admirable goals), graduate student instructors are divided into "classroom instructors" and "document instructors": the former teach in the classroom, and the latter grade papers online under a piecework quota system, as follows: 17 drafts per week to comment on and grade; 18 on which to offer a grade (but not comments); 25 peer critiques to review; and 20 student self-evaluations. This is supposed to take 12 hours a week, which means an average of 10 minutes per document. It also makes the system sound like the Triangle Shirtwaist Factory, but we're assured that this isn't the case.

The system sounds interesting, and it does have the effect of having a student's writing graded strictly on product (this, after years of the comp mantra of teaching that writing is about process, not product). And, as is stressed in the article, it's a gold mine for researchers, though whether that ought to be a primary justification for students' writing isn't clear.

Okay. Here are some questions:

1. The system is rigid enough that a final grade of 89.9 can't be "rounded up" from a B+ to an A-, and yet paper grading, even when it is normed and discussed, is necessarily *somewhat* subjective unless it's done by martinets who deduct one point for a comma error. The system allows for this: if the two graders' grades are apart more than 8 points, the paper is kicked over to a third reader. That's a good system, but I'd still say that it's not a fine enough increment to declare an 89.9 an inflexibly and objectively derived grade, not when that grade is based on much more flexibly assigned points earlier in the semester.

2. One benefit described is that the DI (document instructor) can eliminate repetitive comments by cutting and pasting (or inserting somehow) links to explanations of things like comma splices. That's fine, and we've all done versions of that. But what got my attention was the idea that, as one instructor put it, you can use "search and replace" to hunt for and grade errors.

Sorry--my cognitive dissonance alarm is screeching. This is the same discipline that (rightly) pitches a fit every time someone claims to come out with some kind of grading software that flags basic errors, even if the software isn't used to grade the content of the papers. Automated search/replace or automated software? What's the difference?

3. What do the students write about in these essays? What do they THINK about? How do the instructors know whether this is all just material rehashed from a class discussion? Do they have class discussions that encourage critical thinking? I'm sure that they must.

4. Wouldn't this lead to a hierarchy in which classroom instructors would be considered "above" document instructors? Is there a distinction made when the grad students graduate, or are all considered to have had equivalent teaching experiences? Also, doesn't this reinforce the already and unfortunately prevalent view of grading as a lesser form of work--one that could eventually be outsourced to freelancers or the people at Smarthinking.com or its offshore equivalents? Doesn't this reinforce the sorts of classroom practices that have been so vigorously scorned in pedagogical circles for years, such as lectures?

5. Some classroom instructors say that their authority is undermined because they don't get to grade the writing; Mr. Kemp argues that it turns them into coaches, a good thing. As a veteran of exit exams and portfolio systems, I'd agree with the "coach" analogy and believe that it can be valuable. But what about the conferencing time spent with students, when you can see their work improving draft after draft? And what's lost if their writing gets better but their ideas do not, because there's no one who can say, "Remember when you said X in your second draft? You left it out here, but that was an interesting idea worth developing"?

With all that, the system is surely a worthwhile experiment, and Fred Kemp and TTU ought to be applauded for their boldness in trying something new.

No comments: