Saturday, March 30, 2013

Robot grading and the real student essay

As it always seems to do just before The Season of Heavy Grading, the issue of robot grading is making the rounds again.  Alex Reid has a thoughtful post on it in which he points out that what we mean by machine reading and human reading are two different things. What we want from good writing isn't merely a correct sentence--the "Hitler discovered America in 1315 and published it in the Magna Carta" stellar writing that the machine grading programs love--but something that shows an awareness of the rhetorical complexity beneath expressing something worthwhile. You know, something that shows human thought behind the words.

Let's leave aside for a minute the kinds of tasks that promoters want robot graders to do, including high-stakes testing that determines students' futures. I want to know what the robot graders are going to do with these kinds of papers, which I've gotten plenty of over the years:
  • The Adorable BS Artist paper.  Adorable BS artist is that charming and very bright student who writes beautifully, but because her busy schedule doesn't always allow time for actually reading the text, the essay she produces is a lovely, airy souffle without any actual content in it. Adorable BS Artist will often reassure you that she's never received less than an A on a paper in her life, and you can see why a work-weary high school teacher would be dazzled by the sentences.  But excellence in college writing demands more than that, and if you point out the difference to her early, she can develop into a really good writer, one with substance as well as style.   If Adorable BS Artist puts in a date or two, though, the robot grader will never know the difference, and she'll never learn to think.
  • The Patient Plodder paper. Patient Plodder's paper is almost painfully correct. The sentences are correct. But there is not one idea in the paper, not one, that you have not talked about in class, and there is absolutely no evidence of the writer's original ideas. Yet it is correct, it is of the required length, and it throws in a date here or there. Patient Plodder believes that she should receive an A because "there's nothing wrong." But "there's no there there," and a human grader would point out the difference between "nothing wrong" and "not enough right." Would a robot grader do this?
  • The Icarus paper. The Icarus writer has tried, really tried, and has put herself out there in terms of assertions. Parts of the paper are near-brilliant. Parts of it are a train wreck, in part because the writer has overreached and in part because she doesn't know how to integrate quotations without making a mess of them. If you're a human teacher, you help her sort it out and nurture the brilliance. If you're a robot, you melt those wax wings and let her fall into the sea.
By the way, here's a helpful tip for students: if your essay is being graded by machine and it's not long enough to get full points, copy and paste it again into the essay box. The machine will never know the difference, and you'll get full points!  Win-win!

No comments: