Thursday, December 02, 2010

Theory and experience, or "but it looked good on paper"

A long time ago, when I was a more trusting soul, I read a cat care book that recommended using the upholstery brush attachment of a vacuum cleaner--with the vacuum cleaner running--to groom your cat. What's not to love? The cat hair is magically vacuumed up, and the book assured me that "cats love it!"

Those of you who know how much cats generally love vacuum cleaners can guess the rest. Anyway, the deep scratches on my arms healed after a month or so, but I was never quite so trusting about some kinds of advice again.

Dean Dad has a post up about a similar "it looked good on paper" moment: treating first-year students as an incoming cohort, putting them all in the same classes, and so on. What happened?
It was one of those (retrospectively) glorious exercises in perspective. From the college’s perspective, the idea of group bonding, integrated instruction, and deliberate exposure to extracurriculars should have added ‘good’ to ‘good’ to ‘good.’ To the students, though, it felt like High School II. In high school, they saw the same people over and over again from class to class; they were actually eager to break away from that at college.
The commenters at Dean Dad's say the same thing happened at their schools.

My "it looked good on paper" moment (apart from the cat experience) has been with rubrics. I believe in them, and I've developed what I think are some good ones. But when I've asked for feedback on courses where I've used them for some assignments, the students say things like "I like the comments you give us better than the rubric" or "I like it when you write a long comment." Part of this is doubtless my fault for not using rubrics all the time; maybe if they didn't know they could get long comments, they'd be happy with the rubric.

Have you ever had an experience like this, where something ought to work well based on all the theories, but it just didn't work in the classroom?


Dame Eleanor Hull said...

This doesn't quite answer your question, but I instituted rubrics as a way to make things easier for myself (as well as more transparent for the students), and now I find that while they do enhance transparency, they are a whole lot more trouble for me. I'd rather just read the papers, mark them up, and write a comment, than go down a long list of boxes trying to decide whether various aspects of a paper are excellent, acceptable, or whatever. But this is more "at home" than "in the classroom."

Ink said...

Rubrics plague me. For the same reasons both you and DEH mention. I do think it helps them to be able to see at a glance where they're strong and where they need more work on the assignment. However, it does seems like way more work, and the students prefer comments, they've said, and also? There's always that one thing that somebody does that is so bizarre it doesn't fit into the boxes and I can't figure out how to address it, pointwise.

Arbitrista said...

I'm sorry, I just couldn't get past the cat and vacuum cleaner part of the story. Tee hee.

Jonathan said...

The way academics think is usually: 'Sure it works in practice, but will it work in theory?"

I don't use rubrics because I would just manipulate them to get the result that felt right to be for the paper anyway. It feels dishonest.

undine said...

Dame Eleanor, that's the problem I have. I hate to check that "good" or "adequate" (or whatever) box, since it seems more final and less nuanced than a written comment.

Ink, I've found that, too. I really only use rubrics now for things that have a "countable" component (i.e., citations in MLA format?).

Arbitrista--glad you liked it! It made me laugh, too, mostly at my own stupidity.

Jonathan, I end up doing that, too. I end up with a score that doesn't seem to represent what the grade would be (and should be) if the grading is done by comments.

michele said...

I once tried an annotation exercise that I thought would work well but fell flat.

My advisor when I was new to teaching suggested that getting the students to create a set of footnotes or annotations for a difficult text was a good exercise. The idea was to get them looking up things they didn't know and then writing up an explanation (footnote) for two or three which then would be compiled for the whole class to benefit from.

The students hated it. They thought it was make-work and they mostly just supplied dictionary definitions for the words and terms we'd identified as a class that needed footnoting.

I still don't know why it didn't work - seemed like a pedagogically sound idea. But they weren't buying it!

Carl said...

Agreed, agreed. I use our departmental rubric, which we wrote together to generate tangible likert-scale numbers for the accreditation, to automate the little technical comments I'd otherwise write a lot without adding much intersubjective value. The numbers indicate a magnitude of performance, not a fraction of a final 'score' (no need for Jonathan's jiggering - I still explicitly grade wholistically). This means I can save my writing hand to address their content, which makes them feel attended to and sometimes even teaches them something.

This semester I tried turning the first batch of papers back with only a grade on, and told them encouragingly that I'd like to say more about their papers than I could fit in the margins, so I hoped they'd come by my office to talk with me about their work. It turned out that they were so used to getting papers back with no substantive commentary and so uninterested in a long personal grilling that no one came. I must admit this confirmed two hypotheses, but perhaps it counts as a failed experiment even so?