March 13, 2002

"A mathematician is a machine for turning coffee into theorems."--Paul Erdös

On 13 March 2002, after wrapping up their second group "survey project" for a unit on statistics and data analysis, students worked in new groups to design test questions for the unit. The purpose of this was threefold: first, to give students some in-class time to review the concepts and formulas they had learned; second, to let them know in an underhanded way what the test questions would cover by way of the guidelines for devising questions; and third, to give them some insight into the process of creating tests.

Writing tests is still not my strong point. I acknowledge that I've written less than stellar test items in the past, and I've thrown out questions that I discovered had unexpected ambiguities. I dislike the phenomenon whereby students genuinely are blindsided by test questions, though as a question-writer I've been guilty of this; I too remember being a student staring at a question that might have popped in from Dimension X for all I knew. Tests, I decided, should not be mysterious entities used to punish students. My past attempts at creating "creative" questions to challenge students were misguided. What better remedy than to let the students have a shot?

I expected, of course, that not all the test questions that emerged from these groups would be test-worthy, even with the guidelines. Fair enough. Not all my test questions were test-worthy, so why should theirs be, especially the first time? I did, however, refer students to numerous examples from previous class assignments and homework, as well as the Preview (an open-book, ask-your-friends pretest). I was also prepared to pick and choose among the students' questions and create my own questions to fill in any gaps.

Some groups found this assignment more troublesome than others. A few students decided to use previous homework questions verbatim, which was fine in principle, but less educational than I could have wished. One group, amid giggles, attempted to turn in a question on histograms involving a fictitious survey with responses on whether it's "okay to have sex on the first date" divided by the respondents' sex; after I asked them if they would feel comfortable showing this question to the principal or their parents, the question was quietly retired. (In some ways I regret this, as their invented results pointed to a cynical understanding of the "double standard" for men and women, and would have made a great discussion topic, say, after class.) Other groups, however, came up with interesting scenarios for statistical analysis, such as swimming times or survey biases.

All very well, you may ask, but how to assess this?

First, I wrote down the group configurations and circulated around the groups to eavesdrop on what discussions were taking place. Though I can't claim to know in detail what those discussions involved, I attempted to get "snapshot" pictures of who was involved in what.

Second, I required the groups to turn in all their written work. This way I had a chance to look at individuals' proposed test questions as well as the comments they wrote on each others' questions, and the scribe's notes.

Third, as you might expect, students actually had to take the test, which consisted mostly of their test questions (included verbatim, though anonymously) and partly of mine. The last question on the test, #8, asked them to choose and critique a test question on whether it was relevant, understandable, too easy or too hard or just right, and whether it allowed them to show what they knew about the subject. Some students' responses to this last question were minimalistic, while others wrote eloquently on why a question did or didn't confuse them. Interestingly, students commented both on questions they liked and questions they didn't; as all responses were polite and well-considered (#8 reminded them of the two rules "be polite" and "be constructive"), I photocopied these and gave them (anonymously) to the questions' originators.

Finally, two classes afterward, students in groups had the opportunity to grade my responses to two questions. I had taken the test with them, but sketched out solutions rather than writing them up in detail. In fact, I didn't even finish the thing. I deliberately chose two responses that were not exemplars, including a very incomplete answer to the question on survey design (which most students did quite well on). Despite initial awkwardness--I assured them that I wouldn't be offended if they flunked me--students were soon engaged in heated debates over what they considered important or "correct" in an answer and how to award partial credit. One student wanted to dock me half a point for being "too technical," which the rest of the group disagreed with; another group awarded a point of extra credit, to my amusement. A student who rarely speaks at all in class was speaking passionately as I passed by. Best of all, when I went around to each group to ask what they thought the point of the whole exercise was, students articulated ideas such as: "It gives us a look into the teacher's point-of-view when grading" and "Now we know what it's like to try to put a grade on something."

I may never reach my fondest goal of making my presence near-obsolete in the classroom, but times like this give me hope.

Copyright © 2002 Yoon Ha Lee <requiescat@cityofveils.com>
Last updated on 21 May 2002