Monday, February 02, 2009

The Upside-down Pop Quiz

Remember pop quizzes? "Class, take out a sheet of paper. I assume you've all read Chapter Three. So let's play the Read-My-Mind Game, also known as Gotcha! Bwahahahah...." Even if you'd done the homework and read the chapter, you never knew if what *you'd* gotten out of the chapter matched what the teacher thought was important. Oh, how we hated them.

A couple of years ago I was designing a new course with a History professor, and inspiration struck. Instead of punishing students for not reading the book (or not getting the "right stuff" out of it), why not provide a positive incentive? We want them to read the book. What do they want that we can provide? Grades! A simple transaction, really: You do what I want, I give you something you want. It's worked for generations of parents and corrupt government officials (until they get caught, anyway). It's called BRIBERY: Read the textbook, and it's worth a letter grade to you.

So we turned the pop quiz on its head. We created untimed, open-book online quizzes for the text chapters we assigned. The questions came directly out of the book, and were designed to be answered while looking at the text. (After all, life is an open-book test.) The quizzes were posted on the class website (Blackboard) the week before the discussion of the readings. Students could take the quiz as many times as they wanted. The feedback to incorrect answers directed students to the appropriate page of the text. We did not give the correct answers. The sum of all the quizzes amounted to 10% of the final grade.

Questions were a mix of high-level, conceptual, big-picture items that required students to integrate ideas across an entire section, and nitty-gritty detail questions that could not be answered apart from the text. Case in point: The text contained a passage from an original source document listing government jobs in Massachusets in 1690. The list included baker, brewer, collector of tithes, person to keep dogs out of church, rebuker of boys, and so on. So we put this question on the quiz: "According to the text, all of the following were government jobs in Massachusets in 1690 EXCEPT..." and then we listed the jobs above, inserting the red herring, "Admonisher of young ladies." Sounds reasonable, but it wasn't listed in the text. (I suppose colonial girls were better-behaved than their brothers.)

Now, I guarantee you, if you ask a question like that on a closed-book test, the students would be lighting torches and sharpening pitchforks, and with good reason. Expecting students to memorize a list like that is completely unreasonable. But it's perfectly reasonable to ask them to read that paragraph closely at least once, in order to kickstart a class discussion of colonial attitudes about the role of government.

Ok, fine, sounds great. Innovative teaching strategy, hoo-hah hurray, golf-claps all 'round. But does it work?

Yes. Emphatically so.

The data showed that most students took the quiz more than once, some up to four times in order to improve their scores. The time-stamps on successive quiz attempts showed how students were driven to the text, forcing them to ferret out the answers. The instructor could see which questions the students had missed, showing which concepts needed extra attention in the lecture and discussion.

In class, the discussions made it very clear that the students had read the assigned chapter, and read it carefully. And since the quiz questions reflected what the instructor thought was important, they were all on the same page regarding both the big ideas and the details. The instructor didn't need to spend time rehashing the chapter content; it became a starting point for the discussion.

So, I commend the upside-down pop quiz to you. Give it a try, and if you'd be so kind, let me know how it works for you.