Rationale for Only-Answers-Exams

Let me be honest. I hate marking exams. But I enjoy teaching. For me at least, teaching at the university level is the best possible job. I like the freedom of the it, I enjoy explaining things and thinking about well-known things in a new way. If I should complain, I would like to have some more time for research. However, since I have over the years developed a, what I call “Just-in-Time Unpacking” method of teaching that requires almost no time for preparations, I get time for that too (I will describe the method in another post).

But I simply can’t standing marking exams. I remember as a graduate student when I got a pile of 180 exam papers in Mathematical Physics for Electrical Engineers to mark. I split the pile with a fellow student (Ulf B. if you happen to read this) but still it was almost a week of boring job. But that was extreme. Normally you get at most 50 papers or so to mark. Still I can’t stand the drudgery.

Some years ago while teaching a course in introductory thermodynamics I got the idea of simply doing away with much of the marking job and speed it up. The problem was really, as I had discovered by introspection, that marking exams require too much attention, it can’t be automated. It is sometimes alright to do routine work, like painting a fence or hand-wash dishes, because you can think about something interesting during the work. Some types of work, like hanging wallpaper, is really philosophical and very conducive to deep thinking. But not marking exams. There is no way I can think about something else while marking exams.

Why not design an exam with tick-box questions? It took some thinking to reformulate the typical exam problems in this way. Of course, there must be a correct answer to tick for each question, and it took some work to ensure that I did not myself make any stupid errors (and I did occasionally). What was problematic was funnily enough to decide on the wrong answers. All in all it took considerably longer time to put together an exam, but the extra time also made for higher quality in the questions (I think).

The marking was swift!

No student ever complained about this type of exam. But I was once challenged by one parent (!) who thought that guessing made the exam unreliable. Intuitively I did not think so, I had thought about it when setting up the questions (number of alternatives and way of marking), but now I had to work through the statistics. There was no way you could pass the exam by pure guessing.

However, eventually it turned out to be too complicated to design the problems in this way. It took too much effort to make 100% sure that there always was one correct answer to each question. And also coming up with the wrong answers wasn’t that easy. So I switched from tick-box answers to Only-Answers where the students write down their own answers rather than checking against a list and choosing the correct. That worked well. Designing exams was reliable and the marking still swift.

Next step was to do it in mathematics courses.

The standard written exam in say physics or mathematics requires the student to solve a set of problems. They are supposed to write up their solutions in a way that is possible for the teacher to follow. The solution should lead up to the correct answer. The exam requires that solutions should be well motivated with notation and equations explained et cetera.

Over the years I made the following observations:

  • There is a very strong correlation between good solutions and correct answers.
  • There is a very strong correlation between bad solutions and wrong answers.
  • Even the good solutions are seldom, at least not always, up to the standards of “solutions well motivated with notation and equations explained”.

No colleague I’ve talked to have denied this. And of course it is highly expected. Furthermore, any experienced teacher who is marking a problem starts at the end and checks the answer. If it is correct it takes no time to see that the solution is OK, perhaps not up to the requirements, but OK all the same. If the answer is almost correct it is often easy to spot the error, but not always, sometimes it takes a considerable amount of time to find the error. If the answer is plainly wrong but there is a long solution it often takes a long time to evaluate the solution. Then there is a middle ground which is very time-consuming. It is clear that the student don’t really know how to solve the problem but there is a lot of writing to go through. You get into a mode of working where you try to dig out something that is correct. But this wading through badly formulated solutions leading up to wrong answers in order to locate the exact errors and judging how serious they are, or trying to find out how many points to hand out, is largely pointless. After all, it is wrong.

Some students expect to get some points for half-baked attempts at solutions where it is plain that they don’t know how to solve the problem, or don’t even have the basic skills to do it. Encouraging this is bad pedagogy.

What are the good points of Only-Answers-Exams? Here are some:

  • The focus is on getting it correct: To get 10 points you have to do 10 things 100% correct. It is not enough to do 20 things 50% correct. This is really a matter of Quality Management. 90% correct means 9 correct out of 10 and 1 wrong, not all 10 things 90% correct.
  • The marking is fair and objective. Same rules apply to everyone. There is no bias due to bad or good handwriting.
  • Almost no borderline arguing. If the limit for PASS is 20 points and a student gets 19 points, there is no way to find another extra point because there are no more correct answers on the exam. The result, even though sad, has to be accepted.
  • You can still design the exam problems so that students can get points even if they can’t solve the complete problem. You split up the problem in parts and design the questions in steps. I’ve heard complaints that this gives the students too much help in solving the problem. What’s wrong with that? Building the problem step by step helps the student to show how much he or she knows. I’ve seen too many exam questions that are meant to trip the students.
  • The only thing you lose is checking if students can formulate coherent solutions. But that can be checked in other ways. You can have one fairly simple problem where the focus is precisely on writing up a solution.
  • And marking is swift. You get time for the real job: teaching well.
This entry was posted in Uncategorized. Bookmark the permalink.

One Response to Rationale for Only-Answers-Exams

  1. I’m surprised that somebody identifying as a “mathematical humanist” would advocate multiple choice. Multiple choice is the standard test format in the US at almost all levels of education. It is cheap and allows for mechanized grading. The effect is that students learn to find as many right answers as possible by elimination and guessing. Another effect is that it is usually not possible to provide useful feedback to the student so they can learn from their mistakes. In Mathematics in particular, it seems to me that only low-level subjects can even be tested by multiple choice. In real University-level mathematics, you expect students to come up with proofs and derive results, not just perform calculations that can be checked as right or wrong.

    The time constraint is of course a real issue. When I was teaching, I quickly found out that trying to do justice to each student was simply not realistic within the time budget. I also observed that some professors (not specifically in mathematics) would pose challenging problems to students but then give everybody an “A” rather than actually grading their work because they simply didn’t have the time. That to me is one of the worst things one can do to students. The whole point of testing in my view is to get an honest assessment of the student’s level of competence, AND to give them the chance to learn from their mistakes. That takes time, and certainly from a humanist perspective, the answer would be to provide sufficient time and resources (i. e. teaching assistants), rather than to dumb down the testing procedure.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s