Hand-writing code on exams

Yesterday I posted a question to the SIGCSE mailing list which garnered a lot of responses. I was interested in how my colleagues felt about requiring students to hand-write code on exams. It's a practice I've followed in most of my courses since I started teaching in 1997, and it's something all of my colleagues at Harding still do. Even the Computer Science AP exam requires students to hand-code their solutions.

The reason I asked this question was because this semester I started receiving more push-back than normal from students who complained about having to hand-write code on their exams. One beginner complained that he couldn't write well or quickly. Another said it was too different going from an IDE to paper. One didn't understand why I would ask him to write code that the compiler auto-generates (a getter and setter in a Java class), and another complained this would be the only time in his life that he would be forced to hand-write code, so what was the point?

So I put this to my colleagues: Is requiring our students to hand-write code on exams an out-dated way of assessing our students?

The question received more than 50 responses in the past 24 hours, most of them in favor of hand-writing code on exams. There were a few, however, who abandoned this practice a while ago in favor of exams which were at least partially completed with an IDE in a computer lab.

Below is a summary of the responses. I begin with various reasons why my colleagues have their students hand-write code on exams (the pros) followed by some reasons not to (the cons). At the end I summarize some of the best practices for those who require hand-written solutions and those using an IDE on an in-lab exams.

Pros of Hand-Writing Code

1. Some instructors simply don't have easy access to a lab, so having their students hand-write code is the most feasible solution. And computers are not always reliable; they might have software problems, crash, etc.
2. Many students experience much more stress writing code in an IDE under time pressures than hand-writing code because they can get hung-up on syntax errors. It's easier to give partial credit and forgive minor syntax errors like missing a semicolon when students hand-write their code. It's also difficult to control cheating when using an IDE since the computers are networked.
3. Hand-writing code demonstrates the student's level of mastery without the aid of a crutch. Using an IDE can encourage students to "fish" for the answer... they may try many different things to see which one works without really understanding the problem or solution.
4. When interviewing for a job, students will often be asked to hand-write code to solve various problems. And many programmers in industry collaborate by writing code on whiteboards without the help of an IDE. Of course the interviewers are usually more focused on the thought process and less concerned about syntax correctness. And coding on a whiteboard is also more about communicating ideas, not the syntax.
5. Students should not rely too heavily on an IDE because every IDE is different and they are not always available.
6. Some faculty feel there are some things that every programmer should be able to do, including hand-writing solutions to fundamental CS problems like "how many nodes are in a binary tree?". Surely some computing pioneers like Dijkstra would agree.

Cons of Hand-Writing Code

1. Most real coding is done in an IDE, and writing code on paper is too tedious and foreign.
2. Some instructors admit to needing the help of an IDE when they program because there are so many functions and differences between languages that it's difficult to remember everything. Why should we expect students to do with less?
3. Many students can type much faster than they can write, so they are penalized for writing slow. College students hand-write much less today in general, and many feel more comfortable in front of a keyboard.
4. Grading hand-written code is slow and painful. It's much easier to automate the grading of problem sets in an IDE where problems are either solved or not solved.

Best Practices for Hand-Written Exams

There are some things instructors can do to prepare their students to perform optimally when hand-writing code on an exam.

1. Make sure students get plenty of practice hand-writing code before the exam so it is less foreign. Pre-tests are ideal.
2. Make sure students don't have to write really long segments of code, certainly nothing over a page in length.
3. Make sure hand-written coding exercises focus on the basics, not on memorizing lots of different functions and minutia. If students do need to use a library, include the interface documentation for the library on a supplemental handout.
4. Beginners should be tested for precise syntax, but upper-level students should be given more leeway. Allow students to write in pseudocode if you are more concerned with the thought process rather than the syntax. (If syntax is more important, award pseudo-points for pseudocode. - Richard Pattis)
5. Let students know that if they can't remember how to do something they can write a comment like "and here I resize the vector, I forget how."
6. Instructors should hand-write code on the white board in front of students so they can see how it is done.
7. To quell potential complaints, tell students in advance that they will often be expected to hand-write code in job interviews and on the job when computers are not nearby. Also tell them you will be much more forgiving when grading their code than the compiler will be.

Best Practices for In-Lab Exams

Several of my colleagues reported bad experiences with in-lab exams, but others suggested ways to do it successfully.

1. Don't expect students to be able to solve the same number of problems in an in-lab exam as they would on paper. Reduce the number of problems significantly because they will get hung-up on some syntax errors.
2. Use problems that are easily broken into small discreet steps that are as independent as possible so getting tripped-up on one problem doesn't stop the student from completing other problems. (Note: This is very hard to do.)
3. Make sure students get practice writing code under time constraints, possibly with pre-tests, so they know they must study and use their time wisely. This will make the exam less stressful.
4. You may allow students some Internet access to sites like StackOverflow to remind them of proper syntax or to get ideas on how to formulate a solution. Obviously you will have to trust that students are not collaborating, and you will need to ask problems for which no solution can be found online.

Thank you to the many SIGCSE members whose ideas contributed to this blog post. If you think of something else that should be added to the lists above, please email me or leave a comment.