About the programming task:
- The task should be easy to understand.
- Make sure the task is correct, for example by turning it into a literal programming file, or by deriving it from an actual program.
- Spend a lot of time modifying the problem, tweaking it, trying it out, etc. Have a few groups of students test it.
- Don't finish the task a couple of minutes before the contest starts!
Meta observations about the task:
- It would be nice if the winners to in some way belong to the community, be interested in the community, and definitely show up at ICFP (which is probably a consequence of the first two desirables). This implies that you want to design a task that is not too alien to people in or close to the community, and that it helps to know about programming languages.
- One of the nice things about the ICFP Programming Contests is that lessons can be drawn from most contests afterwards. For example, the 2005 contest, with the twist, was easier to solve in Haskell than in most other languages. It is maybe hard to prove these statements, but it is nice to be able to say something like this to your students. It might be nice to think of the `lesson' that can be drawn from the contest in advance.
About the process:
- It is important to have hackers on the team.
- Try to prevent premature rankings. Maybe set up a mailing list in which contestants can discuss the problem. Otherwise they will do it anyway. Monitor the list, and remove postings that reveal too much.
- Reserve a long slot at ICFP, and spend a lot of time on the problem, several statistics, interesting details about submissions, etc
We've taken most (but not all) of this advice to heart. The mailing lists, and long slots at ICFP have happened for a couple of years now.