What's up?

The process

Ralf Hinze, the general chair of ICFP 2007 asked us already at the end of 2005 if we would like to organise the ICFP Programming Contest 2007. I tried to collect a group of people from the Software Technology group of the Computing Sciences department of Utrecht University that were willing to help. That turned out to be rather easy: the majority of the people I asked were enthusiastic about it. In the first months we had frequent meetings, to discuss the several aspects of organising such a contest. Furthermore, we extended the team with several young colleagues. In particular with people that had participated in a couple of previous contests, which was really helpful.

After two months we stopped meeting to wait for the 2006 contest. We participated in the 2006 contest with four members of the team. As I mentioned in the blog message about the 2006 contest, I think this was a great experience.

We resumed brainstorming at the end of summer 2006. We ended up with three or four problem ideas. Each idea was investigated by a small group of people, and in October we selected one of the ideas, based on the practical requirements for a task we have formulated in a previous blog message.

In the following months we implemented several components for the task, and we tried to solve the several problems we envisaged with our problem idea. In January 2007 we were pretty convinced that our problem idea had no fatal flaws, and we started working out the many details that we had left open.

Since then we have been working on the implementation of the several components, testing, and quality control.

Close encounters of Arrow kind?

What's in it for us?

Why would you ever organize an ICFP Programming Contest? I can assure you it is a lot of work. So why did we agree to do it?

• We are happy to do a service to the community. We benefit from having a community in which we perform our research. The community organises conferences and workshops at which we learn about new ideas, can present our own ideas, and meet colleagues. Furthermore, people from the community review our papers and research proposals. In return, we sometimes have to invest time in projects that support the activities and visibility of the community.
• Visibility for the community is good for us as well. There is very little programming language research in The Netherlands, and we have the impression that programming language research is sometimes not as respected as we would like it to be. Organising a contest that shows that programming languages and tools matter, might help a bit. Especially if previous organisers come from places like Harvard, CMU, and MIT .
• We try to draw some attention to the contest in the Netherlands. It is nice to be able to have a story to present to the media. It is quite a challenge to present our work on a level that is understandable for non computer scientists.
• Organizing the ICFP Programming Contest gives us visibility in the community.
• We will write and publish a paper about the contest. However, the amount of work for producing this paper is considerably higher than for a normal paper.
• We are a team of around 10 people (depending on whom you count), most of them from the Software technology group of the Information and Computing Sciences department of Utrecht University, working on a common goal, with a strict deadline. I think such a common goal creates a good atmosphere in the group.
• We have already learned a lot, and had quite some fun.
• Eternal fame. The ICFP Programming Contest has a good name. We hope that the contestants will like this years' contest, and remember it.

Core dump?

The ICFP Programming Contest 2006

I started collecting a team of people to organize the ICFP Programming Contest already by the end of 2005. Besides collecting and discussing requirements and advice, we didn't start working on a particular task immediately, also because the 2006 contest could well make it necessary to change our ideas. Of course, we took the chance to participate in the ICFP Programming Contest 2006. Only four of our team could make it, but that happened to be exactly the team size that could still qualify for a prize...

I can recommend participating in such a contest to everyone :-). It was really Big Fun. Family life suffered, but participating was both intellectually challenging and good for the team-spirit. Moreover, it was an excellent opportunity to find out how much you can actually do in three days time. Which is a lot!

What was really nice about the 2006 contest:

• entering a virtual world which you could explore
• solving puzzles
• scoring incrementally; small points for small solutions, big points for major achievements
• writing programs in languages which tied your hands on your back (I wrote programs in qbasic and a stupid rewriting engine; and helped thinking a bit about the `rotating' machine.)
• very, very, cleverly done!

What was maybe a bit less nice:

• had to program the machine in C because of speed requirements
• could not use Haskell, our programming language of choice, other than for testing
• small programs
• only teamwork at the start, from then on mostly individual work (but that was our own, not very conscious, decision).

All in all a very nice experience, from which we learned a lot.

Whither, fairest, art thou running?

This garbage collector seems to be going somewhere...

Previous organizers

Once we had agreed to organize the ICFP Programming Contest 2007 (did we really?), I called the previous organizers to ask for advice. I did not talk to all 9 of them, but only the last three: John Hughes (Chalmers, race tracks, 2003), Benjamin Pierce (UPenn, ants, 2004), and Robby Findler (Chicago, cops & robbers, 2005). At this time the 2006 contest hadn't happened yet. Most of the advice corresponded with our own requirements for the task, but they had some useful observations for us.

About the programming task:

• The task should be easy to understand.
• Make sure the task is correct, for example by turning it into a literal programming file, or by deriving it from an actual program.
• Spend a lot of time modifying the problem, tweaking it, trying it out, etc. Have a few groups of students test it.
• Don't finish the task a couple of minutes before the contest starts!

Meta observations about the task:

• It would be nice if the winners to in some way belong to the community, be interested in the community, and definitely show up at ICFP (which is probably a consequence of the first two desirables). This implies that you want to design a task that is not too alien to people in or close to the community, and that it helps to know about programming languages.
• One of the nice things about the ICFP Programming Contests is that lessons can be drawn from most contests afterwards. For example, the 2005 contest, with the twist, was easier to solve in Haskell than in most other languages. It is maybe hard to prove these statements, but it is nice to be able to say something like this to your students. It might be nice to think of the `lesson' that can be drawn from the contest in advance.

About the process:

• It is important to have hackers on the team.
• Try to prevent premature rankings. Maybe set up a mailing list in which contestants can discuss the problem. Otherwise they will do it anyway. Monitor the list, and remove postings that reveal too much.
• Reserve a long slot at ICFP, and spend a lot of time on the problem, several statistics, interesting details about submissions, etc

We've taken most (but not all) of this advice to heart. The mailing lists, and long slots at ICFP have happened for a couple of years now.