Saturday, March 05, 2011

A note to Program Committees

Over the years I've written over 50 papers, many technical reports, quite a few magazine articles and several books, not to mention a PhD thesis. I've sat on enough program committees over the past 20+ years to know that it is a difficult job, particularly if you've got a lot of papers to review and very little time. That's why assembling the program committee is often one of the most challenging tasks for the conference/workshop chairs.

Throughout this time I think I've had a pretty good paper acceptance ratio, but I have had papers rejected by conferences and workshops. I think if you weren't disappointed about a rejection then you probably didn't put in enough effort to the paper writing in the first place, and that's definitely something I do for every paper. So it's true to say that when our WS-REST paper was rejected we were all disappointed.

I'm not going to go into whether or not I agree with the reviewers: that would be inappropriate. But at least one of them did frustrate me with something I've seen done once or twice elsewhere and each time I see it I get really annoyed. As a reviewer, whether for a conference/workshop, a journal or elsewhere, you often want to illustrate your points, both positive and negative, by drawing the attention of the authors to other papers or works. But if you're going to do this make sure that those papers or citations are publicly available or it really is pretty pointless: you may as well be saying "You are wrong; I know it but I refuse to tell you why and you just have to believe me because I am a reviewer!", which is obviously not what you expect from peer review! It also doesn't reflect well on the conference.

Over the past few decades some journals and conference/workshop publications have gone private, only being available to paying readers. This does make it hard, but not impossible, to achieve publicly available copies. I remember us doing this back in the 1980's, by publishing technical reports that covered roughly the same content and were primarily there for others to access when the original papers were unavailable. As a member of a program committee you could then refer to these. But one way or another, if you are on a program committee and you're going to want authors to learn from the experience of others and improve their papers based on your comments, either refer to papers that they can access, or don't bother!

No comments: