What peer review can often feel like.*

What is the purpose of peer review?

Is it to police errors (the “specialist” angle)? Is it to determine how interesting in a broad sense it is (the “generalist” approach). Or is it to determine whether a manuscript is suitable for a particular journal (the “editorial” angle)?

The third is the most subjective and the most awkward, but in theory the other two are both capable of improving a manuscript – either by clarifying its methodology or by clarifying its interpretation. And perhaps this is the function of peer review in an idealistic sense – tough but fair criticism whose intention is to assist the authors in making the paper as good as it possibly can be.

One thing however should be clear – refereeing is not, and should never, be an opportunity to settle scores or vent spleen. The fact that papers are (usually) reviewed anonymously often leads to a road rage effect in which referees are disinhibited from expressing themselves in a manner bordering on the unprofessional. The anonymity is intended so that tough criticism can be made – even of one’s professional seniors – without fear of reprisal. However, vindictive attitudes have meant that it is now commonplace to exclude from the review process many of the exact people who are best-positioned to judge the work – namely, people working closely in the same niche – and the result is to limit the amount of expert criticism.

This creates a chicken and egg problem. Experts are excluded from reviewing a paper, angered when it is published, and this decreases the level of trust between groups and encourages a siege mentality. Each blocks the other from assessing their work, and complains about their own exclusion.

In principle there’s nothing wrong with publishing a paper that’s less than perfect (it’s impossible to satisfy everyone anyway) and anything that stimulates thought and debate is good for science as it gives people something to aim at, if nothing else. The problem is that the hypercompetitive nature of modern science is making publication of imperfect work a source of great bitterness. And bitterness will impair the most priceless requirement of good science – the need for open dialogue.

Perhaps, before reviewing, we should all reflect on what our role is. In particular, we should remember the potential for causing offence and ask whether we would be willing to make the same comments in person, or how we would feel to receive feedback in kind. A referee’s duty is not to kill a paper, but to evaluate it. It is executioners, not judges, who must wear masks.

Note that this posting was stimulated by an article on the history of peer review in the magazine section of Nature on April 21st – highly recommended.

* The statue is actually of St Aquilinas, born in Würzburg, who was martyred following his denunciation of some early Christian sects. An early instance of an unfavourable response to vitriolic peer review…