Getting them aligned isn’t always easy.

A research paper is like a story. There’s a beginning (the introduction), a middle (the results, hopefully with some kind of climax), and an end (the discussion); a narrative arc, hopefully an argument, a rhetorical point to be made. And like Spielberg films (or at least, everything he’s made since Sugarland Express), the endings are almost always happy ones.

With films, you watch and form your own opinion. Reading a scientific paper though is like watching a film with the director’s commentary switched on. Not only do you have the data, but the authors will helpfully explain what it all means. How thoughtful! They summarise everything that happened in the prequels, walk you through this latest instalment, and then at the end tell you why they liked it so much, and what the (next) sequel will be. It’s helpful in that it removes the ambiguity (subjective versus objective truth being the critical difference between the arts and the sciences), but it’s not perfect.

The problem here is that the director’s commentary and the film are not always telling exactly the same story. It’s not a question of omission (though other filmmakers can often be outraged at a failure to acknowledge their previous work) but rather one of alignment. Sometimes what you see on the screen and what the commentary tells you you’re seeing are close, but not exactly the same thing. What the director claims to have produced and what is there to see are not quite identical.*

These slight misalignments present a conundrum for reviewers, those thankless critics of the scientific world, who are tasked with determining whether the work is suitable for public consumption. If the manuscript is saying one thing and the data another, then clearly the authors have to make changes – but to what? To the words, or to the pictures? Should you make the manuscript better fit the data presented, or make the data better fit the words in the text?

There are good reasons for favouring the former. Few people enjoy writing manuscripts, but modifying them at least requires no extra benchwork and no nasty surprises. Demanding extra benchwork may similarly bring the text and data closer into alignment, but brings the risk of mission creep, requires much more – sometimes significantly more – work, and may have a negligible impact on the broad brushstrokes of the story being told. (Until fairly recently, there was a widely-held but mistaken belief that calling for extra experiments from authors was a badge of rigour on the part of the reviewer, but this is a mindset that is dying out and not before time.)

The downside of changing the text to fit the data though is that the significance of a work may be lost. The authors may – in good but mistaken faith – have produced results that do not say quite as much as they would like them to, but requiring them only to modify the text could result in a rather insipid outcome. Instead, by challenging them to perform the experiment that will actually yield the conclusion they erroneously believed they reached the reviewer gives them the opportunity to produce the paper that they wanted to, instead of a drabber compromise piece.

It’s a very difficult balancing act to perfect on the part of the reviewer, and one that requires no small degree of skill and judgement. There are very few scientists who would claim that their manuscripts have not benefitted from the peer review process, and the usual outcome is a detectable tightening of the argument and improved focus.

For all that though, it’s probably the case that most scientists are better wielding a pipette than a pen (and when wielding the latter, often not writing in their native language either – which means nuances of expression can easily be lost). The balance of evidence would suggest that it’s more likely a manuscript is unclear than its data unpublishable. And as any filmmaker knows, it’s easier to change the script than refilm the movie.

*For example – a paper describes depleting a kinase by RNAi, and shows that a particular phosphoprotein then becomes dephosphorylated . The paper concludes that the phosphoprotein is a substrate of the kinase, when in fact it shows that the phosphorylation status of the phosphoprotein depends on the presence of the kinase (the actual phosphorylation might be by another kinase that’s directly or indirectly regulated by the first one).