Being asked to review a paper is one of the biggest but commonest responsibilities a scientist has to take on. Here are some tips for how to go about it.

Peer review is critical to the scientific process. The dispassionate evaluation of others’ work to determine its suitability for publication is one of the biggest responsibilities you will take on, and it’s important that it’s done well.

This is by no means THE way to review papers, but it is one way to review papers, and it’s especially intended for those who have less experience of the activity.

Producing a review is obviously very closely related to the critical reading of scientific manuscripts, and for a fuller discussion of that topic you can read my posting on it here.

I’ve collected together the main points for the review in a summary and template (Word file) that can be downloaded here:

Click HERE to download the reviewing summary and template.

If you want to skip the rest of this posting and jump in, now’s your chance. 😉

1. Before the review
Some things to consider when you are first approached to provide a review:

1. Do you have the right experience? It can sometimes be hard to judge when you only have the Abstract to go on, but try to judge whether the manuscript falls wholly or significantly within your technical and theoretical expertise. If you’re unsure, feel free to consult with the editor – reviewers are often hard to find, so they’ll be happy to help.
2. Do you have the time? Many papers will be the results of someone’s PhD thesis, and careers can be made and broken through publications. Do not under any circumstances agree to review a paper if you do not have the time to do it properly. Don’t worry about the deadlines, as these can invariably be extended (within reason). The question is whether you have the time and attention to do this properly.
3. Do you want to do it alone? Consider including a junior member of your group in the review process – this is invaluable experience for them.
4. If you can’t do the review, suggest some alternative reviewers when you respond, if possible (and don’t provide a list of old white men – try to offer a diverse shortlist of alternative candidates).

2. After you’ve accepted
You’ve got the manuscript PDF – now what do you do?

1. Check the submission form (there’s usually a link in the email you’re sent after accepting the review) to see whether there are defined sections. There may be a checklist, or even defined subsections. This may affect how you structure your review.
2. Do a quick PubMed search to see whether there are any relevant recent papers on the topic, or anything else from the group’s recent work that will provide context. Don’t invest more than a few minutes on this though, especially if you’re already confident that you’re familiar enough with the topic.
3. Prepare your template for the review. You’re going to fill this in as you go through the manuscript, and I’ve provided a template in the downloadable document above. In brief, this will contain:
   1. Summary
   2. Major points
   3. Minor points
   4. Editorial points
4. Extract sections from the manuscript PDF so that you can view things side by side. Keep the original manuscript PDF intact as a reference.
   1. Figures + Figure legends
   2. References
   3. Supplementary material

3. The review begins
Now the real work starts!

1. Again, if you’re unsure of how to critically read scientific papers, I’ve written a “How to…” guide, which you can read here. Basically everything that follows in this section is a condensed version of that posting, so consult that for the long-form reference.
2. As you go through the different sections of the paper, make your notes in the Major/Minor/Editorial sections of your review template.
   1. Major points are serious issues that require addressing.
   2. Minor points are less serious issues, but ones which you think will help improve the paper.
   3. Editorial points are things like typos, grammar, and so on. It’s up to you how fussy you want to be about these things, and remember that the authors of the paper are unlikely to be native English-speakers. Don’t get hung up about syntax and grammar if the message is clear.
3. Read the abstract to get a sense of the context. Note any points that occur to you in your Summary section (always quote line numbers, if available).
4. Go through the Figures and the Figure legends.
   1. A well-constructed paper will be intelligible from its Figures and Figure legends alone, and it’s important to assess the data before you encounter the rhetoric in the Results. Note any small changes that need making in your Editorial points section.
   2. If you’re unsure how Figure Legends should be written, check out my posting on that topic here.
   3. Pay close attention to the quality of the data. Are there appropriate controls? Have the data been quantified?
   4. Highlight any issues with the data presentation or visualisation.
5. Go through the Introduction and References.
   1. Have the Introduction section and the References visible side-by-side (this is why you extracted the References earlier on). This will let you check the citations as you go through.
   2. Check that the background to the project is sketched, that the knowledge gap is defined, and the aim of the project is clearly stated.
   3. Ensure that relevant publications are cited, and always ensure that primary literature is cited as far as possible. Reviews should only be cited for the most general statements.
6. Go through the Results and the Figures.
   1. You’ve seen the figures once already, but now you’re going to revisit them while the authors try to convince you that their interpretation is a valid one. Have the Results and the Figures open side-by-side (again, this is why you extracted the Figures section earlier).
   2. Read the Results carefully, and make sure that the authors’ statements are supported by the data in the figures. If there’s a disconnect, it’s important that you highlight this.
   3. Ensure that the supplementary data really are supplementary. It’s surprisingly common to find important experiments relegated to the supplementary section, where they’re far less likely to be read.
7. Go through the Discussion.
   1. Decide whether the authors have placed their results in context, and provided appropriate interpretations for their observations.
   2. Have they acknowledged shortcomings? Have they highlighted possible alternative interpretations? Have they noted where their observations either match or contradict previously-published work?
8. Go through the Materials & Methods
   1. Ensure that the authors have provided enough detail to make it possible for others to repeat their experiments.
   2. If you’re not sure how M&M sections should be written, have a look at my posting on that topic here.

Phew! That’s probably just over half of the work done now. But it’s downhill from here.

4. After you’ve finished going through the paper:
Now is the time to collect all of your observations into coherent feedback for the authors and editor.

1. First, collect all your Summary observations together and write an overview of the manuscript. Briefly mention its main findings, its relevance, its importance, and especially the quality of the data.
   1. Send a clear signal to the editor whether you are enthusiastic/positive/lukewarm/negative about the manuscript’s suitability for immediate publication.
   2. Mention areas where you feel you don’t have the expertise to review them properly, so that the editor can ensure these are covered by the other reviewers.
2. Next, collect all your Major and Minor points together, and make sure that they are numbered. You may find that there is some reassignment at this stage – things which you first thought were Major issues may have been resolved by the time you reached the end, and things which you may have thought were Minor may be consistent and increasingly problematic issues.
   1. Remember that more than 3 Major Points are usually grounds for rejection by a journal. Think carefully about how Major your major points really are.
   2. Minor points can be more numerous. Group observations together into single points where possible.
3. Check your Editorial points. These will already be in manuscript order, and should require little to no additional work. If there are recurring issues (e.g. no mention of biological replicates and independent experiments in the figure legends) these can potentially be highlighted as a Minor Points.
4. Go back through the review and check you’re happy with it. If you have a problem paper, don’t be too meek in your assessment. It is better to take a firm (but polite) line than to tip-toe around and risk the editor thinking that problems are not as severe as you think. The editor is looking to you for guidance as they make their decision, so make sure that you (politely) send a clear signal. Important points need to be made clearly (they are easily lost in a long review). Don’t be rude, but don’t be afraid of clearly expressing yourself either.
5. Consider signing your review at the end. In my experience, this is a good way of signalling that you are not hiding behind anonymity and are focusing on the science. (It’s worth noting that there are two schools of thought on this point, so don’t feel obliged to do it).
6. Write the confidential comments to the editor.
   1. This should not be a long section and can be left completely blank if need be.
   2. If (like me) you’ve written a pretty long review and raised a lot of points, it’s worth highlighting whether you are generally favourable or not. A good manuscript may still have lots of little things that could be altered.
   3. Acknowledge any other people who contributed to your review.
   4. If you’ve chosen to sign your review, it’s worth highlighting it here.
   5. Disclose any relationships with authors, if necessary. Obviously if you have a conflict of interest then you shouldn’t review in the first place, but in smaller research communities there are often connections that are worth noting just to head off any accusation of bias.

5. Before sending it off:
Do not under any circumstances send the review off just yet! You need to leave it for a day and then come back to it to check that everything is ok.

1. Remember that you’re the midwife – you’re assisting the delivery of this paper, not standing in judgement. It’s your job to try and help the authors, and ensure that it’s as good as it reasonably can be.
2. Read the review again and make sure that it is coherent and makes sense.
3. Check that you have provided constructive criticism that the authors can implement (if they choose do).
4. Check your language very carefully to make sure that you have not been rude or too brief or too dismissive. Remember: this is very likely to be someone’s PhD thesis and nobody out there deliberately sets out to do shoddy work. You are helping the authors make this paper better, not trying to destroy it.
5. When you’re happy with everything, submit the review.

And that’s it! I very much hope that’s of use. Two things to highlight one last time: (1) there are many different ways to go about reviewing papers and this is just one of them, and it’s the approach I use. Feel free to implement as much or as little as you see fit; (2) as already noted several times, much of the content of this posting overlaps with my “How to…” guide to the critical reading of scientific papers, and it’s worth reading that for a longer discussion of many of the points raised here.

Lastly, I’d love to get any feedback (reviews of this posting, as it were), so if you have any suggestions you’d like to make, leave a comment below or send me a message via email or Twitter.

Brooke Morriswood
January 2022