1. Step by step guidelines to reviewers

2. Working with editors

3. How to review revised manuscripts

4. For reviewing a clinical manuscript

5. Reviewing Registered Reports

Step by step guide to reviewers

            The abstract of the paper should be sent to you when you are invited to peer review. This will help you to decide whether you wish to take part in the review. It will be easier to avoid delays if you respond to invitations promptly. As well as declaring any potential conflicts of interest, it is important to detail them at this point.

Overview of the review report format

            The review report’s format differs between journals. Others take a more formal approach, while some adhere to an informal structure.

Informal Structure

            Many journals only want your “analysis of merits” as the review criteria. In this situation, you may want to become familiar with examples of prior reviews written for the journal, which the editor should be able to supply, or you may want to rely on your own developing style as you acquire experience.

Formal Structure

            A more formal approach is necessary for other journals. On occasion, you will be sent a questionnaire and asked to respond to certain questions in your evaluation. Or they can want you to use a scorecard to assess the manuscript according to several criteria. These are frequently hidden until after you check in and submit your review. Therefore, it is wise to check for any journal-specific standards and guidelines before accepting the work. If there are official guidelines, let these govern how your review is organised.

In Both Cases

            You should anticipate compiling comments to writers and perhaps private ones to editors alone, whether the reporting style clearly calls for it or not.

First Read Considerations

            When skimming, keep a pen and paper close at reach.Try to keep the following considerations in mind as they will aid in forming your general opinion:

• What is the primary issue that the study aims to address? Is it fascinating and pertinent?
• How unique is the subject? Compared to other published materials, what does it bring to the field?
• Is the paper written well? Is the writing readable and clear?
• Are the findings in line with the facts and justifications offered? Do they answer the primary query?
• Does the author have a strong case if they strongly disagree with the prevailing academic consensus? If not, what would be necessary to support their position?
• What value do tables and figures add if they are included in the paper? Do they contribute to understanding or are they unnecessary?

Spotting Potential Major flaws

While you should read the whole paper, making the right choice of what to read first can save time by flagging major problems early on.

Editors say, “Specific recommendations for remedying flaws are VERY welcome.”

Examples of possibly major flaws include:

• Drawing a conclusion that is contradicted by the author’s own statistical or qualitative evidence

• The use of a discredited method
• Ignoring a process that is known to have a strong influence on the area under study

If experimental design features prominently in the paper, first check that the methodology is sound – if not, this is likely to be a major flaw.

You might examine:

• The sampling in analytical papers
• The sufficient use of control experiments
• The precision of process data
• The regularity of sampling in time-dependent studies
• The validity of questions, the use of a detailed methodology and the data analysis being done systematically (in qualitative research)
• That qualitative research extends beyond the author’s opinions, with sufficient descriptive elements and appropriate quotes from interviews or focus groups

Major Flaws in Information

If methodology is less of an issue, it’s often a good idea to look at the data tables, figures or images first. Especially in science research, it’s all about the information gathered. If there are critical flaws in this, it’s very likely the manuscript will need to be rejected. Such issues include:

• Insufficient data
• Unclear data tables
• Contradictory data that either are not self-consistent or disagree with the conclusions
• Confirmatory data that adds little, if anything, to current understanding – unless strong arguments for such repetition are made

If you find a major problem, note your reasoning and clear supporting evidence (including citations).

Concluding the First Reading

After the initial read and using your notes, including those of any major flaws you found, draft the first two paragraphs of your review – the first summarizing the research question addressed and the second the contribution of the work. If the journal has a prescribed reporting format, this draft will still help you compose your thoughts.

The First Paragraph

This should state the main question addressed by the research and summarize the goals, approaches, and conclusions of the paper. It should:

• Help the editor properly contextualize the research and add weight to your judgement
• Show the author what key messages are conveyed to the reader, so they can be sure they are achieving what they set out to do
• Focus on successful aspects of the paper so the author gets a sense of what they’ve done well

The Second Paragraph

This should provide a conceptual overview of the contribution of the research. So consider:

• Is the paper’s premise interesting and important?
• Are the methods used appropriate?
• Do the data support the conclusions?

After drafting these two paragraphs, you should be in a position to decide whether this manuscript is seriously flawed and should be rejected (see the next section). Or whether it is publishable in principle and merits a detailed, careful read through.

Rejection After the First Reading

Even if you are coming to the opinion that an article has serious flaws, make sure you read the whole paper. This is very important because you may find some really positive aspects that can be communicated to the author. This could help them with future submissions.

A full read-through will also make sure that any initial concerns are indeed correct and fair. After all, you need the context of the whole paper before deciding to reject. If you still intend to recommend rejection, see the section “When recommending rejection.”

Before Starting the Second Read-Through

Once the paper has passed your first read and you’ve decided the article is publishable in principle, one purpose of the second, detailed read-through is to help prepare the manuscript for publication. Of course, you may still decide to reject it following a second reading.

The benchmark for acceptance is whether the manuscript makes a useful contribution to the knowledge base or understanding of the subject matter. It need not be fully complete research – it may be an interim paper. After all research is an incomplete, on-going project by its nature. The detailed read-through should take no more than an hour for the moderately experienced reviewer.

“Offer clear suggestions for how the authors can address the concerns raised. In other words, if you’re going to raise a problem, provide a solution.” (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Preparation

To save time and simplify the review:

• Don’t rely solely upon inserting comments on the manuscript document – make separate notes
• Try to group similar concerns or praise together
• If using a review program to note directly onto the manuscript, still try grouping the concerns and praise in separate notes – it helps later
• Note line numbers of text upon which your notes are based – this helps you find items again and also aids those reading your review
• Keep images, graphs and data tables in clear view – either print them off or have them in view on a second computer monitor or window

Now that you have completed your preparations, you’re ready to spend an hour or so reading carefully through the manuscript.

Doing the Second Read-Through

As you’re reading through the manuscript for a second time, you’ll need to keep in mind the argument’s construction, the clarity of the language and content.

With regard to the argument’s construction, you should identify:

• Any places where the meaning is unclear or ambiguous
• Any factual errors
• Any invalid arguments

You may also wish to consider:

• Does the title properly reflect the subject of the paper?
• Does the abstract provide an accessible summary of the paper?
• Do the keywords accurately reflect the content?
• Is the paper an appropriate length?
• Are the key messages short, accurate and clear?
• Check the Language.
• Not every submission is well written. Part of your role is to make sure that the text’s meaning is clear.

Editors say, “If a manuscript has many English language and editing issues, please do not try and fix it. If it is too bad, note that in your review and it should be up to the authors to have the manuscript edited.”

If the article is difficult to understand, you should have rejected it already. However, if the language is poor but you understand the core message, see if you can suggest improvements to fix the problem:

• Are there certain aspects that could be communicated better, such as parts of the discussion?
• Should the authors consider resubmitting to the same journal after language improvements?
• Would you consider looking at the paper again once these issues are dealt with?

On Grammar and Punctuation

Your primary role is judging the research content. Don’t spend time polishing grammar or spelling. Editors will make sure that the text is at a high standard before publication. However, if you spot grammatical errors that affect clarity of meaning, then it’s important to highlight these. Expect to suggest such amendments – it’s rare for a manuscript to pass review with no corrections.

A 2010 study of nursing journals found that 79% of recommendations by reviewers were influenced by grammar and writing style (Shattel, et al., 2010).

The Second Read-Through: Section by Section Guidance

1. The Introduction

A well-written introduction:

• Sets out the argument
• Summarizes recent research related to the topic
• Highlights gaps in current understanding or conflicts in current knowledge
• Establishes the originality of the research aims by demonstrating the need for investigations in the topic area
• Gives a clear idea of the target readership, why the research was carried out and the novelty and topicality of the manuscript