How to review a journal article: requirements, tips and strategies

David J. Pannell School of Agricultural and Resource Economics, University of Western Australia 1. What the journals tell you The notes in this section are adapted from instructions provided by the Agronomy Journal, the Journal of Consumer Research, the American Journal of Pharmaceutical Education, and the Canadian Journal of Agricultural Economics. The ideal review will be fair, unbiased, speedy, and confidential. The ideal reviewer will approach the paper in terms of questions such as: "Is the science good?" and "Is it understandable?" or "What is needed to make it clear?" rather than "What are all the little things that annoy me in style or presentation?" 1.1 Ethics Volunteer reviewers and editorial board members are asked to decline from reviewing papers of any authors with whom there is a possibility or appearance of a conflict of interest. Can you can answer "yes" to one or more of the following questions?

Have you had significant and acrimonious disagreements with the authors in the past?

Are the authors and you co-investigators on a current research project?

Have the authors and you jointly published an article in the past 5 years?

Are you close personal friends with one or more of the authors?

Are you working in such a similar area of research as the authors that you might be considered to be a competitor or gain an advantage by reviewing the manuscript?

Did you review the manuscript as a peer reviewer prior to its submission to the journal?

If so, you should respond as follows: (a) if there is a clear problem with you doing the review, withdraw, or (b) if you are uncertain about whether it is appropriate, advise the editor of the situation and ask him or her to judge whether you should review the paper. Reviewers must attempt to be impartial when evaluating a manuscript. Although it is difficult to be completely objective when assessing a paper that may not coincide with one's own beliefs or values, nevertheless, a reviewer must always strive for that goal. If a reviewer cannot separate the evaluation process from a desire to advocate a preferred theory or to reject the manuscript out-of-hand on philosophical grounds, then the reviewer should disqualify himself or herself from that review. Do not allow the manuscript to be reproduced while in your custody. You must not use the manuscript for your personal advantage in any way. You cannot cite it or use its contents in any way until/unless it is published. If it is not published but you wish to use it, you need to contact the author (e.g. via the editor if the journal uses double-blind reviewing). 1.2 Abstract Abstracts accompany articles in most journals, and they are often republished as printed in secondary abstracting services and journals. The abstract, therefore, should meet two requirements. A reader should be able to tell readily the value of the article and whether or not to read it completely. It also should provide the

literature searcher with enough information to assess its value and to index it for later retrieval. According to the Agronomy Journal, the abstract should:

Strive for an impersonal, non-critical, and informative account.

Give a clear, grammatically accurate, exact, and stylistically uniform treatment of the subject.

Provide rationale or justification for the study. The statement should give a brief account of the purpose, need, and significance of the investigation (hypothesis or how the present work differs from previous work).

State the objectives clearly as to what is to be obtained. Give a brief account of the methods, emphasizing

departures from the customary. Be specific.

State key results succinctly. Outline conclusions or recommendations. An emphasis of the significance of This the may work, conclusions, new and

recommendations.

include

theories,

interpretations, evaluations, or applications. Be quantitative and avoid the use of general terms, especially in presenting the method and reporting the results. For example, if two rates of a treatment are used, state what they are.

Never cite references. Contain about 200 to 250 words.

Different journals specify different lengths for their abstracts. Some are not long enough to do all of the points listed above. If cutting aspects out, I'd look at dropping methods, reducing objectives to a minimum, and limiting results and conclusions to absolute highlights.

1.3 Manuscript General Content

Does the title of the paper clearly reflect its contents, and is the title sufficiently succinct? Is an abstract present?

Is it consistent with the length used by this journal? Is the content of the manuscript worthwhile? If not to you, is there a segment of the journal's readership that would find it worthwhile?

Do you feel that the author(s) reviewed the existing literature adequately?

Do you know of any references that authors might want to refer to and discuss? Are references listed according to the journals formatting style?

Are all references cited listed in the reference list and vice versa?

Does the formatting of headings, tables and figures correspond to journal style?

Are the conclusions supported by the data presented? Are there alternative explanations for the findings presented? Should the manuscript be shortened?

Is it well organised?

Across all the review rounds, reviewers should strive to distinguish between what is perceived as correctable versus uncorrectable problems and between major versus minor concerns. The first round of reviews is the time for reviewers to highlight uncorrectable problems or other major concerns about a paper. It is generally inappropriate to raise them in later review rounds if they already existed in the first draft submitted. In most cases, new uncorrectable problems or new major concerns raised in later reviews should only apply to changes in a paper that have emerged through the authors' revision work.

1.4 Quality of Writing

Clarity is vitally important. Whether or not you are an expert in the subject discussed, you should understand the paper's content. Read each paragraph carefully. Is there likely to be confusion? If so, request that the author clarify. If you have some suggested revisions, these are usually appreciated by authors, but please don't feel obliged to rewrite the manuscript.

Do the paragraphs flow smoothly? Is the manuscript readable? Can you make suggestions for improvement? (Suggest using active voice.)

Is there unnecessary repetition? Can you suggest deletion of sentences or phrases or words that add little to the paper?

Are enough examples provided to assist readers in relating to the author' s ideas? Can you suggest some examples that the author might want to include in his or her revision?

What parts of the manuscript do you really like? Let the author(s) know.

1.5 Technical

Is the paper technically accurate in terms of methods, procedures, replications, statistics, and so forth?

1.6 Tables and Figures

Are all the tables and figures really necessary? If so, are they understandable? If not, could you suggest another format? Would you suggest additional tables or figures?

Are figures (or photographs) of good enough quality for reproduction in the journal? Are the numbering and lettering large enough to be readable when reduced?

Are the figure and table captions clear?

1.7 Remember ...

Reviewers should not attempt to rewrite a poorly written manuscript.

Reviewers will remain anonymous. Prompt attention to manuscripts will be appreciated both by the authors and by the Editors.

1.8 What you provide to the editor 1. A brief letter addressed to the editor (not for the authors eyes) with a specific recommendation. Different journals have slight variations in the list of possible recommendations. They are likely to be similar to the following options: a. Accept unconditionally. This rarely used category should be reserved for manuscripts that are virtually flawless in their content. In general, when a reviewer makes this

recommendation, he or she will be regarded as having signed off on the manuscript. b. Accept conditionally, subject to minor revisions, according to accompanying comments. This recommendation should be made when the manuscript is judged to be quite strong and in need of only minor additions, deletions, or corrections. It does not need to be reviewed again, in your view. The editor can judge whether revisions have been made adequately. c. Encourage revision, according to accompanying comments. This recommendation should be used for manuscripts that have a high degree of potential for eventual publication, in addition to significant deficiencies that must be corrected. For instance, the research addresses a vital topic, but is not

presented in a logical or lucid fashion, or questions may exist about some aspects of its method. This recommendation should be used when the reviewer believes that satisfactory resolution to his or her concerns is possible and that the achievement of successful resolution will result in an acceptable manuscript. Detailed Comments to the Authors are extremely important in support of this recommendation, so that the authors can conceivably answer all the concerns in a single revision. (A recommendation in this category should not be construed as a guarantee of eventual publication. In some cases, a seemingly promising

manuscript will not be adequately revised to attain the quality and level of knowledge contribution required for publication.) d. Reject in current form, but allow resubmission of a substantially different version. The essential difference between this recommendation (reject in current form) and the option above (encourage revision) is that the current version of the manuscript is not publishable in anything approximating its present form. Instead of simply rejecting a manuscript as completely unsalvageable, this

recommendation includes thoughtful advice for producing a potentially publishable, but different manuscript. This

recommendation should be used sparingly. Some journals do not include it as an option. For these, you would probably switch to the next option. e. Reject unconditionally, because the likelihood of successful revision is remote. This recommendation is appropriate for papers that have major problems. It is viewed as having virtually no chance of ever reaching a publishable standard, at least for this journal. For example, the topic may be of minor importance; the basic conceptual development may be

extremely weak or incorrect; or the empirical work may have uncorrectable defects. This category is the most common recommendation received by high prestige journals.

Comments to the Authors should be especially polite in explaining the nature of the concerns, but the Comments to the Authors need not be as lengthy as in the previous categories. It is permissible and efficient to articulate only the most serious concerns, and to conserve reviewing energy for other manuscripts that stand to gain from more detailed reviewer input. Briefly indicate your reasons for the decision in the letter to the editor. You might address issues such as the significance of the contribution, quality and thoroughness of the analysis, clarity of presentation, practical importance, originality, depth and appropriateness for the Journal. In some cases, a reviewer may be competent to evaluate either the conceptual or methodological aspects of a paper, but not both. In such instances, the reviewer should indicate this in the cover letter to the Editor. 2. Separate comments to the author. Please ensure that comments are detailed and clear and that they are constructive in nature, even if you are recommending that the paper be rejected. Make suggestions about length, organization, tables, figures and references. Number all comments, to make it easy for the author to refer to them in correspondence with the editor. A quality review should note the strengths as well as the weaknesses of the manuscript. Authors need to know what they have done well and not just what they have done poorly. The reviewer should avoid language that can be easily misconstrued or that appears condescending, including obscure or loaded vocabulary as well as humour, irony, and sarcasm. When the reviewer negatively evaluates different

aspects of a paper, he or she should keep in mind that there are many ways to communicate this to the authors. For example, phrases such as "fatal flaws" or "serious mistakes" might instead be offered as "substantial concerns" or "major issues." Reviewers should be as polite and as diplomatic as they are demanding. If the reviewer has ideas on how to improve the paper and the research, sharing those suggestions in the most precise manner possible will raise the likelihood that the authors will understand, appreciate, and utilize those ideas. Specificity. A quality evaluation must be communicated in sufficient detail to support the reviewer's recommendation to the Editor. For instance, when alluding to previous research to uphold an assertion about some conceptual, methodological, or substantive weakness, it is imperative to provide a complete citation so the Editor and authors can locate it. The Comments to the Authors represent the most important component of the review. They provide the rationale for the reviewer's evaluation of the manuscript, as well as suggestions for the improvement of the paper. The Comments to the Authors should also not contain any semblance of a recommended rejection or acceptance of the manuscript. Such

recommendations should be made only in the confidential Summary Letter to the Editor. It is the Editor's responsibility to make the final decision and also to decide on the best way to communicate that decision to the authors. Comments to the Authors are generally most useful to the authors and the Editor when they begin with an overall assessment of the reviewer's reaction to the manuscript, including prominent strengths and weaknesses. This big picture is valuable in providing a context for the more detailed comments that follow. The summary is especially important in identifying the reviewer's major concerns. After the overall assessment, the detailed comments are then provided that tactfully justify the evaluation and offer constructive, specific guidance for a revision or for future research efforts. It can be useful to provide the most

important of the specific comments before any minor comments on matters of detail. 2. What the journals dont tell you 2.1 Some questions you may wonder about Why were you chosen as a reviewer? There are several possible reasons. An article of yours may have been cited in the manuscript to be reviewed. You may have submitted an article to the journal yourself. The editor may know you or know of you from other sources. You may have been recommended to the editor by someone else. Do you have to be an expert in the subject of the article? No. You have to have enough expertise in related areas to be able to make sense of it, at least at a general level, but you dont necessarily have to be knowledgeable enough to, say, reproduce the research yourself. It is up to the editor to ensure that the set of reviewers covers enough areas of expertise. If there are aspects that you are unable to evaluate (e.g. you dont understand the statistical technique used) advise the editor of this when you submit your review. Do you have to agree to do the review? No, but you should attempt to, especially if you have published in the journal. If you just cant fit it in, decline politely, as early as possible. What degree of quality control do reviewers provide? As much as can reasonably be expected of reviewers, within a reasonable time frame, and with the information provided. But not more than that. Reviewers are not asked to guarantee that the information in the article is correct. They are not asked to reproduce the research, which is the only way that some errors could be

detected. They are asked to assess whether it appears to be correct, judging with the best of their ability. Plenty of published articles have been found to contain errors. This is obviously better avoided if possible, but sometimes the reality is that it is impossible. Why should you review articles? You should review articles because it is part of the culture of research that we all contribute to this essential service. It is likely that in the long run the amount of reviewing you are asked to do will be roughly proportional to the number of articles you submit to journals. Reviewing is also good for you. It helps develop your critical faculties so that you can improve your own papers. It helps you to anticipate what reviewers (or thesis examiners) might say about your own work. 2.2 A suggested strategy Here is a strategy that I have found works for me. a. Work from a printed copy. This makes it easier to make notes on it as you go, and to read it in places where you would not take a computer. b. Read through the paper, making notes at the level of detail that is appropriate for that paper. The better the paper, the finer detail you should go to. For a paper that is excellent and needs only very minor revisions to be accepted, you might even note problems with punctuation and spelling, if they are not too numerous. For a paper that is terrible, you would only note major issues, or even major groups of issues. Sometimes I get hooked into rewording sentences. You should only do this if the paper doesnt need too much of it. If bits need rewriting, say so in your review.

c. If the decision is really obvious and you are confident about your ability to judge the paper, you could write your review immediately after your first reading. Otherwise, leave it a few days or a week and come back to it. Read it again, or read problem sections again, and then write your review. d. When selecting your recommendation, consider the prestige of the journal and the usual standard of articles it includes. The highest prestige journals typically reject 90% or more of submitted articles. You are unlikely to be asked to review for such a journal early in your publishing career, unless you have managed to publish in one yourself. The least selective journals probably accept somewhere around 50-70% of submitted articles (after revision of course), but most and 50% of

journals probably submissions.

accept between 20

e. If you are not going to reject the paper, obtain a copy of a recent issue of the journal, or a paper from a recent issue, to check on formatting of headings, tables, figures, references. Note any problems in these areas in your review. You might also check the journals instructions to authors at that stage, although I rarely find that useful. f. Write the manuscript reference number and title at the top of your review. g. Always number your comments. If you have multiple related comments, number them separately. This allows the editor and the authors to easily refer to your comments in further correspondence. h. Start by saying something positive about the paper, no matter how difficult this is. Say as many positive things as

the paper deserves (or one more than that if it doesnt deserve any) before you get into criticisms. i. Some reviewers provide an overview of the paper. Personally, I don't think this is worth doing. The editor will read the paper too and the authors already know whats in it! j. Next, note any concerns or problems at the big picture level. e.g. relating to the overall approach, the statistical methods, the interpretation of results, the quality of presentation or writing. k. Then, start presenting specific comments on the paper. Note the page number and the line number for each comment if possible. Start with most important or serious of the specific comments, and then include any minor issues. l. There is no need to write a conclusion to your review. Just stop when you run out of comments. m. If you have cited any literature, provide complete reference details. Respond as quickly as you can manage. Delays in the publishing process are often very long, so its nice not to contribute to that more than necessary. Reviews are often between one and two pages single spaced. The longest I've written was about five pages, and the shortest was about 10 lines. Further reading Pannell, D.J. (2002). Prose, psychopaths and persistence: Personal perspectives on publishing, Canadian Journal of Agricultural Economics, 50(2): 101-116. Full paper (66K)

Charry AA, Murray-Prior R and Parton KA (2004). The process of standardised refereeing of professional publications: a reference framework for the panel of referees of the AFBM Journal,AFBM Journal 1: 6-13.

http://www.afbmnetwork.orange.usyd.edu.au/afbmjournal/volume 1/

Citation: Pannell, D.J. (2006). How to review a journal article: requirements, tips and strategies, Discussion Paper, School of Agricultural and Resource Economics, University of Western Australia,

http://dpannell.fnas.uwa.edu.au/reviews.htm

David Pannell home page Copyright Last revised: August 05, 2013. 2006-10 David J. Pannell