External peer review is a hallmark of science. “Published in a peer-reviewed journal” is a sign of quality, meaning that the work has been scrutinised by knowledgeable and independent peers. Moreover, peer review also serves the purpose of improving the work after the authors have done their best and, thus, being a reviewer carries a responsibility. However, it comes with little reward; reviewing is usually done without financial compensation and often outside of work hours. But reviewing is rewarding in itself! Besides being pro bono, it exposes you, as a reviewer, to novel findings and adds a new perspective to your own research and scientific writing. And you can add “Peer reviewer for Journal of X” to your CV.
This Doing Science article will deal with peer reviewing of journal articles and conference abstracts. For those of you familiar with the series, the format will be slightly different. Different people tend to have rather personal approaches to reviewing, and rather than generic tips, this article is built around an interview with a very experienced reviewer, and our own personal reflections as reviewers and editors.
The typical review process for a scientific peer-reviewed journal.
The Associate Editor
After the author, maybe with trembling hands, hits the “Submit article” button, an e-mail titled “Please be associate editor for a new manuscript” appears in my mailbox. After scanning the title and abstract of the manuscript, I click “agree” if the manuscript is not clearly outside my area of expertise. You would be surprised how wide the range of topics of received manuscripts is! As an author, it is mandatory to carefully select the journal to submit to and to choose a clear, descriptive title and adequate key words, to increase your chances of reaching the right Editor immediately, and also aid the Editor in finding the right reviewers.
The reviewers I invite should be experts in the subject area of the manuscript, but also be skilled in reviewing and have the time to do it (and I cannot use the same people too often either). Some manuscripts require reviewers from several fields (e.g. an article called “Lung function in Parkinson’s disease” may require reviewers in physiology, neurology and respirology). I also consider geographical and sometimes cultural factors and, of course, there must be no conflicts of interest.
As a junior researcher, my contact network is still limited, and I often invite reviewers I don’t know personally. Using the provided standard invitation letter, I have sometimes ended up inviting ten or even 15 people before getting two to accept! Besides keeping me busy, this substantially delays the review process. Recently, I instead started checking out potential reviewers, e.g. on PubMed and LinkedIn, and writing individualised, personal invitation letters. This has increased my success rate tremendously, and made me realise the importance of reviewers feeling personally involved.
Optimally, I try to pair one experienced with one more junior reviewer since, at least for me, it was difficult to get reviewer invitations early in my career. For your first few reviewer assignments, ask a senior staff member to read and give feedback before submitting. And always read the other reviewer’s comments to further hone your skills. Improving the quality of reviews is our collective responsibility.
Mina Gaga on reviewing
Mina Gaga is past Secretary General of the ERS and a long-time reviewer for the European Respiratory Journal.
You have a request to review a manuscript, what do you consider before even accepting?
It's a matter of time, obviously. Another important factor is whether I find the abstract interesting and whether I consider that I am a fit reviewer for that particular subject. An abstract that is confusing, badly written or does not provide a new idea or finding will usually cause the reviewer to refuse.
How did the journal find you to be a reviewer?
Usually, the associate editor has an area of expertise and knows the people who are working in the field. If the editors run out of people they consider as experts, they can look at the references of the paper but also the list of reviewers of the particular journal. Senior colleagues may also suggest a junior researcher as reviewer.
If you don't feel fine with the manuscript and you would recommend some corrections, how do you decide on a minor or major revision?
Well, a minor revision means “I am almost happy with the paper”, i.e. the authors just need to add a couple of articles in the references or correct something that’s unclear in the text or badly phrased. A major revision means “I think the paper needs a lot of work”. In many journals, it's either a revision or no revision: accept, reject or revise. At the end of the day, the decision for major revision lies mainly with the editor.
Which approach would you recommend for reviewing?
I start with the abstract and definitely the introduction, because it allows me to understand what the paper is about and then I do a quick scan: methods, results and discussion. Then I do a second reading that is point by point. But I think the approach differs with each individual.
What is the best way to help the authors improve the manuscript?
As a reviewer, you mean? I think the first thing is to state what the important points are in this paper, the good points, the new things and then also describe your concerns: i.e. if there is something controversial in the introduction, unclear methodology or something that is odd or already established, which therefore adds no new info. Then list the minor problems; addressing these will also help the paper. Even if a paper is rejected by a specific journal, the reviewers’ comments can always make a paper better before a new submission. But the end result lies with the authors and how they deal with the comments!
As a reviewer with much experience, can you give some advice for young authors?
It’s important to have a very well written and clear abstract. If English is not your native language, maybe you can have some help, because good language increases the chances of the paper being reviewed favourably. A really important thing is a clear message and a reader-friendly manuscript. Know your field and describe it but do not try to incorporate everything in the paper. Make sure your methods are robust and, if you have used a novel method, make sure you can defend it and use the online supplements if you need to go in depth. Keep results short and clear. Tables and graphs help but there should not be too many. Finally, in the discussion, briefly state your findings and, whether they are novel or confirmatory, discuss them against the existing data and state the strengths and weaknesses of the paper.
Notes from a reviewer of conference abstracts
As a reviewer of conference abstracts, e.g. at the European Respiratory Society (ERS) International Congress, you are “only” dealing with abstract texts. However, due to the number of submitted abstracts and the wide variation in topic and quality, the review process for these is a time-consuming task. Grading abstracts is different from grading a manuscript. Usually there are 50 or more abstracts to read. To be objective, I recommend reading them twice. The first reading should concentrate on each abstract separately and be based on novelty of the study, study design, methodology and style. The second reading is to adjust scores on the basis of average abstract quality. The chance of acceptance as well as the audience can vary between ERS Assemblies; so, for authors, it may be wise to discuss with Assembly officials or other active senior ERS members before submission. These people also serve as a good starting point if you want to become a reviewer.
Along these lines, I also recommend that abstract authors should do an internal review before submission. As an abstract author, your opinion on the content is frequently biased and strongly depends on your personal engagement. To increase your acceptance rate ask a “critical friend” not engaged in the work to read the text for you and propose improvements in due time before final submission. The best candidate for your private reviewer is a friend with native language skills who will be able to find and point out the negative points without prejudice. Choose someone you trust and whose advice you are truly willing to follow.
Table 1 summarises the differences and similarities between the reviewing processes for manuscripts and congress abstract, as well as highlighting the difference in the expectations and experiences of authors and reviewers. Remember, the more and better you write, the sooner you will also be invited as peer reviewer. Good luck with the next one!
The importance of giving useful feedback when reviewing
When reviewing an article, it is important that feedback is useful to author and written in an appropriate way. Some real examples of reviewer comments are shown below. Some are rather impolite but, more importantly, the reviewers in these examples provide no specific criticisms as to where the paper under review is deficient and how it could be improved.
“It seems like you have gone fishing for something interesting in available data you hold and would like to squeeze a little extra out of?!”
“The discussion is not a critical discussion of the data presented but is rather a series of hypotheses easily sustainable by the data of the study”
“The presentation of the data is simply very confusing and hard to follow”
“The point is – this is number magic – but what does it tell?! Never-smokers have more comorbidities, such as tuberculosis???”
“Maybe try to have a go for a national magazine?!”
Footnotes
Statement of Interest
None declared.
- ©ERS 2014
Breathe articles are open access and distributed under the terms of the Creative Commons Attribution Non-Commercial Licence 4.0.
