#Interviews
#Interviews
26.03.2024
Peer review: the challenge of making it more diverse
Laura Feetham-Walker, from the British publishing company IOP Publishing, discusses the importance of engaging early career and female researchers in manuscript revision processes
Image: Shutterstock
For her work dedicated to improving academic training and certification in peer review processes, British Laura Feetham-Walker, Reviewer Engagement Manager at IOP Publishing, one of the world’s leading scientific publishing houses based in the UK, was awarded the 2024 APE Award for Innovation in Scholarly Communication, during the 19th Academic Publishing in Europe Conference (APE), in Berlin, Germany.
With a master’s degree in demography and health from the London School of Hygiene & Tropical Medicine and a career working for journals such as the British Medical Journal (BMJ) and The Lancet, Feetham-Walker is currently working on initiatives to improve the peer review process, in which researchers assess the relevance and accuracy of the content included in a scientific study before it is accepted for publication.
At IOP Publishing, Feetham-Walker employs strategies to engage reviewers, incentivizing early career researchers and women in order to make the review process more equitable, diverse, and inclusive—since this process is still mostly carried out by men and senior researchers.
To this end, she is committed to training and certifying new reviewers so that they gain the necessary confidence and evaluate manuscripts responsibly.
In an interview with Science Arena, Laura Feetham-Walker reflects on the current peer review landscape and the potential impact of artificial intelligence in training the next generation of reviewers. Additionally, she lists the most common mistakes that reviewers should avoid and discusses the challenges inherent to improving peer review in various fields of knowledge.
Science Arena – Are we experiencing a global peer review crisis? If so, at what level?
Laura Feetham-Walker – The peer review system has not reached crisis level, but there are, indeed, a series of challenges that we, the publishers, along with other key players in the scientific publication arena, have been aware of and are working to address. Many of the problems faced by the peer review system have emerged slowly over several years, allowing academic publishers to anticipate, prepare for, and adjust to them.
Other challenges were relatively sudden—such as the abrupt emergence of open-source generative artificial intelligence. However, as a publication company, we responded quickly, implementing policies to guide authors and reviewers. The peer review is imperfect, but it’s resilient.
What are the obstacles to increasing the number of reviewers capable of evaluating studies responsibly, given the high demand for submitted manuscripts? How can we avoid overwhelming the reviewers?
To properly answer this question, we need to analyze the various reasons why some reviewers are overwhelmed. Indeed, there is an increasing number of manuscripts submitted for peer review, and this means that reviewers are, on average, receiving more invitations to review. However, another contributing factor is the historical lack of diversity in peer review.
Until relatively recently, only a select portion of qualified researchers were invited to participate in the peer review process, while professionals from certain geographical regions (such as the Global South), women, and early career researchers were largely excluded.
There are many experts who are perfectly qualified to review manuscripts, but who historically have not been invited to do so.
This is something we are changing. An important element in increasing the number of reviewers is approaching these underrepresented groups and inviting them to participate. This is what we have been actively doing for several years. They are less likely to be overwhelmed by invitations and tend to provide excellent review reports.
That being said, diversifying the reviewer pool is not the only solution. We also need to make the review process as simple and efficient as possible for our reviewers. We have done this by simplifying how we communicate with our reviewers and optimizing our peer review systems.
What does IOP Publishing’s Reviewer Engagement Program consist of and what initiatives have been adopted?
The main goal of IOP Publishing’s Reviewer Engagement Program has always been to improve peer review in the physical sciences, such as physics, chemistry, and Earth sciences. The starting point was to interview a representative group of researchers from these fields and ask them what they wanted from the review process.
Based on their responses, the Peer Review Excellence Program was born. We began by offering reviewers significant recognition for their contributions.
Our community has also requested comprehensive, high-quality training in the peer review process. This request came from early career researchers who wanted to be trained in peer review skills. A number of senior researchers also showed interest, revealing a desire for their doctoral and postdoctoral students to receive high-quality training.
Our peer review training is offered flexibly on various platforms. There are free online courses, and we host interactive workshops, both online and in person.
And lastly, we are constantly seeking to introduce innovations to the peer review process. This includes creating a co-review feature, which allows two or more people to collaborate on a review, for which they will all receive recognition. We also have a new reviewer feedback program—this means that all of our reviewers can request feedback from the publisher about their work.
At a certain point you talk about “giving recognition” to researchers who are willing to work as reviewers. Would this recognition be financial compensation or some other type?
Recognition for reviewers, we launched the first recognition programme for reviewers which links certification to the quality and timeliness of reviews. Reviewers who submit outstanding reviews or are consistently helpful and reliable are awarded with formal IOP Trusted Reviewer certification. The threshold for certification for our training graduates (those who have completed our online training or an interactive Peer Review Excellence workshop) is lower, making them better able to achieve certification.
What are the main challenges inherent to recruiting and training new reviewers?
Academic researchers are under enormous pressure from many angles. In addition to conducting their research, they are expected to do all types of work, including teaching, applying for grants, networking, promoting their work, and, of course, peer review.
The biggest challenge inherent to recruiting and training new reviewers is that they often feel they don’t have time to familiarize themselves with the process. To overcome this hurdle, we offer training in various formats, including our online course, which can be taken gradually, in stages.
Giving reviewers meaningful recognition for their work and a certification they can include in their CVs also helps with recruitment and retention.
Could new technology such as artificial intelligence impact training for future generations of reviewers?
Generative artificial intelligence is developing so rapidly that it is impossible to be certain of its future role in peer review training. It is worth mentioning, however, that effectively training the next generation of peer reviewers involves much more than presenting them with the facts.
Peer review is a practical skill—some even call it an “art”—and our training offers concrete experience, as well as being highly interactive.
In our workshops, participants discuss real peer review opinions with expert publishers, and many participants tell us that this is the most impactful part of the training.
What’s more, not only do we provide trainees with skills, but we also give them confidence. Often, early career researchers have all the knowledge they need to provide an excellent review opinion, but what they lack is the confidence to get involved.
Our training boosts their confidence and lets them know that we will support them during the process of submitting their first opinion. I think it is unlikely that artificial intelligence will be able to fulfill all of these functions any time soon.
What are the most common mistakes made by reviewers and how can they be avoided?
One common mistake is not having a coherent structure for an opinion, which can result in missing important elements of the manuscript. A good, simple structure is as follows:
1) Start with a brief summary of the manuscript—this helps organize your thoughts and shows the publisher and the authors that you understood the overall research objectives and findings; 2) list the major concerns; 3) then, list the minor concerns; 4) include a concluding paragraph summarizing the strengths and weaknesses; 5) end by expressing your recommendation to the publisher.
Another common mistake is for reviewers to get bogged down in correcting minor grammatical or language errors. We value our reviewers for their scientific expertise, and we do not want them to waste their time editing texts.
What are the qualities expected for an academic reviewer and what are the challenges to improving peer review?
The most important quality is scientific knowledge—this is why we approach each of the thousands of people who review for us every year. Another essential quality is honesty and integrity.
Fortunately, in the vast majority of cases, our peer reviewers are incredibly honest and conscientious. They truly want to use their time to push science forward. Peer review—and science itself—depend heavily on the ethical behavior of the individuals involved.
This means being transparent about any conflicts of interest of which the publisher may not be aware; providing impartial reviews; striving to submit opinions within a reasonable timeframe; and reading the manuscripts in full. In other words, providing the type of peer review they would hope to receive for their manuscripts.
*
This article may be republished online under the CC-BY-NC-ND Creative Commons license.
The text must not be edited and the author(s) and source (Science Arena) must be credited.
