The future is now: Navigating artificial intelligence’s role in peer review

THE FUTURE IS NOW: NAVIGATING ARTIFICIAL INTELLIGENCE’S (AI’S) ROLE IN PEER REVIEW

Peer review in academic publishing is fundamentally defined as the process of evaluating new scholarship for its relevance, accuracy, and importance to a specific field.^[1] This rigorous assessment is conducted by experts in the same domain, who independently scrutinize the research to determine its quality and suitability for publication. The process is designed to ensure the validity, quality, and often the originality of the submitted work.

At its core, peer review functions as a foundational safeguard for the quality and integrity of scholarly research. It acts as a critical filter, preventing the dissemination of invalid or poor-quality articles into the academic record.^[1]

As scientific publishing accelerates and the sheer volume of research grows exponentially, the peer review system faces unprecedented strain. Reviewers, often volunteers, struggle with increasing workloads, pressure to deliver timely assessments, and the demanding task of scrutinizing increasingly complex methodologies. Another challenge is finding enough reviewers with the right level of experience to accomplish this task.

EI-Sobky wrote a nice paper highlighting the major pitfalls to avoid when writing journal peer-review reports of medical manuscripts, aiming to provide practical tips on how to overcome these pitfalls and guide junior reviewers through the peer-review process.^[1] Khalifa wrote a commentary on EI-Sobky’s paper, adding a tenth pitfall to avoid the temptation of using AI tools to generate faster peer-review reports.^[1,2]

The introduction of AI is not a replacement for human intellect, but a powerful ally in upholding the rigor and efficiency of scientific discourse. The integration of AI into the peer-review process is no longer a futuristic concept; a growing number of journals and publishers are actively implementing it.^[3] Publishers are increasingly integrating AI and algorithmic screening methods to assist with peer review. These tools, trained on vast bodies of papers and reviews, can check for various elements.

WHAT CAN AI DO (AND NOT DO) IN PEER REVIEW?

At present, AI tools are primarily designed to assist, not replace, human reviewers, or editors. Their strengths lie in tasks that are data-intensive, repetitive, and require rapid analysis:

• Plagiarism and duplicate submission detection: AI algorithms can swiftly scan vast databases of published and pre-print literature, identifying instances of plagiarism or previously published work with remarkable accuracy. This significantly reduces the burden on editors and reviewers to perform such exhaustive checks manually.^[3]

• Initial quality checks and manuscript screening: AI can be trained to identify common methodological flaws, inconsistent reporting, or adherence to specific journal guidelines (e.g., data availability statements, and ethical declarations). This can flag manuscripts that may not meet basic publishing standards, allowing editorial teams to filter them before reaching reviewers, thus saving valuable time.

• Identification of potential reviewers: Leveraging natural language processing, AI can analyze a manuscript’s content and recommend potential reviewers based on their publication history, keywords, and areas of expertise. This can streamline the reviewer selection process, ensuring that more relevant and qualified individuals are invited.^[4]

• Language and readability enhancements: For non-native English speakers, AI-powered tools can help improve the clarity, grammar, and overall readability of a manuscript, ensuring that the scientific message is effectively conveyed. While not directly part of the review, this pre-submission assistance can lead to higher-quality submissions for reviewers to assess.^[2]

• Bias detection (emerging area): Research is ongoing into using AI to identify potential biases within a manuscript, such as gender-biased language or stereotypical representations.^[3] While still in its infancy, this has the potential to foster more inclusive and equitable scientific communication.

It is crucial to emphasize what AI cannot do. AI lacks the capacity for critical thinking, nuanced interpretation, ethical judgment, and a profound understanding of the scientific context that human reviewers possess. It cannot evaluate the originality of a hypothesis, the logical flow of an argument, the significance of findings, or the broader implications of a study.^[2,5] These remain firmly within the domain of human expertise.^[6]

BENEFITS FOR EDITORS AND REVIEWERS

For the diligent editor and reviewer, AI assistance offers several tangible benefits:

• Reduced administrative burden: Automating initial triage frees up reviewers to focus on the core scientific evaluation.^[2,5,7]

• Improved efficiency: Quicker identification of suitable reviewers and faster initial screenings can expedite the overall review process.^[5]

• Enhanced quality control: AI acts as an initial filter, potentially reducing the number of fundamentally flawed manuscripts that reach a reviewer’s desk.

• Focus on substantive issues: With routine checks handled by AI, reviewers can dedicate more time and cognitive effort to the scientific merit, methodology, and novelty of the research.

ETHICAL CONSIDERATIONS AND THE PATH FORWARD

The integration of AI into peer review presents its own challenges and ethical considerations. Transparency is paramount. Journals employing AI tools must clearly communicate their use to authors and reviewers. Reviewers should be informed about which AI tools are being utilized and for what purpose.

Furthermore, issues of data privacy, algorithmic bias, and accountability must be addressed. AI models are only as good as the data they are trained on, and if that data reflects existing biases, the AI may perpetuate them. Rigorous testing, continuous refinement, and human oversight are essential to mitigate these risks.

The integration of AI introduces complex ethical and practical considerations. Concerns include:

• The potential for AI-generated reviews

• The ethical implications of using AI without proper human oversight^[2]

• Confidentiality issues when intellectual property is uploaded to AI systems.^[8]

A CALL TO EMBRACE AND ENGAGE

The role of reviewers remains indispensable. AI is a tool, a powerful augmentation to their expertise, not a replacement. Embracing these technological advancements means a more efficient, robust, and ultimately, more reliable peer-review system. We encourage the reviewers to familiarize themselves with the AI tools, their respective journals may be adopting, and to provide feedback on their effectiveness.

The evolving landscape of scientific publishing demands adaptability. By understanding and constructively engaging with AI-assisted peer review, the reviewers will continue to be the guardians of scientific quality, ensuring that only the most rigorous and impactful research contributes to the ever-expanding body of human knowledge. Their invaluable contributions, now bolstered by intelligent assistance, will continue to shape the future of science.

The ultimate challenge lies in balancing the desire for innovation and efficiency with the enduring need for expert human judgment, which remains irreplaceable in upholding the nuanced standards of quality and integrity across all domains where peer review is applied.

Editors and reviewers are advised to exercise extreme caution, fact-check AI-generated content, disclose its use transparently, and ensure that AI tools maintain the confidentiality of submitted material.