Commentary on “Top nine pitfalls to avoid when writing a journal peer review report”

Dear Editor, with great interest, I read the remarkable brief report by El-Sobky,^[1] where the author elegantly alluded to nine pitfalls related to formulating a peer-review report when judging a scientific manuscript.

I noticed that the author addressed his report mainly to junior reviewers, advising them on some particular pitfalls and how to avoid them. However, I found it suitable to add a tenth pitfall, which is mainly related to using artificial intelligence (AI) and its related application for generating the peer review report. I might call it “Avoid the temptation of using AI tools to generate faster peer review reports.”

It is undeniable that AI and its related applications are heavily involved in clinical and research-related orthopedics activities.^[2-5] However, its judicious and honest usage relies mainly on the user rather than the application itself.

As regards generating a peer review process, a junior reviewer might find him/herself in a position where they already accepted an invitation to review a manuscript based on the title and abstract provided, then it turns out that the manuscript is beyond their capabilities, or in some instances, they have passed the deadline to deliver the peer review report, here an apparently magic and fast solution might arise, which is using AI-related applications to generate the report.

Using AI tools during the peer review process is a double-edged sword where it might have some benefits, but harm is inevitable. Supporters of its usage argue that it helps check language clarity, logical soundness, and research structure, saving the reviewer time to focus on the scientific content rather than correcting the manuscript’s language and structure.^[6] Conversely, concerns regarding the confidentiality and integrity of manuscript evaluation have been raised, as uploading a manuscript to one of the AI tools is a clear breach of the manuscript confidentiality agreement.^[7] Furthermore, in some studies, the authors showed that AI tools could not identify fundamental flaws related to the evaluated study.^[8] Finally, the AI-generated peer review report might present a superficial analysis, provide generic feedback, lack human expert judgment, and be unable to evaluate manuscript creativity and novelty.^[7]

It is worth noting that using AI tools while drafting a peer-review report has been a concern among various scientific bodies and journal editorial boards. According to Cheng et al., apart from the ban implemented by the National Institutes of Health (NIH) on using online AI tools for drafting peer review reports, the same policy was applied by the Australian Research Council; furthermore, the latest recommendations update of the International Committee of Medical Journal Editors stated that uploading a manuscript received for peer review to an online AI platform is considered a breach of confidentiality.^[9,10] With the emergence of concerns related to AI-generated peer-review reports, many journals modified their peer-review regulations to prohibit or restrict using AI tools for generating peer-review reports, and the reviewer must disclose its usage.^[9,11]

Apart from the drawbacks mentioned earlier, AI-assisted peer-review report generation could be helpful in several aspects where human peer reviewers are deficient.^[12] First, it is more effective in detecting research misconduct and authors’ adherence to reporting guidelines (such as CONSORT guidelines for reporting randomized controlled trials). Second, it ensures fairness when evaluating a manuscript and avoids the bias of human reviewers, which might originate in the single-blind peer-review model. Third, AI tools could compensate for the deficiency of human reviewers or at least participate in the peer-review process; Fiorillo and Mehta suggested the integration of AI tools in the peer-review process, where the manuscript will have an AI-assisted peer-review round before it is sent to a human peer reviewer, which shortens time and improves the submitted manuscript quality.^[13]

A further advantage related to AI tools’ involvement in the peer-review process might be its initial triage role, which journal editors usually perform. Some journals receive an extensive number of submissions, and their editors become overwhelmed by this enormous number of manuscripts, which should be initially assessed for sending for peer-review rounds or deciding on immediate rejection; this step might be exhausting and time-consuming, especially for young editors.

Moreover, apart from its well-known roles related to detecting plagiarism and helping locate human reviewers, AI tools (available online for anyone to use or developed explicitly by some publishers) might assist young editors in many other aspects.^[14-16] First, some tools will help rapidly identify the manuscript’s suitability for the journal, speeding up the preliminary manuscript triage. Second, a compliance automation checker tool such as “Penelope.ai” helps in confirming that the submitted manuscript adheres to the journal’s instructions. Third, detecting the AI-generated text (a common practice by paper mills) using a tool such as “Gepetto.” Fourth, the “Snappshot” tool helps identify manipulated figures and images. What are previously mentioned and even more available tools could save the editors time and effort, improve the efficiency of submission processing and workflow, and avoid human-related inherent biases.^[16,17]

One last issue worth mentioning related to the future of implementing AI tools in editorial and peer-review tasks. Considering the plethora of AI tools development, it is undoubtful that their involvement in editorial and peer-review processes is expected to grow. For editors, this might involve 1-introducing a partially or fully AI-operated editorial team, 2-performing all initial checks (language quality assessment, plagiarism detection, and adherence to journal guidelines evaluation), 3-grading the quality of the submitted manuscript, and deciding on priority for publications, and 4-AI-assisted peer reviewer selection based on the best match for a particular manuscript. For the peer-review system, this might include 1-involving solely AI-peer reviewers for AI-generated studies (especially those including analysis of big data or performing network metanalysis), two journals might offer trustable AI tools for reviewers to use while assessing manuscripts, and 3-freely available AI-peer review tools might help authors in evaluating their manuscripts before submitting to specific journals.