AI reviewers are here — we are not ready

How do authors want to use AI for review?

A survey of researchers who compared AI-generated scientific reviews with journal-agnostic human peer review reveals that they overwhelmingly prefer using AI as a self-checking tool before submission rather than as a replacement for human reviewers. It encourages an “author-centric” model in which AI helps researchers improve their manuscripts before they are reviewed by their peers.

Major AI conference flooded with peer reviews written fully by AI

Nature - Controversy has erupted after 21% of manuscript reviews for an international AI conference were found to be generated by artificial intelligence.

Peer review is overwhelmed—can it survive in the AI era?

As research and AI-assisted papers surge, volunteer reviewers struggle to keep up.

AI-assisted Reviewing is Necessary and Should be Open

Peer review is facing a death spiral. AI production tools are speeding it up. AI-assisted reviewing is necessary and should be open.

AI-assisted Reviewing is Necessary and Should be Open

Peer review is facing a death spiral. AI production tools are speeding it up. AI-assisted reviewing is necessary and should be open.

ChicagoHAI/OpenAIReview

AI reviewing paper drafts for improvement.

Guest Post — Now is the Time for AI in Peer Review, and Publishing Policies Need to Recognize This - The Scholarly Kitchen

Today's post asks us to acknowledge the role of AI in peer review and ensure practical guidance and policies that help scholars respond with consistency and confidence.

How well does AI peer review work?

A new evaluation benchmark

Adil on Twitter / X

being assigned to review AI generated slop papers is very frustrating. the way this is supposed to work is that first you take the time to write something and then i take the time to read it. if you don't do the first part then i shouldn't have to do the second part.— Adil (@adilsoubki) August 26, 2026

AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot

Scientific peer review faces mounting strain as submission volumes surge, making it increasingly difficult to sustain review quality, consistency, and timeliness. Recent advances in AI have led the community to consider its use in peer review, yet a key unresolved question is whether AI can generate technically sound reviews at real-world conference scale. Here we report the first large-scale field deployment of AI-assisted peer review: every main-track submission at AAAI-26 received one clearly identified AI review from a state-of-the-art system. The system combined frontier models, tool use, and safeguards in a multi-stage process to generate reviews for all 22,977 full-review papers in less than a day. A large-scale survey of AAAI-26 authors and program committee members showed that participants not only found AI reviews useful, but actually preferred them to human reviews on key dimensions such as technical accuracy and research suggestions. We also introduce a novel benchmark and find that our system substantially outperforms a simple LLM-generated review baseline at detecting a variety of scientific weaknesses. Together, these results show that state-of-the-art AI methods can already make meaningful contributions to scientific peer review at conference scale, opening a path toward the next generation of synergistic human-AI teaming for evaluating research.

AI’s Growing Role as Scientific Peer Reviewer | Stanford HAI

Stanford computer scientist James Zou is exploring how AI can accelerate scientific research and peer review. His finding: AI excels at spotting gaps, but judgment calls still need humans.

ReviewBench: An Extensible Framework for Benchmarking Human and AI Manuscript Review

The volume of scientific manuscripts is rising faster than the available pool of expert reviewers, and AI tools are emerging as a possible response, ranging from frontier large language models applied directly to peer review to purpose-built multi-agent systems. Scalable, standardized benchmarks are needed to regularly evaluate how these tools compare to one another and to human reviewers. We present ReviewBench, an open-source, venue-agnostic framework that compares human and AI reviews across structure, alignment with a paper’s major claims, impact, and critique category. We apply ReviewBench to 145,021 review comments from human reviewers, frontier large language models (GPT-5.2, and Gemini 3 Pro), and Reviewer3.com (R3), a multi-agent peer review system. The dataset spans papers in computer science (ICLR 2025, n = 1,000), social science (Nature Human Behaviour, n = 142), and life science (eLife, n = 1,000). Across disciplines, AI reviews are more structured and engage more directly with a paper’s major claims, with R3 more often surfacing consequential comments, defined as comments capable of undermining those claims. When restricting to critical comments, however, human reviewers rank first on consequential rate on more individual papers than any AI source, despite a lower average. We identify a bimodal reviewer distribution with peaks near 0% and 100%, indicating that many reviewers outperform AI on this metric, but a substantial fraction of reviewers near 0% brings the average down. Critique typing demonstrates systematic differences, where humans emphasize contribution and clarity, while AI emphasizes validity, sufficiency, and transparency. Together, these findings argue against framing AI as a replacement for human review and instead support a complementary model in which AI scales technical verification of major claims while human judgment remains essential for evaluating contribution and shaping editorial decisions.

Accelerating Science with Human+AI Review

This issue of NEJM AI features the first two articles published through our accelerated human+AI review process. In this editorial, we describe the invitation-only “Fast Track” process used to revi...