AI’s Growing Role as Scientific Peer Reviewer | Stanford HAI

Exploring AI’s growing role in scientific peer review

Stanford computer scientist James Zou discusses AI’s strengths at spotting errors and limitations in judging research significance.

AI, peer review and the human activity of science

When researchers cede their scientific judgement to machines, we lose something important.

AI peer review goes pro

A what we're reading spotlight

How do authors want to use AI for review?

A survey of researchers who compared AI-generated scientific reviews with journal-agnostic human peer review reveals that they overwhelmingly prefer using AI as a self-checking tool before submission rather than as a replacement for human reviewers. It encourages an “author-centric” model in which AI helps researchers improve their manuscripts before they are reviewed by their peers.

How to build an AI scientist: first peer-reviewed paper spills the secrets

AI Scientist, an autonomous research tool, first released in 2024, has now undergone peer review, highlighting its strengths and limitations.

How well does AI peer review work?

A new evaluation benchmark

Guest Post — Now is the Time for AI in Peer Review, and Publishing Policies Need to Recognize This - The Scholarly Kitchen

Today's post asks us to acknowledge the role of AI in peer review and ensure practical guidance and policies that help scholars respond with consistency and confidence.

Peer review is overwhelmed—can it survive in the AI era?

As research and AI-assisted papers surge, volunteer reviewers struggle to keep up.

Major AI conference flooded with peer reviews written fully by AI

Nature - Controversy has erupted after 21% of manuscript reviews for an international AI conference were found to be generated by artificial intelligence.

More Versus Better: Artificial Intelligence, Incentives, and the Emerging Crisis in Peer Review | Organization Science

As the AI Task Force for Organization Science, we provide an early account of artificial intelligence’s (AI) impact on both submissions and reviews at a major academic journal. Submission volume ha...

Science Is Drowning in AI Slop

Peer review has met its match.

AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot

Scientific peer review faces mounting strain as submission volumes surge, making it increasingly difficult to sustain review quality, consistency, and timeliness. Recent advances in AI have led the community to consider its use in peer review, yet a key unresolved question is whether AI can generate technically sound reviews at real-world conference scale. Here we report the first large-scale field deployment of AI-assisted peer review: every main-track submission at AAAI-26 received one clearly identified AI review from a state-of-the-art system. The system combined frontier models, tool use, and safeguards in a multi-stage process to generate reviews for all 22,977 full-review papers in less than a day. A large-scale survey of AAAI-26 authors and program committee members showed that participants not only found AI reviews useful, but actually preferred them to human reviews on key dimensions such as technical accuracy and research suggestions. We also introduce a novel benchmark and find that our system substantially outperforms a simple LLM-generated review baseline at detecting a variety of scientific weaknesses. Together, these results show that state-of-the-art AI methods can already make meaningful contributions to scientific peer review at conference scale, opening a path toward the next generation of synergistic human-AI teaming for evaluating research.

No. 023: Co-Scientist, Terminal-Bench Science, AI in peer review - The Hallway Track

The Hallway Track No. 023 - How AI is affecting the practice of science