Exploring AI’s growing role in scientific peer review

AI’s Growing Role as Scientific Peer Reviewer | Stanford HAI

Stanford computer scientist James Zou is exploring how AI can accelerate scientific research and peer review. His finding: AI excels at spotting gaps, but judgment calls still need humans.

AI, peer review and the human activity of science

When researchers cede their scientific judgement to machines, we lose something important.

How to build an AI scientist: first peer-reviewed paper spills the secrets

AI Scientist, an autonomous research tool, first released in 2024, has now undergone peer review, highlighting its strengths and limitations.

No. 030: Google Deep Review, AI research taste, AI in dissertations - The Hallway Track

The Hallway Track No. 030 - How AI is affecting the practice of science

AI peer review goes pro

A what we're reading spotlight

Guest Post — Now is the Time for AI in Peer Review, and Publishing Policies Need to Recognize This - The Scholarly Kitchen

Today's post asks us to acknowledge the role of AI in peer review and ensure practical guidance and policies that help scholars respond with consistency and confidence.

How do authors want to use AI for review?

A survey of researchers who compared AI-generated scientific reviews with journal-agnostic human peer review reveals that they overwhelmingly prefer using AI as a self-checking tool before submission rather than as a replacement for human reviewers. It encourages an “author-centric” model in which AI helps researchers improve their manuscripts before they are reviewed by their peers.

No. 023: Co-Scientist, Terminal-Bench Science, AI in peer review - The Hallway Track

The Hallway Track No. 023 - How AI is affecting the practice of science

AI co-scientists are revolutionizing how research is done

Artificial-intelligence systems can generate hypotheses, design experiments and analyse data — but humans still need to decide what makes sense.

No. 014: Science's AI productivity paradox, Sakana clears peer review, AI proof search on open Erdős problems - The Hallway Track

The Hallway Track No. 014 - How AI is affecting the practice of science

Artificial intelligence tools expand scientists’ impact but contract science’s focus

Developments in artificial intelligence (AI) have accelerated scientific discovery1. Alongside recent AI-oriented Nobel prizes2–9, these trends establish the role of AI tools in science10. This advancement raises questions about the influence of AI tools on scientists and science as a whole, and highlights a potential conflict between individual and collective benefits11. To evaluate these questions, we used a pretrained language model to identify AI-augmented research, with an F1-score of 0.875 in validation against expert-labelled data. Using a dataset of 41.3 million research papers across the natural sciences and covering distinct eras of AI, here we show an accelerated adoption of AI tools among scientists and consistent professional advantages associated with AI usage, but a collective narrowing of scientific focus. Scientists who engage in AI-augmented research publish 3.02 times more papers, receive 4.84 times more citations and become research project leaders 1.37 years earlier than those who do not. By contrast, AI adoption shrinks the collective volume of scientific topics studied by 4.63% and decreases scientists’ engagement with one another by 22%. By consequence, adoption of AI in science presents what seems to be a paradox: an expansion of individual scientists’ impact but a contraction in collective science’s reach, as AI-augmented work moves collectively towards areas richest in data. With reduced follow-on engagement, AI tools seem to automate established fields rather than explore new ones, highlighting a tension between personal advancement and collective scientific progress.

Charting AI’s Role in Scientific Discovery — Renaissance Philanthropy – A brighter future for all through science, technology, and innovation

Renaissance Philanthropy, with support from Google.org , is conducting a landscape study of AI integration in scientific research — and we want your perspective.

Deep Research, information vs. insight, and the nature of science

What AI will accelerate in the scientific process, what it cannot do, and how we can prepare for new manners of scientific investigation.

No. 002: Hallucinated citations at scale, ICML catches AI reviewers, scholarly infrastructure gaps - The Hallway Track

The Hallway Track No. 002 - How AI is affecting the practice of science