AI, peer review and the human activity of science

AI’s Growing Role as Scientific Peer Reviewer | Stanford HAI

Stanford computer scientist James Zou is exploring how AI can accelerate scientific research and peer review. His finding: AI excels at spotting gaps, but judgment calls still need humans.

Science Is Drowning in AI Slop

Peer review has met its match.

How do authors want to use AI for review?

A survey of researchers who compared AI-generated scientific reviews with journal-agnostic human peer review reveals that they overwhelmingly prefer using AI as a self-checking tool before submission rather than as a replacement for human reviewers. It encourages an “author-centric” model in which AI helps researchers improve their manuscripts before they are reviewed by their peers.

AI peer review goes pro

A what we're reading spotlight

Artificial intelligence and illusions of understanding in scientific research

Scientists are enthusiastically imagining ways in which artificial intelligence (AI) tools might improve research. Why are AI tools so attractive and what are the risks of implementing them across the research pipeline? Here we develop a taxonomy of scientists’ visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings. But proposed AI solutions can also exploit our cognitive limitations, making us vulnerable to illusions of understanding in which we believe we understand more about the world than we actually do. Such illusions obscure the scientific community’s ability to see the formation of scientific monocultures, in which some types of methods, questions and viewpoints come to dominate alternative approaches, making science less innovative and more vulnerable to errors. The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less. By analysing the appeal of these tools, we provide a framework for advancing discussions of responsible knowledge production in the age of AI.

AI has supercharged scientists—but may have shrunk science

Analysis of 41 million papers finds that although AI expands individual impact, it narrows collective scientific exploration

Deep Research, information vs. insight, and the nature of science

What AI will accelerate in the scientific process, what it cannot do, and how we can prepare for new manners of scientific investigation.

Major AI conference flooded with peer reviews written fully by AI

Nature - Controversy has erupted after 21% of manuscript reviews for an international AI conference were found to be generated by artificial intelligence.

Artificial intelligence tools expand scientists’ impact but contract science’s focus

Nature - Artificial intelligence boosts individual scientists’ output, citations and career progression, but collectively narrows research diversity and reduces collaboration, concentrating...

A New Paradigm for Scientific Publishing, Peer Review, and Impact Assessment

Scientific publishing and peer review have evolved little in three centuries, while the demands placed on them have grown profoundly. The growing role of artificial intelligence has underscored deep, systemic shortcomings of an aging system that has largely evaded innovation, a system whose origins are appallingly closer to the invention of the printing press than to the internet. We can do better – much better. This article is intended as the beginning of a communal experiment: a living document that critically reviews the modern academic publishing and peer-review system and presents a concrete framework to address what bibliometrics experts¹ have characterized as "the pervasive misapplication of indicators to the evaluation of scientific performance". Building on the Leiden Manifesto, DORA, and a body of scholarship spanning many disciplines and decades, we present a community-governed, non-profit platform organized around three trust-weighted impact factors, for articles, authors, and reviewers, with full algorithmic transparency, an open development log, and structural decoupling of credibility scoring from content moderation and from monetization. We invite the community to discuss, critique, and help shape it.

The Age of Academic Slop is Upon Us

what happens when AI automates "normal science"?

The AI Chemist: To be trustworthy, LLMs need to show their work

Good scientists reveal how they do their experiments and report their results; so should any machine-driven research

More Versus Better: Artificial Intelligence, Incentives, and the Emerging Crisis in Peer Review | Organization Science

As the AI Task Force for Organization Science, we provide an early account of artificial intelligence’s (AI) impact on both submissions and reviews at a major academic journal. Submission volume ha...