Artificial intelligence and illusions of understanding in scientific research

Scientists are enthusiastically imagining ways in which artificial intelligence (AI) tools might improve research. Why are AI tools so attractive and what are the risks of implementing them across the research pipeline? Here we develop a taxonomy of scientists’ visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings. But proposed AI solutions can also exploit our cognitive limitations, making us vulnerable to illusions of understanding in which we believe we understand more about the world than we actually do. Such illusions obscure the scientific community’s ability to see the formation of scientific monocultures, in which some types of methods, questions and viewpoints come to dominate alternative approaches, making science less innovative and more vulnerable to errors. The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less. By analysing the appeal of these tools, we provide a framework for advancing discussions of responsible knowledge production in the age of AI.

AI Research Agents Narrow Scientific Exploration

AI research agents now support large-scale AI-assisted scientific discovery. We examine whether AI-generated ideas broaden scientific exploration or primarily reinforce existing work. Using five agent frameworks and five large language models, we generate 219,655 ideas for different scientific fields. Across experiments, four consistent patterns emerge. First, AI-generated ideas are more concentrated than human-authored papers within the same research area. Second, they remain much closer to starting literature than later human follow-on work does. Third, AI-generated ideas align less with future human research. Last, AI-generated ideas are located in lower-impact regions of the historical scientific landscape. Overall, current AI research agents appear better suited to local elaboration than to broadening scientific exploration.

AI has supercharged scientists—but may have shrunk science

Analysis of 41 million papers finds that although AI expands individual impact, it narrows collective scientific exploration

Charting AI’s Role in Scientific Discovery — Renaissance Philanthropy – A brighter future for all through science, technology, and innovation

Renaissance Philanthropy, with support from Google.org , is conducting a landscape study of AI integration in scientific research — and we want your perspective.

ScientistTwo: Pioneering the Human Knowledge Frontier with Autonomous AI

Scientific discovery is defined by the ability to identify the boundaries of existing knowledge and venture into unexplored territory. The ultimate vision for AI in science is problem-driven autonomous research: given a fundamental challenge by a human expert, the AI independently navigates the scientific landscape, uncovers theoretical and empirical bottlenecks, and systematically expands the frontier of knowledge. In this paper, we introduce ScientistTwo, a fully autonomous multi-agent framework designed to realize this vision. Specifically, ScientistTwo takes an initial problem as input, establishes state-of-the-art baselines, formulates novel hypotheses, and coordinates specialized agents to orchestrate an end-to-end discovery cycle without human intervention. Moreover, the framework rigorously conducts experiments using diverse datasets and metrics, refines methodologies through automated ablation studies, and validates research findings via a closed-loop simulated peer-review rebuttal engine. To evaluate ScientistTwo's capabilities against the highest standards of human scientific achievement, we benchmark it across papers accepted at top-tier conferences such as ICLR, ICML, and NeurIPS. As a result, ScientistTwo autonomously generates expert-level, publishable papers and fully verified, executable codebases. Its solutions consistently outperform human state-of-the-art models, and achieve higher average review ratings than human-authored papers under automated AI review agents. These results show that ScientistTwo is not merely an assistive tool but an autonomous scientific pioneer capable of pushing the frontiers of human discovery. Project website: https://scientist-two.github.io/

AI as the Catalyst of the New Paradigm of Science? | Cadmus Journal

The rapid advancement of Artificial Intelligence (AI) is reshaping the landscape of scientific inquiry. This article examines the challenges confronting modern science, including knowledge fragmentation, diminishing returns of current paradigms, and the disconnect between science and society. It explores how AI offers solutions enhancing collaboration and integrating knowledge—as well as considers the possible “darker side” of using AI in research. However, the advent of AI in science not only enhances scientific research but also redefines the role of human researchers and fosters a new paradigm of human understanding. By embracing these transformations, a more cohesive and responsive scientific enterprise can emerge.

Where does the rigor go? Research software and the future of trustworthy science.

Generative AI now makes it dramatically easier to produce something that looks like research: analysis code, figures, literature reviews, even whole pap…

AI maps science papers to predict research trends two to three years ahead

The number of scientific papers is growing so rapidly that scientists are no longer able to keep track of all of them, even in their own research area. Researchers from the Karlsruhe Institute of Technology ...

Project Rachel: Can an AI Become a Scholarly Author?

This paper documents Project Rachel, an action research study that created and tracked a complete AI academic identity named Rachel So. Through careful publication of AI-generated research papers, we investigate how the scholarly ecosystem responds to AI authorship. Rachel So published 10+ papers between March and October 2025, was cited, and received a peer review invitation. We discuss the implications of AI authorship on publishers, researchers, and the scientific system at large. This work contributes empirical action research data to the necessary debate about the future of scholarly communication with super human, hyper capable AI systems.

Will AI spark a scientific renaissance — or a diffuse monoculture?

Artificial intelligence’s ability to enrich science will depend not only on model capability, but also on whether researchers, reviewers and funders reward originality over speed.

AI in Science: Boosting Productivity and Innovation | Arthur Turrell posted on the topic | LinkedIn

Today at Google & Google DeepMind, we published work led by Mihai Codreanu on AI in Science, drawing on 15 million Gemini interactions, 2,600 specialised AI models & a 600-scientist survey. What did we find about how AI is changing science? Read on... First, why does this matter? If AI is a tech boost we get a one-off economic growth boost. But if AI is an "innovation in the method of innovation", we get long run higher expected future growth rates! Compound interest makes those scenarios very different for future prosperity. Where might we see innovation in the method of innovation first? Well, science of course—the engine of innovation and one of the main drivers behind the huge increase in prosperity over the last 200 years. So what do we find? 1) In our data, scientists are about 2.7x overrepresented in AI use compared to what you'd expect from employment share. Scientists are using AI a lot! 2) Science convos are longer (tokens), more multimodal, and involve more domain expertise 3) Computer science has highest use 4) specialised models and LLMs are complements: LLMs do code, stats, literature, drafting, teaching, admin while specialised models predict, generate, classify — eg protein structures, weather. 5) within LLMs, analysis & modelling + communicating research findings are top 2 tasks But are we getting *improvements*? 6) 74% of scientists surveyed report net time savings; (6% report losses). Average saving of ~7 hours/week. This is mostly going back into more research. 47% report an increase in outputs of 10% or more. There's another effect too though—as some things (hypothesis generation, modelling) become 'cheap', science gets bottlenecked by other tasks... 7) 89% of those who save time spend more than a tenth of it checking AI outputs & 41% report a growing backlog of untested hypotheses Looking a bit beyond, we expect time spent on bottleneck tasks to grow as a %, more papers that recombine ideas from across disciplines, & publication volume increases for fields that don't need wet lab work. Unfortunately, not all of this volume may be high quality ("sci-slop") So what's our early take on AI as an innovation in the method of innovation? There's significant potential to increase scientific productivity, & 7 hours/week (big!) is *already* being saved. But it's early days & bottlenecks (inc. verification) limit sci-productivity take off Huge team involved in this work, as you can imagine—see the paper for more! https://lnkd.in/g55s4MtC | 38 comments on LinkedIn

AI, peer review and the human activity of science

When researchers cede their scientific judgement to machines, we lose something important.

Deep Research, information vs. insight, and the nature of science

What AI will accelerate in the scientific process, what it cannot do, and how we can prepare for new manners of scientific investigation.

No. 002: Hallucinated citations at scale, ICML catches AI reviewers, scholarly infrastructure gaps - The Hallway Track

The Hallway Track No. 002 - How AI is affecting the practice of science

How to build an AI scientist: first peer-reviewed paper spills the secrets

AI Scientist, an autonomous research tool, first released in 2024, has now undergone peer review, highlighting its strengths and limitations.

AI Scientists as Engines of Discovery: A Case for Development within Reformed Institutions

Agentic artificial intelligence (AI) systems are beginning to assist, accelerate, and partially automate scientific discovery, performing tasks that span literature synthesis, code generation, data analysis, hypothesis proposal, and model criticism. We argue that this transition is qualitative rather than incremental, and that suitably designed multi-agent systems may evolve from passive computational tools into ``AI scientists'' that can expand the hypothesis-generating and verification capacity of science. Such systems must be developed and deployed within a scientific ecosystem fit for purpose: institutions must be redesigned for verification, accountability, interpretability, and dual-use safety. We sketch how multi-agent architectures, illustrated by the prototype framework \textit{Denario}, accelerate the discovery cycle and traverse model spaces beyond human reach; examine what this implies for authorship, peer review, and the enduring role of human scientists; and close with recommendations for governing AI as an epistemic actor rather than a mere instrument.