AI research speeds up. Monitoring gets harder. - Sensemaker
OpenAI says coding agents now run 3.1 days for every human research day, while its chief scientist warns the lab’s main monitoring method is weakening.
AI research speeds up. Monitoring gets harder. - Sensemaker
OpenAI says coding agents now run 3.1 days for every human research day, while its chief scientist warns the lab’s main monitoring method is weakening.
Research acceleration: The view inside OpenAI
Inside OpenAI, coding agents are reshaping AI research. Explore early data on agent usage, experiment velocity, task complexity, and research acceleration.
OpenAI News
Stay up to speed on the rapid advancement of AI technology and the benefits it offers to humanity.
Deep Research, information vs. insight, and the nature of science
What AI will accelerate in the scientific process, what it cannot do, and how we can prepare for new manners of scientific investigation.
Open source was not ready for AI-speed contributions
AI did not create the maintainer burden problem in open source. It accelerated it. Contributors are being amplified, but maintainers are still the verification bottleneck.
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
We conduct a randomized controlled trial to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.
Evaluating chain-of-thought monitorability
OpenAI introduces a new framework and evaluation suite for chain-of-thought monitorability, covering 13 evaluations across 24 environments. Our findings show that monitoring a model’s internal reasoning is far more effective than monitoring outputs alone, offering a promising path toward scalable control as AI systems grow more capable.
AI Is Slowing Down
If you liked this piece, you should subscribe to my premium newsletter. It’s $70 a year, or $7 a month, and in return you get a weekly newsletter that’s usually anywhere from 5,000 to 18,000 words, including vast, detailed analyses of NVIDIA, Anthropic and OpenAI’s
Measuring the Impact of Early-2025 AI on Experienced Open-Source...
Despite widespread adoption, the impact of AI tools on software development in the wild remains understudied. We conduct a randomized controlled trial (RCT) to understand how AI tools at the...
OpenAI launches new AI model for life sciences research
Researchers are drowning in data. AI could help.
Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
Despite widespread adoption, the impact of AI tools on software development in the wild remains understudied. We conduct a randomized controlled trial (RCT) to understand how AI tools at the February-June 2025 frontier affect the productivity of experienced open-source developers. 16 developers with moderate AI experience complete 246 tasks in mature projects on which they have an average of 5 years of prior experience. Each task is randomly assigned to allow or disallow usage of early 2025 AI tools. When AI tools are allowed, developers primarily use Cursor Pro, a popular code editor, and Claude 3.5/3.7 Sonnet. Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%--AI tooling slowed developers down. This slowdown also contradicts predictions from experts in economics (39% shorter) and ML (38% shorter). To understand this result, we collect and evaluate evidence for 20 properties of our setting that a priori could contribute to the observed slowdown effect--for example, the size and quality standards of projects, or prior developer experience with AI tooling. Although the influence of experimental artifacts cannot be entirely ruled out, the robustness of the slowdown effect across our analyses suggests it is unlikely to primarily be a function of our experimental design.
AI has supercharged scientists—but may have shrunk science
Analysis of 41 million papers finds that although AI expands individual impact, it narrows collective scientific exploration
Collection
Updated 4h
Primary sources for Sensemaker's September 7 brief on OpenAI's reported research acceleration, chain-of-thought monitoring limits, and safety pacing.
Peer review is facing a death spiral, and AI production tools are speeding it up. AI-assisted reviewing is necessary and should be open. We built OpenAIReview: open AI reviewing for everyone, for the cost of a coffee. openaireview.github.io/blog.html 🧵
AI-assisted Reviewing is Necessary and Should be Open
openaireview.github.io"AI makes it cheaper to contribute to Open Source, but it's not making life easier for maintainers. More contributions are flowing in, but the burden of evaluating them still falls on the same small group of people. That asymmetric pressure risks breaking maintainers." also relevant to slop science
