We are Changing our Developer Productivity Experiment Design

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

We conduct a randomized controlled trial to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.

AI | 2025 Stack Overflow Developer Survey

84% of respondents are using or planning to use AI tools in their development process, an increase over last year (76%). This year we can see 51% of professional developers use AI tools daily.

Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

How do the productivity effects of AI evolve across successive generations of tools, and to what extent do task-level gains ultimately translate into final outp

Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

How do the productivity effects of AI evolve across successive generations of tools, and to what extent do task-level gains ultimately translate into final outp

AI boosts worker productivity — but does that translate to final outputs? | MIT Sloan

A new study found that developers using AI tools can write much more code than those working without AI, but they don’t release as much new software.

The ‘productivity paradox’ of AI adoption in manufacturing firms | MIT Sloan

Companies that adopt industrial artificial intelligence see productivity losses before longer-term gains, according to new research.

Writing Code vs. Shipping Code: Productivity Effects Across Generations of AI Coding Tools

How do the productivity effects of AI evolve across successive generations of tools, and to what extent do task-level gains ultimately translate into final output? We study these questions in the context of software development, using data on more than 100,000 GitHub developers combined with their AI usage telemetry. In a matched event study design, we find that autocomplete, interactive coding agents, and autonomous coding agents each significantly increase coding activity ("commits"), with respective cumulative effects of 40%, 140%, and 180%. These gains, however, attenuate sharply across the production hierarchy: the 180% cumulative effect falls to 50% for the number of projects, and to 30% for actual releases. This pattern is consistent with the weak-link hypothesis: the strong productivity gains from AI are attenuated by human bottlenecks in the production chain, with an estimated elasticity of substitution of 0.25 between AI and human effort, which indicates strong complementarities. We further confirm these results across four major app marketplaces, finding a moderate increase in the number of new apps but no increase in total usage. Large task-level AI productivity gains have therefore translated only partially into shipped and used software thus far.

Arvind Narayanan on Twitter / X

To understand and empathize with how workers in many or most fields outside software experience advances in AI capabilities, I propose a little thought experiment. https://t.co/QZG24aRiBe— Arvind Narayanan (@random_walker) July 28, 2026

Measuring the Impact of Early-2025 AI on Experienced Open-Source...

Despite widespread adoption, the impact of AI tools on software development in the wild remains understudied. We conduct a randomized controlled trial (RCT) to understand how AI tools at the...

Tech companies are cutting jobs and betting on AI. The payoff is far from guaranteed

AI experts say we’re living in an experiment that may fundamentally change the model of work

How building software is changing at Anthropic

A deepdive on what’s changed in how the leading AI lab makes software. Ever more code review and testing is done by AI, two-pizza teams very much alive, and more. Details from inside of Anthropic

Study finds AI tools made open source software developers 19 percent slower

Coders spent more time prompting and reviewing AI generations than they saved on coding.

Women executives warn AI adoption is outpacing workforce development

A Chief and Harris Poll survey reveals how women leaders are shaping AI adoption by prioritizing workforce development and ethical implementation over speed.

The AI Engineering Report 2026: The AI Acceleration Whiplash - Ten Takeaways

What two years of telemetry data from 22,000 developers reveals about AI's real impact on developer productivity, code quality, and business risk in 2026.

Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity

Despite widespread adoption, the impact of AI tools on software development in the wild remains understudied. We conduct a randomized controlled trial (RCT) to understand how AI tools at the February-June 2025 frontier affect the productivity of experienced open-source developers. 16 developers with moderate AI experience complete 246 tasks in mature projects on which they have an average of 5 years of prior experience. Each task is randomly assigned to allow or disallow usage of early 2025 AI tools. When AI tools are allowed, developers primarily use Cursor Pro, a popular code editor, and Claude 3.5/3.7 Sonnet. Before starting tasks, developers forecast that allowing AI will reduce completion time by 24%. After completing the study, developers estimate that allowing AI reduced completion time by 20%. Surprisingly, we find that allowing AI actually increases completion time by 19%--AI tooling slowed developers down. This slowdown also contradicts predictions from experts in economics (39% shorter) and ML (38% shorter). To understand this result, we collect and evaluate evidence for 20 properties of our setting that a priori could contribute to the observed slowdown effect--for example, the size and quality standards of projects, or prior developer experience with AI tooling. Although the influence of experimental artifacts cannot be entirely ruled out, the robustness of the slowdown effect across our analyses suggests it is unlikely to primarily be a function of our experimental design.