Do AI Coding Tools Actually Make Developers Faster? The 2026 Evidence

The most important finding on AI coding tools in 2025 is not that they make developers faster. It is that developers cannot tell whether the tools made them faster.

In July 2025, METR ran the only proper randomized controlled trial on the question. Sixteen experienced open-source maintainers. 246 real tasks on mature codebases — over a million lines of code, over 22,000 GitHub stars on average. Cursor Pro with Claude 3.5 and 3.7 Sonnet. Before the trial, the developers predicted a 24% speedup. After living through the measured slowdown, they still believed they had been 20% faster.

The measured result: AI made them 19% slower.

“When developers are allowed to use AI tools, they take 19% longer to complete issues — a significant slowdown that goes against developer beliefs and expert forecasts.”
— METR, arXiv:2507.09089

A 39-point gap between perception and reality. Every survey that asks developers “did AI make you more productive?” is reading the perception side of that gap. That includes Anthropic’s internal 50% number, Stack Overflow’s 52%, and most of the LinkedIn commentary on the topic.

The rest of this post is what the evidence on AI coding tools looks like once you stop trusting that question.

DORA 2025: AI is an amplifier, not a fix

DORA 2025 surveyed about 5,000 engineering professionals. AI adoption hit 90%. The median developer spends two hours per workday with AI tools, and 65% report “heavy reliance” on them.

The findings are mixed in a way that matters.

Throughput now positively correlates with AI adoption. So do individual effectiveness, code quality, and team performance. That is genuinely new in 2025; earlier DORA reports saw no throughput effect or a negative one.

Stability still negatively correlates with adoption. Quoting the report directly:

“Our data shows AI adoption not only fails to fix instability, it is currently associated with increasing instability.”

DORA’s lead, Nathen Harvey, frames the underlying mechanic this way:

“AI is an amplifier. It’s an amplifier of the things that you already have in your organization. You might be listening to a high school band full of amateurs and you just made it a lot louder; or you might be listening to a band of professional musicians, and that’s way better.”

We see the same pattern across engagements. Teams with working CI/CD and clear platform abstractions get faster and ship cleaner. Teams with long-lived branches and weak testing get faster at shipping the wrong thing.

Claude Code vs Cursor vs GitHub Copilot: what the evidence says

Tool	Strongest evidence	Headline win	Headline risk	Favored by
Claude Code	Anthropic internal (Aug 2025); skill-formation RCT (Feb 2026)	+67% PRs/dev/day post-rollout	−17pt junior comprehension under full delegation	Small teams (46% “most loved”, Pragmatic Engineer 2026)
Cursor	U. Chicago observational (2025); Speed at the Cost of Quality (Nov 2025)	+39% merged PRs, PR size unchanged	Persistent rise in static-analysis warnings and complexity	Mid-market, agentic workflows
GitHub Copilot	NAV IT longitudinal (2025, 26k commits)	High accept rate on routine completion	+40% secrets-leak rate; no measured commit-activity lift	Enterprises (~56% in 10k+ employee orgs)

Claude Code

Anthropic’s August 2025 study covered 132 engineers, 53 in-depth interviews, and 200,000 Claude Code sessions. Self-reported productivity boost: 50%. Daily share of work touched by Claude: 59%. PRs merged per engineer per day after rollout: up 67%. 27% of Claude-assisted work was “tasks that wouldn’t have been done otherwise.”

The honest caveats are in the same study. Over 50% of Anthropic’s engineers could fully delegate only 0–20% of their work.

Then there is Anthropic’s February 2026 RCT on skill formation — 52 mostly-junior developers learning Trio in Python. Those using AI scored 17 percentage points lower on comprehension tests after the task. Full-delegation users came in under 40%. Conceptual users — the ones who asked the AI to explain before generating — landed at 65% or higher.

Claude Code is a strong tutor and a fragile crutch. The difference is whether the developer is doing the thinking.

The Pragmatic Engineer’s March 2026 survey (around 900 respondents) ranked Claude Code as “most loved” at 46%. Small teams favor it. Large enterprises favor Copilot.

Cursor

A University of Chicago study, run across 24 Cursor-adopting orgs and eight matched controls, found +39% merged PRs after the agent became default. PR size did not grow. Revert rate did not move. That is a cleaner result than most of the literature offers.

The counterpoint is a November 2025 paper, Speed at the Cost of Quality, running difference-in-differences on matched GitHub projects:

“A statistically significant, large, but transient increase in project-level development velocity, along with a substantial and persistent increase in static analysis warnings and code complexity.”

Speed up front. Drift over time. The drift is the part most pilots underweight, because thirty days is not long enough to surface it.

Cursor crossed roughly $2B ARR by late 2025. About 25% of generative-AI corporate spending on Ramp now goes to Cursor.

GitHub Copilot

The most rigorous independent study so far is NAV IT’s 2025 longitudinal analysis of 26,317 commits across 703 repositories. The finding is uncomfortable for vendors:

“Individuals who used Copilot were consistently more active than non-users, even prior to Copilot’s introduction. We did not find any statistically significant changes in commit-based activity for Copilot users after they adopted the tool.”
— NAV IT, arXiv:2509.20353

High performers adopted Copilot. Adoption did not make them high performers.

GitGuardian’s 2026 telemetry on roughly 20,000 repositories with Copilot enabled: 6.4% leaked at least one secret. Baseline: 4.6%. A 40% higher leak rate.

Copilot leads enterprise adoption (~56% in 10,000+ employee organizations), largely on procurement inertia and GitHub Enterprise bundling. Among dual users, about 73% rate Copilot as faster for routine completion, and about 39% rate it as more accurate for complex refactors.

The costs that do not make the slides

Cost	Headline number	Source
Code review	PR review time up 441%; PR size +51%; incidents/PR +243%	Faros AI 2026 (22k devs)
Security	2–10× per-dev vulnerabilities; +40% secrets-leak rate on Copilot repos	Snyk RSAC 2026; GitGuardian 2026
Technical debt	Copy-pasted code +48%; refactored code collapsed 44%; duplicated blocks up 8×	GitClear 2025 (211M LOC)
Junior careers	Sub-26 dev employment 20% below late-2022 peak; −17pt comprehension under full delegation	Stanford Aug 2025; Anthropic Feb 2026

Code review is the new bottleneck

Faros AI’s 2026 report covers 22,000 developers. The numbers around AI rollout:

• Median PR review time: up 441%.
• PRs merging with no review at all: up 31%.
• PR size: up 51.3%.
• Bugs per developer: up 54%.
• Incidents per PR: up 242.7%.

Faros calls it “Acceleration Whiplash.” Throughput rises at the top of the funnel; defects multiply at the bottom.

CodeRabbit’s December 2025 analysis of 470 real-world PRs (320 AI-coauthored, 150 human-only) found AI-coauthored PRs contained roughly 1.7× more issues on average. Microsoft Engineering, running an AI reviewer across 5,000 internal repositories, reports 10–20% median PR completion-time improvements on the same workload. The tool now touches almost 90% of Microsoft’s internal PRs.

Review capacity is not a nice-to-have alongside an AI coding tools rollout. It is the bottleneck the rollout creates.

Security is degrading

From Snyk’s Chief Innovation Officer, Manoj Nair, at RSAC 2026:

“Across our customer base, we’re seeing somewhere between two to 10x increase in per-developer actual vulnerabilities over the last year. The only attribution we can give for that is AI-generated code.”

GitGuardian’s State of Secrets Sprawl 2026 recorded 28.65 million secrets leaked on public GitHub in 2025, up 34% year over year.

Slopsquatting is now a real supply-chain attack vector. UT San Antonio, Oklahoma, and Virginia Tech tested 16 LLMs on 576,000 generated Python and JavaScript snippets. About 20% of recommended packages did not exist. 58% of the hallucinated package names repeated across runs. 43% repeated every single time the same prompt was issued. That is a stable target for a malicious package to squat on.

Technical debt compounds

GitClear’s 2025 analysis of 211 million lines of code:

• Copy-pasted code: 12.3% of changes, up 48% relative to the prior baseline.
• Moved (refactored) code: collapsed to 9.5% of changes, down 44% year over year.
• Short-term churn (code reverted within two weeks): up to 5.7%.
• Duplicated code blocks: up eightfold in the most recent year.

GitClear’s reading: “AI-assisted coding may be encouraging less structured development practices, with a focus on immediate output over sustainable architecture.”

Juniors carry the cost

Stanford’s Canaries in the Coal Mine? (Brynjolfsson, Chandar, Chen, August 2025) used ADP payroll data covering 25M+ U.S. workers:

“In July 2025, employment for the youngest software developers was 20 percent below its late fall 2022 peak.”

Developers aged 26+ remained stable. Combined with the Anthropic skill-formation finding, the picture is concrete. Juniors get an immediate output boost. The trade is a measurable hit to comprehension and a shrinking entry-level job market.

How to know if it is working

Most teams measure the wrong things when evaluating AI coding tools. Daily active users, suggestion acceptance rate, lines of code generated. None of those answer the question that pays the bill: is the team shipping more product, with fewer incidents, in less wall-clock time?

The measurable approach is to tag commits with the AI tool and model that produced them, then track those commits through the delivery pipeline. Compare AI-assisted and unassisted work on PR cycle time, review time, change failure rate, and throughput per developer. Datadog shipped an off-the-shelf version of this in May 2026. Internal builds on the same idea work too.

This is how “do you feel faster?” becomes a number you can defend to a board.

What we recommend

We’ve helped engineering teams ship AI coding tools rollouts under both shapes — strong DORA fundamentals where the rollout amplified them, and weaker setups where it amplified the drift. The playbook below comes from those engagements.

Before broad rollout

Baseline cycle time and the rest of the DORA metrics — review time, change failure rate, and defect density. Without the baseline, the post-rollout debate is about perception, not delivery.

Write a clear AI usage policy. Sanctioned tools, data-handling rules, mandatory review depth on AI-generated changes, rules for AI-generated tests.

Turn on secret scanning before the first license is provisioned. GitHub Push Protection, gitleaks, or trufflehog as a pre-commit hook. The 40% leak-rate gap from GitGuardian is the single most actionable security fix in the 2025–2026 evidence.

During pilot

Run a 30–60 day pilot on one team. Measure tasks completed, PR size, review time, defect density, and self-reported satisfaction. Then compare what developers say to what the data shows. The METR perception gap is the part you have to confront directly, or it will eat the post-rollout debate.

Invest in review capacity in the same quarter as the rollout. Internal AI reviewer, CodeRabbit, GitHub auto-review — whatever fits the stack. Throughput gains transfer into review queues if review does not scale. The right engineering team structure determines how much of that load lands on whom.

For juniors, mandate explain-mode use (“ask the AI to explain before it generates”) and schedule weekly no-AI practice. Anthropic’s own data supports this directly, and it is the cheapest intervention available.

Scaling

Roll out broadly only if pilot showed stable or improving change failure rate and review time. If those degraded in pilot, fix the bottleneck before scaling. Rollout amplifies what is already there.

Invest in context. Project-specific CLAUDE.md, Cursor rules, Copilot custom instructions. The best predictor of useful AI output is how much codebase-specific context the model has at request time.

Add software composition analysis and dependency provenance to CI. Slopsquatting is no longer hypothetical.

Track rework rate — DORA’s new fifth metric in 2025 — unplanned deployments divided by total deployments. It catches the “looks productive, actually breaking things” pattern early, and it slots cleanly into an existing velocity-measurement programme if you already have one.

Decision criteria

Metric	Pull back if…	Expand if…
PR review time	> +50% vs. baseline	Flat or declining
Change failure rate	> +2 percentage points	Flat or declining
Secrets-leak incidents	Rising	Flat
Junior comprehension assessments	Falling	Flat or improving
Throughput	—	Rising without stability degradation
Defect density	—	Flat or declining
Developer satisfaction	—	Improving

Frequently asked questions

Do AI coding tools actually make developers faster?

Sometimes. The only randomized controlled trial on the question — METR, July 2025 — measured experienced developers taking 19% longer to complete real tasks on mature codebases when using AI coding tools. Survey-based studies routinely show large self-reported speedups, but the perception gap is the most replicated finding in the 2025–2026 literature. AI coding tools help with boilerplate, scaffolding, debugging assistance, and exploration; they tend to slow experienced engineers down on large, mature codebases.

Which AI coding tool is best: Claude Code, Cursor, or GitHub Copilot?

The choice depends on team size and engagement shape, not on raw quality. Small engineering teams favour Claude Code (46% “most loved” in the Pragmatic Engineer 2026 survey). Cursor has the strongest evidence for agentic workflows — +39% merged PRs in the University of Chicago study with no growth in PR size. GitHub Copilot dominates enterprise adoption (~56% in 10,000-plus employee organisations), largely through procurement bundling. Across all three, the practices around the tool matter more than the tool itself.

How should we roll out AI coding tools to an engineering team?

In three stages. Before any licences are provisioned, baseline DORA metrics, write an AI usage policy, and turn on secret scanning. Run a 30–60 day pilot on one team measuring cycle time, review time, change failure rate, and defect density. Only scale broadly if those metrics held or improved during the pilot. The fastest way to amplify a struggling engineering organisation is to roll out AI coding tools without that gate.

Are AI coding tools worth it for enterprise teams?

If your engineering practices are strong — small batches, working CI/CD, real testing, clear platform abstractions — AI coding tools amplify throughput while holding stability. If those practices are weak, the same rollout doubles incidents per PR and triples review time (Faros AI 2026, 22,000 developers). DORA’s 2025 finding is consistent across 5,000 respondents: AI is an amplifier of what the organisation already does, not a fix for what it does not.

What this means for your roadmap

The choice between Claude Code, Copilot, and Cursor matters less than the choice of practices around them. Small teams favor Claude Code; enterprises favor Copilot; Cursor sits in the middle with the strongest agentic-workflow evidence. None of that decides whether your team ships better software next quarter. Your review capacity, security posture, baseline metrics, and platform investment do.

If you want a second pair of eyes on the rollout — baseline, pilot design, the policy, the review-capacity plan — that is the fractional CTO engagement we run most often in 2026. The deliverable is a rollout plan you could defend to a board, not a tool recommendation.