Duku AI is now Antidote
30 stats every engineering leader should know about AI-era software quality
Hundreds of reports exist on AI and software quality, most published by people selling a fix for it. Here are 30 you can check, and we tell you which is which.
William Macpherson
Member of Ops Team
·
·
8 min read
Most statistics in this field do not survive being looked up. Take the claim that a production bug costs 100 times more to fix than one caught in design. It traces back not to a study, but to IBM internal course notes cited in a 1987 textbook, whose underlying data “could be as old as 1967” (The Register). It has been repeated for forty years as though it were research.
Every figure below names its source, its date and its sample, and links to somewhere you can check it. Each also carries a tag showing what kind of evidence it is. A peer-reviewed paper, a regulatory filing and a vendor’s customer survey are not equally strong, and you should not have to guess which one you are reading. Antidote sells autonomous verification, so read our own framing with the same scepticism.
The verification gap
1
90% of software professionals now use AI, but 30% trust its output little or not at all.
23% trust it "a little" and 7% "not at all". Nearly 5,000 respondents worldwide.
Industry survey, Google-run
2
Only 6.6% of developers trust AI with important decisions.
The most common answer, at 48%, was "I trust it when I can easily verify the output". 14,304 answered the question, in a survey of more than 30,000 people across 169 countries, published October 2026.
Developer survey
3
96% of developers don't fully trust AI code to be correct, yet only 48% always check it before committing.
AI already accounts for 42% of committed code, and developers expect 65% by 2027. More than 1,100 developers, January 2026.
Vendor survey
4
70% of software leaders believe application quality is already suffering.
93% have adopted AI coding tools, and 40% generate more than 40% of their code with AI. Of 273 CTOs, CIOs, developers and QA directors, 86% are raising testing budgets by more than 11% this year.
Vendor survey
5
"Almost right, but not quite" is developers' biggest frustration with AI tools.
66% of 31,476 developers chose it, the most-selected of seven options. Runner-up: 45.2% say debugging AI-generated code takes longer. This is from 2025, the most recent edition to ask the question.
Developer survey
6
AI-co-authored pull requests carry 1.7 times more issues than human ones.
10.83 issues per PR against 6.45, across 320 AI-co-authored and 150 human-only open-source PRs. Logic and correctness issues were 75% more common. Authorship was inferred, not confirmed.
Vendor analysis
7
AI-generated code introduces a known security flaw 45% of the time, no better than two years ago.
More than 95% of it is syntactically correct. Defences against cross-site scripting passed just 15% of the time. 150+ models, four languages, March 2026.
Vendor benchmark
8
Code duplication is up 81% and refactoring has collapsed.
Duplicated blocks rose from 40.3 to 73.0 per million changed lines between 2023 and 2026. Moved or refactored lines fell from 21% of changes in 2022 to 3.8%. 623 million analysed changes.
Vendor telemetry
What AI is doing to delivery
9
At teams with high AI adoption, incidents per merged PR more than tripled.
Up 243%, with monthly incidents up 58% and bugs per developer up 54%. Task throughput per developer rose only 34%. 22,000 developers across 4,000+ teams, two years of data to March 2026. It is a before-and-after comparison within teams, not a controlled experiment.
Vendor telemetry
10
Median time in code review rose 441%, and PRs merged with no review rose 31%.
Same teams, same comparison. The bottleneck has moved from writing code to checking it.
Vendor telemetry
11
Main-branch success fell to 70.8%, its lowest in over five years, before recovering to 78.3%.
Across 28.7 million workflows in September 2025, median teams' main-branch throughput fell 7% while feature-branch throughput rose 15%. By August 2026, across 22.1 million workflows, success had recovered to 78.3%.
12
Developers saving 10+ hours a week with AI fell from 68% to 26% in a year.
65% spend time correcting AI output that missed the original intent. Only 17% report a clear AI strategy across the delivery lifecycle. 1,560 respondents, self-reported.
Vendor survey
13
87% were confident they could tell within 24 hours whether AI code caused an incident. Of those that had one, 34% couldn't.
Harris Poll, 1,528 developers and technology buyers, six countries, June 2026.
Vendor survey
What the tests themselves cost
14
84% of Google's pass-to-fail test transitions involved a flaky test.
That's against a steady background of about 1.5% of all test runs reporting a flaky result.
Company engineering data
15
A quarter of Google's tests using the Android emulator were flaky.
25.46% of them, against a general finding that "the larger the test, the more likely it will be flaky".
Company engineering data
16
Only 1.23% of Google's test targets ever found a real breakage.
In one month covering more than 4 billion test outcomes, 2.07% of 5.5 million targets both passed and failed. Only 1.23% remained once flaky targets were filtered out.
Peer-reviewed
17
Up to half of all builds contain a flaky failure, and 86% of flaky tests only flake in CI.
The figure was 14% to 52% of builds, depending on the project, across five Microsoft projects over 30 days, and 4.6% of individual tests were flaky. Of 315 flaky tests re-run 100 times on a local machine, 86% never flaked there.
Peer-reviewed
18
Two thirds of Mozilla's job failures were not real failures.
25,871 of 38,596 failures across nearly 2 million runs, each classified by hand rather than by heuristic.
Peer-reviewed
19
Flaky tests outnumbered genuine failures four to one at Facebook.
In CI data where almost 99.9% of dependency-selected test targets passed.
Peer-reviewed
What failure costs
20
Poor software quality costs the US at least $2.41 trillion.
On top of that is about $1.52 trillion of accumulated technical debt, reported separately. A modelled estimate from 2022, with no later edition.
Consortium estimate
21
The average US data breach now costs a record $11.5 million, and one in four malicious breaches is AI-enabled.
The global average is $4.99 million, up 12%, across 602 organisations breached between March 2025 and February 2026. AI-enabled attacks rose 56% in a year.
22
Downtime costs Global 2000 companies $600 billion a year, and every technology leader surveyed had an AI-related outage.
That's up 50% in two years, or roughly $300 million per company. Half cited incorrect AI-driven automation. 2,000 executives, 20 countries, 2026.
23
10 of 2025's 16 major cloud outages were caused by configuration or deployment changes.
Covers AWS, Azure, Google Cloud and Cloudflare, mostly from official post-mortems. Two examples: a database permissions change took Cloudflare down for about five and a half hours in November 2025. AWS's October 2025 us-east-1 outage cost US companies an estimated $500–650 million.
24
One bad update cost the Fortune 500 an estimated $5.4 billion, and Delta around $500 million on its own.
CrowdStrike, July 2024; the Fortune 500 figure excludes Microsoft and was modelled by an outage insurer. Delta reported $380 million of lost revenue and $170 million of extra non-fuel cost, offset by $50 million of unburned fuel, across 7,000 cancellations in five days.
25
UK banks logged 803 hours of unplanned outage in two years.
More than 33 days, across at least 158 incidents at nine institutions between January 2023 and February 2025, on their own returns to Parliament.
Parliamentary data
Customer and revenue impact
26
A 31% faster page lifted Vodafone's sales by 8%.
A genuine 50/50 A/B test with no visual or functional differences between versions. Around 34,000 visits per version per day, all paid media traffic.
Controlled experiment
27
A 1.09% crash rate is enough for Google Play to flag your app.
Above that share of daily users experiencing a crash, an app crosses Google's "bad behaviour" threshold. For a single device model, the threshold is 8%.
Platform policy
28
Shoppers cut spending after 47% of bad experiences.
Down 8 points year on year, but 58% in online retail and 62% in fast food. 20,000+ consumers, 14 countries, Q3 2025.
Vendor survey
Where the market is heading
29
Gartner expects 70% of enterprises to have AI-augmented testing tools by 2028.
Up from 20% in early 2025. From Gartner's first Magic Quadrant for AI-Augmented Software Testing Tools, October 2025.
Analyst forecast
30
Only 15% of companies have scaled AI in quality engineering enterprise-wide.
43% are still experimenting and 30% have it in limited use cases. Gen AI non-adopters rose from 4% to 11%. 2,000+ senior executives, 22 countries.
Vendor survey
What this means
Read together, these numbers tell a consistent story. Writing code has stopped being the bottleneck; checking it now is. Developers say they trust AI when they can verify its output (2), but fewer than half verify it (3). Review queues have grown more than fivefold (10), more code ships with no review at all, and incidents per change are rising (9). Meanwhile the test suites meant to catch problems spend much of their time raising false alarms (14–19).
Most failures that reach users show up as behaviour: a broken checkout, a crash, a config change that takes a region down. So the useful question is not whether more code passes review, but whether what you are about to ship still works for the people using it. You can only answer that by checking real behaviour, with evidence, before every release.
Written by
William Macpherson
Member of Ops Team
Leave the testing to Antidote.
Antidote proves your product works every time you ship.
