AI-written code: what the data says about security, quality and why code review has become the bottleneck

Security: models write better code, but not more secure code

The most widely cited source is Veracode’s GenAI Code Security Report from July 2025. It tested more than 100 language models on tasks in Java, Python, C# and JavaScript and concluded that 45% of code samples failed security tests and contained a vulnerability from the OWASP Top 10. Java fared worst with 72% of samples failing, and for cross-site scripting (CWE-80) the models failed in 86% of relevant cases.

The trend matters more than the number itself. The report notes that models were getting better at functional correctness, while security stayed flat regardless of model size or training sophistication.

The March 2026 update, which also covered the newest flagship models of the time, confirms this: the security pass rate remains at roughly 55%. The differences lie mainly between vulnerability types and languages:

According to the same update, the only notable exception is reasoning-focused models, which reached 70–72%.

Quality and maintainability: more copying, less refactoring

The second question is what AI does to a codebase over time. In its report The Maintainability Gap (2026), GitClear analyzed 623 million code changes from 2023–2026. Its duplicated-block metric (per million changed lines) rose from 40.3 in 2023 to 73.0 in 2026, an increase of 81%. The share of moved code, which the authors use as a signal of refactoring, fell from 13% of changed lines in 2023 to 3.8% in 2026, while the share of copy/paste grew from 9.4% in 2022 to 15.7% in the first half of 2026. The report puts the speed-up of heavy AI users compared with their own past at 25%.

Caution applies here too: GitClear is an analytics tool vendor, and this is a correlation over time, not an experiment. But the direction is consistent with how assistants work – adding a new block is easier for a model than finding and reusing an existing function. The debt does not show up on merge day, but at the first larger change.

At the delivery level, this is confirmed by the 2025 DORA report (almost 5,000 respondents, September 2025). 90% of respondents use AI at work and over 80% believe it has increased their productivity. Unlike the previous year, the research now sees a positive relationship between AI and delivery throughput, but the relationship with delivery stability remains negative. The authors’ explanation: AI speeds up development, and the larger volume of changes exposes weaknesses further down the chain – without strong automated testing, mature version control and fast feedback, it leads to instability.

Productivity: less certainty than the headlines suggest

The METR study from July 2025 became the skeptics’ argument: 16 experienced developers from large open-source projects worked on 246 tasks, and with AI tools allowed the work took them 19% longer. Beforehand they expected a 24% speed-up, and even afterwards they believed AI had sped them up by 20%.

The follow-up is less well known. In an update from 24 February 2026, METR reports newer data: for ten developers from the original study the estimate came out at −18% (confidence interval −38% to +9%), and for 47 newly recruited developers at −4% (−15% to +9%); a negative number here means less time with AI. At the same time, METR is changing the experiment design because it stopped being reliable: 30–50% of developers admitted that they did not submit some tasks to the study at all because they did not want to do them without AI. METR itself therefore describes the result as only very weak evidence of the size of the speed-up.

Two things follow for leadership. First, a subjective feeling of speed-up is not a measurement – in the original study it differed from reality by tens of percentage points. Second, even the best available research cannot currently say exactly how much AI speeds up an experienced team working on its own code. Making decisions based on a single study, whichever way it came out, is a mistake.

Why review became the bottleneck

When writing code gets cheaper, the queue moves to where capacity stayed the same. Faros AI telemetry from more than 10,000 developers in 1,255 teams (AI Productivity Paradox, 2025) shows this directly: developers in teams with high AI adoption complete 21% more tasks and merge 98% more pull requests, but review time grows by 91%, average PR size by 154% and bugs per developer by 9%. At the level of the whole company, the authors found no significant correlation between AI adoption and improvement.

DORA’s 2026 report on the return on AI works with the same mechanism. According to InfoQ’s summary, it describes a temporary productivity dip after AI is introduced (a J-curve) and names a “verification tax” – the cost of verifying AI-generated code in review – as one of its three causes. A sample calculation in the report assumes the change failure rate rising from 5% to 6%.

Developers themselves know that verification is real work, not a formality. In the Stack Overflow Developer Survey 2025, 84% of respondents used or planned to use AI, but 46% distrusted the accuracy of its output and 33% trusted it. The most common frustration (66%) was solutions that are “almost right, but not quite”; 45% said debugging AI code takes more time.

The 2026 edition, published on 6 October 2026, asked differently, and the result is telling: 48% of respondents trust AI when they can easily verify the output. Only 7% trust it for many tasks including important work decisions, and 12% do not trust it for most work tasks. Stack Overflow concludes that over 87% of people have “some level of trust”, compared with 31% a year earlier; but because the answer options changed, the year-on-year comparison is only indicative. The practical conclusion is different: trust is conditional on verifiability. And verifiability is a property of the process, not of the model.

Automated gates: what the machine should catch before a human

A reviewer should not be hunting for SQL injection or a forgotten token – tools do that more cheaply and more reliably. The minimum set of checks that should block a merge:

• Secrets scanning – already in a pre-commit hook and again in the pipeline.
• SAST – with rules for the vulnerability classes where models fail most often (XSS, log injection, unsanitized input).
• SCA and dependency checks – known vulnerabilities, licenses and every newly added dependency. More in the article on dependency scanning.
• Tests and coverage of changed code – not a global percentage, but coverage of the diff.
• IaC and container scanning – if the agent also generates Terraform, Dockerfiles or manifests.

A skeleton of such a pipeline (illustrative, in GitHub Actions syntax; pick the specific tools to fit your stack):

name: pr-gates
on: pull_request

jobs:
  gates:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@<commit-sha>
      - name: Secrets
        run: gitleaks detect --redact
      - name: SAST
        run: semgrep scan --config auto --error
      - name: Dependencies (SCA)
        run: trivy fs --scanners vuln --exit-code 1 --severity HIGH,CRITICAL .
      - name: Tests
        run: make test
      - name: PR size
        run: ./scripts/check-pr-size.sh 400   # custom script, diff line limit

Gates must be mandatory (branch protection) and apply equally to people and agents. How to put them together into a whole is described on the ideal pipeline page and in the article on CI/CD security.

Slopsquatting: a dependency that does not exist

A specific risk of AI code is invented packages. A study presented at USENIX Security 2025 generated 576,000 code samples from 16 models: commercial models recommended a non-existent package in at least 5.2% of cases, open-source models in 21.7%. The researchers collected 205,474 unique invented names. An attacker only needs to register such a name in a public registry and wait for someone to install it.

The defense is procedural: install only from a lockfile, bring in new dependencies through an internal proxy registry or allowlist, review every new package (age, author, download count), and forbid the agent from installing packages without approval.

Small PRs, tests and rules for agents

Automation catches known classes of bugs. The rest – whether the change does what it should and whether it fits the architecture – remains with people. For them to keep up, the input to review has to change.

Small batches

An unconstrained agent produces a large PR as easily as a small one. Set a diff size limit and have the agent split the work: one change, one intent. Do not mix refactoring and behavior changes in a single PR.

Tests as specification

A test written by the same model in the same step as the implementation often just confirms what the code does, not what it should do. It is more useful when acceptance criteria or tests are written up front and the reviewer reads them first.

Rules in the repository

The instruction file for the agent (for example AGENTS.md or its equivalent for your tool) should contain what is otherwise passed on verbally: how to run tests and the linter, which libraries to use for input validation, encryption and logging, and what the agent must not do (add dependencies, change CI, touch migrations without being told). Instructions are not a security boundary – the model can ignore them – so the pipeline must enforce the same rules.

Author accountability

Whoever opens a PR is accountable for it, whether they wrote it themselves or an agent did. The PR description should state what changed, why and how it was verified. AI review as a first pass helps filter out obvious mistakes, but it does not replace approval by a human who owns the code.

When these recommendations do not apply, and what critics say

• Prototypes and one-off scripts. For code that will not go to production and does not handle sensitive data, the full set of gates is unnecessary overhead. But define the boundary up front – prototypes have a habit of ending up in production.
• Lab benchmarks are not your codebase. With project context, security instructions and feedback from tools, results may be better than in Veracode’s tests. The only way to verify this is to measure it yourself.
• Most of the data is published by vendors. Veracode, GitClear and Faros AI all sell tools that address the problem described. Independent sources (DORA, METR, USENIX) point in the same direction, but with less certainty about the size of the effect.
• Gates that are too strict do harm. When the pipeline blocks on false positives, people start working around it. Start by blocking high severities and tune the rules.
• The numbers age quickly. Models and tools change month by month; METR itself admits its method stopped being sufficient. What holds is the mechanism (faster writing → pressure on verification) rather than the specific percentages.

What to do on Monday

Key takeaways

• AI code security is stagnating. According to Veracode, roughly 55% of samples pass security tests even with 2026 models; XSS, log injection and Java are the weakest areas.
• The bottleneck has moved. More PRs and larger PRs mean longer review; individual gains may not show up at the organizational level.
• Nobody can measure productivity precisely. A feeling of speed-up is not evidence, and even METR describes its newer results as weak.
• Trust rests on verifiability. Almost half of developers trust AI only when they can easily verify the output – and that is ensured by the pipeline, not the model.
• The machine should catch known bugs; people should judge intent. Secrets, SAST, SCA and tests as mandatory gates; human review for architecture and logic.
• Dependencies are a new attack vector. Invented packages are not a curiosity but a measurable and exploitable phenomenon.