Deployment in the AI era: why shipping became the bottleneck

What actually changed

Until 2023 the equation was simple: writing code was the largest slice of software delivery. Every productivity investment - better IDEs, libraries, frameworks - targeted that slice, because that is where the waiting happened.

AI assistants broke that equation. A developer now generates a CRUD service, a database migration, and a test suite in an afternoon. But writing code was one phase out of eight. The rest of the path to production - review, tests, environment provisioning, configuration, approval, deployment, verification - stayed exactly where it was.

The result is familiar to anyone who has lived it: the team "delivers" more, yet customers get nothing sooner. Work simply accumulates in the queue ahead of deployment. The classic theory-of-constraints rule applies: speeding up a step that is not the constraint does not increase throughput - it only increases work-in-progress piled up in front of the real one.

The numbers behind it

The DORA 2025 report (Google Cloud) puts AI tool adoption at roughly 90% of developers, and for the first time links AI adoption to higher delivery throughput. The same report also finds that AI adoption correlates with higher instability: more failed changes, more rework, and longer time to resolve problems.

A Faros AI telemetry study across more than 10,000 developers and 1,255 teams gives that shift a concrete shape - and shows clearly where the bottleneck moved:

In other words: developers produce twice the changes, the changes are 2.5x larger, and the wait for review got longer. That is the definition of a bottleneck. More recent data from the same source shows the pressure still building - median time in review has multiplied year over year, and a growing share of PRs merge with no review at all, simply because reviewer capacity ran out.

Where the new bottleneck actually forms

Before automating anything, you need to know where work is waiting. In practice it is these five places:

1. Verification instead of authoring

AI moved the effort from creation to verification. A reviewer now reads code nobody on the team wrote, at a volume the process was never designed for. Without automated tests, static analysis, and clear rules, review turns into endless reading.

2. Waiting for environments

If a team has three shared test environments and ten parallel changes, seven of them wait. A shared staging environment is the most expensive constraint you can keep in the AI era.

3. New project setup

A new service is written in a day, but its pipeline, environments, DNS, certificates, secrets, monitoring, and alerts are assembled by hand over two weeks. That is pure overhead, and templates remove nearly all of it.

4. Manual steps in the release process

Manual approvals, manual database migrations, manual traffic switching, "a colleague still has to confirm it". Every manual step carries a fixed cost measured in hours or days, and that cost multiplies as change volume grows.

5. A missing safety net

When a team is afraid to deploy, it deploys less often in larger batches - which raises risk further. Without fast, automatic rollback you cannot raise deployment frequency, no matter who writes the code.

Quick test: is your deployment the brake?

Walk through this list. Every unchecked item is a concrete delay.

Fewer than six checked items means your highest-return investment in development speed right now is not more AI licenses - it is the deployment pipeline.

Fix 1: Template the project, don't copy it

Standardising how projects start has the fastest payback. The goal is a single action after which a working service exists and is deployed to a dev environment - pipeline, health checks, dashboard, and alerts included.

What the template should contain

service-template/
├── .github/workflows/
│   ├── ci.yaml              # build, tests, SAST, SCA, image scan, SBOM, signing
│   └── release.yaml         # tag → push to registry → bump version in GitOps repo
├── deploy/
│   ├── Chart.yaml           # dependency on the shared library chart
│   ├── values.yaml          # only what is specific to this service
│   └── overlays/
│       ├── dev/
│       ├── staging/
│       └── prod/
├── docs/
│   ├── README.md            # how to run the service locally
│   └── runbook.md           # what to do when it's on fire
├── observability/
│   ├── dashboard.json       # Grafana dashboard as code
│   └── alerts.yaml          # SLOs and alerts as code
├── catalog-info.yaml        # service catalog registration (Backstage)
├── Dockerfile               # distroless, non-root, multi-stage
└── CODEOWNERS

Service template structure - everything a new service needs to survive in production.

How to run the template

At small scale cookiecutter, copier, or a native GitHub template repository is enough. From roughly ten teams upward a developer portal (Backstage or equivalent) pays for itself, because creating a service becomes a form and the developer never has to learn the platform's internals:

# Backstage Software Template (abbreviated)
apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: nodejs-service
  title: Node.js service (production standard)
spec:
  parameters:
    - title: Basics
      required: [name, owner, tier]
      properties:
        name:  { type: string, title: Service name }
        owner: { type: string, title: Owning team }
        tier:  { type: string, enum: [tier-1, tier-2, tier-3] }
  steps:
    - id: fetch
      action: fetch:template
      input: { url: ./skeleton, values: { name: '${{ parameters.name }}' } }
    - id: publish
      action: publish:github
      input: { repoUrl: 'github.com?repo=${{ parameters.name }}&owner=acme' }
    - id: register-gitops
      action: github:pull-request        # PR into the GitOps repository
      input: { repoUrl: 'github.com?repo=platform-gitops&owner=acme' }

One form creates the repository, the pipeline, and the GitOps entry.

Fix 2: Kubernetes as a deployment template

In this context Kubernetes is primarily a uniform declarative model. When every service describes its deployment in the same language, the part that is identical for all of them can be factored out into a shared template. A new service then inherits health checks, limits, autoscaling, network policies, and rollback strategy for free.

Layer 1: a shared Helm library chart

A library chart contains templates but is never deployed on its own. It is your platform standard expressed as code:

# platform-library/Chart.yaml
apiVersion: v2
name: platform-library
type: library          # key point: not deployable, only provides templates
version: 2.4.0

# platform-library/templates/_deployment.tpl
{{- define "platform.deployment" -}}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.name }}
spec:
  replicas: {{ .Values.replicas | default 3 }}
  strategy:
    rollingUpdate: { maxUnavailable: 0, maxSurge: 1 }
  template:
    spec:
      securityContext: { runAsNonRoot: true, seccompProfile: { type: RuntimeDefault } }
      containers:
        - name: app
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          readinessProbe:
            httpGet: { path: /healthz/ready, port: http }
            periodSeconds: 5
          livenessProbe:
            httpGet: { path: /healthz/live, port: http }
            periodSeconds: 10
          lifecycle:
            preStop: { exec: { command: ["sleep", "10"] } }   # graceful shutdown
          resources:
            requests: { cpu: {{ .Values.resources.cpu }}, memory: {{ .Values.resources.memory }} }
{{- end -}}

The platform standard: nobody has to think about probes, security context, or rolling update strategy again.

A service then only needs this:

# payments-api/deploy/Chart.yaml
dependencies:
  - name: platform-library
    version: "2.4.0"
    repository: "oci://registry.acme.cz/charts"

# payments-api/deploy/values.yaml
name: payments-api
image:
  repository: registry.acme.cz/payments-api
replicas: 4
resources: { cpu: 500m, memory: 512Mi }

The entire deployment configuration of a new service - six lines instead of six hundred.

Layer 2: Kustomize overlays per environment

Differences between environments do not belong in the template, they belong in overlays. One base, and each overlay changes only what it must:

# overlays/prod/kustomization.yaml
resources:
  - ../../base
replicas:
  - name: payments-api
    count: 8
patches:
  - target: { kind: Deployment, name: payments-api }
    patch: |-
      - op: replace
        path: /spec/template/spec/containers/0/resources/limits/memory
        value: 1Gi
configMapGenerator:
  - name: app-config
    envs: [prod.env]

Dev, staging, and production share one base - configuration drift between environments practically disappears.

Layer 3: one definition for every service

An Argo CD ApplicationSet generates the deployment for every service and every cluster automatically. Adding a service to the platform becomes one directory in the GitOps repo:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: services
  namespace: argocd
spec:
  generators:
    - matrix:
        generators:
          - git:                                  # each service = a directory in the repo
              repoURL: https://github.com/acme/platform-gitops
              revision: HEAD
              directories: [{ path: services/* }]
          - list:                                 # target environments
              elements:
                - env: staging
                  cluster: https://staging.k8s.acme.cz
                - env: prod
                  cluster: https://prod.k8s.acme.cz
  template:
    metadata:
      name: '{{ path.basename }}-{{ env }}'
    spec:
      project: default
      source:
        repoURL: https://github.com/acme/platform-gitops
        path: '{{ path }}/overlays/{{ env }}'
      destination:
        server: '{{ cluster }}'
        namespace: '{{ path.basename }}'
      syncPolicy:
        automated: { prune: true, selfHeal: true }
        retry: { limit: 5 }

New service = new directory. Deployments to every environment appear on their own.

Fix 3: GitOps instead of deployment scripts

If your pipeline deploys by running kubectl apply from a CI runner, you do not have a verifiable production state - you have a history of job runs. The GitOps model inverts this: the desired state lives in Git, and a controller in the cluster is responsible for making reality match it.

• Auditability: every production change is a commit with an author and a review.
• Rollback: git revert instead of reverse-engineering what the last deploy did.
• No drift: a manual change in the cluster is reverted automatically (selfHeal).
• Security: CI needs no cluster access - write access to a repository is enough.

In the AI era this model has one more effect: the audit trail no longer depends on who (or what) wrote the code. Whether a change came from a human or an agent, there is exactly one auditable path into production.

Fix 4: An automated safety net

This is the most important shift in thinking. At the change volume AI generates, a human stops being a usable safeguard. A reviewer will not catch a subtle bug in a 400-line PR that arrives five times a day. A system can - in production, on a small percentage of traffic, and it can pull the change back on its own.

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: payments-api
spec:
  strategy:
    canary:
      analysis:
        templates: [{ templateName: success-rate }]
        startingStep: 1                  # start analysing at 10% traffic
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - setWeight: 50
        - pause: { duration: 10m }
        - setWeight: 100
---
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  metrics:
    - name: success-rate
      interval: 1m
      successCondition: result[0] >= 0.99
      failureLimit: 2                    # 2 failures → automatic rollback
      provider:
        prometheus:
          address: http://prometheus.monitoring:9090
          query: |
            sum(rate(http_requests_total{service="payments-api",code!~"5.."}[2m]))
            /
            sum(rate(http_requests_total{service="payments-api"}[2m]))

Canary with automated analysis: a bad version withdraws itself before most users ever see it.

Fix 5: An ephemeral environment for every pull request

Shared staging is a queue. When every PR gets its own namespace with its own instance of the application, both the waiting and the mutual blocking disappear - and testing moves before merge, where fixing is cheapest.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: preview-environments
spec:
  generators:
    - pullRequest:
        github: { owner: acme, repo: payments-api }
        requeueAfterSeconds: 60
  template:
    metadata:
      name: 'payments-api-pr-{{ number }}'
    spec:
      source:
        repoURL: https://github.com/acme/payments-api
        targetRevision: '{{ head_sha }}'
        path: deploy
        helm:
          parameters:
            - { name: image.tag, value: '{{ head_sha }}' }
            - { name: ingress.host, value: 'pr-{{ number }}.preview.acme.cz' }
      destination:
        namespace: 'preview-pr-{{ number }}'
      syncPolicy:
        automated: { prune: true }
        syncOptions: [CreateNamespace=true]

The environment appears when the PR opens and disappears when it closes - resources cleaned up included.

Two practical notes: preview environments need a hard lifetime limit (or your cloud bill will run away) and must never touch production data - use seeded test data or an anonymised copy.

Fix 6: Guardrails as code

AI generates code quickly, but without the context of your operational and security rules. Those rules therefore have to be enforced by the platform, not by agreement. In practice that means three layers of control:

# Kyverno: production accepts only signed images from our own registry
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-signed-images
spec:
  validationFailureAction: Enforce
  rules:
    - name: verify-signature
      match:
        any:
          - resources: { kinds: [Pod], namespaces: ["prod-*"] }
      verifyImages:
        - imageReferences: ["registry.acme.cz/*"]
          attestors:
            - entries:
                - keys: { publicKeys: |-
                    -----BEGIN PUBLIC KEY-----
                    ...
                    -----END PUBLIC KEY----- }

A rule that cannot be skipped under time pressure or missed in review.

Critically, these checks belong in the project template. If every team has to enable them individually, half of them will not. If they ship in the template, they are everywhere from day one - and the platform team can update them centrally.

Fix 7: Measure queues, not activity

Commit counts and merged PRs say almost nothing about performance in the AI era - those numbers rise even when nothing extra reaches production. Track the four DORA metrics and add per-stage waiting times:

• Deployment frequency - how many times a day a change reaches users.
• Lead time for changes - broken down by stage, not as a single number.
• Change failure rate - the share of deployments that cause a problem.
• Time to restore service - how quickly you get back to a working state.
• Time to first review - the first place a queue forms today.
• Time from merge to production - the pure overhead of your release process.
• Share of PRs merged without review - an early indicator of an overloaded team.

If the first two improve while the rest degrade, you are driving into the wall faster - exactly the pattern DORA 2025 describes as higher throughput with higher instability.

A 90-day plan

You do not have to rebuild the platform at once. This order gives the highest return per unit of work:

Days 1-30: measure, and standardise one path

• Measure real waiting times per stage (commit → review → merge → production).
• Pick one representative service and move it fully onto GitOps deployment.
• Write your deployment standard down as a Helm library chart, version 0.1.

Days 31-60: template it, and remove the queues

• Build the project template with everything: CI, chart, alerts, runbook, policy.
• Create at least two new services from it - a template with no users is dead.
• Turn on preview environments for pull requests in your busiest repository.

Days 61-90: replace the human safeguard with automation

• Roll out canary with automated analysis on one production service.
• Add admission policy (signed images, no root, mandatory limits).
• Remove manual approval steps that automated checks now cover.
• Publish the metrics to teams - visibility is half of the improvement.

Five mistakes that kill this transition

1. A template with no owner. Nobody updates it, six months later it is stale, and teams go back to copy-paste.
2. A platform that is mandatory but not helpful. If a team's own solution is faster than your template, you have lost. The golden path has to be the easiest path.
3. Automating chaos. Automating an inconsistent manual process just produces mistakes faster. Standardise first, automate second.
4. Speed without a safety net. Raising deployment frequency without automatic rollback and observability is the surest route to an outage.
5. AI outside the rules. If AI-generated code bypasses the checks human code goes through, it will show up in production sooner or later.

Frequently asked questions

Key takeaways

• The bottleneck moved. It is not writing code - it is the path from commit to production.
• Faster code generation without faster deployment only grows the queue - and the risk with it.
• Templates are the fastest win. A new service in 30 minutes, with every standard built in.
• Use Kubernetes as a shared template - library chart, overlays, ApplicationSet - not as a goal in itself.
• The safety net has to be automated. Canary, metric analysis, rollback, policy as code.
• Measure queues, not activity. DORA metrics plus per-stage waiting times tell the truth.

Related articles