Cloud Outages 2025–2026: What the Post-Mortems Mean for Your Architecture

What happened: four incidents in seven weeks

The overview below is based solely on the providers’ official post-mortems. They are worth reading in full – they are the best textbooks on operating distributed systems you can get.

AWS: an empty DNS record and what came after it

The AWS summary describes the cause as a “latent race condition” in the DNS management system for DynamoDB: two automated processes interacted in such a way that the result was an empty record for dynamodb.us-east-1.amazonaws.com, which the automation did not repair on its own and which required manual intervention by operators. Elevated DynamoDB API error rates lasted from 23:48 PDT on 19 Oct to 2:40 PDT on 20 Oct – Monday morning in the Czech Republic.

The second half of the story matters more. After DynamoDB recovered, the EC2 subsystem that manages physical servers (DWFM) entered a state AWS calls “congestive collapse”, and engineers had to throttle incoming work and restart its hosts. Launching new EC2 instances fully recovered only at 13:50 PDT, and Network Load Balancer errors ended at 14:09 PDT. The original fault lasted under three hours; its consequences lasted more than fourteen.

And one detail for anyone running in a different region: according to AWS, customers signing in as root or via federation configured to use signin.aws.amazon.com had trouble logging in to the console even outside us-east-1.

Azure Front Door: when the front door goes down, you can’t even reach the controls

Microsoft’s Post Incident Review (tracking ID YKYN-BWZ) states that between 15:41 UTC on 29 October and 00:05 UTC on 30 October, Azure Front Door and Azure CDN customers experienced timeouts and DNS errors. The fault passed validation because not all features were tested across different control plane versions. Rolling back to the “last known good” configuration did not help directly – the faulty metadata was already in it – and the configuration had to be cleaned up manually. Among the affected services, the PIR also lists Azure Portal, Entra ID and Microsoft 365.

Cloudflare: configuration both times, not code

The November post-mortem: a permissions change in ClickHouse caused a query to return duplicate rows, and the Bot Management feature file grew. The proxy had a hard limit of 200 features (it normally used around 60) and panicked when the limit was exceeded. The December post-mortem: while mitigating a vulnerability in React Server Components (CVE-2025-55182), an internal WAF testing feature was switched off through the global configuration system without a gradual rollout, which triggered a bug in an older proxy version that had gone undetected for years.

The common pattern: control fails, not hardware

Three things recur across all four cases.

• The trigger was a change or automation, not hardware. DNS automation, configuration metadata, a generated file, a global switch.
• The change spread faster than it could be stopped. At Cloudflare, the December change went through the global configuration system, which does not perform gradual rollouts. In their remediation actions, Cloudflare and Microsoft both list introducing gradual rollouts and longer validation of configuration changes.
• Recovery took longer than the cause, because the systems needed for recovery depended on what was broken.

This matches the broader data. The Uptime Institute’s Annual Outage Analysis 2026 report (May 2026) states that the outage rate per data center has fallen for the fifth year in a row, but 57% of respondents put the cost of their most recent significant outage at more than $100,000, and one in five at more than a million. According to Uptime, third-party IT and data center service providers account for roughly two thirds of publicly reported outages over nine years of tracking.

Lesson 1: hidden dependencies on a single region and on global services

A workload in Frankfurt or West Europe does not mean you have no dependencies elsewhere. Typical places where they hide:

• Identity and sign-in – sign-in endpoints, federation, an IdP hosted with the same provider or behind the same CDN.
• Global entry layer – CDN, WAF, global load balancer, DNS. A single service for all regions is, by definition, a single failure domain.
• Your suppliers’ suppliers – CI/CD as a service, image registries, monitoring, status page, incident tooling, payment gateway. If they run in the same region or behind the same CDN as you, they go down at the same moment.
• Recovery tooling – the pipeline you deploy from, Terraform state storage, the secrets vault, runbook documentation.

A sensible first step is a simple dependency register for each critical service. It does not have to be a tool; a file in the repository will do:

service: objednavky-api
rto: 60m
rpo: 5m
dependencies:
  - name: primary database
    scope: region (eu-central-1), multi-AZ
    failure_mode: read-only mode from replica
  - name: CDN / WAF
    scope: global, single vendor
    failure_mode: switch DNS directly to origin (runbook RB-12)
  - name: sign-in (IdP)
    scope: global SaaS
    failure_mode: valid sessions run out, new sign-ins fail
  - name: CI/CD
    scope: SaaS, hosted in us-east-1
    failure_mode: manual deployment from break-glass account (runbook RB-07)
recovery_requires_control_plane: true   # candidate for redesign
last_restore_test: 2026-06-18

Orderly infrastructure is a prerequisite – whatever is not described as code is hard to get into such a register. More in the article on infrastructure chaos.

Lesson 2: static stability instead of heroic recovery

AWS defines static stability as a system continuing to operate unchanged when a dependency fails. The recommendation from the same document: instead of relying on the EC2 control plane to launch new instances after a zone failure, have the capacity provisioned in advance – and remove control plane dependencies from the recovery path.

The October incident is a precise illustration. Those who had enough running instances in us-east-1 on 20 October rode it out. Those who relied on autoscaling to add more under load or after a failure waited for hours, because launching new instances did not work.

The cost is obvious: pre-provisioned capacity costs money even when it is not being used. For an internal system that can tolerate an outage of several hours, it does not pay off, and it is legitimate to decide so – but consciously and in writing.

Lesson 3: multi-AZ, multi-region, multi-cloud – what each step solves

At AWS, DynamoDB global tables kept working during the incident: according to the post-mortem, customers connected to replicas in other regions, and only replication to and from us-east-1 was delayed. Multi-region therefore helped those who had it prepared at both the data and the operational level.

For outages of the global entry layer, multi-region would not have helped at all. The answer is an alternative traffic path, but Microsoft itself, in its architecture guide on redundancy in front of Azure Front Door, warns that a poorly designed high-availability solution can make availability worse because it adds points of failure and operational complexity, and that most applications do not need such an architecture.

When full multi-cloud makes no sense: almost always, unless it is required by regulation or a specific business reason. A second cloud deployed only on paper is the most expensive form of false security – a failover that has never been tested behaves like no failover at all during an incident.

Lesson 4: RTO and RPO are decisions, recovery is a skill

RTO (how quickly the service will be running again) and RPO (how much data you can afford to lose) are not technical parameters but business decisions with a price tag. Without them, you cannot say whether multi-AZ is too little or multi-region too much. Set them per service, not for the whole company, and have them signed off by the service owner.

The other half is verification. The post-mortems show that even providers with the best teams ran into recovery paths nobody had walked through in a long time – among its actions, AWS lists adding recovery tests for DWFM, and Microsoft cutting data plane recovery time to about an hour.

For the last point, see the article on monitoring and logging; for deploying changes without impacting operations, see zero-downtime deployment in Kubernetes.

What regulation says

The following is not a legal interpretation, just a guide to where to look for the requirements.

DORA (financial sector)

Regulation (EU) 2022/2554 applies from 17 January 2025. Article 11(6) requires ICT business continuity plans and ICT response and recovery plans to be tested at least once a year and after substantive changes to systems supporting critical or important functions; testing plans must (except for microenterprises) also include scenarios of switching between the primary ICT infrastructure and redundant capacity. Article 12(6) requires that, when setting recovery time and recovery point objectives for each function, account is taken of whether it is a critical or important function. Article 28(8) requires exit strategies for ICT services supporting critical or important functions.

DORA does not mandate multi-cloud. It requires you to be able to demonstrate that you know and manage the risk of depending on a provider, and that your exit and recovery plan has been tested. How strictly the supervisor interprets this in a specific case is a question for compliance and lawyers.

NIS2 and the Czech Cybersecurity Act

The NIS2 Directive, in Article 21(2)(c), lists business continuity, including backup management and disaster recovery, among the mandatory measures, and in point (d) supply chain security. In the Czech Republic it is transposed into Act No. 264/2025 Coll. (the Cybersecurity Act, zákon o kybernetické bezpečnosti) and its implementing decrees. According to explanatory material from NÚKIB (the National Cyber and Information Security Agency) on Decree No. 409/2025 Coll. (the higher-obligations regime), business continuity management is governed by § 15: the obliged entity sets the minimum level of services provided, the recovery time and the data recovery point, and regularly tests its continuity and recovery plans. In other words, exactly what the previous section describes – only with an obligation to prove it.

The scope of obligations differs by regime (higher and lower), and the exact deadlines for implementing the measures depend on the registration date of the specific organization; check them in the Act and with NÚKIB.

What to do on Monday

1. Pick three to five services whose outage really hurts the company, and record the RTO, RPO and owner for each.
2. Write down their dependencies, including global services, SaaS tools and suppliers. For each, mark its scope (zone, region, global) and what happens when it fails.
3. Go through the recovery plan and underline every step that requires a working provider control plane, CI/CD or SSO. These are candidates for pre-provisioning or an alternative path.
4. Schedule one recovery test for the next quarter – start with restoring the database from backup and measuring the time.
5. Decide on the entry layer: either you consciously accept the risk of a CDN/WAF outage, or you have a runbook to bypass it and know which security controls you temporarily lose by doing so.
6. Check your own configuration changes: are they rolled out gradually, and can they be reverted quickly?

Only then does it make sense to open the debate about a second region or a second vendor. Without the first four steps, it would be an investment made blindfolded.

Key takeaways

• The major outages of 2025 were triggered by automation and configuration, not hardware failure; they spread globally within seconds to minutes.
• Recovery took longer than the cause: at AWS, under three hours of DNS fault, more than fourteen hours of impact on EC2 and NLB.
• Multi-AZ is the foundation, not the goal. It does not address a regional service, the global entry layer or the loss of several zones.
• Static stability: a recovery plan that creates or changes nothing during an outage is the most reliable.
• Redundancy has a cost and can do harm. An unrehearsed failover and duplicated configuration add points of failure.
• RTO, RPO and recovery tests are also exactly what DORA and the Czech Cybersecurity Act require of regulated entities.