On Oct. 19, 2025, a latent race condition in the DynamoDB DNS management system left the service's us-east-1 regional endpoint with an incorrect, empty DNS record, and AWS says that failure triggered the disruption that spread through its N. Virginia (us-east-1) Region.
What happened
AWS says customers in the region could not connect to DynamoDB through its public endpoint from 11:48 PM PDT on Oct. 19, because the systems that needed the service hit DNS failures.
AWS describes its DNS system as two parts: a DNS Planner that writes plans of load balancers and weights, and a DNS Enactor that applies each plan to Amazon Route 53. AWS says an Enactor that was unusually delayed applied an older plan just as another process cleaned up old plans. The older plan overwrote the newer one, and the clean-up then deleted it, which was now the active plan, so all IP addresses for the regional endpoint were immediately removed. The check that should have stopped the older plan was out of date by then, and the system needed manual operator intervention to correct.
AWS also says the DynamoDB failure reached EC2. Its state checks depend on DynamoDB, so the leases that track physical servers timed out. After DynamoDB recovered at 2:25 AM PDT, engineers could not rebuild the leases fast enough, and at 4:14 AM they throttled incoming work and began selective restarts of the hosts that manage those leases. Full EC2 recovery came at 1:50 PM PDT on Oct. 20.