Upstash outage: Degraded Performance

Affected components

• Frankfurt, Germany (eu-central-1)
• Ireland (eu-west-1)
• London, UK (eu-west-2)
• Mumbai, India (ap-south-1)
• N. California, USA (us-west-1)
• N. Virginia, USA (us-east-1)
• Ohio, USA (us-east-2)
• Oregon, USA (us-west-2)
• Singapore (ap-southeast-1)
• Sydney, Australia (ap-southeast-2)
• São Paulo, Brazil (sa-east-1)
• Tokyo, Japan (ap-northeast-1)

1. postmortemJun 11, 2025, 11:49 UTC

   A routine system maintenance operation at the OS level led to the application of system updates across multiple EC2 instances in our clusters in several AWS regions. These updates included changes to networking components, which inadvertently triggered restarts. As a result, several EC2 nodes failed health checks and temporarily dropped out of the cluster, disrupting high availability and causing partial connectivity issues for some clients and operations. We have since reproduced the issue in a controlled environment and verified the root cause. To prevent a recurrence, we are updating our node maintenance strategy to ensure greater control over the timing and impact of system-level changes and excluding networking components from automated upgrades.

2. resolvedJun 11, 2025, 10:02 UTC

   This incident has been resolved.

3. monitoringJun 11, 2025, 09:20 UTC

   A fix has been implemented and we are monitoring the results.

4. investigatingJun 11, 2025, 08:24 UTC

   We are continuing to investigate this issue.

5. investigatingJun 11, 2025, 06:51 UTC

   We are currently investigating this issue.

What APIStatus.watch saw

Our 5-minute checks didn't record a change in Upstash's overall status around this incident. Smaller or regional incidents often leave a provider's overall status green.

How often does Upstash have incidents?

Upstash reported 3 incidents in the last 90 days, 1 of them major or critical. See Upstash's uptime and incident history