Minor impactResolvedMedia

LiveKit incident: Investigating reports of elevated API failures in Chicago region

LiveKit reported this minor-impact incident on its official status page on Oct 19, 2026, 17:02 UTC. It was resolved after 32 min.

Right now LiveKit is operational. Live LiveKit status →

🔔 Get alerted when LiveKit has issuesFree · email alerts · no credit card
Oct 19, 2026, 17:02 UTC
started
Oct 19, 2026, 17:34 UTC
resolved
32 min
duration
Minor
impact

Affected components

  • US Central - Real Time Communication
  1. postmortemOct 22, 2026, 00:47 UTC

    ## Summary On October 19, 2026, between 16:34 and 17:03 UTC and again between 17:47 and 17:54 UTC, some room management, agent dispatch and SIP API requests failed for rooms hosted in our US Central \(Chicago\) region. Requests from other regions that targeted those rooms also failed. During these windows some rooms could not be created or joined, and some agents were not dispatched. The cause was CPU exhaustion on the node running one of our message bus instances in US Central. A recently deployed overlay network used more CPU than it requested, and because our monitoring tracked CPU per pod rather than per node, we didn't see the node run out. That message bus instance began dropping messages, and a room-coordination service that depended on it stopped completing requests. We have moved the message bus in US Central onto larger nodes and resized other regions at similar risk. Node-level CPU alerting and automatic recovery for the coordination service are in progress. ## Impact B

  2. resolvedOct 19, 2026, 17:34 UTC

    Between 16:35-17:03 UTC, Chicago region experienced about 11% failure rates in API errors, failed agent dispatches, and issues connecting to rooms. We faced another wave of the same issue between 17:47-17:53 UTC where the failure rates were less than 10%. Room Service APIs called from other regions targeting to a room hosted in Chicago region were also affected during this period. API requests made with recent server SDK versions are automatically retried in other regions, so customers on those versions should not have seen these errors. The last related errors occurred at 17:53 UTC, and all affected services are operating normally since. We will share a detailed RCA with more information soon.

  3. monitoringOct 19, 2026, 17:11 UTC

    A fix has been implemented as of 17:00 UTC and we are seeing the errors return back to baseline. We are continuing to monitor before marking this resolved.

  4. investigatingOct 19, 2026, 17:02 UTC

    We are currently investigating reports of elevated API errors across our services in the Chicago region.

What APIStatus.watch saw

  • Oct 19, 2026, 17:05 UTCoperational → degraded
  • Oct 19, 2026, 17:15 UTCdegraded → operational

Other services with incidents at the same time

Overlapping incidents aren't necessarily related.

How often does LiveKit have incidents?

LiveKit reported 24 incidents in the last 90 days, 3 of them major or critical. A typical incident lasted 1h 26m. See LiveKit's uptime and incident history