Zenoss incident: Zenoss Cloud service is disrupted by the ongoing global Google Cloud issues

Affected components

• Collection Zones
• Dashboards
• Event Ingest
• Metric Ingest
• Model Ingest

1. postmortemJun 13, 2025, 16:42 UTC

   GCP has posted its full RCA for the incident on Thursday June 12th here: [https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW](https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW) In summary, the underlying issue was with GCP Authentication APIs. During the incident, all Zenoss Cloud customers would have been unable to log in for approximately 1 to 3 hours depending on which region their Zenoss Cloud incident is hosted in. CZ functionality and CZ monitoring were unaffected, other than some minor Cloud layer data ingest delays for some CZs. These minor delays were fully resolved by morning of Friday June 13th.

2. resolvedJun 12, 2025, 22:50 UTC

   All Zenoss Cloud instances are fully functional. There is a minor backlog on metric ingest in us-central1 production instance that is still clearing and we expect that to be cleared this evening. We'll post another update with summary of the incident tomorrow morning. Please open a support ticket if you are having any issues at this point.

3. investigatingJun 12, 2025, 20:27 UTC

   Zenoss Cloud instances in Iowa are now functioning normally again as Google's recovery in us-central1 progresses. There may still be some delays to data ingest to the Cloud layer and Data Access but all CZs are operating normally.

4. investigatingJun 12, 2025, 19:50 UTC

   GCP status update indicates that all regions have recovered except for us-central1. As such, Zenoss Cloud customers in EMEA, Australia and in the Vegas US region should now have Zenoss Cloud full functionality restored. Customers in the Iowa US region and all customers dev environments may still be having issues as google continues recovery in their us-central1 region.

5. investigatingJun 12, 2025, 19:35 UTC

   We are closely tracking the Google Cloud outage and for additional insight into Google Cloud issue it is available here https://status.cloud.google.com/incidents/ow5i3PPK96RduMcb1SsW

What APIStatus.watch saw

Our 5-minute checks didn't record a change in Zenoss's overall status around this incident. Smaller or regional incidents often leave a provider's overall status green.

Other services with incidents at the same time

Overlapping incidents aren't necessarily related.

• Vimeo: Degraded ServiceCritical · Jun 12, 2025, 18:20 UTC
• Mixpanel: Mixpanel.com and APIs experiencing elevated errorsCritical · Jun 12, 2025, 18:29 UTC
• Imgix: Elevated rendering errorsMajor · Jun 12, 2025, 18:09 UTC
• npm: Issues with package install, package publish, and login.Major · Jun 12, 2025, 18:14 UTC
• Smartsheet: Google Cloud Service Incident affecting Smartsheet FunctionalityMajor · Jun 12, 2025, 18:54 UTC
• LogRocket: Platform wide issuesMinor · Jun 12, 2025, 18:15 UTC
• Cloudinary: Google Cloud outage affecting features including Documentation, Add-Ons, Use of Google Cloud Storage for account backupsMinor · Jun 12, 2025, 18:26 UTC
• npm: Issue with package publish with provenanceMinor · Jun 12, 2025, 21:00 UTC

How often does Zenoss have incidents?

Zenoss reported 0 incidents in the last 90 days. See Zenoss's uptime and incident history