Chameleon application errors / performance

Incident Report for Chameleon

Resolved

This incident has been resolved.

Posted Feb 03, 2025 - 06:44 PST

Monitoring

Response times and error rates are returning to normal and will continue to monitor. At this time, we know that a large increase in latency when writing to the database caused the latency of essentially all other operations to increase.

We will leave this incident open until we're sure that the changes have a chance to full stabilize, the background jobs have processed, and the workload returns to normal levels/latencies.

Posted Feb 03, 2025 - 03:57 PST

Identified

We are seeing elevated error rates, and various systems being slow or not loading. At the moment this includes the Chameleon dashboard, identifying users, and sending data to Chameleon. We know this is related to the performance of our primary database and will have an update again shortly.

Posted Feb 03, 2025 - 03:13 PST

This incident affected: User profile API, Sidebar API, and Data API.