Chameleon application errors / performance

Incident Report for Chameleon

Resolved

This incident has been resolved, and all systems are operating as expected.
Posted Feb 28, 2025 - 11:55 PST

Monitoring

We are continuing to monitor system performance as response times and error rates have improved over the past hour. No new end-user-facing timeouts have been observed recently, but we are still actively investigating to ensure stability.
Posted Feb 28, 2025 - 09:52 PST

Investigating

We are currently investigating slow downs and 503 errors on profile requests.
Posted Feb 28, 2025 - 07:39 PST

Monitoring

Our new capacity has been online for several hours now and is performing as expected. We will keep this incident open for another 24 hours in case of any new developments or issues. Thanks for out patience
Posted Feb 27, 2025 - 10:41 PST

Identified

Chameleon continues to experience extremely high load and is taking steps to increase capacity above normal limits to address this. The current impact is that some requests to Chameleon are queued for a long time and/or timing out. In the UI this looks like page not loading, auto-saving not working, and for end-users experiences may not display. This currently appears to be happening for a < 10% of traffic but we'll continue to monitor as we work to resolve the issue
Posted Feb 27, 2025 - 05:48 PST
This incident affected: User profile API, JavaScript CDN, Sidebar API, Data API, and Chameleon Builder.