Profile Write Latency and Errors

Incident Report for Chameleon

Postmortem

Available here

Posted Feb 11, 2025 - 12:30 PST

Resolved

This incident is over but will remain open while the final data is processed
Posted Jan 31, 2025 - 17:13 PST

Update

This incident is has now been partially resolved, some data ingestion delay will be processed over the next few hours as more capacity is brought online to assist in working through a backlog of events and updates from integrations
Posted Jan 31, 2025 - 14:41 PST

Update

We're still actively monitoring this incident and will report any new information we have
Posted Jan 31, 2025 - 14:01 PST

Monitoring

This incident is still ongoing, we will have more updates within the hour. The main impact is that requests that typically take 30-50ms are varying from 30ms to 3s, with some timing out and never completing, this is causing some Chameleon experiences to not start when they otherwise would
Posted Jan 31, 2025 - 10:34 PST

Investigating

Intermittent slowness and timeouts resulting in occasional Application errors
Posted Jan 31, 2025 - 06:28 PST
This incident affected: Heroku.