The main impact of this incident is now Resolved for the Chameleon Data API. For a period of ~6 hours today the autoscaler (and and turn the manual scaler) was not able to add additional resources to meet the typical morning/afternoon needs of the Data API. Any data that was received by Chameleon is now available for display on the dashboard and for access via API.
As mentioned previously, all other aspects of Chameleon remained online during this incident (we used Chameleon to update our customers in the Chameleon dashboard 🎉 http://is.am/4qxa).
We will hold this incident open until AWS and Heroku mark it as resolved, but since the impact to Chameleon is resolved, this will be the last email/slack alert you receive about this.
Thank you for your patience today.
Posted Dec 07, 2021 - 14:50 PST
Update
The referenced issue on heroku is here => https://status.heroku.com/incidents/2390 - we're still keeping up with updates as they come and will report back here when we have a resolution.
Posted Dec 07, 2021 - 14:13 PST
Identified
Due to the API and console outage at AWS us-east-1 and downstream for the Heroku API etc. the Data API will likely remain unavailable until those issues are resolved.
Data sent to Chameleon, your Chameleon Experiences (live to your end-users), editing and publishing Experiences are not impacted at this time.
Posted Dec 07, 2021 - 09:04 PST
Investigating
We are currently seeing Segmentation and Analytics data fail to load on the Chameleon dashboard and via API requests. We're currently investigating the cause of the issue, and will continue to provide updates as more information becomes available.