While we're confident this incident is over for the immediate future, and while everything is catching back up (for a few types of data updates, mainly to companies) as everything returns to normal, we will be monitoring a few things more closely over the next few days.
Posted May 02, 2023 - 09:53 PDT
Monitoring
This incident is now on it's way to resolution; from our testing and metrics the Chameleon dashboard (app.chameleon.io) is now loading and Experiences are being displayed (e.g. Launchers, Tours). We will provide a final update here when all of our metrics/signals have returned to normal.
Posted May 02, 2023 - 06:45 PDT
Update
As an update to the ongoing issue we're experiencing, our database provider's team (MongoDB Atlas) has identified the underlying issue is actively working on the resolution. Chameleon, for the most part, will continue to be unavailable until this is resolved. The primary impact is that new Chameleon Experiences will not be displayed to your users and the Chameleon dashboard (app.chameleon.io) is not loading.
Posted May 02, 2023 - 06:33 PDT
Identified
app.chameleon.io continues to timeout when loading -- the underlying cause appears to be a recent maintenance event at our MongoDB instance which has left it only partially available and are working with their team toward resolution
Posted May 02, 2023 - 06:03 PDT
Investigating
We're investigating a slowdown in serving API requests to Chameleon
Posted May 02, 2023 - 05:44 PDT
This incident affected: User profile API, Sidebar API, and Data API.