Nov 9, 17:00 CST
Storefront error rates and latency have been back in the normal range for nearly 4 hours now, we're closing this incident out.
Nov 9, 00:02 CST
Update: At 5:36 PM CST, we received a large surge of traffic which overwhelmed our servers and caused our automatic scaling rules to react too slowly to accommodate the traffic.
We attempted to manually intervene and it took some time to identify and isolate the incoming traffic.
We have put provisions in place so that, going forward, we will be able to identify and isolate large traffic spikes much quicker in order to prevent this problem from occurring again.
We are still seeing heavy traffic usage and are prioritizing storefronts while we monitor. Access to the Merchant Dashboard may be degraded for now.
Nov 8, 22:08 CST
We're seeing recovery of errors across most storefronts, and page response times getting back to normal.
Nov 8, 19:38 CST
We've managed to correct the store scaling and error rates are dropping, but still elevated
Nov 8, 18:40 CST
One of our scaling groups appears to not be expanding as fast as intended, we're looking into it
Nov 8, 17:52 CST