On Thursday, February 6th, from 13:46 ET to 14:09 ET, Site Service experienced elevated latency. This disrupted client site navigation from successfully loading and failures in Composer and WebSked when loading content.
The elevated latency in the service was caused by a sudden increase in traffic and the service's inability to scale up sufficiently to handle the load. Capacity was restored, and the service returned to normal operation. The cause of the traffic spike is still under investigation.
All times ET + 24 hour clock
| Time | Event |
|---|---|
| 13:42 | Traffic Increase |
| 13:46 | Automated alerts notify engineering team of the elevated latency |
| 13:46 | Engineering team begins investigating |
| 13:56 | Engineering team increases the service capacity |
| 14:02 | Engineering team restricts traffic so service can recover |
| 14:02 | Clients notify Arc that there are issues with the platform |
| 14:05 | Service recovers |
| 14:09 | Engineering team restores traffic and latency returns to normal levels |
Autoscaling configurations will be updated for Site Service to ensure that its capacity is not limited when handling traffic spikes.