End users of Arc XP customer sites received 403 errors on about 8% of requests when browsing pages and related assets. Services such as image resizing, API delivery (including Subs API and View API), and video delivery were not affected.
Arc XP relies on a centralized cache service within its CDN provider to ensure consistent content delivery across users. This cache is a shared (multi-tenant) service. A surge in traffic from another customer caused the shared resource limit to be reached.
When limit was hit, the centralized cache returned 403 errors to parent and edge servers that did not already have the requested content stored locally. This led to failed requests for end users until the surge subsided.
All times ET + 24 hour clock
| Time | Event |
|---|---|
| 14:31 | Increase in 403 errors began. |
| 14:39 | First customer reports received. |
| 14:50 | Error rate returned to normal. |
Arc XP is working with its CDN provider to migrate the centralized cache to a dedicated tenant, which will prevent external customers' traffic surges from impacting Arc XP customers. This change will be included in the next Web Delivery release.