Customer Impact

On Thursday, June 12 at around 4:16 PM ET, several customers experienced deployment failures while using PageBuilder Deployer. This prevented them from promoting their latest code changes to the Arc environment.

Root Cause

The issue was caused by an outage on the npm registry, which is used to download packages during deployment.

The npm outage started at 3:53 PM ET and ended at 6:14 PM ET. During that time, there were high error rates when trying to download or install packages. Because PageBuilder Deployer depends on npm to pull in necessary code, the outage disrupted deployments for all customers during that window.

Timeline

All times ET + 24 hour clock

Time Event 15:53 NPM outage began 16:16 Customers started reporting deployment issues 16:25 Arc XP notified customers of the npm outage 17:12 Services began operating normally again 18:14 Continued monitoring until npm confirmed the outage was resolved

Arc XP Next Steps

• Making deployments more resilient - We’re updating our deployment process to better handle outages from external services like npm.
• Faster issue detection - We’re adding checks to quickly detect problems with npm and display clear warnings when deployments might be affected.
• Improved communication - We’re speeding up how quickly we notify customers when external issues affect deployments.