Author is disappearing from stories when editing stories in Composer V2

Incident Report for Arc XP

Postmortem

Customer Impact

On Tuesday, January 28th, at 15:00 ET, Composer released version 2.3.15, which included several improvements and hotfixes for Composer 2.0. Following this release, users began noticing unexpected behavior in the authors field. The field sometimes failed to display selected authors, caused authors to disappear when changes were made, and exhibited other related inconsistencies.

Root Cause

On Wednesday, January 29th, at 9:15 ET, users began reporting issues with the authors field. The problem stemmed from recent quality of life changes designed to differentiate guest authors from regular authors, along with internal code modifications. In response, at 13:00 ET, Arc XP engineers implemented a fix, which was expected to resolve the issue.

By January 31st at 11:23 ET, some users reported that the issue persisted, though at a much lower frequency. Arc XP engineers resumed their investigation at 11:30 ET. The issue reemerged on February 3rd at 11:30 ET, when additional users reported continued problems. After further analysis, the root cause was identified as a race condition in Composer 2.0’s initialization process, which could cause authors to disappear under certain conditions. Although the team was unable to reproduce the issue, a potential fix was deployed at 17:54 ET in an effort to resolve the problem.

Despite this fix, reports of the issue continued. On February 4th at 10:35 ET, users once again reported that the problem was still occurring. Arc XP engineers immediately resumed their investigation and, after a few hours, were able to consistently reproduce the issue. This breakthrough led to the development of a thoroughly tested fix, which was successfully deployed at 13:51 ET. Following this deployment, no further reports were received, and the issue is now considered fully resolved.

Timeline

All times ET + 24 hour clock

Time Event
09:15 User's reports indicate that authors have been vanishing from stories.
12:18 Root cause identified by Arc XP engineering team
12:39 Arc XP engineering team begins deployment with a resolution to the issue.
13:00 Hotfix successfully deployed
13:09 Users has confirmed the issues was solved
Jan 31th  
11:23 Some users reported the issue was still occurring intermittently and inconsistently
11:30 Arc XP engineering team begins investigation, and determines a race condition is the root cause, but is unable to reproduce the issue.
Feb 3rd  
11:41 Additional users report the issue is still occurring
17:00 Arc XP engineering team suspects there could be an issue where data is lost when loading the page, but is unable to reproduce the issue; however a hotifx is prepared to address the potential issue
17:54 ArcXP engineering team deploys a hotfix hoping it may resolve the issue
Feb 4th  
10:35 Users reported that some author data was still missing when saving a story on Composer.
12:49 Composer engineers started working on a fix to solve this issue.
13:51 Solution was deployed fixing the mentioned issue early the day

Arc Next Steps

To prevent similar issues in the future, Arc XP engineers are improving testing strategies to better reproduce intermittent bugs, enhancing safeguards against race conditions in Composer 2.0’s initialization process, and refining QA workflows for high-impact UI changes like the authors field. Additionally, we are assessing monitoring and alerting mechanisms to detect anomalies more quickly and reviewing internal coordination processes to streamline debugging and deployment efforts. These steps aim to improve system stability and accelerate issue resolution moving forward.

Posted Feb 14, 2025 - 12:24 EST

Resolved

A Hot fix has been pushed out for this incident.
Posted Jan 29, 2025 - 13:04 EST

Identified

We have identified the issue in the author component, and it is a change introduced in yesterday's release ("The guest authoring experience"):

https://docs.arcxp.com/en/what-s-new-in-arc-xp.html?_hsenc=p2ANqtz-_aGqzO-pUkvCp0NfS-5F1WqvDKWjIzPOGdChA71ZEHZ_6QGDthWCuFHE0JazXSRluEWNWRUPbnyxtlWkJpNGPHOoXojyug9kMp_tmHYSPs4nmajwQ#UUID-111b75fa-fa2f-6248-456f-096f002d0975_UUID-8e5607b6-7d37-2678-ffc4-c919fdada933

Yesterday's release introduced important critical fixes, and cannot be reverted without other detrimental effects. The Composer team is actively working on fixing the author component and a hotfix release is expected later today.

In the meantime, it is possible to move the author widget to the Metadata tab as the bug appears isolated to the Planning tab. We understand this is disruptive to your workflow, and will be prioritizing the hotfix today as quickly as possible.
Posted Jan 29, 2025 - 10:00 EST

Investigating

We are currently investigating reports that author information is disappearing from stories after making edits in Composer V2.
Posted Jan 29, 2025 - 09:23 EST
This incident affected: Creator Apps (Composer).