Destiny 2 is back online after a second incident in which a bug had caused players’ inventories to lose items. Rather than simply fixing the problem, developer Bungie has detailed what has happened… and more importantly, why it won’t happen again.
“Several months ago, players reported that quest log sorting wasn’t working properly, and we wanted to fix that. The team investigated and found that the clean-up process was resetting the timestamp on a subset of quests, which was breaking chronological sorting. We decided to fix this by disabling the timestamp-resetting behavior for quests,” Bungie detailed.
“That fix was conceptually reasonable but, through subtle side effects, it ended up disabling too much of the clean-up process. The net result was that the game calculated the wrong cap quantity for stacked items (such as currencies and materials), which caused items above the cap to be lost. We knew this code was critical and, per our typical process, we had two domain experts provide code reviews for the change – but sadly, we didn’t spot the bug.”
Bungie said it initially chalked up the bug as a false-positive when it, obviously, was not.
“Fast-forward to two weeks ago. The 2.7.1 update had the aforementioned bug that caused character data corruption and resulted in our first ever rollback of character data. To fix that issue quickly, we applied a patch to the servers instead of trying to get a full build of the game code deployed. This involved making a change to a server setting to override the game code used to process character data and then restarting the WorldServers to pick up that change,” the explaination continues.
“Fast forward again to today, February 11th, when we rolled out the 126.96.36.199 update coinciding with the launch of Crimson Days. After launch, some of the WorldServers once again crashed on startup because of a high volume of servers starting simultaneously. Once again we manually restarted those servers and thought everything was fine. We were wrong.”
Bungie then details how they’ll prevent this issue from happening again — that includes safeguards, a permanent fix and ways to better deal with rollback scenarios.
“We know today’s outage and character rollback has been frustrating for you, especially with launch of Crimson Days, just as it’s been frustrating for us to realize that this is a problem we should have been able to avoid,” the post concludes.
Destiny 2 is available now on Windows PC, Xbox One, PS4 and Stadia.