Now that the 10th anniversary celebrations are behind us and we've all had a bit of time to interact with the new achievements and pursuits systems, we wanted to drop a quick Dev Update with some reflections.
As is common within game and software development, our team meets for a retrospective after we launch major features. The purpose of these exercises is to reflect on the development process itself, up to and including the launch rollout and the real time feedback we get from you. We open the floor for everyone to say what's on their mind about what went well, what could use improvement, and lessons we might learn to benefit the next project we embark on. Generally, we try to conduct these meetings not long after a feature has been deployed so that thoughts are fresh in our minds.
We wanted to share a few insights from our recent retrospective on the achievements system launch. This is an extremely pared down version (as much of a retrospective is internal), but there are a few things we thought it might be nice to share with you.
What Went Well
We saw this echoed in a lot of community discussions, but we feel like we made the right choices in communicating the launch date, progress, and intentions of this project. While the achievements themselves were a surprise, we felt that it was a step in the right direction to update you on the technical challenges, caveats, and goals we had for the feature. We hope to continue that as best we can.
We had meetings. A lot. Dozens of meetings. So many meetings. But this brought us together as a team, frequently, so that pivots or iterations in the development process came with less confusion. This was vital given how large the project was, and how long it took. Burnout was a real concern due to the massive nature of this feature (it truly was a marathon in every sense of the word), so we took every possible opportunity to keep everyone as in-the-know as possible.
We shot for the moon, and are elated to say that 99% of the achievements we scoped as far back as November made it into the final release. When you're talking about over 600 "things" to put in a feature, that is an absolutely smashing success in terms of planning, implementation, and testing. Collectively, we are very happy we were able to deliver nearly everything to you that we wanted to.
By necessity, our team became extremely familiar (or refamiliarized) with every single function and feature on the site. This not only is good for keeping ourselves fresh on the inner workings of the site, but will aid heavily in making quality of life and accessibility updates in the future.
Creating new achievements and pursuits also becomes fundamentally less challenging and risky now that the system is in place. We know exactly how to create all the technical hooks necessary and can proceed with coding future features with these systems fully in mind. This means far less "try really hard to make an old feature work with achievements" and far more "build the feature expressly with achievements factored in".
The "live ops" period for this feature, while tiring, went very well. We'd like to thank everyone who dove right into interacting with the systems and reporting any issues they experienced in the Bug Report Forums. That kind of feedback, right away, was integral for helping our engineering team turn around fixes much more quickly.
What Could Be Improved
There simply was no way real way for us to know what "launch day load" would look like until it got here. As described in one of our earlier updates, using live player data for testing was useful, but in the end, we couldn't truly emulate the server hugs we were going to eventually get, which is what led us to dial back (temporarily) how often achievement progress was calculated.
A double-edged sword if we ever saw one: in working hard to maximize the number of achievements all players could claim as complete on launch day, we did not adequately predict that it might be frustrating to claim them one-at-a-time. There were extremely important reasons why we didn't implement a "Claim All" button in the feature, but it still did end up being a temporary pain point for many of you.
There were a few integrations and "nice-to-haves" that didn't quite make it in (ideas for expanded Game Database tools, Which Waystone, etc.). In the end, we felt it was important on launch day to have more achievements and pursuits to interact with than tie-in or referential tools. We still want to create a lot of these.
For better or for worse, this project was a marathon. Aside from launching the website itself, this was the biggest and most challenging task our development has ever been through. While we still successfully navigated and avoided crunch, it was still easy to get exhausted with something that took as long to make as this. Thankfully, future feature work will have a narrower scope, but we feel there's something applicable we can still learn from this process.
Every game development studio utilizes retrospectives in different ways, but the ones who excel use them to guide their work and processes going into the future. Thankfully, we got a wealth of information, and it led to many changes just after launch. It also helped advise the following fixes and updates that have been rolled out most recently:
- The achievement formerly titled Diamond in the Stuff has been replaced by Battle Ready. The former achievement was not possible to achieve.
- The achievement Electrifying Result now requires a score of 700,000 rather than 1,000,000.
- A new achievement, Teacher's Pet, has been added.
- A new achievement, Know It All, has been added.
- A new achievement, Treasure for Trash, has been added.
- A cute "confetti" animation has been added behind the claim window to make completing an achievement a little more exciting.
- Achievements that involve collecting/possessing a list of items more clearly indicate which items in the criteria you've already obtained.
Work isn't done yet, and you can look forward to more achievements and pursuits as time goes on, but we wanted to thank everyone for engaging with them, enjoying them, and giving feedback on them. Congratulations on what you've already achieved, and best of luck going forward!
Flight Rising Developer
Project Management | Engineering Team Cheerleader | UI Designer
Project Management | Engineering Team Cheerleader | UI Designer