Building a new twitter.com required thinking about everything we wanted to bring to the new site that was an integral part of the experience. We wanted to think about how to bring the new twitter.com to people while also continually incorporating their feedback at each step.
Creating the milestones
When we first kicked off this effort, we looked at every single feature - there were hundreds of them! - available on the old site and prioritized them using criteria. Here’s how we evaluated each feature using prioritized criteria:
The first three criteria were non-negotiable; beyond that, the more criteria a feature fell under, the higher it was likely to get prioritized. There were nuances of the magnitude of impact within the criteria themselves, but using this framework enabled us to make decisions objectively and faster. We didn’t have to debate each features inclusion, we could just apply the criteria. If we felt the criteria didn’t prioritize a certain feature correctly, we could provide an argument for making an exception. This criteria also allowed us to sequence and build features to build the product iteratively without taking away from the core Twitter experience that people know and love.
Our goals with the milestones were two-fold: (i) divide the amount of work we had to do into milestones that could be completed over the course of a few months each, and (ii) create a feedback loop between the milestones to gather customer feedback and quantitative data to inform what we might want to change in the next milestone.
With those goals in mind, these were our milestones:
Milestone 1
When we were planning out the milestones for the project, we wanted to make sure we were getting feedback at every step of the way to help us iterate and improve on the experience. We started with getting feedback with a small group - our internal web team - and once we felt like the product was both a functional and reliable experience to share with all Twitter employees we started collecting feedback from them.
Milestone 2
The feedback from Twitter employees was important in helping us get the product ready, especially because they tend to use twitter.com on their laptops while they are at work. This served as a good starting point to iterate on the experience, however, we also recognized that we needed to capture the different types of customers, they way they use Twitter, their devices, and geographies when taking feedback into account.
Milestone 3
We knew the Twitter employee feedback would be biased due to it coming from people who are building and improving the product daily, so we slowly started unveiling the experience to more people through an opt-in.
Before rolling out the opt-in we prioritized key features that we needed to have before sharing it publicly and we descoped the rest to avoid delaying getting feedback. We collected feedback through a survey that people could access in the product and when they chose to return to the old experience. We received thousands of feedback submissions over the course of this milestone and learned a lot about what was important for the desktop experience.
One of the items we descoped was the emoji picker in the composer because a workaround existed: a person could still add emojis to their Tweet using the native emoji keyboard. We didn't expect this to be the biggest complaint about the new twitter.com during this early milestone. However, through collecting and listening to feedback we found that it was the #1 pain point for people because people weren’t able to express themselves as easily without access to emojis. We used feedback from people to prioritize bringing emoji picker to the new experience faster so that people could send 🔥Tweets.
There were other themes of feedback that we continued to read and use to influence our roadmap. One big theme of feedback from the early iteration made us think about how we can use the screen real estate on the device. This led to the side navigation design where one goal was to take advantage of the available screen space; it also provided the opportunity to bring forward features that felt hidden before, like Lists and Bookmarks, as well as building the framework for additions in the future. More on this from our lead designer and engineer soon!
Another piece of feedback we heard a lot, especially from Japan, was the desire to easily switch between accounts. People on Twitter use account switching in different ways, and through user research we’ve found that in Japan (where we have some of the highest usage of account switching), people use it to keep up to date on their interests. Account switching is available on our Android and iOS clients, but the cost and complexity of bringing it to web was high and we often debated the cost-benefit of this feature. Customer feedback influenced our decision to prioritize it and include this feature in the new Twitter.com.
Milestone 4
While opt-in testing allowed us to incorporate user feedback into our roadmap, we quickly observed a new bias: the people who opted in skewed towards power users and their patterns of usage are different from people who are newer or lighter users of Twitter. We uncovered this with the help of our data science team who did an analysis on how people were using the site. We are building a global product for everyone who uses Twitter so it was important that we got feedback and data from people that were representative of our entire user base.
We took a couple of steps to mitigate this bias from the opt-in experience: (1) qualitative research in the form of in person interviews and (2) experiments that defaulted people into the new experience rather than giving them an option to opt-in. We used qualitative research to understand the why behind people’s actions and patterns, particularly when testing out the side navigation layout. We used experimentation to gather quantitative data that revealed what people were doing at scale.
In this milestone we also experimented with two navigation styles and to collect feedback and make a decision about which UI we wanted to ship. Some people got the new side navigation, others got top navigation.
Through experimentation, we learned that people were able to navigate the site smoothly in both treatments, and we didn’t see a significant change in core metrics. With the exposure of Bookmarks and Lists in the navigation, we saw an increase in the usage of those features and are actively thinking about making those experiences better. We also identified areas of improvement like Tweeting: we found that the modal compose pattern created more friction when Tweeting which impacted some of our Tweeting related metrics and also heard this in feedback. We used this information to bring an inline composer to the new twitter.com and invest in our composition experience.
Milestone 5
After synthesizing the experiment data and qualitative feedback, we set out to address the most critical pain points before we launched this experience to everyone. We had a tough choice to make: when to ship? We knew there was still feedback that we needed to address, but we also knew that most of the people using twitter.com were on the old site and weren’t getting updates like Account switching, Bookmarks and Explore. We made the decision to address what we thought were the most important points of feedback (which came down to a list of 4-5 items) before shipping it to everyone, followed by a list of fast follows that we prioritized using insights from experimentation and feedback.
We started rolling out the new twitter.com on July 15th, 2019 and have been watching metrics and listening to feedback as we have with every other milestone. Milestone 6 is still in progress, so you’ll hear about that soon!
Takeaways
Over the course of this project, we learned that setting criteria early in the process to prioritize our milestones was important to drive the product forward and continuously add value for people who use twitter.com. Criteria helps you take an objective approach to prioritization that is consistent and fast.
In addition to our internal prioritization framework, developing publicly and in the open was important too - we knew we might not have made the right prioritization calls and listening to feedback from people who use the product can help us course correct earlier rather than later. With the logged in launch, the feedback loop hasn’t stopped and we’re excited to keep improving twitter.com together.
Did someone say … cookies?
X and its partners use cookies to provide you with a better, safer and
faster service and to support our business. Some cookies are necessary to use
our services, improve our services, and make sure they work properly.
Show more about your choices.
