Some of the most memorable interactions on Twitter occur because of the public conversations that take place on our service. Lately, we’ve been hearing feedback about a complication with these conversations – sometimes, when you reply to a Tweet, you aren’t able to see your reply in the conversation. Due to a longstanding technical capacity limitation, when there is an overwhelming volume of replies to a Tweet, our platform is unable to show all of these replies on the Tweet’s page.
In the last infrastructure blog post we focused on some of the key efficiency and optimization efforts that we’ve made over the years and in this one, we’ll cover the challenges we’ve had to overcome to scale some of our key services and network.
In this post, we discuss how we have evolved our search technology to accommodate diverse document types, the surprising performance impact of these changes, and how we are using this improved technology to power Twitter’s latest product efforts.
Our search infrastructure team is building Omnisearch, a new information retrieval system to power the next generation of relevance-based, personalized products. We recently launched SuperRoot, the first major architectural component of Omnisearch. In this blog post, we detail the path to building and launching a high-SLA production system at Twitter.
As one of the most critical infrastructure at Twitter, Observability provides highly scalable data collection and visualization services. This blog post gives overview of our architecture and shares our experience in developing and operating our systems.