In the last infrastructure blog post we focused on some of the key efficiency and optimization efforts that we’ve made over the years and in this one, we’ll cover the challenges we’ve had to overcome to scale some of our key services and network.
In this post, we discuss how we have evolved our search technology to accommodate diverse document types, the surprising performance impact of these changes, and how we are using this improved technology to power Twitter’s latest product efforts.
Our search infrastructure team is building Omnisearch, a new information retrieval system to power the next generation of relevance-based, personalized products. We recently launched SuperRoot, the first major architectural component of Omnisearch. In this blog post, we detail the path to building and launching a high-SLA production system at Twitter.