The Twitter Engineering Blog

Information from Twitter's engineering team about our technology, tools and events.

Posts from Engineering: web scale

Manhattan software deployments: how we deploy Twitter’s large scale distributed database

We elaborate on the challenges involved in conducting software deployments for our distributed database and share our approach to solving them.

Read more...

Resilient ad serving at Twitter-scale

“Adaptive quality factor” is a technique used to make the ad server resilient and scalable, and at the same time achieve revenue optimality.

Read more...

Strong consistency in Manhattan

We explore lessons we learned while adding strong consistency to Manhattan and describe several problems that had to be solved along the way (implementing TTLs in a strongly consistent manner, doing distributed log truncations).

Read more...

How we break things at Twitter: failure testing

The design, architecture and implementation of Twitter’s failure testing framework.

Read more...

Push our limits - reliability testing at Twitter

At Twitter, we strive to prepare for sustained traffic as well as spikes - some of which we can plan for, some of which comes at unexpected times or in unexpected ways. To help us prepare for these varied types of traffic, we continuously run tests against our infrastructure to ensure it remains a scalable and highly available system.

Read more...

TSAR, a TimeSeries AggregatoR

Twitter is a global real-time communications platform that processes many billions of events every day. Aggregating these events in real time presents a massive challenge of scale. Classic time-series applications include site traffic, service health, and user engagement monitoring; these are increasingly complemented by a range of analytics products and features such as Tweet activity, Followers, and Twitter Cards that surface aggregated time-series data directly to end users, publishers, and advertisers.

Read more...

Manhattan, our real-time, multi-tenant distributed database for Twitter scale

Twitter has unique and intensive storage requirements. We built a database, called Manhattan, that meets those requirements and allows self-service access for our engineers.

Read more...