The Twitter Engineering Blog

Information from Twitter's engineering team about our technology, tools and events.

Twitter at @MesosCon 2015

Once again, we’re pleased to sponsor and participate in #MesosCon. As heavy users of both Mesos and Apache Aurora to power our cloud infrastructure, we’re excited to be part of this growing community event.

Read more...

Flying faster with Twitter Heron

A new real-time analytics platform at Twitter scale.

Read more...

Graduating Apache Parquet

Apache Parquet, a columnar storage format for Hadoop, is graduating from the Apache incubator.

Read more...

Building a new trends experience

Learn how we revamped trends and rewrote the entire trends backend system.

Read more...

Another look at MySQL at Twitter and incubating Mysos

We’re sharing updates on how Twitter uses MySQL, as well our plans to open source Mysos, a new MySQL on Apache Mesos framework.

Read more...

Handling five billion sessions a day – in real time

An inside look at the Answers architecture that provides app developers with reliable, real-time and actionable insights into their mobile apps.

Read more...

All about Apache Aurora

As @ApacheAurora announces their 0.7.0 release, we share the story of Aurora at Twitter and in the open source community.

Read more...

Introducing practical and robust anomaly detection in a time series

A novel way to detect anomalies in big data.

Read more...

Building a complete Tweet index

Today, we are pleased to announce that Twitter now indexes every public Tweet since 2006.

Since that first simple Tweet over eight years ago, hundreds of billions of Tweets have captured everyday human experiences and major historical events. Our search engine excelled at surfacing breaking news and events in real time, and our search index infrastructure reflected this strong emphasis on recency. But our long-standing goal has been to let people search through every Tweet ever published.

Read more...

Breakout detection in the wild

A novel statistically rigorous approach for early detection of breakout(s) in web-scale cloud systems.

Read more...

Pages