The Twitter Engineering Blog

Information from Twitter's engineering team about our technology, tools and events.

Results from Engineering for: September 2013

Dremel made simple with Parquet

Columnar storage is a popular technique to optimize analytical workloads in parallel RDBMs. The performance and compression benefits for storing and processing large amounts of data are well documented in academic literature as well as several commercialanalyticaldatabases.

Read more...

Observability at Twitter

As Twitter has moved from a monolithic to a distributed architecture, our scalability has increased dramatically.

Read more...

Streaming MapReduce with Summingbird

Today we are open sourcing Summingbird on GitHub under the ALv2.

Read more...