The Twitter Engineering Blog

Information from Twitter's engineering team about our technology, tools and events.

Results from Engineering for: February 2010

Link: Cassandra at Twitter

Storage Team Lead Ryan King recently spoke to MyNoSQL about how we’re using Cassandra at Twitter. In the interview Ryan talks about the reasons for the switch and how we plan to migrate tweets from MySQL to Cassandra.



The Anatomy of a Whale

Sometimes it’s really hard to figure out what’s causing problems in a web site like Twitter. But over time we have learned some techniques that help us to solve the variety of problems that occur in our complex web site.


Links: Relational Algebra and Scala DI

Wondering what other technical concerns we face here at Twitter?

Infrastructure engineer Nick Kallen has a pair of posts on his personal blog: one about the new relational algebra system behind the Rails 3 ORM, and one about dependency injection in Scala.



Introducing the Open Source Twitter Text libraries

Over time Tweets have acquired a language all their own. Some of these have been around a long time (like @username at the beginning of a Tweet) and some of these are relatively recent (such as lists) but all of them make the language of Tweets unique. Extracting these Tweet-specific components from a Tweet is relatively simple for the majority of Tweets, but like most text parsing issues the devil is in the details.


WOEIDs in Twitter's Trends

How do you represent a “place”? That’s what we were wondering when we were putting together our API for Trends on Twitter. We needed a way to represent a place in a permanent and language-independent way - we didn’t want to be caught in using an identifier that may change over time, and we didn’t want to be caught in internationalization issues. Where we landed was on using Yahoo!’s Where On Earth IDs, or WOEIDs, as our representation.


Hello World

Welcome! I’m Ben, and I’m an engineer at Twitter. We’ve started this blog to show some of the cool things we’re creating and tough problems we’re solving.