The Twitter Engineering Blog

Information from Twitter's engineering team about our technology, tools and events.

Results from Engineering for: June 2013

hRaven and the @HadoopSummit

Today marks the start of the Hadoop Summit, and we are thrilled to be a part of it. A few of our engineers will be participating in talks about our Hadoop usage at the summit:

Read more...

Zippy Traces with Zipkin in your Browser

Last summer, we open-sourced Zipkin, a distributed tracing system that helps us gather timing and dependency data for the many services involved in managing requests to Twitter. As we continually improve Zipkin, today we’re adding a Firefox extension to Zipkin that makes it easy to see trace visualizations in your browser as you navigate your website.

Read more...

Libcrunch and CRUSH Maps

When we introduced our in-house photo storage system Blobstore to the world, we discussed a mapping framework called libcrunch for Blobstore that maps virtual buckets to storage nodes. The libcrunch implementation was heavily inspired by the seminal paper on the CRUSH algorithm.

Read more...

CSP to the Rescue: Leveraging the Browser for Security

Programming is difficult — and difficult things generally don’t have a perfect solution. As an example, cross-site scripting (XSS) is still very much unsolved. It’s very easy to think you’re doing the right thing at the right time, but there are two opportunities to fail here: the fix might not be correct, and it might not be applied correctly. Escaping content (while still the most effective way to mitigate XSS) has a lot of “gotchas” (such as contextual differences and browser quirks) that show up time and time again.

Read more...