Posts from all blogs: java

Simplify Service Dependencies with Nodes

A history of how we handle service dependency graphs in search backend, and introducing a new library to make the implementation of asynchronous services in Java easier.

Read more...

Omnisearch Index Formats

In this post, we discuss how we have evolved our search technology to accommodate diverse document types, the surprising performance impact of these changes, and how we are using this improved technology to power Twitter’s latest product efforts.

Read more...

SuperRoot: Launching a High-SLA Production Service at Twitter

Our search infrastructure team is building Omnisearch, a new information retrieval system to power the next generation of relevance-based, personalized products. We recently launched SuperRoot, the first major architectural component of Omnisearch. In this blog post, we detail the path to building and launching a high-SLA production system at Twitter.

Read more...

Introducing Omnisearch

Our search infrastructure team is building a new information retrieval system to power the next generation of relevance-based, personalized products.

Read more...

The release of Pants 1.0

After five years of development, open source project Pants 1.0.0 is here and we can’t wait for you to check it out.

Read more...

TwitterDev code samples on GitHub

Visit TwitterDev on GitHub to help you jumpstart your Twitter integration or brainstorm ideas for your next great project.

Read more...

Netty at Twitter with Finagle

Finagle is our fault tolerant, protocol-agnostic RPC framework built atop Netty. Twitter’s core services are built on Finagle, from backends serving user profile information, Tweets, and timelines to front end API endpoints handling HTTP requests.

Read more...

Dremel made simple with Parquet

Columnar storage is a popular technique to optimize analytical workloads in parallel RDBMs. The performance and compression benefits for storing and processing large amounts of data are well documented in academic literature as well as several commercialanalyticaldatabases.

Read more...

Drinking from the Streaming API

Today we’re open-sourcing the Hosebird Client (hbc) under the ALv2 license to provide a robust Java HTTP library for consuming Twitter’s Streaming API. The client is full featured: it offers support for GZip, OAuth and partitioning; automatic reconnections with appropriate backfill counts; access to raw bytes payload; proper retry schemes, and relevant statistics.

Read more...

Visualizing Hadoop with HDFS-DU

We are a heavy adopter of Apache Hadoop with a large set of data that resides in its clusters, so it’s important for us to understand how these resources are utilized. At our July Hack Week, we experimented with developing HDFS-DU to provide us an interactive visualization of the underlying Hadoop Distributed File System (HDFS).

Read more...

Pages