The Hosebird Client streaming library

Today we’re releasing Hosebird Client, a Java-based client for Twitter’s Streaming APIs. While Twitter has open-sourced a considerable amount of software, this is the first API client library we’ve developed in-house and are releasing to the world.

As you might expect from a streaming client developed by Twitter, many of the Streaming APIs’ features are built-in, including simple OAuth integration, GZip support, and reconnect logic which follows our best practices. In fact, since these features are built directly into the client, writing a Java application which consumes streaming data is remarkably direct:

// Connect to the filter endpoint, tracking the term "twitterapi"
Hosts host = new BasicHost(Constants.STREAM_HOST);
StreamingEndpoint endpoint = new StatusesFilterEndpoint();
endpoint.trackTerms(Lists.newArrayList("twitterapi"));
 
// Drop in the oauth credentials for your app, available on dev.twitter.com
Authentication auth = new OAuth1("consumerKey", "consumerSecret", 
                                 "token", "tokenSecret");
 
// Initialize a queue to collect messages from the stream
BlockingQueue<String> messages = new LinkedBlockingQueue<String>(100000);
 
// Build a client and read messages until the connection closes.
ClientBuilder builder = new ClientBuilder()
    .name("FooBarBaz-StreamingClient")
    .hosts(host)
    .authentication(auth)
    .endpoint(endpoint)
    .processor(new StringDelimitedProcessor(messages));
Client client = builder.build;
client.connect();
 
while (!client.isDone()) {
  String message = messages.take();
  // Do something with message
}

Hosebird Client has been primarily authored by Steven Liu (@steven) and Kevin Oliver (@kevino). Thanks to their hard work, the client is suitable for any level of streaming access — from a low-volume track connection all the way up to the entire stream of public Tweets. It is used by internal teams at Twitter for real-time processing, so it has already been tested in a production environment. We hope you find it useful for your own projects as well!