Open source
Introducing a new Swift package for Apache Thrift
By
Friday, 4 February 2022
We are introducing a new library for iOS, macOS, and Linux for ApacheThrift using Swift Codable. This new Thrift Library is an open-source, standalone, lightweight, data encoding library. This is the encoding library we now use in our clients when Apple platforms need to communicate with specific endpoints on the Twitter backend.
Today, we’re happy to share Twitter Apache Thrift so iOS developers outside Twitter can start using Thrift data.
Thrift is an open-source interface description language similar to Google’s Protobuf. Thrift and Protobuf are very similar; the main difference is in the serializer implementations to convert it to binary. At Twitter, we mainly use it for storing data and communicating between services.
For example, when someone makes a request for a Tweet, the Tweet service will talk to the Tweet metadata services all in Thrift. Then the Tweet service will merge the metadata into a Thrift model. Then Scrooge, our backend Thrift library, will convert that Thrift model to JSON for the clients.
We wanted to use Swift in our new telemetry system because most new features in the Twitter app are developed in Swift. The library we previously used for Thrift only supported Objective-C, and it wasn’t intuitive to use.
Our previous system used a mixture of JSON and Thrift in Objective-C. One of the goals of the new system was to have more structured data. JSON was not a good fit for our purposes because it lacks strong typing and the structure we needed. We also chose Thrift because the encoded data is smaller than JSON.
The initial draft of the library was created during a Hack Week. Initially, our goal was to get familiar with implementing a Swift Encoder and Decoder. Then we integrated it into the Twitter App and have been using it for over a year without any major issues. We have also added new features to it, such as support for Compact Thrift.
Twitter Apache Thrift is implemented as a Swift Package that you can easily add to your project in Xcode. It doesn’t have any dependencies beyond foundation system frameworks.
Twitter Apache Thrift is built on the protocol Codable. If you have used the built-in Swift JSON encoder/decoder, using the Thrift one should be intuitive. Encode Thrift data using the ThriftEncoder class and encode method:
struct GroceryProduct: ThriftCodable {
var name: String
var points: Int
var description: String?
}
// Setup the model
let pear = GroceryProduct(name: "Pear", points: 250, description: "A ripe pear.")
// Create an encoder
let encoder = ThriftEncoder()
// Encode our model
let data = try encoder.encode(pear)
Decoding is just as easy. Use the ThriftDecoder class and decode method to decode Thrift data:
struct GroceryProduct: ThriftCodable {
var name: String
var points: Int
var description: String?
}
let thrift = // some thrift data
let decoder = ThriftDecoder()
let product = try decoder.decode(GroceryProduct.self, from: thrift)
print(product.name) Building on Swift Codable also gives us free JSON encoding for our Thrift models. JSON encoding is helpful when debugging our telemetry system because it’s a human-readable format.
We also support automatic code generation with the Twitter Scrooge project. When generating code with Scrooge, it automatically creates a Swift Package with only TwitterApacheThrift as a dependency, allowing for easy integration into your projects. We have options for Swift classes or structs and Objective-C support for your models. You can find more information in the Scrooge documentation.
Building this library made it easy to use Thrift in the Twitter app. When we were building a new feature, we didn’t have to think about how to implement the models and get them decoded from Thrift; it just worked. We even saw wins on the backend with this feature. For example, we saw no significant increase in the amount of data leaving the data center. We were also able to refactor some code that used the Objective-C library, and we simply dropped in the new library.
The Twitter Apache Thrift project can be found on Twitter’s GitHub page.
Check out our other iOS open source projects, including:
We would love to get contributions from you! Please refer to CONTRIBUTING.md for the details.
Did someone say … cookies?
X and its partners use cookies to provide you with a better, safer and
faster service and to support our business. Some cookies are necessary to use
our services, improve our services, and make sure they work properly.
Show more about your choices.