Twitter at @MesosCon 2015

Friday, 31 July 2015

Once again, we’re pleased to sponsor and participate in #MesosCon. As heavy users of both Mesos and Apache Aurora to power our cloud infrastructure, we’re excited to be part of this growing community event.

The conference, organized by the Apache Mesos community, features talks on the popular open source cluster management software and its ecosystem of software for running distributed applications at the scale of tens of thousands of servers.

Conference highlights
This year’s #MesosCon will be significantly larger than last year and features simultaneous tracks including beginner talks, the Mesos core, frameworks, and operations. We have a stellar lineup of invited keynote speakers including Adrian Cockcroft (@adrianco, Battery Ventures), Neha Narula (@neha, MIT), Peter Bailis (@pbailis, UC Berkeley), and Benjamin Hindman (@benh, Mesosphere).

We’re also pleased that Twitter will have a strong presence. We’ll be sharing our latest work as well as best practices from the last four-plus years of using Apache Mesos and Apache Aurora. And if you’re interested in learning more about engineering opportunities, stop by our booth.

There’s a pre-conference hackathon that several of us Twitter folks will be attending. We’re also hosting a #MesosSocial in our Seattle office on Wednesday, August 19 to kick off the conference. You can follow @TwitterOSS for updates when we announce more details next week. See you at #MesosCon!

Twitter speakers

The New Mesos HTTP API - Vinod Kone, Twitter, Isabel Jimenez (@ijimene), Mesosphere
This session will provide a comprehensive walkthrough of recent advancements with the Mesos API, explaining the design rationale and highlighting specific improvements that simplify writing frameworks to Mesos.

Twitter’s Production Scale: Mesos and Aurora Operations - Joe Smith, Twitter
This talk will offer an operations perspective on the management of a Mesos + Aurora cluster, and cover many of the cluster management best practices that have evolved here from real-world production experience.

Supporting Stateful Services on Mesos using Persistence Primitives - Jie Yu, Twitter, and Michael Park, Mesosphere
This talk will cover the persistence primitives recently built into Mesos, which provide native support for running stateful services like Cassandra and MySQL in Mesos. The goal of persistent primitives is to allow a framework to have assured access to its lost state even after task failover or slave restart.

Apache Cotton MySQL on Mesos - Yan Xu, Twitter
Cotton is a framework for launching and managing MySQL clusters within a Mesos cluster. Recently open-sourced by Twitter as Mysos and later renamed, Cotton dramatically simplifies the management of MySQL instances and is one of the first frameworks that leverages Mesos’ persistent resources API. We’ll share our experience using this framework. It’s our hope that this is helpful to other Mesos framework developers, especially those wanting to leverage Mesos’ persistent resources API.

Tactical Mesos: How Internet-Scale Ad Bidding Works on Mesos/Aurora - Dobromir Montauk, TellApart
Dobromir will present TellApart’s full stack in detail, which includes Mesos/Aurora, ZK service discovery, Finagle-Mux RPC, and a Lambda architecture with Voldemort as the serving layer.

Scaling a Highly-Available Scheduler Using the Mesos Replicated Log: Pitfalls and Lessons Learned - Kevin Sweeney, Twitter
This talk will give you tools for writing a framework scheduler for a large-scale Mesos cluster using Apache Aurora as a case study. It will also explore the tools the Aurora scheduler has used to meet these challenges, including Apache Thrift for schema management.

Simplifying Maintenance with Mesos - Benjamin Mahler, Twitter
Today, individual frameworks are responsible for maintenance which poses challenges when running multiple frameworks (e.g. services, storage, batch compute). We’ll explore a current proposal for adding maintenance primitive in Mesos to address these concerns, enabling tooling for automated maintenance.

Generalizing Software Deployment - The Many Meanings of “Update” - Bill Farner, Twitter
Bill will present the evolution of how Apache Aurora managed deployments and describe some of the challenges imposed by wide variance in requirements. This talk will also share how deployments on Aurora currently run major services at Twitter.

Per Container Network Monitoring and Isolation in Mesos - Jie Yu, Twitter
This talk will discuss the per container network monitoring and isolation feature introduced in Mesos 0.21.0. We’ll show you the implications of this approach and lessons we learned during the deployment and use of this feature.

Join us!
Good news: there’s still time to register for #MesosCon and join us in Seattle on August 20-21.

There’s a pre-conference hackathon that several of us Twitter folks will be attending. We’re also hosting a #MesosSocial in our Seattle office on Wednesday, August 19 to kick off the conference. You can follow @TwitterOSS for updates when we announce more details next week. See you at #MesosCon!