In 2014, we began development on Heron, a real-time streaming and analytics platform, to reliably process billions of events generated at Twitter every day. Today, we are proud to donate Heron to the Apache Incubator where the community will continue to grow and thrive under the guidance of the Apache Software Foundation. We are excited to see usage of Heron grow beyond our use cases here at Twitter, and look forward to continued community growth and collaborative support.
Heron is the next generation distributed streaming engine that was built to be backwards compatible with Apache Storm, which we open sourced in 2011 and donated to Apache. It was built to improve our developer and operational experiences with Storm and introduced a wide array of architectural improvements and native support for Apache Aurora. Heron has become our primary streaming system, reliabily powering all of Twitter’s real-time analytics and running hundreds of development and production topologies deployed on thousands of nodes.
Twitter Heron galvanized the streaming community by introducing several new ideas in stream processing including:
In 2016, we were excited to open source Heron, which enabled the project to grow into the vibrant and active community it is today. Some of the significant contributions from collaboration with community include:
We’ve worked collaboratively with the open source community and have benefited from contributed features that are now running in production at Twitter. We shared our experiences in streaming by publishing several papers in premier conferences.
Thank you to the Real-Time Compute team at Twitter, and to the Heron Community, for your continued support as the project moves onto the next phase of its life-cycle. Please follow us on Twitter at @heronstreaming, subscribe to the mailing list, and help support Heron in its new home!
 Dhalion: Self-Regulating Stream Processing in Heron, Proceedings of the Very Large Database(VLDB), September 2017.
 Twitter Heron: Towards Extensible Streaming Engines, IEEE 33rd International Conference on Data Engineering (ICDE), May 2017.
 Optimizing Twitter Heron, Twitter Engineering Blog, March 2017.
 Open Sourcing Twitter Heron - Twitter Engineering Blog, May 2016.
 Streaming@Twitter, Bulletin of the Technical Committee on Data Engineering, IEEE Computer Society, December 2015.
 Twitter Heron: Streaming at Scale, Proceedings of ACM SIGMOD Conference, Melbourne, Australia, June 2015.
 Flying Faster with Heron, Twitter Engineering Blog, May 2015.
 Storm@Twitter, Proceedings of ACM SIGMOD Conference, Snowbird, Utah, June 2014.