As Oliver Wyman were platinum sponsors of this years’ RabbitMQ Summit, a group of us were in attendance to contribute and hear the latest news from today’s RabbitMQ users.
Our own Alex Thomas gave a talk: Using the Source-oriented Exchanges Pattern to Keep Events in Order. Another talk I particularly enjoyed was Lifting the Lid on Quorum Queues, because it was cleverly framed to fit in the time: Karl Nilsson described the command log and tackled how it’s efficiently truncated. This is critical because the command log includes message bodies. Once you combine what we learnt in the talk with an understanding of RAFT, you’ll be well equipped to reason about the performance of quorum queues.
I also went to see Don’t Guess—Trace, which covered generating flame graphs using the output of Erlang tracing to investigate RabbitMQ performance problems. If you don’t have time for a video, just look at eflame on Github. Definitely a useful addition to my toolbox.
Rapidly Building Event Driven and Streaming Applications with RabbitMQ is well worth a watch if you’re interested in Spring Cloud Stream. It gives a good feel for the framework in a short period of time. Depending on what your requirements are, this is a crowded space, but Spring Cloud Stream might be attractive if you want to remain agnostic to your infrastructure: It supports various open source and cloud based messaging infrastructure as well as Kafka.
I’ve since watched WeWork’s ‘Good Enough’ Order Guarantee, it was on at the same time as as Lifting the lid on Quorum Queues. It is a real world example of of the more general talk Alex gave, with sharding mixed in. It doesn’t seem to address the problem in the first slide: The booking has two topics: the account and the office, but it’s a useful pattern in any case. It points out how useful exclusive consumers are in the absence of some other consensus system. It’s a lot easier to look after than Zookeeper, for example. In fact, RabbitMQ has a feature I wasn’t aware of: single active consumers, which offers better availability.
I’m yet to watch The Consistent Hash Exchange: Making RabbitMQ a better broker, which doesn’t seem to be on my printed schedule. I don’t what the story is there. Anyone? I mention this last talk because it almost adds up to a theme: using RabbitMQ where ordering within a topic is critical. Alex described the pattern, WeWork provided a case study. RabbitMQ has always had a strong storey here: because exchanges process messages in order within a node, as long as publishers bind to only one node at a time, RabbitMQ provides a good enough ordering guarantee. I’ve used this property on several projects very successfully. New features in RabbitMQ are making this scale (consistent hash exchange), and be more resilient (quorum queues, single active consumers), with little investment on the part of developers and operators.
Thanks to all the people who gave talks, and also all the people I spoke to who shared interesting RabbitMQ stories and use cases. It was a really stimulating day.