Misadventures in Concurrent and Parallel programming, plus random comments on software performance and various OSS contributions.
Where Aeron catches up on the goss. The Need For Naming A few months ago a pull request appeared on Aeron's Github site that added the ability to request Aeron to resolve or re-resolve host names to IP addresses. In cloud environments, especially when using Kubernetes, when nodes fail and restart it is not uncommon for a node with the same host name to restart with a different IP address.…
One of my more recent projects has led me to become more involved in the Aeron project. If you are unaware of Aeron, then head over to the Github site and check it out. At its core is an reliable messaging system that works over UDP, Multicast UDP and IPC. It also contains an archiving feature for recording and replay and (still under active development) an implementation of the Raft protocol for…
If you have been a long time follower of my blog you will have noticed that is has been a really long time since I posted any new content. Since my last post six years ago at lot has changed. I no longer work at LMAX an have started out as an independent consultant. If you are part of an organisation looking for assistance in building software, especially if you facing challenges around…
While preparing for my talk at QCon SF 2014, I wanted to investigate a theory around how micro-benchmarks are not a useful reflection of how software may behave when run as part of a larger application. Specifically due contention in the last-level cache (L3* in current Intel CPUs). Read more »
I'll be giving some talks over the next few months: 8 Oct: Auckland Software Craftsmanship - 6 Years of test automation . 16 Oct: Auckland JVM Group - Stuff I learned about performance . 5 Nov: QCon San Francisco 2014 - Stuff I learned about performance .
My talk on the Disruptor at YOW! 2013 is now available at the YOW! Eventer site.
If, like me, you prefer to run the Oracle version of Java on your Linux machine as the default JDK, you will often find that the Linux distro will have other ideas. Fedora for example has a number of Java based applications as part of the distribution which will include a dependency on the OpenJDK. When the distro installs OpenJDK is will generally be setup as the default for executing the various…
Recently on InfoQ, Aliaksei Papou posted an article on some of his experiments with high performance interchange of messages between threads. There were a number of examples within the article, but I am going to focus on the multi-producer case. One of the optimisations that the article showed was that if you knew the number of producers that you have at initialisation time you can build a…
I recently upgraded my personal workstation from Ubuntu 12.04 to Fedora Core 19. I'm very happy with the change, I prefer the standard Gnome 3 to Canonical's Unity desktop. Probably something to do with my workstation not being a tablet. However, one of the annoyances with Fedora Core has always been its font rendering. It was never as nice a default Ubuntu install. However after a couple of days…
I've decided that I'm a bit bored of the whole putting a beta tag on various versions of the Disruptor so I've decide to send forth Disruptor 3.0.0 into the world. The big challenges of this release were to clean up the code and come up with a better algorithm for handling multiple producers. If I was lucky, make it even faster. I went down a couple of dark alleys initially with this release, but…
I'm happy to announce that I will speaking at Tech Mesh in December. I'll be speaking about the Disruptor from two perspectives, firstly looking briefly back at some of the history and motivations behind the Disruptor. Then spending some time explaining at the challenges of building high performance concurrent systems (like the Disruptor) and delving into how the JVM and hardware could change to…
Last week I gave a talk on non-blocking concurrency at JAX London . Here are the slides: The video is also available
During a recent conversation on the LJC mailing list around a proposal for adding a library to the JDK that would add support for handling integer overflow a question arose. Would the JVM be able to optimise this code to make efficient use of the hardware support for overflow detection if this functionality was implemented as a library. I made the comment that this is problem is probably one best…
I've be working sporadically on the next major revision of the Disruptor, but still making steady progress. I've merged my experimental branch into the main line and I'm working on ensure comprehensive test coverage and re-implement the Disruptor DSL. As a matter of course I've been running performance tests to ensure that we don't regress performance. While I've not been focusing on performance,…
The Disruptor version 2.10 has been released . It is available from the Google Code download page and has been submitted to Maven central. Changes Remove deprecated timeout methods. Added OSGI metadata to jar file. Removed PaddedAtomicLong and use Sequence in all places. Fix various generics warnings. Change Sequence implementation to work around IBM JDK bug and improve performance by ~10%. Add a…
Last month the guys from the Distributed Podcast interviewed me about LMAX and the Disruptor. Many thanks to Jonathan and Rinat, I had a great time.
Along with Martin Thompson , I'm speaking about non-blocking concurrency at QCon London in March.
My slides from JAX London - Beginner's Guide to Hardcore Concurrency: Beginners guide-concurrency View more presentations from Michael Barker Video: LJC@Playfish , JAX London My slides from Devoxx - Disruptor Tools In Action: Disruptor tools in action View more presentations from Michael Barker Video: Devoxx (Payment required)
Within a project I've been working on I've had the need to simulate the capabilities of Linux's /proc/cpuinfo on Mac OS. Specifically I needed to build a topology of the CPUs on a given system. I.e. I need to map the operating system's processors to hardware threads, then build a picture of which cores and sockets those threads reside. For example my Mac looks something like: CPU0 (Thread 0) ---+…
A few weeks ago one of the users of the Disruptor posted some worrying benchmarks: ThreePublisherToOneProcessorSequencedThroughputTest run 0: BlockingQueue=645,161 Disruptor=1,772 ops/sec run 1: BlockingQueue=1,250,000 Disruptor=20,000,000 ops/sec run 2: BlockingQueue=1,250,000 Disruptor=56 ops/sec It appears under heavy contention with fewer available cores than busy threads the Disruptor can…
I've decided to rename my blog. I plan to focus my blog efforts more on concurrency than any other particular subject and I thought it would be fun to pay homage to the bad science blog by Ben Goldacre. It will probably kill my traffic for a while, until I get the few blog aggregators that carry my blog to update their feeds. So welcome to my rebranded blog, I'll start posting again in the next…
I've had a few speaking events confirmed for the end of this year: Trisha Gee and I are speaking at JAX London on November 2nd . I'm presenting the disruptor in a half hour "Tools in Action" slot at Devoxx 2011 on November 14th . I'm speaking on Technology and Performance Folklore at StackOverflow Dev Days on November 15th . Trisha and I are hoping to preview our talk to the LJC sometime in…
Ever wanted to know what flags could be set and their default values for the Hotspot VM? Try this: java -XX:+PrintFlagsFinal
Being someone who's interested in Open Source and very focused on performance, I've been looking at the Java & the JVM and wondering about what I would most like to see added. IMHO one of the fundamental missing pieces is support for complex stack allocated types. This is not a new idea. C# calls them structs. In C, C++ and Go stack & heap allocation is orthogonal to the definition of the type. An…
The Disruptor , the design pattern at the core of the financial exchange I'm helping build at LMAX has been open sourced . If you've seen the presentation that Martin Thompson and I gave at QCon or the one I did for the LJC @Skillsmatter, this is the ring-buffer based code that we've been banging on about for a while. What is the Disruptor? At its simplest, it's an alternative to a queue, but can…