Hardware and software working together in harmony
Financial systems communicate by sending and receiving vast numbers of messages in many different formats. When people use terms like "vast" I normally think, "really..how many?" So lets quantify "vast" for the finance industry. Market data feeds from financial exchanges typically can be emitting tens or hundreds of thousands of message per second, and aggregate feeds like OPRA can peak at over 10…
Last week I attended a review session of the new JSR166 StampedLock run by Heinz Kabutz at the excellent JCrete unconference. StampedLock is an attempt to address the contention issues that arise in a system when multiple readers concurrently access shared state. StampedLock is designed to perform better than ReentrantReadWriteLock by taking an optimistic read approach. While attending the session…
Serial, Parallel, Concurrent, CMS, G1, Young Gen, New Gen, Old Gen, Perm Gen, Eden, Tenured, Survivor Spaces, Safepoints, and the hundreds of JVM startup flags. Does this all baffle you when trying to tune the garbage collector while trying to get the required throughput and latency from your Java application? If it does then do not worry, you are not alone. Documentation describing garbage…
Sometimes when profiling a Java application it is necessary to understand the assembly code generated by the Hotspot JIT compiler. This can be useful in determining what optimisation decisions have been made and how our code changes can affect the generated assembly code. It is also useful at times knowing what instructions are emitted when debugging a concurrent algorithm to ensure visibility…
Even from highly experienced technologists I often hear talk about how certain operations cause a CPU cache to "flush". This seems to be illustrating a very common fallacy about how CPU caches work, and how the cache sub-system interacts with the execution cores. In this article I will attempt to explain the function CPU caches fulfil, and how the cores, which execute our programs of instructions,…
In a previous article I reported what appeared to be a performance issue with CAS/LOCK instructions on the Sandy Bridge microarchitecture compared to the previous Nehalem microarchitecture. Since then I've worked with the good people of Intel to understand what was going on and I'm now pleased to be able to shine some light on the previous results. I observed a small drop in throughput with the…
Lately a number of people have suggested I start a discussion group on the subject of mechanical sympathy, so I've taken the plunge and done it! The group can be a place to discuss topics related to writing software which works in harmony with the underlying hardware to gain great performance. https://groups.google.com/forum/?fromgroups#!forum/mechanical-sympathy
In my last post I detailed the implications of the access patterns your code takes to main memory. Since then I've had a lot of questions about what can be done in Java to enable more predictable memory layout. There are patterns that can be applied using array backed structures which I will discuss in another post. This post will explore how to simulate a feature sorely missing in Java - arrays…
In high-performance computing it is often said that the cost of a cache-miss is the largest performance penalty for an algorithm. For many years the increase in speed of our processors has greatly outstripped latency gains to main-memory. Bandwidth to main-memory has greatly increased via wider, and multi-channel, buses however the latency has not significantly reduced. To hide this latency our…
Do you ever wish you could turn a Java object into a stream of bytes as fast as it can be done in a native language like C++? If you use standard Java Serialization you could be disappointed with the performance. Java Serialization was designed for a very different purpose than serialising objects as quickly and compactly as possible. Why do we need fast and compact serialisation? Many of our…
How should a system respond when under sustained load? Should it keep accepting requests until its response times follow the deadly hockey stick, followed by a crash? All too often this is what happens unless a system is designed to cope with the case of more requests arriving than it is capable of processing. If we are seeing a sustained arrival rate of requests, greater than our system is…
I'm often asked about the performance differences between Java, C, and C++, and which is better. As with most things in life there is no black and white answer. A lot is often discussed about how managed runtime based languages offer less performance than their statically compiled compatriots. There are however a few tricks available to managed runtimes that can provide optimisation opportunities…
Since leaving LMAX I have been neglecting my blog a bit. This is not because I have not been doing anything interesting. Quite the opposite really, things have been so busy the blog has taken a back seat. I’ve been consulting for a number of hedge funds and product companies, most of which are super secretive. One company I have been spending quite a bit of time with is my-Channels , a messaging…
Many applications record a series of events to file-based storage for later use. This can be anything from logging and auditing, through to keeping a transaction redo log in an event sourced design or its close relative CQRS . Java has a number of means by which a file can be sequentially written to, or read back again. This article explores some of these mechanisms to understand their performance…
After my last post on Java Lock Implementations , I got a lot of good feedback about my results and micro-benchmark design approach. As a result I now understand JVM warmup, On Stack Replacement (OSR) and Biased Locking somewhat better than before. Special thanks to Dave Dice from Oracle, and Cliff Click & Gil Tene from Azul, for their very useful feedback. In the last post I concluded, based on…
We all use 3rd party libraries as a normal part of development. Generally, we have no control over their internals. The libraries provided with the JDK are a typical example. Many of these libraries employ locks to manage contention. JDK locks come with two implementations. One uses atomic CAS style instructions to manage the claim process. CAS instructions tend to be the most expensive type of…
In a previous article on Inter-Thread Latency I showed how it is possible to signal a state change between 2 threads with less than 50ns of latency. To many developers, writing concurrent code using locks is a scary experience. Writing concurrent code using lock-free algorithms, i.e. algorithms that rely on the use of memory barriers and an intimate understanding of the underlying memory models,…
How often have we all heard that “batching” will increase latency? As someone with a passion for low-latency systems this surprises me. In my experience when batching is done correctly, not only does it increase throughput, it can also reduce average latency and keep it consistent. Well then, how can batching magically reduce latency? It comes down to what algorithm and data structures are…
When trying to build a highly scalable system the single biggest limitation on scalability is having multiple writers contend for any item of data or resource. Sure, algorithms can be bad, but let’s assume they have a reasonable Big O notation so we'll focus on the scalability limitations of the systems design. I keep seeing people just accept having multiple writers as the norm. There is a lot of…
Sequencing events between threads is a common operation for many multi-threaded algorithms. These sequences could be used for assigning identity to orders, trades, transactions, messages, events, etc. Within the Disruptor we use a monotonic sequence for all events which is implemented as AtomicLong incrementAndGet for the multi-threaded publishing scenario. While working on the latest version of…
I’m often asked, “What is the best way to learn about building high-performance systems”? There are many perfectly valid answers to this question but there is one thing that stands out for me above everything else, and that is modelling. Modelling what you need to implement is the most important and effective step in the process. I’d go further and say this principle applies to any development and…
Significantly improved performance and a cleaner API are the key takeaways for the Disruptor 2.0 concurrent programming framework for Java. This release is the result of all the great feedback we have received from the community. Feedback is very welcome and really improves the end product so please keep it coming. You can find the Disruptor project here , plus we have a wiki with links to…
Within our industry we use a huge range of terminology. Unfortunately we don’t all agree on what individual terms actually mean. I so often hear people misuse the term “ Refactoring ” which has come to make the business in many organisations recoil in fear. The reason for this fear I’ve observed is because of what people often mean when misusing this term. I feel we are holding back our industry…
In my previous post on False Sharing I suggested it can be avoided by padding the cache line with unused long fields. It seems Java 7 got clever and eliminated or re-ordered the unused fields, thus re-introducing false sharing. I've experimented with a number of techniques on different platforms and found the following code to be the most reliable. import java.util.concurrent.atomic.AtomicLong;…
Message rates between threads are fundamentally determined by the latency of memory exchange between CPU cores. The minimum unit of transfer will be a cache line exchanged via shared caches or socket interconnects. In a previous article I explained Memory Barriers and why they are important to concurrent programming between threads. These are the instructions that cause a CPU to make memory…