The Compute Express Link (CXL) microconference was held, for a second straight time, at this year's Linux Plumbers Conference. The goals for the track were to openly discuss current on-going development efforts around the core driver, as well as experimental memory management topics which lead to accommodating kernel infrastructure for new technology and use cases. CXL session at LPC23 (i) CXL…
locking/rwsem: optimize trylocking for the uncontended case This applies the idea that in most cases, a rwsem will be uncontended (single threaded). For example, experimentation showed that page fault paths really expect this. The change itself makes the code basically not read in a cacheline in a tight loop over and over. Note however that this can be a double edged sword, as microbenchmarks have…
sched/wake_q: reduce atomic operations for special users Some core users of wake_qs , futex and rwsems were incurring in double task reference counting - which was a side effect for safety reasons. This change levels the call's performance with the rest of the users. [Commit 07879c6a3740 ] irq: Speedup for interrupt statistics in /proc/stat On large systems with a large amount of interrupts the…
mm/page-alloc: reduce zone->lock contention Contention in the page allocator was seen in a network traffic report, in which order-0 allocations are being freed by back to the directly to the buddy, instead of making use of percpu-pages in the page_frag_free() call. Aside from eliminating the contention, it was seen to improve some microbenchmarks. [Commit 65895b67ad27 ] mm/mremap: improve…
With v4.20 out for almost the entire v5.0 rc-cycle, here are some of the more interesting performance related changes that made their way in. signal: Use a smaller struct siginfo in the kernel Reduces the memory footprint of ' struct siginfo ' most of which is just reserved. Ultimately this avoid spanning two cachelines to just one. [Commit 4ce5f9c9e754 ] sched/fair: Fix cpu_util_wake() for…
This post marks one year since I began doing these kernel performance goodies write ups, starting from v4.14 . And this week Greg released Linux v4.19, so here are some of the changes related to software optimizations, performance and scalability topics across various subsystems. epoll: loosen irq safety when possible The epoll code uses an irq-safe spinlock to protect concurrent operations to the…
Linux v4.18 has been out a two months now; making this post a bit late, but still in time before the next release. Also so much drama in the CoC to care about performance topics :P As always comes with a series of performance enhancements and optimizations across subsystems. locking: avoid pointless TEST instructions A number of places within locking primitives have been optimized to avoid…
With Linux v4.17 now released, there are some interesting performance changes that went worth looking at. As always, the term ' performance ' can be vague in that some gains in one area can negatively affect another so take everything with a grain of salt. sysvipc: introduce STAT_ANY commands There was a permission discrepancy when consulting shm ipc object metadata between /proc/sysvipc/shm…
Linux v4.16 was released a few weeks ago and continues the mitigation of meltdown and spectre bugs for x86-64, as well as for arm64 and IBM s390. While v4.16 is not the most exciting kernel version in terms of performance and scalability, the following is an unsorted and incomplete list of changes that went in which I have cherry-picked. As always, the term ' performance ' can be vague in that…
With the Meltdown and Spectre fiascos , performance isn't a very hot topic at the moment. In fact, with Linux v4.15 released, it is one of the rare times I've seen security win over performance in such a one sided way. Normally security features are tucked away under a kernel config option nobody really uses. Of course the software fixes are also backported in one way or another, so this isn't…
Last week Linus released the v4.14 kernel with some noticeable performance changes. The following is an unsorted and incomplete list of changes that went in. Note that the term ' performance ' can be vague in that some gains in one area can negatively affect another, so take everything with a grain of salt and reach your own conclusions. sysvipc: scale key management We began using relativistic…
The complexity of futexes, their non-trivial interactions and semantics, very much serve as a good candidate for applying fuzzy testing techniques to them. In general futex code is poorly understood and audited, both at a kernel implementation level and by the respective userland callers, normally trying to implement some sort of locking primitive. Unsurprisingly, bugs related to this call will…
With the need for better scaling on increasingly larger multi-core systems, we've continued to extend our CPU barriers in the kernel. Two important variants to prevent CPU reordering for lock-free shared memory synchronization are pairs of load/acquire and store/release barriers ; also known as LOCK/UNLOCK barriers. These enable threads to cooperate between each other. Multiple, yet pretty much…
This year I had the privilege of leading the Performance and Scalability micro-conference for Linux Plumbers. The goals and motivation behind organi zing this track were threefold. First present relevant work-in-progress ideas that can improve performance in core kernel subsystems, and need some face to face discussion -- as such, this requires previous debate on lkml. Similarly, learn about real…
Hash tables are popular data structures that efficiently handle dictionary operations (search and insert/delete). The Linux kernel relies on them for a number of subsystems, including major core kernel areas, such as dcache/inode lookups, workqueues, timers, the PID table, TCP/UDP and futexes. This last being used as common building blocks for implementing userspace locking primitives, pthreads…
EFI's GPT disklabels present a number of benefits to the traditional MBR scheme. For instance, not having to deal with CHS addressing, better data integrity (including a backup header as data redundancy) and 64bit LBA addressing, allowing partitions to go beyond the 2Tb limit all the way up to 9.4 Zb. These nice features don't come free, however, having to deal with older legacy systems (normally…
Back in June I submitted a talk to FOSS.IN 2012 conference in Bangalore, India. Unfortunately my talk was not included in the list of accepted proposals, in other words, it was rejected. But that's not the reason why I'm writing, or why I am most disappointed in how things were handled by FOSS.IN's organizing team. The program's call for participation stated that the list of accepted proposals…
The fdisk tool is perhaps the most recognized disk partitioner in the world, as it has historically been present in Windows and all Unix flavors, among other OSs. While this tool has proven useful for its Linux variant, it as been subject to intense patching along its 20 years of existence, and it is a product of multiple authors, coding styles and concepts. Because of this, extending fdisk , to…
Traditional x86 architecture implicitly requires TLB flushing upon context switching ( CR3 writes) so the new process-to-run's address space does not conflict with lineal to physical translations cached by previous processes. When using shadow pages for MMU virtualization, it can be quite expensive to throw away. Intel introduced Virtual Processor ID ( vpid) into its VT-x technology in order to…
The lslk(8) program has been unmaintained and deprecated for over a decade now, since 2001. I've recently rewritten the tool, now called lslocks(8) that allows an easier and up-to-date way of seeing all the current file held locks in a Linux system. This program will be shipping soon with standard system tools and available in your favorite distribution. Some important modifications include…
One of the initialization steps that KVM does when a virtual machine (VM) is started, is setting up the vCPU's memory management unit (MMU) to translate virtual (lineal) addresses into physical ones within the guest's domain. For x86, which is what will be covered here, most of the corresponding code is in <kernel>/arch/x86/kvm/mmu.c . Disclaimer: Although this document requires at least some…
CPU vendors began adding hardware virtual memory management unit (vMMU) support circa 2009, with Intel's VT-x (vmx flag) addition. Historically, the guest's physical (gpa) to host physical (hpa) addresses where translated through software, using shadow page tables. These tables are kept synchronized with the guest's page tables, and are one of the main sources of overhead in virtual machines, as…
We normally have a file's canonical/absolute path, and with that we can get just about any details from it, usually through stat(2)-family . What about when we have the inode number? I had to come up with this little ugly function to parse (luckily I also had the PID) procfs and go comparing all the files... we can do better! I'm hoping someone can tell me a more straightforward way of doing this…
If you're interested in a career in computer systems, here is an unsorted list of books you should get your hands on. Some are UNIX related, but hey, that's my area of knowledge and, in one way or another, they have all helped me grow as a computer scientist. Kernighan, Brian and Ritchie, Dennis. The C Programming Language (2nd Ed.) . Hennessy, John L. and Patterson, David A. Computer…
I was having some trouble finding my CPU's TLB page size and data entries a few days ago, and it's no mystery that Intel provides very poor specs in this specific area. I couldn't see it exported from Linux either (although it *does* list it in /proc/cpuinfo, depending on the L1/L2 cache sizes and attributes, but that's another story). To overcome this I was forced to write my own little program…