👋🏼 This is part of series on concurrency, parallelism and asynchronous programming in Ruby. It’s a deep dive, so it’s divided into several parts: Your Ruby programs are always multi-threaded: Part 1 Your Ruby programs are always multi-threaded: Part 2 Consistent, request-local state Ruby methods are colorless The Thread API Bitmasks, Ruby Threads and Interrupts, oh my! When good threads go bad…
👋🏼 This is part of series on concurrency, parallelism and asynchronous programming in Ruby. It’s a deep dive, so it’s divided into several parts: Your Ruby programs are always multi-threaded: Part 1 Your Ruby programs are always multi-threaded: Part 2 Consistent, request-local state Ruby methods are colorless The Thread API Bitmasks, Ruby Threads and Interrupts, oh my! When good threads go bad…
Your code using the /o modifier Source: wikipedia Hi there! Do you like Regex? Do you like performance? Do you like creating confounding bugs for yourself rooted in the mechanics of the Ruby VM itself? If you said yes to all of the above, have I got a feature for you! But first, let’s start with a story. The cliffs of insanity I was recently reviewing some code, and part of the functionality was…
In part 5 , we finally got our new instruction defined and outputting as part of our bytecode. if you didn’t run it yourself, you just had to trust me that it really did run. But, I just dropped most of the implementation code in without explaining it. Let’s start off by walking through the basic version, then start planning for the true optimization. The progress so far Here’s our sample Ruby…
In Peephole optimizations: adding opt_respond_to to the Ruby VM, part 4 , we dug deep. We found the connection between prism compilation and the specialization we need for our new bytecode, called “peephole optimization”. We learned how to debug and step through C code in the Ruby runtime, and we added some logic for matching the “pattern” of the instruction we want to change. Now that we know…
In The Ruby Syntax Holy Grail: adding opt_respond_to to the Ruby VM, part 3 , I found what I referred to as the “Holy Grail” of Ruby syntax. I’m way overstating it, but it’s a readable, sequential way of viewing how a large portion of the Ruby syntax is compiled. Here’s a snippet of it as a reminder: // prism_compile.c static void pm_compile_node (rb_iseq_t * iseq,…
In Finding the compiler: adding opt_respond_to to the Ruby VM, part 2 , I found the entrypoint into the compiler! It takes the root of our abstract syntax tree - pm->node - and produces a rb_iseq_t . rb_iseq_t is an “InstructionSequence”, which represents our virtual machine bytecode. Here’s the code where we left off: // ruby.c static VALUE process_options ( int argc, char **…
In Adding opt_respond_to to the Ruby VM: part 1 , inspired by recent JSON gem optimizations, I setup my goal: I want to add a new bytecode instruction to the Ruby VM which optimizes respond_to? calls. I took this Ruby code: if $stdout . respond_to?( :write ) puts 'Did you know you can write to $stdout?' end And identified what bytecode instructions matter most (I think): == disasm:…
@byroot has been posting a series on optimizations he and others have made to the json gem, and it’s been 🔥🔥🔥. Enjoyable and informative, I highly recommend reading what he’s posted so far. In his second post , he mentions the possibility of improving performance by adding an additional method cache. This would involve compiling respond_to? calls in a special way: It actually…
Over on BlueSky, Gregory Brown suggested a #20daygemchallenge . Post gems you’ve either used time and time again, or have inspired you in some way, in no particular order. Mohit Sindhwani suggested writing about them at the end, which sounded like a great idea! I’m breaking it into two parts. Here’s my breakdown of my first 10 gems posted: First is HTTParty : https://github.com/jnunemaker/httparty…
I was honored to present a talk on Ruby concurrency at RubyConf 2024. It represents a high-level distillation of much of my writing and research over the past year. The conference itself was great, and presenting was such a fun experience. Here is the talk, titled “In-Depth Ruby Concurrency: Navigating the Ruby Concurrency Landscape”: If you want the slides, I’ve published them as a short YouTube…
There is a recent language comparison repo which has been getting shared a lot. In it, CRuby was the third slowest option, only beating out R and Python. The repo author, @BenjDicken , created a fun visualization of each language’s performance. Here’s one of the visualizations, which shows Ruby as the third slowest language benchmarked: The code for this visualization is from…
I recently posted my docker setup for hacking on CRuby , which showed how I test Linux features when working on CRuby. But most of the time, I just build CRuby directly on MacOS. The Building Ruby guide from ruby-lang.org is the most up-to-date guide on doing this, but I like to spell it out exactly in order of how I do it day-to-day. So this is for me more than anything, but you may find it…
There is a central macro in CRuby, RUBY_VM_CHECK_INTS , which is a very hot path for the Ruby runtime. It’s an important part of how threads are managed, and it’s called constantly . I was curious just how often it was called, and it turned out CRuby comes with some handy debugging functionality for just this scenario. Inside of debug_counter.h , I changed #define USE_DEBUG_COUNTER 0…
I run on MacOS, but I often want to test Linux behaviors when working on the CRuby implementation. Here’s the Dockerfile I use: FROM ubuntu:24.04 # Preventing dialog prompts when installing packages ENV DEBIAN_FRONTEND=noninteractive # Update and install basic build dependencies and Rust RUN apt-get update && apt-get install -y \ git \ curl \ build-essential \ autoconf \ libreadline-dev \…
Looking to impress your Ruby friends by calculating the largest known prime, 2 ** 136_279_841-1 ? On Ruby 3.4.0-preview2 and earlier, 2 ** 136_279_841-1 logs a warning and returns Infinity 😔: 2 ** 136_279_841 - 1 # warning: in a**b, b may be too big # => Infinity Thanks to @mametter , Ruby 3.4 will handle this calculation just fine! See Do not round a**b to infinity . Knowing this, you excitedly…
👋🏼 This is part of series on concurrency, parallelism and asynchronous programming in Ruby. It’s a deep dive, so it’s divided into 12 main parts: Your Ruby programs are always multi-threaded: Part 1 Your Ruby programs are always multi-threaded: Part 2 Consistent, request-local state Ruby methods are colorless The Thread API Bitmasks, Ruby Threads and Interrupts, oh my! When good threads go bad…
👋🏼 This is part of series on concurrency, parallelism and asynchronous programming in Ruby. It’s a deep dive, so it’s divided into 12 main parts: Your Ruby programs are always multi-threaded: Part 1 Your Ruby programs are always multi-threaded: Part 2 Consistent, request-local state Ruby methods are colorless The Thread API Bitmasks, Ruby Threads and Interrupts, oh my! (Concurrent, colorless…
👋🏼 This is a series on concurrency, parallelism and asynchronous programming in Ruby. It’s a deep dive, so it’s divided into 12 main parts: Your Ruby programs are always multi-threaded: Part 1 Your Ruby programs are always multi-threaded: Part 2 Consistent, request-local state Ruby methods are colorless The Thread API Bitmasks, Ruby Threads and Interrupts, oh my! (Concurrent, colorless Ruby)…
👋🏼 This is a series on concurrency, parallelism and asynchronous programming in Ruby. It’s a deep dive, so it’s divided into 12 main parts: Your Ruby programs are always multi-threaded: Part 1 Your Ruby programs are always multi-threaded: Part 2 Consistent, request-local state Ruby methods are colorless The Thread API Bitmasks, Ruby Threads and Interrupts, oh my! (Concurrent, colorless Ruby)…
👋🏼 This is a series on concurrency, parallelism and asynchronous programming in Ruby. It’s a deep dive, so it’s divided into 12 main parts: Your Ruby programs are always multi-threaded: Part 1 Your Ruby programs are always multi-threaded: Part 2 Consistent, request-local state Ruby methods are colorless The Thread API Bitmasks, Ruby Threads and Interrupts, oh my! When good threads go bad Thread…
My previous post, PgBouncer is important, useful, and fraught with peril , was a deep dive into Postgres feature compatibility with different modes of PgBouncer. I’m happy with how it came out and it was well received. I think it is the most comprehensive guide to Postgres/PgBouncer compatibility that exists. But it might be a daunting read for some (9k words 😅), and the name doesn’t clearly…
Updated 2024-09-17 to reflect updated PgBouncer support for protocol-level prepared statements 🐘 To start, I want to say that I’m appreciative that PgBouncer exists and the work its open source maintainers put into it. I also love working with PostgreSQL, and I’m thankful for the incredible amount of work and improvements that go into it as well. I also think community and industry enthusiasm…
A few months back I was working on a Javascript frontend for a large dataset using Tanstack Table . The relevant constraints were: Up to 50k rows of content Grouped by up to 3 columns Using react and virtualized rendering, showing 50k rows was performing well. But when the Tanstack Table grouping feature was enabled, I was seeing slowdowns on a few thousand rows, and huge slowdowns on 50k rows. It…
Twitter. Oh how I could stroll that infinite corridor of information. Ever since having kids my personal time has been squeezed into a tiny ball. Still, I’d finish putting the kids to bed, open my phone, and where would I end up? On Twitter for 45 minutes. An hour. Hour 30. I’d sit there feeding whatever meager morsels of free time I had left to that tiny blue bird, it gobbling them up like some…