Recent content on Code - a programming blog
When I was younger, my favorite painting was The Building of FTL1 by John Harris: 
 
 Looking back, I think what captivated me wasn’t the painting itself, but the feelings of awe and wonder that it evoked in me. In philosophy this phenomenon is called the sublime. 
 The sublime occurs when the imagination reaches its limits. When we can’t fully picture what we’re…
A few years ago when DevOps and microservices were all the rage, it seemed like the software industry had reached a consensus on what good practice looked like. No book captured this better than Accelerate , a favorite of mine that guided my own work and thousands of other professionals. Based on survey data, the book lists 24 “capabilities” that every high performing engineering org…
Why do we build monolithic applications? And when do we decide they’ve become too large and start breaking them up? 
 Ronald Coase asked a similar question about firms. If markets are so efficient, why do firms exist at all? His answer was transaction costs. Coordinating work through the market incurs costs—discovering suppliers, negotiating contracts, enforcing agreements. Firms exist…
Prompting coding agents has quickly become normal. It feels so natural: just tell the agent what you want! The problem is that the prompt is not the unit of work. 
 You see, a prompt isn’t just instructions. It’s a compressed representation of everything you know about the work: the goal, the state, and what should happen next. The problem is that most of that knowledge never…
It’s not every day that a business strategy essay reshapes an industry, but Jack Dorsey and Roelof Botha’s From Hierarchy to Intelligence is having an unusual run. Published in March, it laid out a vision of AI-era organizations — companies built as intelligences rather than hierarchies, with thin layers of edge workers replacing traditional management. Two months later, Coinbase CEO…
FOR IMMEDIATE RELEASE - Advanced AI Tier Now Generally Available 
 San Francisco, CA - Tianthropic is thrilled to announce the general availability of our Advanced AI tier for cognitively demanding tasks. Following an extended research preview, AAI is now available to enterprise customers on annual commitments. 
 In third-party evaluations conducted by the Frontier Capabilities Consortium,…

 To become angry belongs to everyone, and is an easy thing as is also giving and spending money; but to whom [one ought to do so], how much, when, for the sake of what and how - these no longer belong to everyone nor are easy. 

 — Aristotle, Nichomachean Ethics 2.9 

 
 Earlier this year an employee at Meta’s “Superintelligence” division watched in…
I’ve been using Claude code daily for a while now. I give it a task, it writes code, runs commands, edits files. It’s genuinely productive — I ship things faster than I used to. 
 But there’s a problem. These agents run commands on your machine. Real commands, with real consequences. rm , git push , curl to production endpoints. To mitigate that, the agent proposes an action,…
In his book The Nature of Technology , W. Brian Arthur describes technology as combinations of other technologies. This description poses a couple of questions for computing. 
 First, take a software program like Git. To figure out what technologies Git is made of, perhaps all we have to do is inspect its dependencies? Surprisingly, Git has few real dependencies.¹ 
 Second, note how…
Specifically the Brother HL-L2350DW. I bought it to print coloring pages and puzzles for my kids, and it was great for that. Cheap and quick. 
 But then I started printing out technical papers, reading them, marking them up, putting them down, and referring to them later. I retained more of the material, drew more cross-inferences with other papers, but more than that, I enjoyed the reading…
In the 1940s Bell Aircraft used an experimental approach to build the first commercial helicopter:¹ 
 
 At Gardenville, we built things, tested them, modified them until they worked, and then made the drawings. 

 — The Bell Notes, Arthur M. Young, 

 
 The project manager contrasted this with how they built airplanes: 
 
 The main engineering group made…
Twenty five years ago the most popular solution for web apps was known as “LAMP” (Linux, Apache, MySQL, PHP). A typical web app today, deployed with Kubernetes might comprise of 5x as many technologies. That we don’t use such acronyms anymore tells us something about how software engineering has changed. 
 This has mostly been a journey from implicit to explicit state. For…
We were asked to build an app in Go,
Fortunately we were experienced programmers,
Unfortunately our experience was with Java,
Fortunately Go is simpler than Java,
Unfortunately we wrote thousands of lines of code,
Fortunately adding dependencies was easy,
Unfortunately we accumulated many CVEs,
Fortunately our CICD tracks vulnerabilities,
Unfortunately it kept…
Chapter 15 from The Art of Prolog contains this dictionary implementation: 
 

 
 
 
lookup(Key, dict(Key,X,Left,Right), Value) :-
 !,X=Value.
lookup(Key, dict(Keyl,X,Left,Right), Value) :-
 Key < Keyl, lookup(Key,Left,Value).
lookup(Key, dict(Keyl,X,Left,Right), Value) :-
 Key > Keyl, lookup(Key,Right,Value). 
 
 

Of all the AWS services, IAM seems to be the least well-understood. There is something about its data types, logical rules and runtime conditions that make for an arcane, abstract service that most developers treat like a black box. 
 And that’s unfortunate because IAM secures organizations’ cloud resources, and can be the difference between having a data moat and a data breach.…
Search engine startup Perplexity have introduced a new type of LLM model, which they call an “Online LLM” . 
 Online LLMs combine the capabilities of an LLM with factual data scraped from the web, that a traditional search engine would provide. The combination is powerful - you get the retrieval power of search combined with the natural language skills of an LLM. 
 For example,…
When things are broken, it’s easy to know what to do. Less so when things seem to be ticking along. 
 One useful source I’ve found is the 24 capabilities described in Accelerate . Benchmark your org against them, see what’s deficient and get to work. 
 Often the actual issue is a few degrees of separation away from the capability. For example capability #14 is…
A few years ago I was working on a team that owned a web app for our enterprise customers. It had been forked from our original app specifically to meet the needs of larger clients. Despite both apps’ shared heritage, they were not similarly reliable. My team’s app build almost never failed, yet the original app build was red a third of the time. Since a red build prevented new…
Recently at work we’ve instituted a change to require code review by default. The merge request owner can disable the review requirement in GitLab but the intention is to always review. Some co-workers think this change is great, others worry about the additional overhead and loss of velocity. 
 I used to think stuff like this matters - I don’t anymore. Moreover I don’t think…
For the past two years I’ve been experimenting with note-taking techniques, and have settled on one that works quite well. I call it “Right-Page-Only” note-taking, where I only write on the right page of the notebook. Let me tell you why. 
 I want to take quick and neat notes and therefore it’s essential I have a consistent and comfortable writing space. This means that…
I wrote a shell script called ch which lets you use ChatGPT from the command line. I’ve been using it for the past few months on my laptop and mobile phone. 
 This video has more background on the rationale for ch and some examples. 
 
 

Recruiters aside, does anybody actually enjoy using LinkedIn? Who reads the stream of corporate updates (“Big Corp were awarded 2nd place for ‘best office’ how wonderful”). I suspect most people who have a LinkedIn account are there because they have to be. 
 A few months ago I quit LinkedIn. When I’m asked for my LinkedIn page, I send them to my version . I…
Hugo, the static site generation engine (which generates this site) has a useful function called safeHTML which marks an HTML string as “safe”, preventing it from being escaped and displayed literally. 
 The other day I needed this same behavior but for a variable declared in a post’s frontmatter. “I’ll use that handy safeHTML function” I thought. Wrong!…
Roughly a year ago I wrote about my attempts to tackle Go’s error handling problem using generics. I wasn’t very successful: 
 
 This code is almost as long as the original and frankly, hurts my eyes. 
 
 Since then I’ve continued experimenting with the idea and think I have something worth sharing. But first, let me recap the problem with an example. 
…
At work we’re in the midst of a big project to move our data analysis out of relational databases and into a datalake. We’re an AWS customer, so instead of running queries in RDS or Redshift, analysts will be using Athena. 
 Whilst Athena provides an analyst-friendly SQL interface for running queries, it has a “pay-as-you-go” model which charges for the amount of data…
N.B. The main example in this post has been updated to use POSIX shell instead of bash. 
 Last week at work I had to whip up a script to process several thousand ids for product analysis on a new feature we’re launching. The processing included making HTTP requests to a service, and I was on a deadline, so the script had to be concurrent. Here’s what I came up with:
Apache Spark is a chatty application; it logs early and often . Sure, some log messages increase Spark’s observability, but others seem questionable. Like logging query plans, which can add over 1MB of text to a single log message.¹ 
 Writing enormous log messages causes issues with our logging pipeline, so at $work we set the Spark config option spark.sql.maxPlanStringLength to 0. This…
The release of Delta Lake 2.0.0 came with an exciting new feature called Change data feed which tracks row-level changes to a delta table. This ability unlocks new use cases: 
 
 Batch selection of new data; previously new rows could be streamed from a delta table, but streams are opaque. Batch mode enables operations on a whole set of changes. 
 Streaming Change Data Capture (CDC)…
Most programming language grammars are not context-free. That means the parser has to maintain state in order to process a snippet of the language. 
 Often when discussing context somebody will point out the parsing strings, parentheses and so on also require a kind of state in order to match their terminators. That is not the same thing. The question here is given valid input, can a parser…
It sounds like such a simple question - but virtual memory makes measuring real memory use complicated. Popular tools like ps and top report virtual, resident and shared memory consumption, but these are unsatisfactory: 
 
 Shared memory is not apportioned per process using it so it is overstated. 
 Resident memory includes shared memory, so it overstates as well. 
 Virtual memory…
One of the most complained about aspects of programming in Go is error handling. However - Go is getting generics ! Generics have already been merged to master and are slated for inclusion in the 1.18 release. So can generics raise¹ Go’s error handling out of the doldrums? 
 The problem 
 Here’s some typical error-handling code: 
 str, err := foobar()
if err != nil…
A few weeks ago I was writing a macro loader for my JSON Processor , and I ran into an odd case where code I entered in the terminal was splitting differently to code I loaded from a file. It turns out that Bash’s word splitting behaves differently for arguments than it does for variables. 
 When processing args Bash will not split quoted words that contain a delimiter: 
 printf…
I usually program with vim inside a tmux session. I’m a big fan of tmux , but the copy-and-paste feature is decidedly clunky. And my version of vim is compiled without clipboard support . What to do? I could install a vim with clipboard support, but surely this is just a small matter of programming. 
 Copy text into the clipboard 
 The clipboard is controlled by the windowing system;…
The POSIX Shell specification defines a minimalist shell language with few features compared to stalwarts like Bash. Yet POSIX shells are everywhere: Dash is the default on Ubuntu, and many Docker containers only have busybox ash . Minimalist shells use less resources and are faster than featureful shells like Bash and Zsh ¹. They are more likely to be secure too, as the reduced feature sets are…
Truthiness is the behavior of interpreting non-boolean values as true or false in a conditional context. E.g. in this loop: while 1 { ... } , the 1 is interpreted as true . Dynamic programming languages provide this behavior as a convenience to save the programmer from typing more than they need to. However, this week I wasted a bunch of time tracking down a bug that had a several root causes:
At any given time I have several projects on the go, and each one gets its own tmux session. This makes it easy to keep my work organized and I can quickly switch to another project without having to close any programs. When I’m ready to resume work on a project I attach to its tmux session and vim and any other terminals or programs I’ve opened are exactly as I left them.
As of October 1st, the Linux kernel source¹ is comprised of over 22 million lines of code (excludes comments, blank lines)². In the past 15 years that figure has increased an average of over 10% per year. At this rate, it will reach 50 million lines of code by 2030. If you studied 2,000 lines of kernel source code per day, you’d never catch up. So is the kernel growing out of all proportion?
The most common way we introduce the stack data structure is by saying it’s like a spring-loaded plate dispenser, the kind found in a cafeteria: 
 
 Indeed when I looked up stacks in an old computer architecture textbook¹ of mine I found: 
 
 stacks are frequently described by the way plates are stored and used in a cafeteria. New plates are added to the top of the stack, or…
Code review is one of those hygienic things most developers believe they ought to do, like kitchen workers washing their hands, but don’t always do. Yet tech articles tend to be unanimously in favor of code review, extolling its benefits (“catch bugs”, “maintain code consistency”, “transfer knowledge”). So if restaurants can require workers to wash their…
Unit testing is great way to build confidence that your code works. Usually you’d write unit tests for libraries; but if you have a particularly long or complicated shell script, it might be worth converting it to a modulino just so you can write tests for it. And if you are writing a library in shell code, you need all the help you can get anyway. 
 So let’s assume you have some…
The B 5000 mainframe computer was the first in a series of special machines released by the Burroughs Corporation . A stack machine with relative memory addressing, tagged architecture and a single-pass Algol compiler, the B 5000 was designed to make software development easier. I first heard about it watching Alan Kay’s Vannevar Bush lecture . 
 As I was researching the machine, I found…
A common misconception about Bash is that function names must follow the same rules that variables do. The Bash manual even suggests this: 
 
 A word consisting solely of letters, numbers, and underscores, and beginning with a letter or underscore. Names are used as shell variable and function names. Also referred to as an identifier. 
 
 In fact Bash function names can be almost…
Unix man pages are a mixed bag, but they’re faster than Google if you know where to look. That’s where the apropos program comes in. E.G. to search for man pages for errno: 
 $ apropos errno
clnt_perrno (3) - library routines for remote procedure calls
clnt_sperrno (3) - library routines for remote procedure calls
errno (3) - number of last error
h_errno (3) - get…
The Unix epoch time is the number of elapsed seconds since January 1st, 1970. Epoch times are handy because they’re not subject to timezone offsets, and they’re numbers, unlike ISO 8601 timestamps for example (“1970-01-01T00:00:00Z”). That makes them memory efficient, and more importantly, easy to do math with. 
 Here are three ways to get the epoch time in Bash. 
…
A modulino is a file which behaves like a library when it is imported, and like a script when executed. I first read about them in Mastering Perl , but you can create them in other languages too. Here’s how to do it in Bash. 
 Let’s say you have a simple script: 
 #!/bin/bash

echo "Hello, World!"
 
 A common refactor with scripts is to encapsulate all the code…
Perl uses a simple form of garbage collection (GC) called reference counting . Every variable created by a Perl program has a refcnt associated with it. If the program creates a reference to the variable, Perl increments its refcnt . Whenever Perl exits a block it reclaims any variables that belong to the block scope. If any are references, their referenced values’ refcnt are either…
Perl’s object system is not one of its most admired qualities. Included in the 1993 Perl 5.0 release, objects were a bolt-on. A big improvement at the time, in today’s context the Perl 5 object system requires too much boilerplate and is under-powered compared to other language offerings (no private state, no type checking, no traits, no multimethods). Perl programmers have been trying…
One of the convenient things about Catalyst is it lets you register controller methods as actions, so you don’t need a separate routing table of URIs to controller methods. A consequence of this though, is that when the web app gets large enough it can be tricky to keep track of all the different paths the app is responding to. 
 Catalyst’s chained dispatch promotes code reuse, but…
Recently I’ve been researching how the Internet is organized, and working with whois data. I’ve been creating simple tools that process IP addresses without any help from CPAN. At work we tend to use Net::IP::XS for these tasks, but sometimes it’s fun to figure out how things work under the hood. 
 Converting to decimal 
 I’m sure you’re familiar with the IPv4…
Last week I was analyzing server log data at work, and my boss asked me to plot it on a scatter chart. “No problem!” I thought, firing up Google Sheets. But Sheets slowed to a crawl when I uploaded 250,000 records to plot. Using CPAN I found something better: Chart::Plot . It has a simple interface and is really fast. Here’s how you use it. 
 A scatter plot script 
…