Managing a Project's Tool Dependencies with Nix (and direnv)
For the past six years, I have lived in a world where:
Cuddly, Octo-Palm Tree
Blog on software engineering, mostly.
Latest posts
Notes on Optimizing Clojure Code: Arrays
Arrays are the most primitive collections on the JVM, and therefore the ones with the fastest operations. This doesn't mean they are right for every situation, but when they are, they can greatly enhance performance.Working with arrays in Clojure is surprisingly easy once you get used to it, but there are a few gotchas worth pointing out.
Notes on Optimizing Clojure Code: Type Hints
Last week I wrote about how reflection works in Clojure and how to avoid it. The primary way to do that is through using appropriate type hints, but I didn't go very deep into how and what those are.There's a surprising amount to know about them, and I don't know of a good unified resource on the topic, so here's my take on it.
Notes on Optimizing Clojure Code: Reflection
Clojure is a dynamic language. That's great for (some notion of) expressivity, but sometimes it can get in the way of performance.To get the best performance out of JVM Clojure, we have to understand how Clojure's brand of dynamic typing meshes with the JVM's. This is what this post is about.
Notes on Optimizing Clojure Code: Data Structures
If you want to write efficient code, it's important to understand the performance characteristics of the building blocks you're using to write that code.clojure.core: abstractions and performance
Notes on Optimizing Clojure Code: Numerics
Pretty much every program will, at some point, need to deal with numbers. Numbers are deceptively complicated in programming: on the one hand, numerical operations are often one of the first things introduced in programming textbooks, on the vague hope that this will make it easier for beginners because "they already know maths"; on the other hand, programming with floating-point numbers is…
Notes on Optimizing Clojure Code: Example
Last week, I presented three different ways to measure what's taking up time in a Clojure code base. This week, I'll walk through how to apply those three types of measurements on a concrete use-case: optimizing my AoC day 12 solution.Starting point
Notes on Optimizing Clojure Code: Measurements
Picking up from last week, we'll delve into the three types of measurements I talked about. Let's jump right in; if you want more context, you can read the previous post.Performance logging
Notes on Optimizing Clojure Code: Overview
Over the month of December, I've spent a lot of time trying to optimize my solutions to Advent of Code, with reasonable success. I thought I'd collect my learnings in a set of notes for my future self, and share that with you. Not all of this is Clojure-specific, but the details mostly are.Why optimize code?
AoC 24, part 2: A proper solution
Last week I explained how I got the right answer in a very unsatisfying way. In this post, I'll show how I eventually got to a proper solution.I'm going to jump right in, so if you're not familiar with the problem or where we left off, this may not make much sense.
AoC 24, part 1: Did I cheat?
I must admit, when I finished reading the problem description, I had no idea how to proceed. In fact, while I did get my two stars on December 24, I would not find a solution to the problem until the evening of December 27. Intrigued? Read on.Spoilers ahead. This post is about my journey through the day 24 problem of this year's Advent of Code. In this post, I'll describe how I approached the…
Graph search: Dijkstra, A*
Over the past few weeks I've been taking part in Advent of Code, which has been a lot of fun. Graph search has come up a couple times, and it took me an embarrassingly long time to remember the basics, so I'm writing this for my future self. Both for future reference and in the hope that writing it down will help me remember it all better.The problem
Tools You Should Know About: nix-shell
Recently, I've discovered how to leverage Nix to reap a lot of its benefits with a very minimal investment; specifically, one that fits in one blog post.I've been aware of the Nix toolset for over a decade now, but until recently it's always looked like it required a pretty big investment. Most of the documentation I've come across explains what the benefits are and how the tooling works in order…
Tools You Should Know About: direnv
In a nutshellFrom the homepage:
"The Genuine Sieve of Eratosthenes" in Clojure
After my previous two posts on primes (first, second), a few people have reached out through various channel to suggest alternative ways to compute them. Eventually I ended up with a Haskell paper in my hands, and I'm apparently starting to make a habit out of translating those to Clojure, so I thought I'd give this one a try too.There is some overlap with my previous posts, but I'm going to write…
Primes in Clojure part 2: Interop
A couple weeks ago I published a blog post on computing prime numbers with Clojure. I got some good feedback on it, including pointers to other implementation techniques I'd overlooked.In this post, I want to explore some of those, notably two based on the Java standard library.
Circular Programming in Clojure
A couple years ago, I decided to learn Haskell. This led me to develop an interest for lambda calculus, interpreters, and laziness. (And monads, in general as well as in Clojure, but that is less relevant for today's topic.) While researching those topics, I came across a paper meshing all those topics together by explaining a nice technique to implement embedded functional languages in Haskell.…
The Clojure Promise
I've recently rediscovered the Clojure promise construct. I don't use it very often, but I find it very neat in its simplicity. In this post, I'll describe what Clojure promises are (with a word on how they differ from what other languages might call "promises"), talk about related Clojure concepts (delay, future), and talk a bit about use-cases.What's a promise?
Computing prime numbers with Clojure
This post is not about prime numbers, really, but they offer a nice, compact problem space to explore a number of Clojure language features.We'll start with defining what prime numbers are, using a naïve Clojure implementation. Then, we'll discuss the performance characteristics of that implementation, and explore various ways in which that can be improved.
Bash functions are better than I thought
I've been using Bash functions quite liberally over the past couple years, but last week I discovered that they can be much better than I thought. In this post, I'll explain how I thought they worked, why I was wrong, and how that makes them better.Function syntax
Forwarding emails with SES and Terraform
A few weeks ago, I had a bit of a scramble when I realized, on Saturday, that the corporate emails I'm responsible for would suddenly stop working on the following Monday, two days later.A few weeks prior, I had decided to switch DNS provider. I had been unhappy with the service I was getting for about a year, but DNS is a bit of a dangerous thing to tinker with, so I'd put it off. It also has a…
A short overview of DNS
I've recently changed DNS provider for this blog, and that forced me to look into how DNS works a bit closer. I did manage a DNS server for a couple years circa 2006, but I have to say I'd forgotten most of it. In case I forget it again, I'm recording my notes here.DNS zone
Tools You Should Know About: Terraform
In a nutshellFrom the homepage:
Monads for Clojure programmers
This post is strictly about the programming technique. There is no philosophy or category theory. If you're interested in a more in-depth explanation of all the concepts related to monad, I have a longer series on the subject.My goal with this post is not to convince you to start using monads everywhere; I only aim to make the technique easy to understand, using Clojure notation to explain it.
Tools You Should Know About: jq
In a nutshellFrom the official page
The elements of git
This is not a tutorial. If you're looking for a quick, easy, "how to use git" kind of post, look elsewhere.The goal of this post is to give you just enough understanding of the git internals that you can build up a correct intuition of what various git commands actually do under the hood.
Tools You Should Know About: tmux
In a nutshellFrom the official wiki:
Monads, part six: But, really, what is a monad?
In the first entry in this series, I argued that "what is a monad?" is not a useful question for the working programmer. In the rest of the series so far, I have explained how to recognize situations in which a monad could be useful, how to apply monads, and how to create bespoke ones.But I still have not really defined what a monad is, and, while I stand by the argument that the answer is not…
Data-first API design
I used to be a Clojure programmer. I've been using it as my main programming language from the first time I took a serious look at it in early 2012 up until the end of 2019, when I decided to check out the competition.I joined a company with a strong static typing culture in what was meant to be a full-time Haskell position.1 For the past three years, I've tried really hard to understand the…
Cheap interpreter, part 10: fastest one yet, then a hundred times faster
Last week we started looking at dynamic code generation for a host (dynamic) language, and how that can yield faster code than any interpreting we'd be doing ourselves in said language. But the code we generated was still using the exact same logic.In this final post in the series, we'll look into how we can improve on that, yielding much faster code than last week (and thus leaving our Haskell…
Cheap interpreter, part 9: even faster register machines
Last week I showed a few ways in which to improve the performance of a Haskell intepreter for a register machine. In this post, we start with the exact same bytecode (same register language, same compiler) and show how to use a much slower language (Clojure) to end up with a much faster interpreter.This series is based on Neil Mitchell's talk "Cheaply writing a fast interpeter". The talk compares…
Cheap interpreter, part 8: faster register machines
Last week I briefly presented the basics of register machines: what they are and how to write a simple compiler and interpreter for an example instruction set. In this post, I'll show a few tricks that can be used to make this example quite a bit faster.This series is based on Neil Mitchell's talk "Cheaply writing a fast interpeter". The talk compares a number of approaches to writing an…
Cheap interpreter, part 7: register machines
[EDIT 2021-08-15]: The original version of RegEmitBefore did not work for nested loops. Huge thanks to Reddit user JeffJeffJeffersonson for spotting and reporting the issue!Last week I presented a few ways to improve the performance of the very simple stack machine we had previously defined. In this post, we move further into the list of techniques presentetd in Neil's talk and take a look at…
Cheap interpreter, part 6: faster stack machines
Last week was all about me screwing up my benchmarks. There have been some very interesting discussions on reddit, including suggestions on how to do better benchmarking in Haskell, and how to better implement the approaches presented in the previous parts of this series. I highly recommend reading them.Two weeks ago, before I so rudely interrupted myself with that diversion on benchmarking, I…
Cheap interpreter, part 5: mea culpa
Last week, I presented the basics of a stack machine: what it is, what a basic stack language looks like, how to write a simple interpreter for it, and how to compile a higher-level language down to it.This week, I had planned to talk about how to make such a stack machine fast, as last week's version was slow enough to raise questions about including the topic in a series about speed at all. This…
Cheap interpreter, part 4: stack machines
Last week, I showed how to use host-language first-class functions as a way to improve the efficiency of an interpreter. The code changes were pretty small and local, which is good, but the performance benefits were not huge. In this post, we're going to explore a completely different approach: we're first going to design a stack machine, then write an interpreter for the stack machine language,…
Cheap interpreter, part 3: host-language functions
Last week I presented a simple strategy for building an interpreter, written in two different styles. This week we're going to start on the main purpose of this series: discussing optimization techniques, with a particular focus on the tradeoff between their added complexity and their added performance.This series is based on Neil Mitchell's talk "Cheaply writing a fast interpeter". The talk…
Cheap interpreter, part 2: tree-walking
Last week I described the general structure of an interpreter, and gave a cursory introduction to parsing. From this point on, I will simply assume I am starting with a parse tree, and ignore how it was produced.This series is based on Neil Mitchell's talk "Cheaply writing a fast interpeter". The talk compares a number of approaches to writing an interpreter and tries to find a good balance…
Cheap interpreter, part 1: overview
A few months ago, my former boss Neil Mitchell gave a talk entitled "Cheaply Writing a Fast Interpreter". It's a very good talk, and I encourage you to watch it. The basic premise of the talk is that it is given to people who already know many ways to write an interpreter, and the talk itself is presenting the results of a study Neil and his team did to compare a number of different techniques on…
Church-Turing equivalence
In the first post of this series, I presented the notion of a Turing machine, which I argued is important because we can build physical representations of them. In the second post, I presented lambda calculus, which I argued is important because it gives us the fundamental tools to manage complexity (i.e. abstraction). In this post, I explain why I think knowing about both of them is important to…
What is functional programming?
Last week I wrote a concise, informal introduction to Turing machines and the associated programming model, imperative programming. In this post, I present a similarly informal introduction to lambda calculus and the associated "functional programming" model.Lambda calculus
What is imperative programming?
In this post, I explain why you should care about Turing machines and what they are. In the next parts of this series, I will contrast Turing machines with lambda calculus and expand on why understanding both models is useful for everyday programming.Functional v. object-oriented
Asking questions as an interviewee
Many candidates approach interviews like they approach oral exams at school: a stressful, one-sided evaluation of their knowledge, their skills, and perhaps even their worth as a human being. This is misguided and detrimental: on the one hand, this tends to generate a lot more stress than is warranted, and on the other hand, it prevents the candidate from evaluating the interviewer.In fact, an…
Monads, part five: What if I'm not writing Haskell?
In the past two posts of this series, I have shown what I believe to be a simple, effective technique for creating monads in Haskell. At this point one may be wondering how that translates to other languages, if indeed it applies at all.In this post, I will take a look at what exactly we need from a language in order to apply this "structural monad" approach, with a detailed example in Clojure.
Monads, part four: examples
This post is part of a series on monads. In the previous post, I presented a technique for implementing new, bespoke monads and applied that technique to well-known, standard ones. In this post, I am going to show two examples of bespoke monads I have recently crafted using this technique, comparing code with and without a monad for the same use-cases and walking through the process of defining…
Monads, part three: monads made easy
In the previous three parts of this series (zero, one, two), I have tried to explain what makes something "a monad" and why and when you could want to use one. In this post, I will show you a simple process to create a brand new monad tailor-made for a particular use-case, and walk through a couple examples.Factoring monads
Monads, part two: Okay, I guess, but what is it <i>for</i>?
Monads are abstract machinesHopefully, after reading parts zero and one of this series, you understand what the monad abstraction is and how bind works. What may not be so clear yet is why anyone would ever want to work with a monad. In other words, what can you do with a monad? More importantly, what can you do with a monad that you cannot do more easily without one? This is what I'm going to try…
Monads, part one: <strike>What's a monad?</strike>
If you've read part zero, you know that the question "What is a monad?" is not a very good one, because "monad" is not a thing, it's a set of properties. A better question would be "what are the properties of a thing that make it a monad?". This is the question I will answer in this post.It will probably be a fairly unsatisfying answer. By the time you reach the end of this post, you may be left…
Monads, part zero: Flavours of abstractions
It's come to my attention that the internet is lacking in monad tutorials, so I've decided to write my own. In this first, preliminary part, we'll look at different flavours of abstractions in order to explain what flavour a monad is.I'm using "flavour" here for a bunch of things with some common properties because "class", "type", "kind" and "category" all have very specific meanings in related…
How to break a computer
For as long as I can remember, people have come to me with computer problems. I'm not quite sure why they do it at first — probably my awkward social skills tipping them off — but it's quite clear why they keep coming: I'm usually able to help. A couple of days ago, I was reminded of why I usually know how to help: I do stupidly reckless things that land me in trouble, then I just don't give up…