We argue that the sum total of everything LLMs do is a form of mutation or fuzzing, and that exactly one hard benchmark will suffice: number of tokens expended to converge to the mathematically correct output. We certainly don't have heated arguments about tar , ls , or any other computer program that takes some input and produces some output. The difference is that the program in question…
Humans have divisive tendencies, whether it's one human against another, one tribe against another, one race against another, one religion against another, or one country against another. When we're not fighting one another, we theorize about the world around us, implicitly placing us at its center. Throughout human history, we have consistently demonstrated anthropocentric tendencies. Historical…
Starting from early childhood, I've constantly been made to feel inadequate. Other children in my school would neglect me, and I would often try to seek attention through eccentric acts, as any child would — my personality today is a product of being treated as an exotic animal in my formative years, and I continue to feel like an exotic animal well into my adulthood. I had almost no friends in…
I arrived in London fifteen years ago from a very poor farming village in Eritrea on a refugee status. Growing up, we didn't have access to a regular supply of clean drinking water, and went hungry on many occasions. I didn't even get the chance to go to high school, and indeed, very few Eritrean families can afford to send their kids abroad for a modest university degree. Virtually everyone in…
People who have good outcomes in life have a tendency to explain their choices in language around personal merit. Conversely, people who have poor outcomes have a tendency to explain their situations in language around personal demerit. The positive language often centers around "difficult educational attainment", "complex job function", "hard work", "ambition", and "intelligence", and the…
There are innumerable reasons to hate AI, starting with its environmental impact: the AI boom has been estimated to consume as much power as New York City, and as much water as the global consumption of bottled water in 2025. The reasons could also be personal, from leading to a loss of livelihood, to threatening the way we pursue our passions. The distaste is fueled by visible misuse, including…
As the year draws to a close, it helps to look back and reflect on some key changes in LLVM. The article is written from my perspective, summarizing work I landed, along with adjacent work, in chronological order. In September 2024, Nikita Popov proposed signedness-independent integer-compares , and the necessary work for enabling optimizations with it occupied the first two months of my 2025. In…
In a follow-up to the introductory article on auto-vectorization , we discuss how the loop vectorizer works internally. Nearly all transforms in LLVM work by directly manipulating IR, using the results of various analyses on the IR, while the loop vectorizer operates on a layer of abstraction above the IR: it lifts the LLVM IR to an overlay IR, analyzes and transforms the overlay IR, and finally…
Capitalism works by getting a small number of skilled humans to produce something or offer some service that can then be marketed and sold to a vast consumer class. We live in very exciting times, for corporations have stumbled on a piece of technology that will take the capitalist wet dream to its absolute end: eliminating a vast number of skilled humans from the process, generating vast amounts…
Any modern compiler has three main components: a front-end that parses the source language, and translates it to a middle-end intermediate representation, a middle-end that operates on this IR, performing target-independent optimizations with a bit of target-specific information, and a backend that takes the final optimized middle-end IR and emits target-specific assembly. In this article, we…
Most modern general purpose CPUs have a vector processing unit (VPU). This unit contains vector registers that can hold multiple integers or floats, and instructions that operate on vector registers. Given two vector registers containing N floats each †, the VPU can perform a single instruction to operate on these two vector registers and store the result in another vector register. Without a VPU,…
I'm eighty today, but the sequence of events I'm going to narrate took place when I was in my early twenties. It was a pleasant Friday in San Francisco, and as many people will tell you, SanFran has the ideal weather. The mood at my workplace was jubilant. Added to the fact that the weekend was coming up, our CTO had announced that we would be going public the following Monday. Everyone was…
I woke up feeling like I had a brand new body and mind. We were asleep in a little cabin in the middle of nowhere, in a little village in Bhutan. We'd arrived the previous day, by air, and ridden on top of a bus while jet-lagged to get to find this cabin. We'd all arrived from different countries: Sven from Svalbard, Aamod from Brooklyn, and I'd made it from Montpellier. To add to the chaos of our…
My heart started racing the moment I saw her. She was standing right there by the coffee machine. The first thing I noticed about her was her hair. Curly hair, that was neatly combed back, with a couple of ornamental clips, pinned haphazardly. Neatly trimmed in line at the neck. She was short, had soft glowing skin, and was speaking very softly, in Russian, with another Russian girl who worked…
I couldn't sleep that night. Tired and frustrated, I got out of bed, took a shower, and dressed for the snowy streets of Boston. Thermals, t-shirt, jeans, and a quirky jacket I'd picked up at the thrift store, that I was quite fond of. I started making plans in my head. I'd take the 5:30am train to Natick after finishing breakfast with a Bloody Mary. There was a diner close to my place that was…
A picture of artwork.
We investigate the limitations of statistical methods, a subclass of which is called "machine learning", taking the opportunity to touch on several aspects of the industry, including the future of employment, various industry practices, and privacy. It's written as a wide-audience article, and only assumes a passing familiarity with software and statistics. We start from a seemingly hardline…
We illustrate how different programming languages influnced each other, and include a quick FAQ write up. An FAQ follows. Why would someone pick C++ over Rust today? C++ templates are incredibly powerful, and with the introduction of compile-time expressions, vast portions of modern C++ programs are just compile-time tables. Examples like this make Rust look very tiny: template < size_t i ,…
"I'm sorry, but we have to let you go", she says. "Right. What do I need to ink?", I respond, prepared. I don't need this job to pay the bills, and I don't enjoy it. I'd been contemplating resignation for a while now, but a fat severance cheque can't hurt. The HR woman hands me a crisp document on her tablet. "Do you know what you'll do next?", she asks robotically. My mind wanders, and I…
Today, we write a quick specialized note on what impredicativity exactly means, for those reasonably familiar with the Rocq syntax. Historically speaking, Rocq started out with making Set impredicative and they still carry around the flag --set-impredicative to maintain impredicativity in Sets. Let's check it quickly: (* Set is predicative. *) Fail Definition SetPred := ( forall X : Set , X ) :…
Here, we talk about equalities, and provide illustrative examples in vanilla Rocq, SProp, and HoTT. Equality in Mathematics In zfc-based mathematics , say in abelian groups, $A \oplus B \simeq B \oplus A$, where the equality is a set-based equality. In mathematics based on category theory, equality is too strong a notion on objects, and only speak about objects "upto unique isomorphism"; morphisms…
Starting from the question of what consequence Gödel's incompleteness theorem has on the foundations of mathematics, we argue for new foundations, and build some intuition for these new foundations in Coq. Mathematics can be thought of as a game where you start with some objects (say, sets), some axioms on the objects (typically, ZFC), and use a calculus of logical deductions (such as classical…
Using a mélange of mathematical syntaxes, we mechanize some exercises from Tao's excellent book Overture Let us informally define sets and set membership $\in$. A set is an unordered collection of objects , with the set itself being an object. Set membership checks whether a given object is contained within a set: The axiom of substitution uses the definition of equality: The axiom of substitution…
Both Lean and Rocq are proof assistants based on pCIC. Here, we argue that what sets them apart are, in essence, cultural differences. Lean has much lower startup-cost for pure mathematicians, since its built-in features and math library are great for doing undergraduate-level group theory & topology, masters-level commutative algebra & category theory, but it plateaus quickly thereafter. Lean…
An input program function is represented using a Control Flow Graph (CFG). It's composed of different Statements, which decompose into Expressions, but we will not concern ourselves with Expressions. The main control flow constructs are SplitStatement and MergeStatement. The SplitStatement has outgoing edges to N (N = 2 in the case of IfStatement) other Statements, and the MergeStatement has…
Detecting loops in a directed graph can be tricky, depending on how you define your loop. If you only want to admit "natural loops", where the header of the loop dominates every node in the body, as well as the footer, we have a simple algorithm. If you want to go to the other extreme, and define the most general "strongly connected components", without regard for loops, you have Tarjan's SCC…
We are going to be discussing LLVM's Fast Register Allocator: you might like to open RegAllocFast.cpp and refer to it as we go through the article. FastRegAlloc allocates linearly, going through instructions and their operands in order. It uses PhysRegState to keep track of the state of various physical registers: they can be 0 ( regDisabled ), 1 ( regFree ), 2 ( regReserved ), or a virtual…
Our custom max function returns the maximum of two positive integers, and returns the negative integer, given a positive and negative integer. Classic signed wrapping, you'd think. It's not so simple, as the problem reproduces only under the following circumstances: LLVM code is calling the max function. The library containing the max function is compiled without debugging information. The entire…