Recent content on Internals for Interns
In the previous article we looked at what inference actually is — how a model takes a prompt and builds a response one token at a time. I mentioned two Go projects that let you run that loop locally: yzma , which lets Go call the llama.cpp inference engine directly, and kronk , which builds a friendly SDK on top of it. Today we open up yzma and see how the whole thing is put together.
The first time I reached for testing/synctest , I assumed it was a sleep helper with better manners. The public API is small enough to support that assumption: synctest . Test ( t , func ( t * testing . T ) { ... }) synctest . Wait () synctest . Sleep ( d ) // new in Go 1.27 Three functions, one obvious guess: fake time, wrapped in test scaffolding. That guess is wrong. It isn’t a sleep…
In the previous article we followed a read() all the way down through the VFS: from the syscall, to the struct file , to ext4, and into the page cache. And there was exactly one moment where we skipped over the details. When the page cache missed — the bytes weren’t in memory yet — the filesystem had to actually go and fetch them, and all we said was that it “reads the disk, through…
Welcome to a new series! For most developers today, using a large language model means one thing: an HTTP call to somebody else’s computer. You send a prompt to an API, tokens come back, and everything in between is somebody else’s magic. But here’s what I find much more interesting: you can run these models locally , inside your own process, on your own hardware — and if…
In the previous article we saw how the scheduler decides which task gets the CPU — so at this point we know how programs get to run . But running isn’t enough: for a program to be useful, it almost always needs to read and store data — its config from /etc , a shared library from /usr/lib , the rows of a database file, the document you’re saving. So the natural next question is: what…
When you build a storage engine in Go, sooner or later you need to answer a very plain question: “How should the code read bytes from files?” This sounds too low-level to matter. A database has bigger ideas: partitions, blocks, indexes, filters, compression, compaction, caches, query planning. But all of these ideas end up doing the same simple action many times: “Read N bytes…
In the previous article we took apart the reflect package and found that its magic is mostly the compiler leaving very good notes — type descriptors frozen into read-only data at build time, and a package that knows how to walk them. The whole article was about reading metadata that was already sitting in memory before main even started. Today we shift the perspective. Profiling is the runtime…
In the previous article we looked at how the kernel gives every process its own private view of memory. But memory is only half of what a process needs to actually run . The other half is the CPU itself — and there are only so many CPUs in a machine, while there are usually hundreds or thousands of things that want to run on them. So somebody has to decide, constantly, who gets a CPU and for how…
In the previous article we watched the runtime rebuild an entire stack trace out of metadata the compiler and linker had frozen into the binary at build time. I told you at the end that reflect works on exactly the same trick — metadata baked into the binary, only pointed at your data instead of your call stack. Today we’re going to cash that promise in. Let’s start with a program…
In the previous article we looked at how a user program crosses the ring 3 → ring 0 boundary to ask the kernel for help. The example we used was read() — a file descriptor, a buffer pointer, a byte count. But we glossed over something important: what is that buffer? Who decided it existed? Who owns the physical RAM behind it? Those questions are what the memory manager answers. And it answers them…
In the previous article we took apart the select statement and saw how it’s really two features in one, with the compiler rewriting the easy shapes away and only the hard cases falling through to the runtime’s selectgo . The recurring theme there was coordination — the compiler and the runtime each doing half the work and meeting in the middle. Today we’re going to lean on a…
In the previous article we followed the kernel from the very first instruction the bootloader handed us all the way to the moment kernel_init called execve() on /sbin/init . That was a long ride, but it ended with a quiet handover: the kernel stepped aside, userspace took the wheel, and /sbin/init started spawning the rest of the services. Here’s the thing though. Those processes that just…
In the previous article we walked through slices, maps, and channels, and how each of them is structured under the hood. Out of those three, channels are probably the most involved one in terms of how you actually use them — and there’s one language construct that really stands out when working with channels: the select statement. That’s what we’re going to be talking about in…
Have you ever wondered what really happens between the moment you press the power button and the moment your login screen shows up? That gap—usually some seconds—hides one of the most intricate initialization sequences in computing. Today I want to walk you through it. This is the first article in a series where I’ll try to make sense of the Linux kernel internals together with you.…
So far in this series we’ve looked at the parts of the Go runtime that orchestrate execution — the memory allocator, the scheduler, the garbage collector, sysmon, the netpoller. Today we’re switching gears and looking at three of the most ordinary things in Go: slices , maps , and channels . They are the bread and butter of every Go program. You probably write all three of them several…
In the previous article , we explored Btrfs—a copy-on-write filesystem built around a single kind of B-tree, where every file, extent, checksum and chunk mapping lives as a tagged item in some tree, and snapshots fall out of the reference-counted extent design. Btrfs took a lot of inspiration from an older system that pioneered most of these ideas: ZFS . ZFS started life at Sun Microsystems in the…
In the previous article we saw how sysmon steps in every 10ms to call netpoll(0) on behalf of busy Ps, making sure network I/O doesn’t stall when the scheduler is too busy to poll on its own. We glossed over what that network poller actually is. Today we’re fixing that. Go’s networking story is one of its most quietly impressive tricks. You write code that looks blocking —…
In the previous article , we explored XFS—a filesystem built for extreme scale that divides the disk into independent Allocation Groups, each with its own B+ trees for free space, inodes, and extent tracking. XFS, like every filesystem we’ve covered in this series, shares one fundamental characteristic with ext4, NTFS, and FAT32: it modifies data in place . When you update a block, the new…
In the previous articles we explored the scheduler — how goroutines get multiplexed onto OS threads through the GMP model — and the garbage collector — how Go tracks and reclaims memory using a concurrent, tri-color mark-and-sweep approach. Both of these systems are impressive, but they have real blind spots. The scheduler can’t reclaim a P that’s stuck in a syscall, because no Go code…
In the previous article , we explored NTFS—a filesystem where everything is a file, from your documents to the Master File Table itself. NTFS centralized all metadata into the MFT, using attribute-based records and a single journal to keep Windows volumes consistent and feature-rich. Now let me introduce you to XFS , the filesystem designed for extreme scale . Originally built by Silicon Graphics…
In the previous article we explored the Go scheduler — how goroutines get multiplexed onto OS threads, the GMP model, and all the tricks the runtime uses to keep your cores busy. But there’s a fundamental problem we haven’t addressed yet: all those goroutines allocate memory, and somebody has to clean it up. That’s the garbage collector’s job, and that’s what…
In the previous article , we explored ext4—a filesystem that divides the disk into block groups and separates metadata from data, with inodes in one place and file content in another. NTFS takes a radically different approach. In NTFS, everything is a file —the allocation bitmap, the journal, even the Master File Table that tracks all the other files. This single idea shapes the entire design: the…
In the previous article we explored how Go’s memory allocator manages heap memory — grabbing large arenas from the OS, dividing them into spans and size classes, and using a three-level hierarchy (mcache, mcentral, mheap) to make most allocations lock-free. A key detail was that each P (processor) gets its own memory cache. But we never really explained what a P is , or how the runtime…
In the previous article , we explored FAT32—a filesystem that conquered the world through simplicity. Its linked-list approach gets the job done, but it comes with real limitations: no crash protection, linear directory searches that slow down as directories grow, and a hard 4GB ceiling on file sizes. Ext4 takes a fundamentally different approach. Where FAT32 uses a single table as both its…
In the previous article we explored how the Go runtime bootstraps itself — how a Go binary goes from the operating system handing it control to your func main() running. During that bootstrap, one of the first things the runtime sets up is the memory allocator . And that’s what we’re going to explore today. Think of the memory allocator as a warehouse manager. Your program constantly…
In the introduction to this series , we covered the foundational concepts every filesystem needs: sectors, blocks, partitions, and the four essential services—naming, allocation, metadata, and crash recovery. Now let’s see how a real filesystem puts these into practice. FAT32 is everywhere. USB drives, SD cards, digital cameras—if you’ve ever moved files between a Windows PC, a Mac, a…
When you write Go, a lot happens behind the scenes. Goroutines are lightweight, channels just work, memory is managed for you, and you never think about thread pools. All of that is powered by the Go runtime —a sophisticated piece of infrastructure that gets compiled into every Go binary. This is the first article in a series where we’ll explore the Go runtime from the inside. We’ll…
Every time you save a document, download a photo, or install an application, you’re trusting a filesystem to keep your data safe. Think about it—filesystems are this invisible layer between your applications and raw storage hardware. They turn a sea of numbered blocks into the familiar world of files and folders that we all take for granted. But what exactly does a filesystem do? And what…
In the previous post , we watched the compiler transform optimized SSA into machine code bytes and package them into object files. Each .o file contains the compiled code for one package—complete with machine instructions, symbol definitions, and relocations marking addresses that need fixing. But your program isn’t just one package. Even a simple “hello world” imports fmt ,…
When you create an index in PostgreSQL, you might think there’s just one type of index structure working behind the scenes. But PostgreSQL actually provides six different index types , each engineered for specific use cases with completely different internal structures and access patterns. This is the final stop on our SQL Query Roadtrip. In the previous article , we explored how…
In the previous post , we explored how the compiler transforms IR into SSA—a representation where every variable is assigned exactly once. We saw how the compiler builds SSA using Values and Blocks, then runs 30+ optimization passes. We watched the lowering pass convert generic operations into architecture-specific instructions like AMD64ADDQ and ARM64ADD . Now we’re at the final stretch.…
In the previous article , we explored how PostgreSQL’s planner chooses the optimal execution strategy. The planner produces an abstract plan tree—nodes like “Sequential Scan,” “Hash Join,” “Sort”—that describe what to do. Now the execution engine needs to actually do the work: read pages from disk, follow indexes, join tables, and produce results. The…
In the previous post , we explored the IR—the compiler’s working format where devirtualization, inlining, and escape analysis happen. The IR optimizes your code at a high level, making smart decisions about which functions to inline and where values should live—on the heap or stack. But the IR still looks a lot like your source code. It has variables that can be assigned multiple times,…
In the previous article , we explored how PostgreSQL’s rewriter transforms queries—expanding views, applying security policies, and executing custom rules. By the end of that phase, your query has been fully expanded and secured, ready for execution. But here’s the million-dollar question: How should PostgreSQL actually execute your query? Let me show you why this matters. Take this…
In the previous posts , we’ve explored how the Go compiler processes your code: the scanner breaks it into tokens, the parser builds an Abstract Syntax Tree, the type checker validates everything, and the Unified IR format serializes the type-checked AST into a compact binary representation. Now we’re at a critical transformation point. The compiler takes that Unified IR—whether it was…
In the previous article , we explored how PostgreSQL transforms SQL text into a validated Query tree through parsing and semantic analysis. By the end of that journey, PostgreSQL knows that your tables exist, your columns are valid, your types match up, and your query makes sense. But before the planner can figure out how to execute your query, there’s one more critical transformation step:…
In the previous post , we explored how the Go compiler’s type checker analyzes your code. We saw how it resolves identifiers, checks type compatibility, and ensures your program is semantically correct. Now that we have a fully type-checked AST, the next logical step would be to generate the compiler’s Intermediate Representation (IR)—the form it uses for optimization and code…
In the previous article , we explored how PostgreSQL establishes connections and communicates using its wire protocol. Once your connection is established and the backend process is ready, you can finally send queries. But when PostgreSQL receives your SQL, it’s just a string of text—the database can’t execute text directly. Let me show you what happens when PostgreSQL receives this…
In the previous posts , we explored the scanner—which converts source code into tokens—and the parser —which takes those tokens and builds an Abstract Syntax Tree. In future posts, I’ll cover the Intermediate Representation (IR) —how the compiler transforms the AST into an intermediate lower-level form. But before we can get there, we need to talk about two crucial intermediate steps: type…
In the previous article , we explored the complete journey a SQL query takes through PostgreSQL—from parsing to execution. But before any of that can happen, your application needs to establish a connection with the database. This might seem like a simple handshake, but there’s actually a sophisticated process happening behind the scenes—involving process management, authentication, and a…
In the previous blog post , we explored the scanner—the component that converts your source code from a stream of characters into a stream of tokens. Now we’re ready for the next step: the parser . Here’s the challenge the parser solves: right now, we have a flat list of tokens with no relationships between them. The scanner gave us package , main , { , fmt , . , Println … but…
Ever wonder what happens when you type SELECT * FROM users WHERE id = 42; and hit Enter? That simple query triggers a fascinating journey through PostgreSQL’s internals—a complex series of operations involving multiple processes, sophisticated memory management, and decades of optimization research. This is the first article in a series where we’ll explore PostgreSQL’s query…
This is part of a series where I’ll walk you through the entire Go compiler, covering each phase from source code to executable. If you’ve ever wondered what happens when you run go build , you’re in the right place. Note : This article is based on Go 1.25.3 . The compiler internals may change in future versions, but the core concepts will likely remain the same. I’m going…
Hi! I’m Jesús Espino , a software developer passionate about understanding how things work under the hood. My Background I’m currently working as a software developer at VictoriaMetrics , the company behind the open source time series database of the same name, and behind VictoriaLogs, a high-performance log database — both written in Go. Over the years, I’ve worked with various…
Privacy Policy Last updated: October 18, 2025 Overview This website (“Internals for Interns”) respects your privacy and is committed to protecting your personal data. This privacy policy explains how we collect, use, and safeguard your information when you visit our website. Information We Collect Analytics Data When you consent to cookies, we use Google Analytics to collect: Pages you…
Get weekly deep dives into software internals delivered straight to your inbox. Learn how compilers, databases, and systems work under the hood—explained in an approachable way. What You’ll Get 📬 One email per week with a new deep dive into software internals 🎯 Focused topics like Go compiler phases, PostgreSQL query execution, Git internals, and more
Welcome! I’m thrilled to finally launch this project—something I’ve been thinking about for almost a decade. What This Is All About For over 10 years, I’ve been giving talks at conferences about how things work under the hood. I started in the Python community, exploring topics like the object model, garbage collection, and the CPython interpreter internals. Later, I expanded…
Have you ever wondered what actually happens inside PostgreSQL when you run a SQL query? Behind a simple SELECT statement lies a sophisticated pipeline involving parsing, semantic analysis, query rewriting, cost-based optimization, and a pull-based execution engine — built on decades of database research. This book follows a query through every stage of that pipeline: lexical analysis and parsing,…
Have you ever wanted to build a REST API in Go that is fast, secure, observable, and actually pleasant to maintain? This hands-on book walks you through designing and implementing a production-ready REST API using Go’s standard library as the foundation. Instead of hiding complexity behind heavy frameworks, you will see how Go’s built-in HTTP tooling is more than enough to build…