miniTPU: building Google's TPU from scratch
An educational build log on constructing the systolic array at the heart of Google's TPU from scratch — starting with a single NAND gate and working up to a running 2×2 array in a digital logic simulator.
Erik Steiger
Portfolio and blog posts mostly about technical things
Latest posts
Softmax attention is a local constant estimator
Deriving the view of softmax attention as Nadaraya-Watson kernel regression, why local constant estimators have boundary bias, and how Parallax adds a learned local-linear correction on top of softmax.
Chunkwise linear attention and DeltaNet (Part 3)
Why per-token recurrences are slow on GPUs, how chunkwise DeltaNet uses the WY representation, and how MQAR shows empirically the improvements of delta rule and gating in fixed-state recurrences.
Non-Linear Attention and Test-Time-Training (Part 2)
From TTT and Titans to the four axes of MIRAS — reframing the hidden state as a tiny model trained online, and placing every modern sub-quadratic architecture (DeltaNet, Mamba2, Gated DeltaNet, TTT, Titans, KDA, ATLAS) on four design choices.
Deriving Linear Attention and DeltaNet (Part 1)
Showing that softmax attention and other sub-quadratic models (linear attention, SSMs, etc.) belong to the same class of models that differ in how they store, optimize, and forget memories.
Clerk Auth via the CLI in Rust
A step-by-step guide to wiring Clerk’s polished web flows into a headless Rust CLI so users can `mycli auth login`, grab a long-lived JWT, and hit protected endpoints without sacrificing security or sanity.
Benchmarking LLM Inference Engines
...
Building Docker Hub for Typst Templates
Escaping CrystalReport. Reinventing PDF Stack and Building Docker Hub for Typst Templates on the way.
Europe, please get your incentives straight
From our phones to our cloud services, Europe depends on American tech. Here's why our brightest minds keep choosing Silicon Valley over Stuttgart - and what we can do about it.
How to use Plausible in Remix with a Proxy
Generating 1 Million PDFs in 10 Minutes with Serverless Rust
How to build a modern and scalable PDF rendering service using AWS Lambda.
Cross-modal search for 1 million Zalando products
Exploring how AI and cross-modal search can revolutionize e-commerce search, moving beyond traditional text-based methods to understand visual product features and customer intent.