Hunting My Hand-Crafted Induction Head in Real Models
Last time I hand-crafted an induction head, and I ended on a question I could not answer from inside the toy: if I go looking in a real trained model, will I find the circuit I just built?
barmag
Engineering leadership, neurodivergence, and technology -- by Yasser Makram
Latest posts
I Didn’t Understand QKV, So I Hand-Crafted an Induction Head
I understood select name from Person where k=42, and I still could not map it to Q, K, V. So I built a 15-dimensional toy transformer by hand, two attention heads, eight matrices, no training, and watched it predict the next token correctly 87% of the time.
Your Wearable Is a Witness, Not a Judge
Wearables arrive with the kind of confidence that two-decimal precision allows. They are describing one substrate, not the whole organism. This post is about how to demote the watch from judge to witness, and when to call it back to the stand.
Panopticon vs Agency: What I Catch Myself Doing in Management
Most engineering ceremonies are panoptic: architectures of visibility that get teams to police themselves on behalf of the org. The Adlerian alternative isn't no visibility; it's separation of tasks. What I catch myself doing as a manager when I forget the difference.
Code Is Still Documentation. Docs Can Finally Keep Up.
Prose docs were abandoned because keeping them fresh cost more than they were worth when stale — not because they were useless. Agent-generated docs collapse that cost. Old use cases get rescued; one new one emerges: catching agents in polite lies.
Your Standup Is Industrial Clock-Time Cosplay: Rethinking Engineering Ceremonies for the Agent Era
The daily standup — and most engineering ceremonies — are products of industrial clock-time, a coordination technology invented in the 18th century to synchronise factory labour. AI agents are the first technology in 250 years that can hold coordination continuity on behalf of humans rather than requiring it from them.
The Courage to Be Disliked: Adlerian Psychology for Engineering Teams That Burn Bright and Burn Out
I spent ten years reading stack traces; now I'm supposed to read rooms. Rooms don't have error messages. How Adlerian psychology gave a neurodivergent engineering manager the explicit framework that intuition-based advice never could.
Lessons from the Trenches: How We Paid Less and Got More from AWS
I am old enough to have used CORBA, and COM+. Lucky enough to be building distributed loosely coupled systems for 20 years. Now witnessing how easy and accessible distributed technology have become. Many of the principles have not changed, but reflecting on my experience with modern cloud native architecture that building resilient and scalable system with PaaS cloud platforms is much cheaper than…
Microservices Patterns Chapter 1 Notes
Microservices Patterns