Thoughts on digital government, democracy, tech, public institutions, and the occasional unrelated rabbit hole.
This post originally appeared on the Ad Hoc blog , coauthored with my delightful colleague Ari Hershowitz . I'm reproducing it here, lightly edited for formatting and clarity. Over the last 15 years, government has made meaningful progress modernizing its services. Agencies and their digital service teams have moved critical processes online. Cloud computing, accessibility, and user-centered…
Grady Booch, one of the genuine elders of the software engineering discipline 1 , recently did a two-hour conversation with Gergely Orosz on the Pragmatic Engineer podcast. The whole thing is worth listening to, but one thing that particularly struck me was the way Booch described the discipline of software engineering: Ours is a field that tries to build reasonably optimal solutions...that…
This post was co-authored with Mark Headd, my colleague at Ad Hoc and a brilliant thinker and doer on technology and government. It's less a prediction than a provocation, playing out one possible consequence of generative AI to its logical endpoint. Check out Mark's blog and follow him on LinkedIn for more of his great writing. There are lots of ways to answer the question "what does a civic…
I'm increasingly intrigued by a concept called Hierarchical Reasoning Models (HRM), a potential alternative architecture to traditional LLMs. 1 Last month, Sapient released an open-source HRM and accompanying paper . The paper is written in the kind of prose you'd expect from nine mathematics PhDs 2 , but as far as I can tell, the basic idea is this: LLMs "reason" one word or concept at a time.…
Because my buddy Todd is rad, I got to see The Pixies live at The Anthem in DC this weekend . It was genuinely one of the best shows I've ever been to. A few reflections in no particular order: They are incredible musicians. Our mental model of The Pixies is distortion-screamy-quiet-twangy but this team knows and loves their instruments and plays with precision and care. Dave Lovering could not…
I had to read this one twice: In our main experiments, a "teacher" model with some trait T (such as liking owls or being misaligned) generates a dataset consisting solely of number sequences. Remarkably, a "student" model trained on this dataset learns T. This occurs even when the data is filtered to remove references to T. The paper describes a subtle but profound behavior in LLM training that I…
I was futzing around with ChatGPT the other day to see how much info I could get it to cram into a single answer 1 . That didn't really get anywhere, but led to a fun prompt that generates a readable account of any historical event. I made a project folder called History By The Hour and gave it the instructions below: You are a meticulous historical analyst and global news correspondent, with a…
Right after WWDC, I was feeling underwhelmed and a little baffled. But with a week of hindsight, I think the commentary was so focused on the aesthetics of Liquid Glass, it missed the function. Liquid Glass tells me that Apple thinks your AI device of the future isn’t a new wearable, but the iPhone you already own. Google, OpenAI, and Meta have world-class LLMs, but an LLM isn’t a product until…
Everyone’s having a good laugh at this image going around Twitter: But I think it’s actually a surprisingly useful way of thinking about AI: Not as a technology but as a product methodology. A rocket-powered onion peeler. Go with me here a minute. There’s an old joke: "How do you carve a statue of an elephant? You start with a block of marble and chisel away everything that doesn’t look like an…