Build logs, patterns, and opinions about platform engineering and internal developer tools.
Six months after we standardized on OpenSearch, a pull request introduced Datadog into a service. The ADR existed. It had been discussed, approved, and stored in the repo. The PR was still green. That is architecture drift. Not because engineers are careless. Because memory does not scale across hundreds of people and dozens of repositories. After we started checking ADRs in CI, we caught several…
Archbot flagged a "blocker" on a PR. It cited the diff, built a plausible chain of reasoning, and suggested a fix. It was completely wrong. Not "LLMs are sometimes wrong" wrong — more like convincing enough that a senior engineer spent 20 minutes disproving it . The missing detail wasn't subtle. It was a guard clause sitting in a helper two files away. Archbot just didn't have that file. That…
Here's a story I tell my mentees often—and it's one where I almost made an expensive mistake. A few years ago, my team was tasked with rebuilding the authentication system for one of our core products. We walked into that room with a massive assumption already locked and loaded: "Authentication isn't our core business. We should buy, not build." It made perfect sense on paper. We assumed buying a…
You don’t “need retries in Kafka” until the day one of your handlers starts failing and you’re forced into a choice: block consumption (and watch lag climb) or keep consuming and retry somewhere else. This post is about one very pragmatic approach: commit the Kafka offset even when processing fails , then push the failed message into a Go retry queue. Kafka keeps moving, and your application owns…
[ TIP ] A Note from the Future (2026) While writing this third part, I realized the original posts in this series needed a refresh to match the depth we're hitting here. So, I've gone back and significantly updated Part 1 (Career Growth) and Part 2 (Architecture) with new frameworks, diagrams, and actionable "workout plans." If you haven't read them in a while (or ever), now is a great time to…
Two weeks after deploying our custom AI code reviewer, it caught a bug that would have crashed our payment processing. A senior engineer's PR looked clean—it compiled, passed linting, followed Go idioms. But the AI spotted a subtle pattern: we'd accidentally turned a soft dependency into a hard one. If our config service blinked, our main service would die. The engineer's response? "Oh." That's…
Hey there! You've reached a significant milestone in your career journey. But if "Architecture" feels like a scary word reserved for people in ivory towers, I've got news for you: you're already an architect. Every time you choose a library, decide between a SQL or NoSQL database, or structure your API response, you are making architectural decisions. The difference between a Junior and a Senior…
Hey there! First off, take a moment to celebrate. You've earned the rank of Senior Engineer. That's no small feat, and it's proof of your technical chops and dedication. But if you're like most engineers I mentor, the champagne toast is quickly followed by a nagging thought: "Now what?" It's the classic "what got you here won't get you there" moment. As a Principal Engineer, I see many talented…