A feed of the latest blog posts
Hey again! Welcome back to part two of me reflecting on the past few quarters and writing down my answers to John Cutler and Tom Kerwin’s questions on how leaders navigate uncertainty and ambiguity. If you’re lost, part one is here . I started writing my reflections to this a while ago (almost two years now!) and decided that I actually wanted to separate out every edition of this by a few years.…
Picture with me, if you will, the absurdity of finding yourself swimming in the middle of the ocean. First think about the ocean and how deep and infinitely vast it is; then about how improbable it is to even fully grasp the notion of how large the ocean is, of how deep it is, of how wide it is; think about how there is so much of it we will never know, and so much we can never know. Let that sink…
Innovation is a tricky subject. The precise details of how to do it are not well studied, at least not under the name “innovation”. In addition, multiple disciplines have advanced research that overlaps significantly, but the multidisciplinary integration lags behind by decades, making shared empirical research or identifying more global patterns difficult. Nevertheless, I consider it dearly…
I’ve been reading this week about how humans learn, and effective ways of transferring knowledge. In addition, I’ve also had AI in the back of my mind, and recently I’ve come to the realization that not only is our industry building AI tools poorly, we’re building them backwards. Which, honestly, is really depressing to me because there is so much unrealised potential that we have available–is it…
Observability, so hot right now. Over the years, we’ve seen observability go from an unknown concept to a ubiquitous phrase that everyone is desperate to stamp on their products. We’ve seen projects come, evolve, and die. We’ve seen technologies emerge out of the ashes, born from the tears of SREs long departed. Yet, amongst all of this growth, all of this innovation, one question remains: and…
I was originally going to call this post “What makes a programming language real?” because I saw some people picking a fight on the internet about this type of topic, yet again, and it got me thinking as to why we even broach the topic in the first place. Surely, one might think, a programming language can just exist peacefully without being questioned as to its legitimacy, right? Well, clearly…
Long running spans are one of my biggest “we don’t really actually have a good standard solution for this” issues in opentelemetry. They’re something I’ve run into before, weirdly frequently, and especially so when attempting to try and instrument front-end or mobile facing systems. It turns out, though, that the issues here with long running spans are actually pretty similar to the issues with…
When going on an observability journey, there tends to be a few concrete phases that every company goes through. Understanding how those unfold and take shape as you mature your observability practices can help you identify when you’ll run into certain types of challenges, and when you’ll start really wanting certain tools and practices to help address those challenges. That said, when you’re…
Caching is hard. So hard. But also, we are so fucking bad at it. Every time I have to use a public wifi setup I have a joker moment. Does absolutely nobody test shit on anything less than wired symmetric gigabit anymore? Web SPA apps are some of the worst for this. Motherfucker, you have the same fucking iconography for three years, why does it load correctly and then ALL OF THE ICONS FAIL ONCE I…
I like to home roll abstractions, but commoditise implementations. What I mean by that is a fairly simple rule that has a very powerful effect, but can be tricky to find the right balance. Home rolling the abstraction, to me, means deeply exploring and fleshing out out an abstraction from whole cloth, whether it be an interface, or a mental model, or a collaborative workflow, or a template, or……
One of the hardest things for people to understand with distributed systems is that eventual consistency is the same thing as eventual inconsistency. The very same pattern that lets you non atomically deal with things also ensures that eventually you’ll have a system that doesn’t match your understanding. Resources will go stale, things will go missing, stuff will exist without ever having been…
I miss the forums . I miss the forums so much it hurts. I miss when research was about discovery and learning and sharing . I miss when humanity felt like it had hope , when human interaction was plentiful, when genuine connection wasn’t rarer than gold. I miss the days before our souls were destroyed for the sake of the market , before our knowledge was plundered , before our humanity exploited .…
If there’s one thing I wish I could burn entirely to the ground and wipe away all traces and remnants of, its the misplaced notion that the productivity of Knowledge Work can be managed, measured, analysed, and optimised as if all one needed to do was drip feed heroin up the arse of their hapless workers. What is Knowledge Work™, you ask? There’s two concepts of Knowledge Work that I’m thinking…
Observability is a bit of a hot topic, and while it’s increasingly been playing a larger role in engineering strategy, I think the way it’s presented can often cause a lot of leaders to miss the value or to over-index on the wrong things. I’m going to present the current definitions of observability that are widely used in engineering and other disciplines, and then introduce my definition; I’ll…
Something that I find missing in almost every software company is this thing that I’m not sure I’ve seen explicitly called out anywhere, but I’m going to call it an Engineering Language. This Engineering Language is something that I’m going to attempt to describe, motivate, outline, and then illustrate with an example. Engineering Language The Engineering Language is something that I would…
I read this post from John Cutler and Tom Kerwin recently on how leaders navigate uncertainty and ambiguity and it intrigued me. I decided to give my shot at answering these as a writing exercise and as an opportunity for self reflection. The past few quarters have seen a lot of change for me, and haven’t taken the time I need to reflect as much as I would otherwise wish; this seems like as good…
When starting a new job as a software engineer, it’s natural to feel the pressure of delivering immediate value and meeting the expectations of your role. However, there’s a unique opportunity during this initial period that often goes unnoticed: nobody expects you to actually do useful work right away. So not only can you can feel free to identify and solve problems that others might have grown…
I see you want to hire a developer to work on internal developer tooling, developer experience, and the generally intangible but admirable goal of “making life better for devs”. That’s awesome; you’ve got one hell of a challenge ahead of you. This role is extremely difficult to hire for. In my opinion, and in my experience, it’s been the most difficult role in the company outside of senior…
So the big thing that makes everything so difficult for browsers is that opentelemetry has a concept of a lifecycle for telemetry that doesn’t map very well to how you ergonomically propagate context and correlate traces together. Opentelemetry works super super well in cases where you have a very linear callstack that’s fully synchronous in design. Something like request -> function A(a1, a2,…
I was asked recently for my thoughts on a wonderful article about software correctness, human convenience, and flossing, and I ended up dumping out an entire blog post worth of thoughts. So, this blog post serves as both a reminder to myself to write more, and also a sincere apology to my wonderfully patient friend, Kelly , who graciously puts up with me dumping absolutely unholy amounts of text…
Y’all ready for a story about one of the wildest fuckups production outages I ever took part in? Buckle up; we’re going for a ride far, far away from any security cameras. Setting the Scene At a previous job we had some fairly intense mismanagement. No tech debt was ever allowed to be handled. No good deed was ever unpunished. No non-white-male person was paid a market salary. Y’know, the usual.…
This blog post will be kept up to date as I find out more information and publish my findings. It’s currently organised in no particular order, as a collection of several fragmented thoughts. Nginx Nginx config for object storage The nginx config used to proxy to an object storage with a cache you will have to tune nginx by increasing its worker_rlimit_nofile and worker_connections values. –…