I have been... stuck... in a funk... worried... anxious... It has been particularly frustrating because I used to publish here a lot. I know how to get unstuck in code. Why is this so hard in prose? I found one answer in a post I published all the way back in 2002 —quoting an article written by my mother-in-law. Perhaps the most insidious tyranny is the silence that occurs from caving in to the…
I used an LLM to summarize an incident slack channel. At a casual glance it looked like a decent enough summary. But on closer analysis, the LLM got it exactly backwards. This scares me. Casual inspection will mislead people to trust the summaries. And actions taken in response will make things worse. We have many kubernetes clusters. Security was on a months-long journey to carefully introduce a…
Conventional thinking in the software industry—what we call our best practices—sees incidents as an irritating interruption that slows progress against our roadmap. Whatever the most recent failure is, we seek minimum viable fixes to prevent the failure from ever happening again. The range of solutions that can be considered are limited to whatever minimizes disruption to our existing plan. We add…
Resilience is human skills and human relationships. Reliability is what we build into our software. There are four different concepts that get lumped into conversations about resilience and reliability. This post uses software engineering examples to clarify the four different meanings of resilience from Dr. David Woods paper: Four concepts for resilience and the implications for the future of…
Husband, dad, friend, brother, son, uncle, cousin; programmer, martial artist, teacher, downhill skier, and thinker. I used to write and publish a lot of prose here back when I was a lot more sure the world needed to hear what I was thinking. I write a fair bit of code: github.com/dobbs . I sometimes microblog: rls.social/@dobbs . I used to tweet: @dobbs . In the early 2000s, I contributed to the…
An experience report from measuring uptime. Many software people believe measuring uptime to be a useful tool to support or assess improvement in software reliability. My experience is different. I joined a team at an internet scale company whose job was to manage an incident chat bot and related incident database. The main job of the database was to track uptime for teams and products. It’s the…
Consider the optic nerve and the optic tract. Original image is from: biology-forums.com gallery Light strikes the retina and signals fire along the optic nerves, through the optic chasm, through the optic tracts and into the left and right thalamus on route to the visual cortex at the back of our brain. Before the signals reach the visual cortex, they must first pass through the limbic sections…
π in base eleven : 3.16150702865a4 Today's date, numerically encoded according to US conventions: 3/16/15 The time this article posted (Mounatin Daylight Time): 07:02 Woo Hoo! π day all over again. If you casually ignore the first two digits of the Year of Our Lord. And ignore that I actually faked the publication date and time . And ignore that the Year of Our Lord is at best an…
I rushed out to catch the bus for fear of missing it. There was only one other person waiting. I needn't have worried. There's a whole story in there about unnecessary fear. But that's not today's story. I recognized the woman waiting at the bus stop. I've seen her fairly often on the bus. Our schedules are similar. She's beautiful. We're waiting together for the bus. Just the two of us. It's dark…
Time itself is an infinite scroll. past you are here wish you were here future