Personal blog of Ryan James Spencer
Systems fail. We know this, yet we often conflate the terms we use to discuss failure. "Reliability" gets mixed up with robustness and resilience, making it harder to reason about what's actually broken and how to fix it. I want to define these terms, which I'll affectionately refer to as R³ from here on: Reliability : The probability a system performs correctly during a specific duration. Think…
Organizations that learn are organizations that, luck granted, succeed. While success requires sharp teams, healthy cultures, sound strategies, and adequate resources, what separates thriving organizations from stagnant ones is their ability to adapt and learn from experience. But excellence requires action. Competitors are constantly innovating and improving. This is physics: systems naturally…
If you are on a team and are tasked with delivering code, record or document your deliverables. Do this when you finish whole initiatives as part of closing bookends, but also as you make progress at stepping stones along the way. Every kind of artifact is fair game for a demo: design documents, diagrams, videos, CSVs of data, dashboards, and so on. Be creative, but be mindful that the outcome…
If you're a software developer, you've likely been there; the manic, not to mention frantic, working through complex input cases and systems interactions, all to fix or extend some potentially tiny bit of code. This sort of "global reasoning" is often considered business as usual for software systems at a certain size, and I'm genuinely surprised there isn't more active discussion around the…
People working in tech often, but perhaps not always, want to show off their proficiencies. Advertising these proficiencies is a tool used to demonstrate capability to those with promotion power, usually in hopes that the advertisement will lead to involvement in activities that they can bundle in a brag document of some kind. The more "altruistic" are those who advertise so that they can feel…
Software (professionally) involves a lot of theorizing; doing X and Y will roll up to a feature we expect users to love and, thus, will want to pay us gobs of money to use to alleviate their pains and/or make them feel like a superstar. Sometimes the theorizing feels airy, and we try to tether it to real customers, real (anonymized) user stories, real data of some kind. There’s another way we, at…
Cleaning makes my mind nimble. Untethered by the visual distractions of my environment, I am free to focus on the tasks that I set out to accomplish. Cleanliness is easier to acheive if you've identified a safe way to give up on the mess. One way I clean digital clutter is to concoct lists, but then my time is sunk in curating lists. This is no good; I want to spend my time savoring the things I…
It has taken me awhile to summarise 2022. I intended to finish writing my book, "Just Perf", but family took precedence, as it always should. Instead, I drastically improved my curation skills, and did get a lot of writing practice in, while also honing my craft at work. Admittedly, I was feeling a bit burnt-out, as one does from time to time working in tech, and I decided at the start of 2022…
Time management advice tends to devolve into the treating time as an expendable resource (it is) that must be cost-analysed (it should), but when people look up "time management tips" I want to propose that they aren't looking for the how or what of time expenditure, but rather are looking to become better aware of time. I base this argument on the fact that those working in jobs that are…
You may get to the point in your career where you see enough things break and assume everything is fundamentally flawed. Paranoia sets in, robust architectures are designed and implemented, and you witness a spectacular explosion of failure on the faintest breeze of unexpected input. Alternatively, the paranoia bonds with apathy and you sit on our hands, unsure what to do, trying to settle on an…
Before the plane takes off you’re told to attach your own oxygen mask first before helping those around you. Putting the oxygen mask on first feels counterintuitive. Giving is about putting others first, is it not? The vision blurs, our senses numb, and we no longer clearly identify how to help others, let alone ourselves. You are a unique individual, and a team is a stitching of the unique into…
Thorsten Ball had this interesting thought bubble about having a personal reading coach: Why is picking new books to read so hard? What I need is a personal trainer, but instead of writing workout programs they tell me what to read next. It’s based on what I read in past and what my goals are. They also make sure enough fiction is in there, cause it‘s important. — Thorsten Ball…
You need to make changes to produce new states of the system you expose. Code change velocity is a metric people sometimes use to determine health of the platform. While a high change rate may seem good, it can also be a sign of a lot of churn. Change is inevitable, but need not be unsafe. To avoid releasing code you regret, try these approaches to making changes in the future. At a macro level,…
shipping things can be done in a way that isn’t tasteless, where you won’t have to go scrambling back to fix things. shipping work can be done in an incremental way, forming progress towards the things you want to achieve. how do you publish articles rapidly? where is the threshold for “this is done?”. is there even a threshold that must be upheld? a thought experiment. what is the smallest…
Commonly the ideal way to work with code changes is to do them immutably. What i mean by this is that you begin the work as an independent thing provide a way to switch from the old thing to the new thing (feature flag, fork, and so on) switch to the thing under particular conditions (sampling users, explicit flag curation, other rules) the point of (2) is that it gives you the ability to make a…
Instead of focusing on the entirety or majority of books I’ve read this year, I wanted to focus, instead, on the books that made a mark on me in some way or another, and the key takeaways for each book. Summaries for books vary drastically. Good writing is an interplay between compression of information on the part of the author as well as compression of information on the part of the reader. To…
Pull requests aren’t precious things. They propose changes, and there is often an unfair assumption in a company setting that they will always make their way to trunk. Pull requests, just like issues, go stale. Not just in respect to the being up to date with the latest changes that may have surfaced since the pull request has been raised, but also in terms of general age. Why do pull requests…
Benchmarks and profiler in hand, you’re ready to start taking stabs to improve your program’s performance. Looking around, there are some large numbers that seems obvious to attack, but this is the start of a broken perception about performance analysis. Bottlenecks are worth evaluating, but they aren’t the only thing you should be evaluating. The title of this article may seem obvious at first…
Not all of our decisions around performance are going to be driven after the thing we want to optimise has been built. In order to design programs with performance in mind at the start, we need a set of heuristics to guide us to make better choices. Often performance gurus have a set of principles they mention early on in their seminal texts. These are useful, but usually only given a short…
Do benchmarks feel really really hard to get right and that you are never sure you are measuring what you want to measure? It’s likely you feel this way because benchmarks are like mini experiments; you run the experiment to see if there are reproducible results given some fixed parameters. Yet despite the relative ease of running experiments and collating data, experiments can be a net negative…
You've got a slow Rust program you're sure you can improve, but you have no idea where to start. If you begin changing things and rerunning the program to test with something like time , it will take you ages. Isn't there anything better? Profilers and hardware architectures are complex beasts. It can take awhile to understand how everything connects together and what metrics a profiling tool is…
For newcomers to Rust unfamiliar with the empty tuple ( () ), it can be a confusing type; what's the point of this thing and how can I use it idiomatically rather than having to endure its presence? Here are some common patterns where unit actually plays a helpful role in both guiding program construction, helping with readability, and even reducing unnecessary memory allocations. A Primer If you…
With the growing number of resources to learn Rust nowadays, where does one even begin? How does one get into writing idiomatic Rust code as quickly as possible? If passively consuming content isn't for you, here's some tips and ideas to kickstart your carcinisation . Tips For Breaking Down Your First Bit Of Metal You're excited to learn Rust and you want to build something; awesome! Here's the…
Have you ever spent a considerable amount of time tracking down the meaning of an error flag or code after a program has crashed? In languages that don't let you break down values with pattern matching, booleans and error codes run rampant and require extra investigative effort on the part of the programmer. Diagnosing problems in programs doesn't have to be hard in Rust given we have Result to…
In the past I've advocated for the default use of criterion in Rust projects as the but it isn't always the fastest to run in a development loop for quick feedback on optimisations. First, let's make our release builds incremental by specifying the following in your project's Cargo.toml file. [profile.release] incremental = true Next, let's setup a baseline benchmark. I've made a template here you…
In imaging, there's a notion called "separation of tone" or sometimes curtly referred to as "separation". Images that clearly distinguish objects from one another possess this quality. Achieving clear tonal separation enables the author of an image to put more effort into composition of the contained objects. Separation and composition are also ways to think about system design. Systems are,…
Imagine you have a generic collection that holds values, but the struct mentioned in the generic is not the actual values of the holder (astute observers will realize this is a bit like ArrowPrimitiveType in arrow ). : trait NumberLike { type Native; } impl NumberLike for Int64 { type Native = i64; } impl NumberLike for Float64 { type Native = f64; } #[derive(Debug, Clone, PartialEq)] struct…
Have you ever felt like you could use some sort of lifetime visualizer or debugger for tricky borrow checker issues in Rust? Has building up a mental model for complex lifetime interactions and their rules felt like one of the hardest things about writing Rust? I am by no means impervious to particularly tricky borrow checking woes, but I have built up a way of thinking about lifetimes and…
Changing function interfaces to get different return values is a chore and can weigh down refactoring sessions. There is a simpler way to "slot in" different functionality based on types and traits! If you've used Rust long enough you are likely aware of the pattern with iterators where you can collect into different collections based on types. Iterators themselves don't have to keep around all of…
Assembly is the last "human readable" frontier before reaching the representation of a program that a machine consumes. Times have certainly changed; the PDP-11 had only 16 instructions in total, whereas the Intel x86_64 and x86 manual is well over 2000 pages . You may not ever need to write assembly yourself, but if you ever want to understand what comes out of a compiler, you'll need to know how…
Incremental builds can get huge in Rust and there is no builtin way for them to get pruned over time. This can be particularly important for developers or build bots working with a limited sized disk drive. I have seen some build caches on projects I work on in production into the range of several gigabytes and this is the size after compression! The easiest way to keep these projects clean is to…
Not all profiling experiences are alike. Some are filled with friction around the tooling. Others are around doubt about whether or not intermediate layers are inflating or shifting numbers in unfair ways. Perhaps you work in a security or data-compliance critical environment and all you want is numbers on what is running in production without having to breach agreements by downloading live data…
If you lived in a village with only one mode of transport, say, a single car per family, then there would be resources that take longer to get at then others. On a computer, the same is true, where the village is the CPU, and the resources are things where data lives or is communicated over, such as a disk drive or a network socket. Things in your own home are quick to fetch; these would be the…
Are you confused about what to learn when it comes to the fundamentals of programming? Do you constantly get told that algorithms and data structures are the gold standard for what will help propel your understanding? How do you get better at designing solutions rather than operating preexisting ones? Most of what people try to shill as fundamentals does not end up empowering people with a…
I consider Ben Eater's series on building an 8-bit CPU from scratch to be a remarkable learning journey that anyone keen enough to better understand the hardware they build software for should watch. I liked Ben's course so much that I thought I'd write a little mini summary of it. This isn't trying to be satisfactory replacement to Ben's video course and I strongly urge you to go watch the whole…
A problem has happened due to some offending code landing on your main, production branch. You use git and your best bet is to keep rolling back commits until the system finds itself in a steady state. You come late into this picture and you're unsure how far back you need to go. Firstly, you ought to be using something that alleviates the need for running through out an entire CI pipeline in…
I recently had to feature flag some code in JavaScript and felt myself wishing I had Rust's expression-based blocks. In the JavaScript code, I didn't want to break out the logic into its own function or module as the code was a proof of concept and deciding on an interface early on would distract me. In languages that aren't expression and block oriented, you have one of two choices: Use an…
Cargo's initial project layout is good for bootstrapping a project but as time goes on there is a growing need to automate chores, wrestle with compile times, and increase discoverability for maintainers and contributors. I've previously written about how I personally orchestrate chores on a Rust project but this article will focus on the latter two points. The highest leverage act you can do with…
Occasionally you want to pull out a specific field or variant out of one or more enums. Doing this with pattern matching is tedious and verbose, but there's a simple way to use methods and combinators to do exactly what you want. There's a pattern I use to solve both of these. If you already have the collection in hand, you can write a simple method that operates on the type named something like…
What's the point of pattern matching if we already have conditionals and variable assignment in a language? Pattern matching helps tease apart values and construct control flow using the shape of data rather than bespoke logic, methods on types, or special fields on a struct. For example, in languages that don't have first-class support for sum types, enums in Rust, you'd have to encode the…
Performance is something users feel . Maybe you're in an organization that doesn't care about performance, and you think they should. Perhaps there are lofty claims that performance does matter, but loose arguments for one approach over another filled with baseless claims like "X is fast !". The only way to get past this conjecture is with numbers, and we collect numbers with benchmarks.…
Compilers are complicated beasts. Our high-level source code goes through many transformations until it winds up becoming machine code that runs on real or virtual hardware. Assembly is the final destination before machine code, and it doesn't have to be menacing! Whether you intend to write assembly directly or not, knowing how your code translates to assembly can drastically improve your ability…
At the start of a program, it is straightforward to clone data all over the place to get things working, and, soon enough, the program is overrun by them. Switching away from clones can be hard because it requires fighting with the borrow checker, and alternative solutions aren't quite right for the job. How do you cut down allocations from cloning as if you were borrowing without winding up in…
Do you find it far too easy to reach for panics or shoehorn pre-existing errors to fit your needs? Is it unsatisfying that there are no exceptions in Rust and challenging to adjust to handling errors with Result ? Here is a fundamental method for modeling data that will help untangle error handling in your programs. Programs laden with unwrap , expect , assert , and panic are quick to gain…
Rust's standard library offers a lot of neat dynamically-sized data structures for use in Rust programs. They are quite performant, but the allocations they conduct behind the scenes to grow may add up and cause performance issues in your programs. Rust intentionally avoids costly uses of new in a program by having the allocation be empty by default, including types outside of std::collections ,…
We call the wildcard variable, denoted by an underscore ( _ ), the "don't care" variable to throw away values we don't care to keep. Wildcards don't bind any values, so wildcards have specific support in the language, as opposed to other languages where an underscore may be yet another variable name. I'll discuss three ways bugs can lurk innocently behind wildcards. Wildcards are useful, but…
Are your errors type devolving into grab bags with varying degrees of categorization? Frequently who is at fault is not clear, and that can be one of the most ergonomically essential classifications. If a program makes it clear that an error is due to a user mistake, an internal complication, or possibly a bug, that dramatically eases the usability of the service/library/tool in question. Making…
It would seem natural to always pick stable Rust, but how much awesome new stuff do beta and nightly have and how unstable are they? It can be confusing that such a plethora of feature flags sits in nightly but we don't want to sacrifice stability. Stability is a guarantee that something won't change. With that said, unstable features theoretically have no guarantees, but in practice there is…
Having crates panic on you feels random because the specific conditions that trigger the panic may not seem clear. Having external crates bring down your program is a pain, but there is currently no static analysis tool to help us easily find panics in external crates. You can get pretty close with fuzzing, though! The format for fuzzing is generally: Get some random bytes Shape them into the…
Iterators are handy ways to describe the potential for looping over data but without eagerly evaluating it. In other words, we can describe the shape of looping over things but we only do work when we call something like collect . The compiler can do several optimizations around iterators and each collect needs to allocate memory to store our results. To avoid a lot of needless allocations it pays…