Let’s play a game. Warning: it is not as fun as Ghost Recon :: Wildlands.
I want you to picture the following:
office fire marshall
I’ll create some space for you to not see something:
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
You likely thought of someone like this:
If you aren’t familiar with this character, their properties are:
Good at planning and organization
Takes everything very seriously; things must be done the Right Way
Enjoys having power and uses it to full effect to make sure everyone is following The Rules
she says that she "enjoys being judged" and that she participated in beauty pageants as a child
Fire marshalls are good people that work hard to prevent or react to something that is unlikely to happen, like a fire at your office, or a security incident, or a fault in a computer program setting a bunch of money and careers on fire.
Now I’d like you to imagine this:
fire fighter
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
tick tock
You likely thought of something like this:
Hero
Cool outfit
Mysterious, can’t see face, might carry an axe
Reacts to something that has happened when others aren’t sure what to do
The Truth
Fire marshals (and inspectors) are made fun of behind their backs, and firefighters get sexy calendars. If you prevent problems from happening or make everyone 10% less likely to create a problem, you are not rewarded like the people who clean up the mess. This is because management acts on incomplete information; they work from afar and cannot hear whispers.1 Only people deeply involved in the day-to-day work know about problems that were prevented, avoided, or solved while they were small.
This is the world you live in, and you need to get used to it.
The Shape of Teams
But on software teams, the team that inspects and builds the house is also the team that fixes it when there is a problem. This would be like a fire inspector, who also shows up if there is a fire. Overall, this is A Good Thing and should lead to fewer fires, as they learn from their oversights and mistakes.
Software teams:
BUILD: write features and bugs.
TEST: find bugs, make sure features work.
SUPPORT: fix faults that make it to production, and recover from failures2.
Different teams have different strengths and weaknesses across these three things. Let’s look at three different teams:
Team 1: Ships a lot of features, weak testing, lots of support
Team 2: Ships less, and tests really well, little support
Team 3: Ships almost nothing, vigorous testing prevents most errors, but support is still a nightmare
Thanks for reading Bad Software Advice! This post is public so feel free to share it.
Anti-hero
Team 1 has an anti-hero. A flawed hero. A firefighter that maybe starts some fires by their carelessness and their confidence in being able to fix things.
If you have someone on the team who is very good at fixing problems (or more than one), you can have a lot of fires. Like, why are there so many fires. The next team over doesn’t have this many fires. They are terrible fire marshalls, but great firefighters. People play to their strengths subconsciously, even creating the situations in which they can shine, without thinking about it.
This person can be very dangerous to certain teams, and absolutely essential on certain teams. Where you place such people matters a great deal.
Extra Employees
You might be thinking, “but David, what about systems with a lot of faults that require greater support?” or “when is lunch?” or “Team 3 doesn’t make any sense, why is there so much support?”.
In my example from above we assumed that all teams had some baseline that they started with that was equal. But teams support different things, and the team needs to match the task. Existing code has a huge impact here - building a Version 1 of something is very different than building Version 38.
Legacy systems have grumpy employees that you cannot fire. They are called The Existing Code. They affect the shape of your team and how you should design your teams.
Let’s use some terrible metaphors.
Your existing code is a large nearby planet that provides gravity, affecting all your actions.
It is geography, changing where you can live because of floodplains.
It is a starting point that is the 4th bar on our graph:
Not that we have made the code quality poor on all three teams, which team will fail?
Team 3 has terrible starting code quality, and has a lot of support already, and weak testing. What should it focus on? Improving testing and support to make sure the fixes are real fixes.
Team 2 is more mature and can ship features, because they have strong testing.
Team 1 is in a death spiral - they have bad code, ship a lot, and have poor testing. Their support could double overnight with a bad release. Uh-oh. Only one thing can stop a death spiral. Time for a hero.
Building Teams
Personal opinions on how these interact vary, and it is a fascinating thing to stare at and think about.
What do you need to increase if you move from senior developers to junior ones? More testing, or more senior support?
What are the relative sizes of these things? If initial code is 10 million lines, and you have two developers supporting it, what do you need more of to keep two developers and not have to hire more people?
At what rate can you improve these things? Hiring stronger developers: how quickly does this make an impact?
Which of these items requires the most, highest-quality teamwork?
If you have a pre-existing, high-quality team, which situation do you drop them into?
What is the effect of making the goal to reduce support, with no other changes - more testing, less features? More experienced staff?
What happens when you cover up weak QA with strong development and automated testing?
How do good documentation and training help?
If you have to be very weak at one, which one would you choose?
Great taste, less filling?
One thing you can do immediately in almost any team scenario: make sure to have a balance of fire marshalls (that are rewarded) and firefighters (that are also rewarded). Perhaps they rotate jobs; you need a balance of both. If you are just starting a company, most people are going to have the firefighter personality and be weak planners. But the firefighters need to establish the first broad rules: no grilling on wooden decks, no smoking at gas station pumps, and no flamethrowers on hayrides, etc.
Established companies are going to have more existing control, process, and organization. That is fire inspector/fire marshall territory. Moving from one to the other requires rotating people around so that firefighters don’t work on the systems that become stable thanks to their help.
Take an honest look at what system you are dealing with, and what plans you have for it. Do you need to save this building? Do you need your best builders/testers/fixers here?
There is much more to building a team than deciding who doesn’t like sitting next to each other.
Bad Software Advice is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.
Being a great manager is about establishing a great intelligence-gathering system; that is a topic for another post.
I’m using Gerald M. Weinberg’s definitions here:
fault is a problem in the software (~bug).
failure is a problem caused by this fault.
Normally (number of failures) > (number of faults). Support recovers from failures, and sometimes also fixes faults.
A fault would be a bug that fails to release some memory, eventually causing a memory leak. Failures occur each time the system stops working because it runs out of memory. Support sometimes means the person who restarts the system four times a day, and sometimes it means the person who fixes the memory leak. Sometimes the person who wrote the memory leak is the same person who didn’t test it well enough, restarts the system 4x a day until they fix it, then doesn’t create a test after it’s shipped.

Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.