Failed software projects are strategic failures

This article was very kindly sponsored by Hermit Tech, who sent me some money so that I could have a month or two to look after my mental health and write a few posts that would be useful to them without stressing too much. They are very capable data engineers (and other kinds of engineers too, but as I understand it, they mostly focus on data), and both wise and good people besides that. If you have pressing needs in the data space, I unreservedly recommend them to you.

My first software job was for an acoustics company in New Zealand, who've worked on quite a number of high-profile projects in their time. As a new intern, fresh out of university and working through the work experience hours I needed to graduate, I was tasked with implementing some new sound propagation models for a piece of software that the company maintained.

This was not an amazingly well-written piece of software. It was written, for a start, in Embarcadero Delphi (for those of you ignorant of what this means, it's essentially object-oriented Pascal). Even ten years ago, this was an archaic language, and looking back at it now, it's even more dated than it was then. The GUI builder was of the VB.net drag-and-drop variety, the syntax is noticeably different from C-like languages in ways that consistently trip you up, and there's a whole slew of what I can only describe as bad design decisions (there's no variable length array type and no way of defining one, which, if you're building a graphics program for FEM work, is kinda a problem). Even taking the language out of the equation, the code wasn't version controlled, there were no unit tests and the codebase was essentially unstructured in the sense that it was just a pile of functions in a couple of massive files. Beyond this, from an organisational perspective, finding Delphi developers in this day and age is bound to be a real challenge. If we were to judge the software purely by code quality and development practices, we would quite fairly judge it to be terrible.

The software, however, was one of the more popular tools for acoustic modelling in the world, and has been used to design an awful lot of skyscrapers, office blocks, apartments and other such things. It's relatively easy for acousticians to use, gives accurate results and consequently, it sold really well. From a strategic perspective, or even an engineering one, this software was thus clearly good, no matter how shoddy the underlying software engineering was.

We might compare this to the Australian Bureau of Meteorology's recent rollout of the new website that Accenture built for them. That software project used similarly dated technologies (who in this day and age uses Drupal for a new build?), was similarly developed by people whom I would not consider to be good software engineers, and while it's plausible that this project used version control, I very much doubt that they used it at all capably or effectively. Unlike the acoustics software, though, the BOM website failed in exactly the ways you'd expect it to and left the Australian government holding an expensive, messy failure.

So if projects that are shoddy from a software perspective can actively succeed, even when the software engineering is really quite bad, why don't they? After all, there's a lot of shitty code out there, and even if only a very small proportion of projects with bad code succeeded, there'd still be an awful lot of properly useful software out there that just happens to be shoddy. And yet we don't tend to observe this: products with shoddy code but important use cases are, on the whole, pretty damn rare.

The reason for this is fairly simple: software projects tend to fail on the strategic level, not the execution level.

What does it look like for a software project to fail?

There's an inaccurate but persistent perception in industry that software projects fail because of failures of execution: the software equivalent to a military failing because, on a level of soldiers or squads, they're tactically unskilled. It makes sense, to some extent, that this would happen: leaders and non-technical staff don't understand what goes into writing software and tend to see it as a recondite, almost arcane skill, meaning that they can't tell what good or bad work looks like. In such a situation, the temptation to blame the people doing the scary work that they depend on but don't understand becomes irresistible. On the software engineering end, we consistently get frustrated by people doing work badly, not writing good commit messages, not using version control, not testing and all of the other irritating things that make for a software engineer who's unbearable to work with. By contrast, unless we work at a particularly unusual workplace, we don't usually have a huge amount of input into or contact with the strategic underpinnings of the projects we work on. This means that once again, it's easier for us to blame failures on the idiots who wrote some unholy spaghetti-code abomination than to attribute fault to anything else.

The thing is, projects don't usually fail like that: I'd be hard-pressed to think of any projects where the strategic underpinnings of the project are sound, the supporting logistics and suchlike behind the company work as expected and the project simply fails because despite all this being in place, the software engineers assigned to the project just aren't good enough. What usually sinks projects are mistakes like a lack of clarity about what a project is actually meant to achieve for a business, a failure to properly understand requirements, under-resourcing or a failure to provide missing capabilities, poor management and organisation and a failure to update the strategy underpinning the project when conditions change. These are all strategy-level mistakes much more than they're tactical ones.

Let's look at the acoustics software for a moment. Why did it succeed despite the software in itself being awful? First off, there's a clear strategic need for the thing: we need to design large buildings, and to do that we need to understand how sound propagates through the building so that the guy working night shifts doesn't find himself having to listen to the couple next door doing their duty to the nation night after night. Given that doing the calculations manually is both error-prone and tedious, there's an obvious reason why we'd want software for this. Secondly, while the company building this didn't exactly have the best software engineers, they do have some of the best acousticians in the world on their staff and they're pretty effective concrete-and-steel engineers. This means that they a) have the domain knowledge to accurately assess whether their software is good enough for their purposes and gives accurate results and b) they can organise a project effectively, set reasonable timelines and effectively do the logistical work that it takes to deliver a software project. With all that in place, then, even indifferent skill in actually writing software was able to lead to a successful result, in the same way that effective strategic and operational thinking can allow a good general to win military victories even with indifferent troops.

The BOM website rebuild, by contrast, had almost none of this going for it. We might start by looking at what the strategic underpinning behind the project was: so far as I can tell, the inception of the project appears to have been a major data breach that occurred in 2015 and triggered a review of the BOM's IT systems. The strategic goal, then, would probably be something like "ensure that the BOM's IT infrastructure is secure, modern and fit for purpose". This is a fairly standard modernisation goal, and we have, as it happens, a fairly good idea of how modernisation projects operate. And here we can already start to see a lack of strategic clarity creeping in. It's unclear to me how the breach initially occurred, but it seems to me much more likely to have been a phishing attach or something similar than a website breach. Already, then the fact that a data breach triggered a focus on the website is questionable: it probably is the case that the website needed a rebuild, but initiating projects with the wrong motivation is risky, as emotive strategy always is: if you wish to improve the security of the BOM's systems, a website overhaul probably isn't your first-order priority.

Even then, the focus of the website overhaul is questionable. The single biggest security hole in the old website, after all, was that it wasn't served over TLS, exposing visitors to a whole host of potential Man-in-the-Middle attacks and other unpleasant things. If the primary focus was on security, this should have been a first-order priority to deal with. It's also very easy: these days you can basically set it up with Caddy and LetsEncrypt in a few minutes, and they could easily have just done that and reverse-proxied to the existing site. Instead, the current website still has a bunch of pages being served over standard HTTP, which means that the most glaring vulnerabilities are still there.

Eliminating those things and focusing in entirely on the front-end redesign, the strategic value of the redesign is unclear at best: the layout of the old website appears to have been basically fit for purpose. The rationale, as best as I can make it out, was to replace the old back-end with something more modern. This could make sense: many older CMSs tend to couple content and presentation in an inconvenient way, so rebuilding the website to decouple it more effectively and implement a modern API based architecture makes sense and would be wise. Unfortunately, a moment's look at the source code reveals that the main page of the new BOM website appears to have Drupal as a dependency. There's really no way in which Drupal could possibly be an appropriate CMS for a new build in 2025, and choosing it is essentially repeatedly shooting yourself in the foot. Not only is finding competent Drupal devs to maintain this work going to be a massive pain in the behind, Drupal is notorious for being full of security vulnerabilities (the CMS advertises itself as providing "enterprise-level security", so of course it fucking is), and is in general a bizarre, Accenture-worthy choice that actively makes the coupling problem worse. Of course, it could be that the legacy system was based in Drupal and they simply couldn't migrate the CMS to something more modern, but while I might have believed that with a budget of 96 million they could quite reasonably have migrated the fucking thing over.

While each of these points is bad but not a complete condemnation of the BOM's strategic vision, taken together it seems clear that the BOM had no idea what they were doing and no clear vision for how their rework was going to meet their policy aims. They had, in short, no real idea of what a successful rework would look like, and in such a situation, they quite literally had no option but to fail.

Compounding on all of this lack of strategic clarity is the fact that unlike the acousticians, the Bureau of Meteorology clearly lacked the domain knowledge to accurately judge whether what they were doing was fit for purpose. They lacked the UX capability to accurately judge whether or not people could find what they needed to on the website, the security expertise to accurately understand their risk model or the software engineering knowledge needed to accurately identify the flaws in their architecture. Lacking all of this knowledge, they decided (as you do) to farm out the work to Accenture, which any competent engineer would have told them to run away from as fast as possible. And of course, without having at least some of that expertise in-house, they found themselves completely unable to identify that Accenture was either incompetent, actively gouging them or both.

To summarise, the BOM found themselves in a situation where they had a vague strategic direction to go in. However, thanks to an unidentified lack of capability on their end, they found themselves unable to translate that into concrete strategic goals. As a result, they focused largely on the wrong areas in their plan, trusted the wrong people, and were sold tools and equipment unfit for the task (which, thanks to the aforementioned lack of capability, they didn't notice). Thanks to their ignorance and trusting the wrong people, they also found themselves unable to identify missing or neglected areas of work that was sorely needed (the UX research that was performed was clearly inadequate, for one thing, and a more capable organisation would have caught it). There's really no level of tactical skill in writing code or building systems that could have saved this debacle.

This pattern is common. Translating vague policy directions into coherent strategic aims is a difficult skill, and one that our leaders tend not to develop. In that situation, it's easy enough to come up with some vague aim like becoming "AI-forward" or "data-driven" because they're seen as fashionable without giving any thought to how either of those things would look in practice or how they can be put to use to help an organisation achieve its strategic goals. Becoming data-driven, for example, means having a fairly tight, coherent story that tells you how you're going to a) collect data from your business and b) use it to increase revenues, reduce costs or make better choices. As you might have picked up from my last article, this takes a lot more detailed knowledge than you might expect. And once you're at that point, well... it's impossible to even really infer what capabilities you might need because you don't know what you're doing, everything else comes crashing down and you wind up spending $96 million on a Drupal site without even registering how fucked-up that is.

The eagle-eyed among you may have spotted something I've not discussed so far: even if the strategy is sound, sufficiently bad tactics should, in principle, be able to kill a project, and if my argument that software engineering failures are almost all strategic is to be believed, I need to be able to give an explanation for why this doesn't happen. The explanation, however, is pretty elegant: a sound, frequently updated strategy will lead to people making choices that head off the probability of it happening.

Take, once again, the acoustics software that I discussed. The company knew what they wanted and what a good outcome looked like, and thus, when keeping an eye on the project, they had a clear idea of what adequate progress looked like. As it happened, they were able to push through well enough with indifferent code that strategic success was attainable and attained, but if too much money and effort was being wasted on no results, they would likely have either cut the project or reoriented it around different, better-understood technologies. They might, for example, have simply implemented a bunch of VBA modules for an Excel spreadsheet, or used some other kind of low-code solution. Alternatively, finding themselves lacking strong software engineers, they would have been aware enough of the situation to engage some, and they would have known at least enough about hiring them to get people who were minimally capable.

A lot of what seem like tactical failures, then, are in fact a direct result of strategic mistakes. The BOM website situation, for example, stems largely from the BOM not having the built capability they needed to judge whether Accenture was actually capable of delivering what they promised on. Given that, Accenture was essentially allowed to do whatever with basically zero discipline or accountability. In this kind of environment, shitty programmers flourish: this is after all the kind of place that judges programming effectiveness by lines of code. Similarly, absent a clear idea of what software you want the team to produce or what it should look like, it's very easy to burn a whole lot of programmer time and money on bad code which, because it's not at all clear what's actually wanted, just kind of sits around, festering and growing like a tumour. Importantly, from the outside, these situations look entirely like the "bad software engineers made us fail" case that we're discussing, when in reality, this is a strategic failure: if you don't see results relatively quickly, it's probably due to lack of clarity, and if you're clear on what you want and you're not getting the results you need, this is a very strong signal that you should probably either get rid of your dev team or pivot to lower-code solutions that do similar things.

So, given that failed software projects are almost always strategic failures first and foremost, how do you avoid strategic failure?

How to never have another failed software project again

The glib response is, of course, that to avoid strategic failure, you need to know how to formulate a strategy. This is unfortunate, because strategic training and education generally isn't something that we as a society do outside of a few military colleges, which I think is a shame. Now, obviously it'd be impossible to condense a full strategic education into one blog post, but we can share a simple framework that can help you make stronger strategic decisions in the context of software engineering.

The framework I'm sharing was developed by the 19th century military theorist Carl von Clausewitz (drink!). Those of you who've been following along will know, of course, that I find Clausewitz an excellent reference on strategy, and for those of you who haven't been, I highly recommend reading On War. In any instance, Clausewitz begins his analysis at the level of policy or statecraft: this is where you decide what your basic goals are, consider your positioning in your environment and look at your strengths and weaknesses as a strategic entity. In the case of a business that's writing some tech, your first sweeping goal is your continued survival as an entity, followed, in almost all cases, by maximising your total profitability. If you're a non-profit organisation, a b-corp or some other manner of values-based organisation, your profits/revenues will wind up reinvested into the organisation and you'll have secondary ideological goals that you'll want to pursue (usually this is outwardly the core purpose of your organisation), but to a first order of approximation, you're aiming at survival and access to resources in a competitive environment.

Given this information, and with your knowledge of your environment and the resources available to you, you will be able to think of a bunch of different things that you can do in your circumstances to achieve your strategic goals. This might be something like developing a new product line, trying out new marketing tactics, improving the efficiency of certain processes that have been identified as vital: fairly concrete, specific actions, but still on quite a wide scale. In the case of the BOM, for example, the continued existence of the Bureau is dependent on the prevention or mitigation of further major data breaches, so they might think of a number of discrete things they could do to achieve this, such as, for example, using TLS to serve the website. An organisation usually has a fairly hard limit on the number of goals they can pursue at once, so having thought of these things that you can do with your resources, the next step will be to rank them in order of impact and choose a set of objectives that will get you the maximum amount of impact for the resources you have available. This in itself is a fairly complex optimisation task, as different tasks will require different resources: if the most impactful single thing will take up all your resources, but you can do the next four most impactful points on the list with the same time and for the same resources, it might be worth sacrificing the single big thing to get a whole lot of moderately impactful things done. In any case, this being done, you'll have a short list of relatively specific tasks that you think will have the biggest impact on your strategic position.

These tasks, in industry, more or less correspond to project-level objectives. When describing them, the broad outline should be something along the lines of a) the current state of the area in which your project objective sits, b) the end-state you want, or what you want that area of your organisation to look like once the objective has been achieved and c) what barriers exist to getting from a) to b). There will by definition be barriers: after all, if getting from a) to b) was meaningfully beneficial and there were no barriers to making it happen, it would have already been done. Clausewitz calls this barrier the Centre of Gravity: in a military context, it refers to whatever it is that allows your enemy to keep effectively mounting a resistance to political goals that you're trying to achieve through the use of military force: war, after all, is the continuation of politics through other means. The goal, militarily, is to destroy Centres of Gravity in order to take away your opponent's ability to resist and force them to come to the negotiating table and capitulate.

While obviously this isn't usually relevant to situations in industry, loosening the definition a bit gives us something useful to work with. In an industry situation, the Centre of Gravity is whatever it is that is preventing a project from having already been implemented. The goal, then, is to use your resources to neutralise said Centre of Gravity and eliminate the barrier to the objective being achieved. And only then do you start thinking about how exactly you're going to shift the barrier and achieve the goal? Training? Hiring more staff? Partnerships with other organisations? Buying a product off the shelf? Or are we actually going to write some bespoke code?

This probably seems like an awful lot of thinking to do before writing code, but with practice and effort, it becomes a pretty intuitive process. And doing this work almost guarantees that you're highly unlikely to end up with your project failing. The reasoning is simple: you know your current state and your desired end state, and you can see whether the project that you're running is actually getting you there or not. The biggest single impact of this is that you'll simply not embark on a lot of projects that were highly likely to fail, saving you an awful lot of time, money and resources that you can put towards things that are more likely to work. When you do embark on a project, you can be highly confident that the impact of the project will be meaningful, and you'll be able to grasp whether it's actually on track or not much better than you'd have been able to had you not done this work. And if you do hit difficulties, doing this work makes it a lot easier to pivot, develop mitigating strategies or otherwise get something out of the effort you've put in. While in war this is a question of the play of probabilities, and we talk a lot about the fact that sometimes it is correct to take a risk even when it doesn't pan out, industry is more forgiving: if you put some effort into thinking strategically, you can be quite confident in almost everything you do having a positive outcome.

A comparatively small amount of planning and strategic thinking will consistently lead to better outcomes for you, and while we can't guarantee that it'll always lead to you getting exactly what you want, the techniques I've discussed here will make sure that you never, ever blow $96 million on a website that's obviously not fit for purpose. At best, you might conquer all before you: at worst, you will suffer a lot less damage simply by virtue of not gambling things you can't lose.

In the next lesson, we'll talk about the concept of friction, and why it's important to build slack and time for recuperation into your systems when you do execute on a project. In the interim, thank you for reading, and we hope, as always, that you've taken something of value from this piece.

---

Developing the skills and engineering mindset that I've described here is much easier to do with a coach or mentor, and this is very much a service that I provide. If you feel that you could benefit from developing a more resilient set of engineering skills and mindsets, or if you know someone in your life who's about to be thrown into what's a very tough engineering market, get in touch!