Engineering judgement and the Claude Code paradox

Prelude: notes on Claude Code

Brecht's Dreigroschenoper puts it best: "first the grub, then the morals.". Being a trans woman who lives largely off a mixture of writing income and money from freelancing or mentoring engagements, my access to grub is somewhat precarious (more so than usual in our current tech economy), and I dream of moderately lucrative tech jobs and long contract engagements. Of course, in this AI age of ours, these are precisely the things that are hard to find, and this holds especially true if you've, say, previously expressedextreme scepticism about LLMs and LLM coding tools in public forums.

So it came to pass that, being at a bit of an impasse, I picked up Claude Code and the cheapest Claude Code subscription I could find, hoping against hope that maybe it'd be good enough that I'd be converted, I could do the whole "Hallelujah, I've seen the light!" thing and embark on a long and prosperous (or at least prosperous) career as an AI grifter in the Gas Town mould. Unfortunately, this didn't happen.

I will say that Claude Code is a noticeable step change over the Opencode-based GLM models that I tried a few months ago. Those were bad enough to not be usable for anything serious or of real complexity. Claude Code, by contrast, can do some relatively complex things: it was able to perform the bulk of my website migration from Nuxt 3 to Nuxt 4 and to the newest version of Nuxt-Content for instance, which the GLM models had utterly barfed on. It wasn't perfect: I had to do quite a bit of tinkering after the fact to make things work, but unlike the GLM attempt, Claude did actually save me some time. Claude also did pretty well with adding Schema.org markup to my website and performing a number of other stupid-but-necessary SEO-related tasks on the same website: none of these tasks are really groundbreaking, but they're the kind of relatively complex pain-in-the-ass task that you'd hope we could automate: at this point in my practice, there's little of value that I can glean from doing those tasks by hand. If it's possible to make this kind of tool accessible to people at an acceptable, sustainable cost and without burning too many resources, then, I think that'd be a good thing.

Claude Code also does quite a bit better than Opencode tools did at CI/CD tasks, IaC in OpenTofu, building Dockerfiles and that kind of task. It's still something I'm quite iffy on, but unlike the GLM models I tried earlier, what Claude Code produces basically works, at least initially. This probably makes it a lot more immediately useful than the GLM models were, as this kind of task is a large part of what tech workers actually do day-to-day, and it's the kind of task that a software engineer might have to perform while not having a very good grasp of the underlying technology.

Many of my earlier criticisms of GLM models still apply to Claude Code, though: despite what people keep saying about the models working better on greenfield projects than on existing projects, I'm still not seeing it. Claude Code consistently makes some very strange architectural decisions, it still struggles with consistent code style and I still don't quite know how maintainable the code it produces will be long-term (GLM code often behaves well initially, but once you've left it running for a week or so you begin to see increasing numbers of strange bugs pop up, and eventually the code becomes unusable and you have to go in and fix it manually). As mentioned earlier, I did have to perform quite a lot of manual tinkering to get exactly the code I wanted as well, though I will say in Claude Code's favour that the code it generates is significantly easier to understand and thus more amenable to such tinkering than what GLM produced.

Taking all of that into consideration, my guess would be that Claude Code is currently at the technological level of a machine lathe: it's a useful internal tool for engineers to have on hand for some tasks and it allows for an engineer to make things significantly more complex or involved than they might be able to without one, but to get a decent result out of one requires considerable knowledge (more on this later), and without it you're as much liable to lose fingers as you are to produce anything worthwhile. It also, despite appearances, doesn't actually make you meaningfully faster at building real systems than doing it manually: it expands the range of systems you can build, but getting the system to the point where you can release it externally still seems to take about the same time as it would otherwise. Take the version migration for example: I was able to do it with the aid of Claude when I otherwise wouldn't have thanks to a lack of mental energy, but if I'd actually had the spoons to do the damned thing manually? It probably would have taken about the same amount of time and I'd have gotten a better result to boot.

Moreover, looking at my outcomes compared to what other people get, it's hard not to come to the conclusion that in one way or another, I'm unusually good at getting adequate results out of these machines. I keep them on a tight leash, provide a lot of architectural input and model code, I tell them exactly what libraries and frameworks to use, I'm usually working on established codebases and I have approximately zero compunctions about rewriting large parts of what the coding agent generates (for that matter, I also generate code in very small chunks). None of this comes about because I'm particularly clever about using the coding agent: it's because I trust it significantly less far than I could throw it and I am not letting it do anything without being very sure that it's not going to do anything stupid (and even then I feel bad about giving it the access that I have). Coincidentally, though, it seems that this leads to better results than people who are outwardly a lot more optimistic and willing to try things get, at least from an engineering perspective: when they produce things using Claude Code it's often unstable, insecure, not fit for purpose, difficult to deploy or just flat doesn't work.

Looking at my peers, this seems like a consistent pattern: the people who get the best engineering results out of coding agents are not only not the boosters, they're the people who think the whole enterprise is dubious and that basically everyone who manufactures these tools is deeply morally compromised. This is a downright weird phenomenon: while a similar thing has happened with other technologies before, it's surpassingly uncommon, and for most technologies, the people most excited about them are also the most adept at using them effectively. What the hell is going on?

Code that shouldn't be written

We can cast some light on this phenomenon by looking at what exactly it was that I was getting Claude to write. Notably, none of what I was getting the machine to generate was core business logic: the route structure was handled by Nuxt already, as was the CMS, I wrote all my database operations and data models manually (and building a good model of your application's data is one thing that is both extremely important from an engineering perspective and that coding agents are consistently bad at) and the content, obviously, is something that I wouldn't let an LLM get even remotely close to. Rather, it was almost all pain-in-the-behind cruft that we need to write but that adds very little value to what I'm shipping from a systems perspective.

Schema.org markup, for example, is something that we could reasonably do without in a sensible world. It adds very little of value to someone actually reading one of my blog posts. Unfortunately, the main ways that people discover my blog posts and the business website that they're attached to at the moment are a) social media and b) search engines (as enshittified as they are). Social media needs Schema.org and og:image tags to present my blog posts as a rich result: without them, significantly fewer people click on my posts, which means that fewer people read them and that, eventually, hits me in the pocket. Similarly, search engines need it to be able to infer (given just how enshittified both the internet and search itself have become) what my pages are actually about, link my disparate presences on the internet and once again show rich search results that people are more likely to see and click on than they otherwise would have. Similarly with the migration, while some amount of breaking changes are probably inevitable in the maintenance of any large software framework or library, there's little direct value that the migration offers: for the most part it fixes bugs and organises things in a more sensible manner than was done previously. A well-designed search engine and better-designed social media could plausibly design out a lot of this cruft, eliminating the need to write it, and while Nuxt and Nuxt-Content are pretty well-designed, I suspect that I would have had to write much less code during the migration process had I maintained my codebase more in small, regular chunks rather than putting it off until there was a lot of work to do.

From the perspective of someone coming from a much more physical engineering discipline, this is quite simply bad systems design. A lot of this work is, in essence, writing an ad-hoc, messy and ill-defined compatibility layer that's meant to match a system that's constantly shifting and utterly lacking in stability: it's as thought you're trying to design consistent pipe connectors between a distillation column and a catalytic cracker at an oil refinery when the catalytic cracker keeps on changing its design every other week and the width of the pipes isn't firmly defined at all. The vast bulk of the code we write is, in fact, glue code of this kind, desperately trying to make disparate system components work together when the interface between them was designed poorly to begin with and now keeps shifting on a regular basis. While some amount of this code is inevitable (things never fit together perfectly, even in the real world), the goal should generally be to write as little of this code as possible, and in general, with good data modelling, that's a lot more doable than people tend to think.

LLM tools are good at generating precisely this kind of glue code that, with better engineering of core systems components, we wouldn't have to write in the first place and that, in some sense, shouldn't be written. This, I think, explains our observed phenomenon. Producing robust, secure and above all useful systems simply isn't a question of coding: it's a question of engineering. You need to know what data you're keeping track of and what you want to do with it, how you're going to store it and how you want to present it when it's done. You need to choose components that are appropriate, that do what you need them to consistently and that interface well together with a minimum of glue code. This means that to build a truly good system, with or without a coding agent, you need really strong engineering judgement, and it's doubly important when a coding agent is involved. This means that people with engineering judgement, who make good systems choices, are going to get better results out of the coding agent because they've made those choices deliberately and constrained the coding agent to fit it. This means that they tend to get less code out of the agent, but it's code that is significantly better and that does what they want it to do.

However, this same engineering judgement is going to lead to people being a lot more sceptical of coding agents in the first place because, of course, we can see that when you aren't keeping it on an extremely tight leash, it does silly things. We're hyperaware, in short, of all of the things that it can't do well, and all of the bad things that it does when we don't explicitly tell it not to. Moreover, between the good judgement and the sheer manual skill that most of us pick up in producing code, we can usually write what we want to pretty fast without the aid of the agent, and the stuff that the coding agent is useful to us for... well, we're generally somewhat pissed off, to a greater or a lesser extent, by the existence of the thing at all. Sure, Claude Code might be great at sorting out the whole Schema.org thing, but we'd much rather that Schema.org didn't exist at all so that we didn't have to write it in order to be minimally competitive in a job market that's basically turned into a content creator economy. This means that even when we acknowledge that a coding agent is useful for something, we treat the agent with a level of barely-concealed resentment because we don't want to be living in a world or working in an industry where what it's good at is valued anywhere near as much as it is.

This explains well enough why the people who are most sceptical of coding agents are often also the people best-placed to get good results out of the damned things. This, however, probably isn't enough: after all, while it makes sense that people who currently have good engineering judgement dislike LLM tools, the question of whether people who have good engineering judgement and still like the tools as presently constituted remains open, and I imagine a lot of engineering managers would probably like it if they existed or could be created. Personally, I have my doubts.

The elements of engineering judgement

To explain why I doubt that sound engineering judgement is compatible with the current state of coding agents and Large Language Models more generally, we have to explain what we think engineering judgement actually is. Obviously, this is difficult: judgement of any kind is mostly intuitive and unless you have some experience of it, it's very difficult to communicate in anything other than a teacher-student type situation. What I've tried to do here is explain how engineering judgement is built and, using that as a lens, discuss the effects that that's liable to have on attitudes to LLMs.

Developing good engineering judgement begins with having some experience of coding or of building things. As I'll discuss later, I don't actually believe you need that much experience, but if you have no experience at all, none of the other ways of developing judgement will work. Reading books, for example, can be very good for developing engineering judgement, but if you don't have a baseline experience of actually writing code, you will have no common experience to draw on when understanding the text, and in the end, you'll get nothing from it. Similarly, it's hard to learn from more experienced engineers or coders unless you have some experience of building systems yourself.

I don't think you need all that much experience: six months to a year of working on your own coding projects will usually get you to the point where you can start learning effectively from other people. What I will note, though, is that the type of experience is important. You really need to be working on your own projects, where you choose the technologies and the architectures that you use, and you need to actually deploy what you work on. Working on uninspired code projects at the bulk of tech companies, for example, is unlikely to get you the experience you need: not only are you working on small parts of a system with little understanding of how it fits together, you lose any feedback mechanisms that a smaller project would have. The watchword seems to be responsibility: you have to have worked on writing and deploying software products that you're responsible for, and where you have to deal with the consequences if they break, even if you're the only person affected.

Having that baseline, you can begin working with other people. None of the people I know with truly good engineering judgement have built that alone: all of them work in the context of a community of other people working on building that same engineering judgement. For some of them, it was working at a place like Pivotal Labs where they could learn from highly capable seniors. For me, it was largely peer education via a shadowy Discord cabal. Some people might even learn a lot from going to meetups and such, though this can be hit-and-miss, or from interacting with capable engineers over social media and blogs. On a similar note, all of the engineers I know with truly good judgement are extremely well-read. Ludic's article You must read at least one book to ride puts the phenomenon pretty well: with enough experience under your belt, you can learn a lot from reading a sufficient number of books: they are, after all, written by people.

The whole process seems to be as follows: you begin by trying some things with your technologies, making mistakes and learning to correct them, which gives you some idea of what the shape of the field is like and how things tend to go wrong. Having that impression, you can then learn effectively from talking to other people and reading books, which gives you access to a much wider set of experiences and situations to learn from than you'd have access to on your own. This then lets you work on more ambitious projects, make more recondite and uncommon mistakes and lets you learn more about the shape of the field. Rinse and repeat that for a few cycles, and you'll quickly build up a strong intuitive sense for what works in practice, what doesn't, and what the best way to build a system that achieves certain things is: engineering judgement, in short.

While everyone can go through this process and develop the sense for what makes for a well-engineered solution, this process does shape everyone who goes through it in some ways. First of all, one develops an acute awareness that, as Pavel Samsonov put it in an excellent article, when you do engineering you're building a system made up of multiple components, including human ones. When you develop that, seeing code as a thing that mediates between system components or as a constituent of a component rather than as an undifferentiated product starts to come naturally, which will naturally alter how you see coding agents. When a coding agent is producing code, which is the thing of value in itself, they look quite attractive. When you're using the agent to weld, bolt or rivet together two existing components, or to machine a new one which is going to sit in a larger system, the tool begins to look quite different, and honestly, much less attractive.

Secondly, the fact that building a system means, to an extent, taking responsibility for it, means that the bar for what you put out into the world becomes much higher when you have good judgement. You become, all things considered, a lot pickier about what you put out into the world when you realise that it's going to be you or someone in your position who's going to have to fix the problem when it breaks. This has two consequences: firstly, you become quite attached to understanding how exactly your code does what it does and how it's likely to fail. You want, more than anything, to understand what could happen and prepare a plan for if it does. You also flip your perspective on code production: where someone who's merely interested in writing code is happy or scared that the coding agent can produce more code than them, as an engineer you want as little code, as few components and as few moving parts as possible: each component and each line of code introduces the potential for failure. In short, you're going to develop an acute sense for when not to write code that shouldn't be written.

All of these points are going to introduce a dislike of coding agents in their current state. After all, the agents are overly verbose, unreliable, opaque when subject to analysis and have a tendency to prioritise the production of code over the design of the system. If, in this situation, you're going to use them at all, they're going to be used in a highly constrained manner, told exactly what to do and simply not used for certain critical tasks: a far cry from the claims of the vibe coders and everyone who tells us that they're going to revolutionise the profession. All told, you're liable to realise that what the coding agent is good for is mostly writing code that you shouldn't be having to write in the first place, and consequently use the coding agent only for that and as little as possible.

This might, of course, be all so much hot air if volume of code was what actually mattered. However, it doesn't: it's still well-engineered solutions that fix a human problem that are what's actually valuable from a social and economic standpoint. The moment you start drifting away from that (Microsoft Windows, Microsoft Office, Google, every other damned tool that's had AI shoved into it somehow), people will stop using whatever you wrote, and you'll consequently start losing money. Socially and economically, you can't get away from the need for engineering judgement.

The contradiction is (to some extent) systemic

This leaves coding agent adopters with a problem. In order to do something that's actually worthwhile with the coding agent, you have to have good engineering judgement: the better of an engineer, the better outcomes you get. However, the better you get at engineering, the more suspicious you'll be of the coding agent, and the less likely you are to use it. Moreover, this is a systemic issue: as established, the people best-placed to use a coding agent to do good work are precisely the people who are the least likely to touch the coding agents at all, and to use them the least when they do use them. This is the fundamental tension at the centre of the current pro-AI push in industry and in the tech community.

There are a few possible resolutions to this. The first, and probably the best one, is acceptance: as a company, you might simply have to accept that the best engineers won't want to use coding agents much at all. In that situation, if you're dead-set on having people use LLMs for some reason, you might have your best engineers work on architecting the system, building the data model and working on defining and constraining the system as a whole. With that work being done, you can then get people who are more willing to use coding agents to fill in the blanks, do the stuff that annoys the good engineers but that you feel that you need to have for one reason or another and get them to extend the initial work within the constraints that your better engineers have built. You could also not use coding agents at all, but if you're an organisation of any size, well... I think you shouldn't use coding agents. I think I've made a good case for it across much of my writing. But realistically, you're going to call me a Luddite and do it anyway. Accepting that your best engineers are likely to be suspicious of coding agents, and that you need them anyway because even working with coding agents requires their skills, at least limits the damage.

Most people and organisations, unfortunately, have taken the second approach: force people to use the damned things on pain of firing. This is having the obvious effects: people with sound engineering judgement are being driven out of organisations, and those organisations are consequently going the way of Boeing. Any sense of code as an engineered system that involves humans and solves human needs is being lost, and people primarily focus on the volume of code produced far more than whether the code actually does anything of value to us. The extreme case of this is exemplified in projects such as Gas Town: it's currently unclear as to whether Gas Town and its derivatives have done anything other than create more Gas Town derivatives, and the whole enterprise seems as though it really isn't aiming to produce anything useful other than feelings of intense productivity for the people involved. Even in less drastic cases, building systems that deliver value tends to go out the window as people with a systems engineering mindset get driven out of organisations in favour of people who are, for the most part, easily impressed by volume of code and the intense feeling of productivity that they engender, and you can see the results in almost every software product you use.

If we began with Brecht, we end with Zizek's paraphrase of Gramsci: "The old is dying and the new cannot be born: now is the time of monsters". The coding agents have finally broken the old model of the tech industry, bloated on capital and already exhibiting most of the flaws of LLMs before LLMs were even a thing. And yet new ways of building code (whether they end up using tools like coding agents or not) cannot be born, because we're all drowning in a sea of vibe-coded slop. The end result is the proverbial monsters: technology that becomes increasingly hostile to anything that might be remotely human. If we're to clear the monstrosities away and figure out what our way of building code is going to look like, we would do well to preserve as much engineering judgement in our spaces as we can.

If you are interested in developing stronger engineering judgement, this is a field in which I offer mentoring and I've had many happy customers work with me on it at this point. If you'd like to join them, get in touch here.

If you lead a team that's wrestling with this at organisational scale rather than as an individual practice, I also offer a fixed-fee AI tooling audit — an honest written assessment of how LLM tooling is actually working inside your codebase and engineering process, with a prioritised remediation plan.