Tech ain't that deep - and that's a problem

The seed of this essay came about when I was writing a note and some exercises for my learning platform Arca(we're officially in early access now, by the way!) about pointers and memory management, using C as a demonstration language. I dedicated about half a day to it, and had quite a bit of fun doing it: at the end of the process I had about 3,000 good words of work done, and had another two articles of similar length sketched out. Moreover, I still don't feel as though I've reached the end of what I could productively say about these subjects by any stretch. This is unusual for me: I usually struggle to write much about purely technical details, and in fact find the whole thing quite boring.

Which raises the question: why can I quite happily write 10,000 words about the finer details of pointers and manual memory management in C, but struggle to write a twentieth that number about React? And why is learning C so much more generally useful than learning React? What I mean by the last point is that learning C tends to develop skills that are useful in a lot of other parts of software and tech work: if you know how to write decent C, you'll have a much easier time of understanding PostgreSQL internals, for example, while if you've learned React, it's anyone's guess as to whether or not that'll actually improve your HTML skills, for example.

The conclusion that I've come to is that it has to do with the relative depth of the fields. Depth is obviously quite an elusive concept, but if you've been around for long enough, you will have had the experience that some areas of study, some artworks, some writing... generally any kind of thing that you can subject to examination... in any case, some of these things have more depth than others. Some books you can re-read every year and still find something new, or bear reading line-by-line and underlining. Others, meanwhile, you can read once and then never pick up again, and you don't feel as though you've missed much or have any need to read them again. Some skills you can work at for years and still, even after all that time, feel as though you're improving and learning new things, and perhaps be a little in awe of how far you still have to go. With others, you've learned everything in six months and while you might still be improving a bit, you're hardly improving all that much. I have a definite preference in most cases for deeper fields over shallower ones, and realising that, some pieces began to click into place.

With that in mind, it became fairly clear to me that the tech industry these days has a depth problem: as much as tech is becoming quite a bit more complex, it's also becoming shallower. Learning new things in tech these days, as often as not, simply doesn't give me the same feeling of satisfaction or achievement as learning or writing C back in the first years of my career did. Being effective in tech these days feels as though it's less about understanding than it is about memorising a big list of buzzwords and all of the catches associated with them. And while LLMs have certainly pushed this into overdrive, the issue has existed, in one form or another, for quite a while now.

What do we mean when we say something has depth?

The depth of a subject or a field of study is not something obviously or easily defined. We have some intuitive notion of depth: mathematics is a deeper subject than business studies, just as Ulysses is a deeper text than, say, The Day my Bum went Psycho (sincere apologies to Andy Griffith: I loved the book when I was like, ten, but I've aged out of your audience a bit). Similarly, there's a clear perception in the tech world that writing Assembly or C, for example, is deeper magic than writing Java. if you were to ask for a formal definition of depth in this context, though, you might find yourself struggling.

The things that we call deep, however, do have some clear commonalities: as much as Ulysses, Gravity's Rainbow, group theory and the music of JS Bach are all very different kinds of thing, you get the sense that they have commonalities which aren't shared with Mills and Boon romances or with, say, Cardi B's musical output. This suggests that there are some criteria by which we're deciding whether something has depth or not, and those bear examination. Given that, let's look at some candidate criteria now.

Our first candidate criterion is that of active engagement. Looking at fields or works that we consider to be deep, all of them reward and almost demand careful, deliberate engagement with them. While you might listen to Bach in the background while doing something else, you get a lot more out of something like his Magnificat if you listen to it undistracted and paying attention to nothing but the piece, with a degree of focus. Gravity's Rainbow, similarly, bares reading and re-reading, cross-referencing and paying attention to exact details of sentence construction in a way that you simply wouldn't get from a novel by John Grisham. And of course, you simply cannot make headway in mathematics or the classical languages without dedicating uninterrupted, focused time to them. A deep field doesn't have to be a difficult or complex one (Bach is actually pretty accessible most of the time), but in general, a deep field is one where you can't really half-ass it or get much of it without dedicating effort to it.

Our second criterion is that of unity (yes, this is also an important criterion in the field of aesthetics: I borrow a fair bit from Kulka's Art and Kitsch here). Fields that are considered deep have a their core a comparatively small set of underlying unifying principles that underpin the rest of the field and that the other content in the field stems from. That means that a deep field, as often as not, will have a sense of coherence to it that less deep fields don't: it'll feel like the whole thing holds together in a way that makes sense because those unifying principles hold it together. Ulysses, for example, as much as it's a book that touches on and engages with a thousand different things, is held together in part by the organising principle of Homer's Odyssey: what does it mean for the Odyssey to be compressed down to a single day in Dublin? What are we to make of Odysseus being Jewish? And which incident is being depicted in any given chapter? Similarly, this is why sciences such as physics or biology often have a feeling of greater depth than a field like economics: the underlying unifying principles (the Principle of Least Action, the Theory of Evolution by means of Natural Selection) are generally a lot better known in these sciences than they are in economics, where we still haven't found one and our attempts to do so have thus far led mostly to unpleasantness.

The third criterion of interest is applicability: a deep field, while not universally applicable, usually feels highly transferable in that you can apply ideas from it to other fields. A deep thing, then, can speak effectively to a whole lot of different people and can apply to a lot of different things, while a shallow one tends to be quite narrow and specific in its applicability. The works of Shakespeare have the staying power that they do, for instance, in part because their applicability is universal: they have something to say to almost every person and every society. Akira Kurosawa can make Throne of Blood, transposing the plot of Macbeth to Feudal Japan, changing the language and about all of the stylistic elements, and the work still has exactly the same force that it had in the Globe back in the reign of James I. Verdi can write it as an opera and bring it to a continental audience, and it remains superlative. While a lot of high school students are, I'm sure, bored by it, the existence of theatre kids suggests strongly that it can speak to them as well. The text has something to say to everyone. Professionally, mathematics and logic can be applied, with only a little thought, to almost everything that you do. To be able to reason through predicate logic, to be able to pull an argument out of a speech, formalise it and see if it stands up, to be able to abstract away from content and see form; these things make you more effective personally, professionally, socially and politically. It's hard to say, after all, that we would not all be better off if more people were able to pull away the content of a political argument that they happen to like and notice that formally, what's being stated is identical to a Nazi argument. Conversely, as much as a skill like web development comes up a lot and is on the face of it useful, knowledge transfer from React development towards philosophy or mathematics is not something that happens all that much.

My final candidate criterion is that of time to mastery. Tests, fields and objects of study in general tend to differ in how long it takes for a person to master the field: in some cases you can dedicate a lifetime to the study of a thing and still not cover more than a small fraction of the subject, while in others you can master basically everything there is to know about a field in six months or so (I'm using a definition of mastery here wherein, past a certain point, more experience and learning leads to diminishing returns: mastery is the point where you hit that). The first type of object of study is, naturally, considered deeper than the second. This isn't necessarily a question of making a start: the rules of chess are simple enough that you can learn them in an hour or so, but mastering that game would likely take several lifetimes. Tic-tac-toe, by contrast, is a game with a low mastery ceiling: once you know the optimal play pattern, every game ends in a win or a draw.

I don't think that it's possible to create a total ordering of all fields in order of depth: I think that would be a definite mistake and any such ordering would be wrong. I also don't believe that you can definitively say that any given thing is "deep" or not. What I believe we are entitled to say, however, is that, in general, fields with greater degrees of the following traits admit more depth:

• The field requires deliberate engagement to enter into it at all (i.e. you can't half-ass it, at least in the initial stages);
• The field admits general principles that can be understood and worked with that underlie the complexity of the field, allowing for a unity of understanding to be developed;
• The field has wide applicability to human life and being beyond the direct technical uses of the field; and,
• The field has a high ceiling for mastery (you can study it for a very long time without hitting a point where you run out of things to learn)

Treated in the same way as Umberto Eco's 14 signs of Ur-Fascism (a field doesn't need all of these to be deep and can maintain depth with any mixture of these, so long as there's enough of it), this is a pretty decent definition of depth for our purposes.

Complexity, depth and the perils of mixing them up

It's a common perception in the general population that things that are deep are also things that are complex: this is, I suspect, largely a factor of the place where depth is most visible being in objects of study that are both. Ulysses, for example, is a paradigmatic example of a work that's both deep and complex. Depth does not, however, necessarily bespeak any inherent complexity or complication, and a lot of things that are quite deep are, in fact, startlingly accessible. To go back to Macbeth, for example, in addition to Kurosawa, Verdi and the hundreds of other directors who've each made their own take on the text, we have, of all things, The Lion King. Nobody would say that The Lion King is inaccessible, and in fact the text in that case is quite simple, but given that the text still has that link to Macbeth, the film still maintains that level of depth in the wings, ready for when the viewer (who's often a child) has developed enough to be able to reach into those depths. The Lord of the Rings, similarly, is a text that's largely accessible at the outset but allows you to delve deeper and deeper (within limits, of course: we don't want to awaken Durin's Bane) as you grow and learn more, both about Tolkien's Legendarium and the world. Every time I re-read the book as I grow older, I find myself reinterpreting the characters and what I think of them as I grow older and somewhat wiser. This is true, I think, for almost all truly deep objects of study.

And now, finally, we come back to technology and the writing of software, because we can address our initial question. While React and C have similar levels of complexity (in fact, I'd argue that C is quite a bit simpler: more on this later), C is a hell of a lot deeper than React is. To compare against our earlier points:

• React is quite a bit easier to half-ass than C: you can, between knowing how to scaffold an app in Next and being able to write a bit of JavaScript and HTML, get something that basically works up and running within a day without much knowledge. By contrast, C demands that you at least know how a compiler and executables work, and to get to a point where you can build and run a simple C program, you've learned a non-trivial amount about how the whole field holds together.
• While learning C is unlikely to change your life, you do learn a lot more that's generally applicable to how a computer works when you write C than when you write React. What a compiler is, what language your operating system was written in, how manual memory management works, how JavaScript and thus by extension React do a lot of the things that they do... learning even a little C makes you that much more able to understand what's going on when you do a whole lot of things on a computer and builds general, foundational knowledge that you can use when you do almost anything with a computer. Front-end work, databases, networking, OS design, driver troubles... knowing C can help you make sense of all of them. React, by contrast, teaches you React and very little else: I have in fact known React devs that struggled with HTML and CSS, or even with writing JavaScript that wasn't React-based, and no amount of exposure to React seemed to shift this.
• The striking thing about C is that it's actually quite a small language: the whole thing basically fits into K&R, and if you've read it, you'll know that. This means that you can more or less fit the whole thing into your memory and understand how it fits together, giving the whole thing a pleasing kind of unity. React, on the other hand... good luck.

All of this, taken together, makes it a hell of a lot easier to write interesting things about C than it is to write anything interesting about React. C is small and simple, which means that you can explain things about it easily, but it grows out to touch an awful lot of things, which means that with a simple example written in C (ten lines or so), I can explain why certain array operations in React are making your browser hang. With only a little bit more work, I can explain why device drivers so often struggle with Linux or what the different kinds of index in PostgreSQL do and when you should use them. And I can explain all this working from the low-level fundamentals of the machine, so that you can understand what's going on all the way down from the browser window you're looking at to the bytecode that your CPU is actually executing. From a small and simple example, I can grow to touch on almost the entire computing world.

The language that we have in the tech world, though, isn't very good at capturing this kind of distinction: we really just have "simple" and "complex", with maybe "complicated" if you're lucky. And this makes it easy for us to make some pretty silly mistakes. C is deep and simple: React is complex and shallow. Moreover, the forms of complexity that C and React admit are quite different. C's complexity is emergent: the thing in itself is simple, but you need to understand the fundamentals perfectly because as soon as you start engaging with real code written in C, those simple primitives grow and combine to form some gloriously, beautifully complex edifices. React's complexity, on the other hand, is largely inherent: there's no unity to them, but rather there's just a large pile of arbitrary weird shit that you kinda just have to deal with. When you learn about one thing in C, you can re-use the lesson for the rest of your learning: when you learn about one thing in React, you've learned about one thing in React.

Or, to put it more crudely: the operating system I'm writing this on is written in C. LinkedIn is written in React (or well, I don't actually know if LinkedIn is written in React or not, but it feels very much like a React-based mudball). One of these things lets me write articles that make the lives of thousands of people across the world a little better. The other persists in trying to sell me training in "prompting AI".

This lack of distinction combines with the fundamentally lazy nature of many tech professionals in some pretty noxious ways. On the whole, our industry doesn't have a great work ethic: while devs and executives are at each others' throats most of the time, a large proportion of both are united in that they've never seen something they don't want to half-ass. Most of the time we have an antipathy to depth and a desire for learning to feel easy and effortless, which comes out both in the general state of the industry and more widely in general social attitudes to media and to the humanities in the tech world. What this means is that much of the time when we're writing software or tooling to supposedly make things simpler, what we're actually doing is working to make them shallower.

To present just one example, we might look at recent trends in PaaS services: the Vercels and Netlifys of this world. A PaaS sells itself primarily as being simpler and more half-assable than hosting an application yourself on a server, and it's true that the initial barrier to entry is lower than doing a more manual version of hosting: just point it at the repository! It's so simple! Understanding how a PaaS offering works is quite clearly a lot shallower as a field of study than learning how to host something on your own box is. Is it simpler, though? I'm honestly not sold on that. PaaS offerings, after all, require that your application be written in a particular language, in a particular way, and they tend to lock you into a particular architecture. There's still a lot of complexity, but now it's inherent complexity in the form of weird shit that you just have to route around as opposed to the complexity of manual hosting, where you might manually learn something. Errors, failures and weird scaling behaviour go from being something that you can figure out, fix and learn from to arbitrary platform behaviour that you just have to deal with. And platform-specific learning tends not to be easily transferable to anything else, while learning how to administer a Linux box teaches lessons that you can re-use in quite a lot of different places. The PaaS thus, quite clearly, trades off depth for the appearance of simplicity. I leave it to you as to whether you think this is a good thing or not.

Our brave new shallow world

This persistent push towards shallowness explains a large part of modern tech culture, and we're very much seeing the results. After all, when you have a need to make any individual stack element half-assable but still need to actually make things work, the end result is that the number of tools tends to proliferate rather quickly, which is how you end up in situations where building a relatively simple website means consulting seventy different documentation sites. React creates a need for Tailwind, the two of them together create a need for Vite, once you have all of that you need Docker to be able to deploy that, and then keeping track of a mess of containers becomes difficult so you pull in Kubernetes...

Each individual tool here is quite easy to half-ass and you don't need to develop a particularly deep understanding of it, but the interactions between all the tools rapidly get completely out of hand because each tool becomes a point of failure. And then, of course, while subjects with depth tend to create at least some alignment on what is and isn't important, "easy to half-ass" is entirely a subjective judgement that's largely unique to each individual person. Which means, guess what? More tools! It gets worse: while understanding a deep subject requires the development of at least a bit of taste, learning fifty or so different tools in a half-assed way makes no such demands on you, nor does it create a requirement for a sense of unity or anything that might require some sense of composition. This leads to brittle, poorly-engineered systems because after all, almost everything in the stack was half-assed: not a place you really want to be.

I can't help but also suspect that this antipathy towards depth explains why nobody in tech seems to be fucking learning anything. Now, I'm not one of those people who'd say that if you don't know how to write complex programs in assembly, you're a failure of a person and don't count as a software person: that's rank arrogance and to be condemned. But you ought to learn something in real depth. Whether that's SQL and the operation of databases, HTML and CSS or systems administration in general, you really need to learn something that admits at least a bit of depth. The issue is that without learning that, you don't develop the kinds of abstraction that is useful for learning and understanding other things later, and you fail to learn how to generalise between problems. If the only things you ever come into contact with are shallow things that you have to handle on a case-by-case basis, you'll treat everything like that, and that tends to mean that, for example, you will not only write bad code, but you'll write way too much of it because you can't identify that say, half a dozen similar classes of problem can be solved with the same code. Given that this is basically how the majority of the industry operates at the moment... well, you see where this is going.

And then of course there are the LLMs. Of course there are the LLMs. There are always the fucking LLMs. The LLM is about the shallowest kind of object of study there is: while the underlying transformer architecture is pretty interesting (and even then, I don't rate machine learning particularly highly as a field compared to statistics, mostly due to the relative lack of wide applicability in the underlying ideas), actually using the thing hasn't changed since ChatGPT was first released and there's basically one thing to know about it: "write a prompt telling it what to do and to make no mistakes". There's nothing to learn, no skills to develop, and even in places like software development where you can use them to get meaningful work done, the skills that you need to learn to use the machine can only be developed by using the task without it. It is every trend towards shallowness discussed here, taken to its final conclusion. It is the ultimate half-assing tool.

... no wonder everyone in tech is all over the fucking things.

The issue is that almost any kind of real value you might think of requires at least a little depth. Even the authors whom I was a bit rude about earlier have to maintain the depth to write a) a coherent plot that b) appeals emotionally to the reader, even if it's on a surface level. That's non-trivial. Similarly, even something like neofetch requires quite a bit of depth (you have to decide what the correct interface is, what information people would like to know, what features people would like - maybe it's very important that you can display your information in the colours of the trans pride flag, say). Given that, the fact that we're in a situation where an awful lot of the code that's being written and the software that's being deployed does not have this level of depth to it is more than a little concerning.

In the end, this trend seems to be creating an industry of lazy, careless people who feel entitled to claim the status of building useful technical products without actually having to learn anything in depth. They're boring, unpleasant to be around, and honestly, these days it's unclear a lot of the time what it is that a lot of coders actually know or do. And as I've discussed before, the people who want depth and seek it out wind up getting, increment by increment, pushed to the margins. This is a world that produces nothing that anyone wants, nor anything that edifies anyone: just mile upon mile of shallow, worthless code that we pretend is important because it's taken on value as an elaborate kind of Veblen Good. We're learning to do tech and producing software not because it serves any need, but as an elaborate kind of status game.

... I really don't want to live in this kind of world.

Rebuilding depth

Of course, this isn't long-term sustainable: it's an elusive but empirically verifiable fact that the disciplines that admit depth tend, in the long run, to be the ones that hold things like, for example, a modern society together. Even in the field of software, the truly deep fields tend to be the relatively low-level core technologies like C, SQL, systems administration and HTML, and however much they're denigrated as being foolish, not-real-tech, too abstruse and complicated or whatever new line the software industry's come up with today, the fact is that we'd feel the loss of these technologies and people skilled in them an awful lot more than we'd feel it if all React developers suddenly disappeared overnight. The fact that a lot of the people in the industry at the moment lack much depth of understanding in anything, then, is a problem that needs to be swiftly addressed.

While this has all been happening, the value of knowledge in individual, specific shallow skills (the use of a tool, knowing React, that kind of thing) has been declining sharply of late. Part of this is, to be fair, because this is precisely the kind of skill easily automated by an LLM (or that an LLM won't fuck up too badly at least), but part of it also is that the proliferation of all these fucking things means that the value of skill in any individual one doesn't look too impressive. The end result of this is that we have people filling up their CVs with fifty or so different skills (many of which are the same skill with slightly different names), all of which they learned just about enough of to be able to half-ass competence with them. At the same time as we're seeing a degradation of the skills we need to build deep understanding, the economic value of the simple vocational skills of shallow type is going downhill.

Much software education, however, is still aimed at developing precisely this kind of shallow skill. The sense I get is that this is a problem that's worse in boot camps than in more rigorous academic programs, but even in those there's been a definite push towards more practical, employable, innovation-ready skills over the last few decades. As bad as the current issue we're having with the bottom dropping out of the software developer labour market is, then, this also represents an opportunity to reform the way we teach and train professionals for the better. Specifically, we should make a real effort to reorient tech education around the building of skills that admit depth.

This is, of course, what a liberal arts education was meant to be up until the beginning of the last century. It makes sense: Latin and Greek are very much deep subjects in the vein that I've discussed on here, the skills that they teach generalise well and are widely applicable, and you can't half-ass a translation of di belli Gallico (well, I suppose you can these days, but the point stands, and the quality of machine translation for classical languages is generally terrible, even now). If we started the educations of all our tech people by teaching the classics, this would probably work.

Of course, nobody in their right minds is actually going to do this, so we'll have to consider different ideas for building depth. For this, my general experience has been that rather than teaching what we think is simplest or easiest to explain, we should start off by teaching the things that are most fundamental: the basic ideas that you need to know in order to build up more complex ones down the line. The hello, world program is an example of this: the first thing that you need to know when using a computer for coding is how to talk to the thing and how to get it to talk back, and without that it's basically impossible to teach anything else, so naturally that's where we start with. This is still sensible: unfortunately, after that we go off the rails a bit (I should note, as an aside, that this is my thinking about how to teach the writing of code: my thoughts about how to teach systems design are a bit different, though they come from the same basic principles). What I'd like to see a lot more of once we're past control flow and such is teaching from examples: for all of the things that a computer might do that we're interested in, we look at how a simple version of it is built in a low-level language and what goes into making it work in some detail. This, I think would both teach nascent devs some much-needed intellectual humility and develop precisely the kind of generalisable deep knowledge that we want from our engineers. Even if you're never going to use the knowledge, knowing how a basic device driver works would, I think, spare a lot of people a lot of grief.

Naturally, as a result of this, I also think we should be much less squeamish than we are about teaching in relatively low-level systems-oriented languages like C. It's very easy not to appreciate the guardrails that higher-level languages offer, or even know that they're there, unless you've worked fairly extensively in a language without one. It does an engineer good, I think, to have had to manually allocate and free memory at least once, or to have to think carefully about the byte representation of the variables they're storing. And honestly, even when it all goes wrong... segfaults are character-forming, and it's probably good that people see them, learn how to fix them and importantly, learn how to recognise them, otherwise when one comes up in real life, they'll be helpless.

Finally, it's a commonplace that what Computer Science departments teach is mostly some disparate chunks of discrete mathematics, taught badly. This is less true, so far as I can tell, than it used to be, and a lot of Computer Science departments these days are more glorified vocational schools than anything, but while it was true that sometimes Computer Science degrees overdid the theory at the expense of practical skills, I think a decent dose of the mathematics is probably good for most people. After all, one of the core things we use when writing code is the function, and it'd probably be useful for engineers to have a good idea of what a mathematical function is. If nothing else, it'd save us a lot of grief trying to explain what a side-effect is and why we might want to define functions without ones. Similarly, big-O notation and some knowledge of basic algorithms will not kill a person and will teach them skills useful in all kinds of things down the line: being able to implement Quicksort from scratch, for example, will neatly head off a whole class of nasty mistakes that you might make in a higher-level language down the line. While we give people shit for prematurely organising, having done this kind of work will often prevent you from writing the inefficient code in the first place, potentially saving a lot in terms of time and grief.

Teaching of this kind, I think, is likely to produce much more effective tech people who have a decent idea of how most parts of the tech that they're working on actually functions, and who have both wide and deep enough experience to be able to transfer some of the lessons learned to their actual work. This is worth doing, I think, even at the expense of teaching the particular technologies in question. And as an aside, this kind of education, even if it's harder, is a hell of a lot more fun for students. Whether they're at university, in a boot camp or simply professionals working to better themselves after work, they get to face real challenges, experience a sense of shared suffering and achievement, build real bonds with the other people also doing that and come out the other end a hell of a lot more competent than the people who just did the easy thing.

I think that all humans have a real need, whether acknowledged or not, for depth. We have a need for mastery, a need to learn things without half-assing them and a need to learn things that we can apply widely rather than just narrowly: the kind of thing where learning it reshapes you, one way or another, into a slightly different and better person. The world in general, and tech in particular, has been starved of that for a while now: let's try and change that. Or I fear that the current state of our computer technology is the best we can expect for the foreseeable future.