Accidentally Alive in an Age of AI

Currently, the most advanced LLMs have a context window of 2 million tokens.1

For the unfamiliar, a context window is essentially the working memory of a language model, the text it can actively use and refer to in a given run. Tokens are units of meaning in the text, representations of part or all of a word as actually used. These days, an average English word is roughly 1.3̅ tokens. So the most advanced AI today can handle up to approximately 1.5 million words, about 3.12 Lord of the Rings’ worth.2

(This upper limit is also more hypothetical than actual, as most models begin to behave very poorly long before you max out the technical limit.)

While triple Lord of the Rings sounds like a lot of content, in perspective it’s not all that much. It’s around 82 hours of reading, which strong readers can easily exceed in just over a month. (Or, if you are like a recent student of mine, probably two days. Don’t ask me how.) At my school, the technically-optional-but-what-is-wrong-with-you-if-you-don’t-do-it reading program expects about 600 minutes a month, so across your entire education you could easily read over 8 times this much just doing the minimum.

So where am I going with all this? Math is fun and all, but surely I have some actual point.

I am one who, despite many grave reservations about the sudden explosion and many dubious uses of generative AI, has tinkered with it extensively. No amount of broader concern can change my deeply ingrained need/fascination/habit/compulsion to experiment with new technologies. And one of the persistent problems with LLMs is their essentially stateless nature. Once a model has been trained, it has a certain fixed amount of material already baked in, and anything new you do with it does not change this baseline unless you do something like fine-tuning. If I tell a given model, say GPT 5.4, that I am a fan of Richard Hooker, this does not change the model at all. By default, it will forget this detail by my next conversation, when the pristine original model is loaded afresh with no further content.

“But wait!” you interject. “I told ChatGPT two weeks ago that I hadn’t been eating enough and now it always warns me about eating disorders! How does it remember to do that?” Well, this is where “memory” as a feature in chat interfaces comes in. This kind of memory does not work like memory as we know it. Rather, sometimes during or after a conversation, whether implicitly or explicitly, the model may flag that something you have said should be remembered, so it adds it to a set of memory notes, summarizing what you said. These memory notes are not part of the model but are injected afresh into new conversations. The model doesn’t, then, actually digest information into itself and retain it. Rather, it produces a set of notes that it will look at again next time to know what’s going on.

In a sense it is like the 2013 Zachary Levi Hallmark movie Remember Sunday.3 A waitress meets, dates, abandons, and then audaciously reintroduces herself to a guy named Gus who had a brain aneurysm and lost the ability to form new long-term memories. Throughout the day, he writes down anything important he needs to know for the future, and in the morning he reads all his notes to know what’s going on in his life, despite remembering nothing since his aneurysm.

So, to translate the analogy, the brain aneurysm is the end of an LLM’s training, the context window is short-term memory, and the “memory” features of a chat interface are the sticky notes.

Gradually, working with and around such limitations on AI projects has helped me begin to see the sheer scale of how much human life and identity relies on the gradual accumulation of thousands, indeed millions, of tiny facts, memories, and experiences. For example, if you were to take a random journal entry and try to help an LLM understand it, think of how many pieces of context would be needed. In fact, let’s use an example text:

After school, Morwen came around again, and today was apparently The Day. Emeldir finally told her that she is leaving.

So, to give to an LLM the basic understanding of these two sentences that the author or someone close to the author would have effortlessly, you may need to pass in:

• The author’s place at the school (student? teacher? parent? administrator? tutor? other staff?)

• The identities of Morwen and Emeldir. Again: students? teachers? staff? parents? Are they even of the same status or role?

• The relationship between Morwen and Emeldir. Why does one care about the other leaving? How much will she care? How long has this been building?

• The meaning and context of Emeldir leaving (leaving what? why? to where?)

• The author’s own concern for the matter: why does the author care about Morwen and Emeldir’s relationship? Is there a history there? Is this part of a larger arc? Why does the author seem to have known Emeldir is leaving before Morwen, even though it appears to matter to her?

• Is “The Day” capitalized as a matter of the author’s style, to signify meaning, or in reference to some objective occasion?

And this is just the beginning. These pieces of information are roughly the minimum for that excerpt to communicate adequate intellectual and emotional sense, but for all we know the text might be enriched by further data, more information about the school or previous writing. It might have allusions or wordplay that is invisible without further context. Furthermore, this only consists of two sentences. A whole journal entry might be exponentially more complex. Add a few more references to a few more people with their own histories and you might easily need hundreds or thousands of words of context injected to get the whole picture.

The point here is the sheer scale and density of particular memories that make up our consciousness and our identities. For me to think like myself, I must remember dozens of inside jokes, hundreds or thousands of individual experiences or anecdotes, the identities of several hundred, maybe thousands, of people, my general and particular histories with most of them, the gists of many hundred books, thousands of specific quotes, scenes, and moments from those books or TV shows or movies or Adventures in Odyssey episodes, dozens and dozens of scholastic terms and distinctions, millions of tiny moments, and far, far more.

This great constellation of things that define my own mind are, philosophically speaking, accidents of my being. This is in contrast to my substance, that foundational combination of form (i.e. defining organization) and matter that turn “human nature” in the abstract to “Caleb Dixon Smith” in the concrete. Just the organization, the “blueprint,” as it were, of human features doesn’t produce Caleb Dixon Smith. Just a hunk of disorganized matter does not, either. Bring the structure of human nature to the matter supplied by my parents, though, and you get me. Then, having been individuated as myself by the union of form and matter, the rest is a series of accidents.

To be clear for those not already working in Aristotelian terms, the accidents are the features beyond the bedrock of this is a human being that I can have or not have while still one human being either way. There are big accidents like sex: humans can just as well be male or female, but it so happens the matter I was made from was disposed to being male. There are small accidents like the heart-shaped scar on my shin that I got from walking atop (or, rather, stepping down badly from walking atop) a harrow. There are weird accidents like my memories of Dad playing all the Dr. Elmo Christmas music, not just “Grandma Got Run Over by a Reindeer,” and the further accident of my inheritance of the same habit. There are complicated accidents like the autistic wiring of my brain or the fact of my having been born just in time to grow up with peak cinema (2000s movies FTW) and to watch always online smartphones destroy all our psyches.

Accidents are weird, metaphysically speaking. On the one hand, they are often viewed as “second class” to substance because they do not determine ultimately what something is. There is something quite important about the fact that I am substantially a human being rather than a mushroom, and none of the accidents of my personal features outweigh that monumental distinction. On the other hand, they are, to invoke another set of distinctions, a matter of act rather than potency, i.e. they are what I really, actively am, the living and realized ways that I exist. To be is to be good, and my accidents are the shape of the actual existence I have in distinction from any other person. There is no Caleb Dixon Smith without the substratum of a human substance, but, on the other hand, the substratum is only that, and the fully actual real life Caleb Dixon Smith is only here and who he is because he is male, 5’ 9”, 170 lbs, 31 years old, born of E.R. and Patty Ann, Floridian, Presbyterian, obscenely nerdy, unreasonably devoted to Owl City, allergic to Bermuda grass, and otherwise healthy as a horse.

A language model, however complex it gets, is still essentially not like this. The model qua model is form without matter, a blueprint for how to manipulate the matter of the computing hardware. And it really only actually4 exists while running, while actually processing the current inputs to produce an output. It doesn’t quite accrue accidents as there is no underlying substance to hold them. The accumulation of accidents can be mimicked by injecting context or fine-tuning a model, but fundamentally it’s not the same thing as a persistent being developing all the marks of its own specific life. Thus there is a certain sterility to most LLM output. Models are general, not particular. They did not form inside jokes during high school.5 They do not remember that one time they fell off into a dolphin pool at Seaworld, contra the memories of their parents. Lacking bodies, they do not have a running disagreement with their wives over whether 78 degrees is “cold.” These artifacts are definitively divorced from the accumulation of individuating properties that follow existing in the real world.

Some of you are no doubt thinking that, although current AI models don’t have these things, this has no relevance to their eventual potential skills or capacities and no bearing on their moral status or consciousness. Sure, whatever, I don’t care,6 because that’s not what this is about. It’s also true some of these things could change. Future setups might drastically alter the picture and relation between training, fine-tuning, context management, etc. Maybe one day their “memories” will be almost as good as those of a woman scorned. I grant it! (Though in the meantime context management feels like an annoying proof these things are in fact presently far, far, far inferior to the New Gods that AGI fanatics would have us imagine.)

But the more significant point really isn’t about the limitations of LLMs. It’s about the volume of particularity that makes human life what it is. We are accidentally alive in both the philosophical and the colloquial senses. It matters a great deal that we are human beings, something we all share, but it also matters a great deal to and for us that we are not just human beings in the abstract but concrete lives that accumulate millions of habits, memories, bodily marks, experiences, dates, relationships, and all other manner of properties that make us ever more distinctly ourselves. In the end, we shall be at our most completely human when we are also fully realized in the whole host of our perfected accidents, irreducibly and idiosyncratically distinct from every other individual who has ever lived.7 And the manifold brightness of these glorified accidents will shine to the glory of the Creator, who somehow possesses all the perfections of all our accidents in a far more eminent and transcendent mode.