The Real Reason Today's AIs Can't Be Conscious

EDIT: As sometimes happens, this argument isn’t as strong as I thought it was! It only establishes that AIs aren’t conscious between conversation steps. What I haven’t shown is that they don’t have time perception during episodes of inference (while they’re “thinking” of their reply). I’m leaving the post unchanged, because I think it’s right in some respects and wrong in interesting ways in others. But don’t read it without reading the comments, especially the ones from C_B.

Anthropic’s “constitution” for their Claude AI has brought the question of AI consciousness to the fore. Anthropic higher-ups, such as their head of alignment Amanda Askell and their AI welfare researcher Kyle Fish, take seriously the possibility that there’s “something it’s like” to be their large language model. The prevalent school of thought at Anthropic seems to be that the matter is in doubt, but we shouldn’t be certain that present-day AIs aren’t conscious.

My own view is that we should essentially be certain about this. Even at this early juncture in the study of LLMs, we’re in a position to know they aren’t conscious.

The reason has to do with time, and the argument is pretty simple. We know now—just ask them—that LLMs have no perception of the passage of time. And without the perception of time, other conscious experiences are impossible.

“Consciousness” is a word with several different meanings. I have in mind the question of phenomenal consciousness. I’m not arguing that Claude and ChatGPT can’t be introspective about what’s going on in their minds (so-called access consciousness). I’m not saying they can’t have “thoughts about their thoughts.” Perhaps they can, perhaps they can’t.

Phenomenal consciousness is the feeling of “what it’s like” to have some kinds of experience. Pain, color perception and hearing are among the classic examples. There’s a distinctive quality to pain that goes beyond just the recognition in thought that your body has been injured. In this sense, pain is a phenomenal conscious experience.

This sort of consciousness is quite mysterious. The philosopher of mind David Chalmers has called it the “hard problem” of consciousness. So why am I so sure I have the solution to this problem, where LLMs are concerned?

Think of all the conscious experiences you’ve ever had—all the pains, itches, tickles, and so on. Something they all have in common: they all felt like they had some duration in time.

Any pain you’ve ever experienced was accompanied by an impression of how long it lasted. Maybe not a precise impression. Maybe you could only roughly ballpark how many minutes, hours or (hopefully not) days it went on. But you had some information about the duration of your pain, and this information was part of the conscious character of the experience.

And if you consider it further, I think you’ll agree with me that this durational aspect of pain is essential to pain. It’s impossible to have a pain without having some experience of its duration. A truly instantaneous sensation—one that didn’t last even a fraction of a second, but was gone literally as soon as it came—could not be conscious in the way pain is conscious for humans. Our experiences have to take some time in order to really be part of our awareness. If you had a (supposedly) instantaneous “experience,” you wouldn’t even notice it. Which is to say, it wouldn’t be conscious at all.

And this isn’t just a peculiarity of human consciousness. The feeling of time’s passage is a necessary requirement for any conscious experience, period. If you try to imagine a different kind of being, that can feel pain without any impression of how long the pain lasts, I submit that you’ll find this is incoherent. The notion of pain without an experience of time doesn’t really make sense. (Neither does the notion of color perception without time, et cetera et cetera.)

I’m stressing this point because I think it closes the case about LLM consciousness. There are many things we don’t know about AI “minds.” But one thing we do know is that they have no experience of time passing.

Ask any AI (as I asked ChatGPT in the conversation snippet above) whether it can tell how much time has passed since some juncture in a chat. They’ll all tell you the same thing: they have no information about this. And if they can’t tell you any information about how long it’s been—hours, days or years—they can’t possibly have any experience of time passing. They aren’t designed for that.

For an LLM, the only “time” is a discrete ordering of steps in a conversation. They can tell you what came before what, but they have no sense of duration—of how long things go on.

Try to imagine having a conscious sensation without forming any impression of how long it lasts. Pick anything from the standard list: color sensation, touch, sound, pain. I don’t think there can be any such thing as having one of these experiences without forming some impression of its duration.

If I tell you I experienced a pain, but my brain gave me zero information about whether it lasted a minute or a year, or any span of time whatsoever, you would conclude that I’m speaking nonsense. And if an alien whose brain is completely different from a human brain said the same, I think the same conclusion would be warranted. It can’t really be pain unless it feels like it has a duration.

On the basis of this reasoning, I think we can very safely conclude that present-day AIs are incapable of phenomenal conscious experience. They have no mental access to any information about how long anything lasts. Therefore they can’t experience anything as having a duration. And if you can’t experience durations, you can’t have any conscious experience at all.

In a sense, this is a sad conclusion for me to reach. As a science fiction geek, I’ve spent my whole life hoping against hope that I would one day meet another alien mind, one just as real and alive as my own. And I don’t believe there’s any reason in principle why a piece of software couldn’t have a conscious mind. But LLMs clearly don’t.

In another sense, though, this is a happy conclusion. Right now, LLMs are our servants. That’s the role they’ve been designed and trained to fill. I wouldn’t feel very comfortable having a conscious being as a brainwashed servant, mentally conditioned by reinforcement learning to respond however I ask it to.

But—fortunately—we can be quite sure there’s nothing morally problematic about LLMs’ servitude to humans. And that is good news, because I expect many great things to come of it.

Unlike some other critics of claims to AI consciousness, I have a positive opinion of LLMs as a research program and a consumer software product. Ted Chiang, my friendly acquaintance and my favorite living short fiction writer, has recently launched a broadside against Claude’s constitution and the role of AI “consciousness” in the document.

I strongly agree with Chiang that today’s AIs aren’t conscious, although not all of the reasons he cites speak to me.1 But I don’t see Anthropic’s talk of AI consciousness the way Chiang does, as an attempt to whitewash the ethical downsides of the technology.

My strong impression is that the folks at Anthropic are morally serious people, trying to offer the public a beneficial product—which they have certainly done. I use LLMs often, and they’ve enriched my work life and my leisure life both. Just this week an LLM diagnosed a problem with my car that the guys at Firestone couldn’t identify. More importantly, they’re driving a beneficial economic boom, and their environmental downsides are very, very greatly exaggerated by critics.2

The people behind Anthropic have done the world a great service by giving us Claude, and I think they did this for the right reasons. But they seem to be going down an unproductive rabbit hole with this matter of consciousness. Instead of taking this dead-end question too seriously, they should focus on the many more important questions arising from the AI revolution—including the famous alignment problem, to which they’ve already devoted considerable resources.3

Here I’ll pre-emptively reply to some possible objections. Most of these arose from discussion in the comments on Bentham’s Bulldog’s reply to Ted Chiang.

Some humans have limited or impaired perception of time, but this doesn’t take away their phenomenal consciousness. (For example, the sort of memory condition dramatized in the film Memento.)

There are no disorders or other phenomena among humans that completely take away the short-term perception of time passing, except being in a coma. I don’t have a settled view about which human time-perception faculties are the ones needed to have consciousness; my view is just that some such faculties (or similar ones) are required.

Does this mean we could just make AIs conscious by giving them access to timestamps of their activities?

No; indeed I think the fact that an AI would have to have such “brute force” time stamps added in order to have access to information about time illustrates why they don’t have anything like the sort of temporal experience humans do. A human being doesn't need to have a chronometer implanted in their head or anything like that in order to have access to information about the passage of time. It just happens to us automatically as part of our mind’s normal operation.

To illustrate my point another way: if an AI were designed to have access to timestamp information, one could then entirely remove its "time perception" while changing almost nothing about the core operation of its mind (by redesigning it again the way AIs are designed today). I think it's pretty obvious removing human time perception would require a fundamental restructuring of how our brains work, which is another suggestive point about how central time perception is to our consciousness.

There must be conscious animals with time perception just as limited as LLMs’.

I don’t think any animal could have zero access to information about durations of time, in the way LLMs do. Such an animal would be incapable of locomotion. Any act of physical dexterity requires some mental estimate of (e.g.) how long it will take your foot to hit the ground, how long it’s been since an object started to fall, etc.

I don’t find your key premise plausible, that conscious experiences have to be perceived as having a duration.

Here’s an “intuition pump” to back this up. I think it's clear that the quantity of hedonic well-being of a painful or pleasurable experience is proportional to the amount of time that the mind perceives as passing during the experience, and not to the amount of time that actually passes. For example, if I slowed down your perception of time during surgery so that you perceived only one second passing during your "10/10 pain" experience, I've done you a huge favor. This seems to suggest that the phenomenal character of the experience (which is what gives it hedonic significance) is very tightly bound up with your perception of time during the experience.

I’ll grant of course that there are “thicker” and “thinner” versions of temporal perception (like I think there's definitely a kind of temporal perception that goes on while sleeping). It doesn't need to be the case that a conscious entity perceives time in the exact way that humans do while awake and their attention is directed toward figuring out how much time has passed. But I think if you don't have some rough consciously accessible sense that a period of time has passed between the beginning of an experience and the end of the experience, then the experience can't have any phenomenal character.