Why Today’s AI Systems Can’t Be Conscious

There seems to be a claim going around that AI systems could become conscious because brains create consciousness, and the brain is a machine—therefore since a computer is also a machine, then AI systems could be conscious. Here I explain why this is a false inference, and a misleading one: it is irrelevant whether a brain is a machine—instead I believe the important question is whether we really think machines like AI chatbots could be conscious.

I worry that the current debate around AI consciousness gets stuck on definitions, leaving bigger questions unasked. We have to ask big questions to protect ourselves long-term against tech company overreach by using democratically-decided regulations. But we cannot regulate what we do not understand, and time and resources are limited. We have to be choosy.

So, I suggest a different approach so we can move on from this sticky debate and focus on higher-impact questions: let’s assume that consciousness requires a machine to generate it, or otherwise cause its existence: we assume that only a machine can think. This isn’t about believing it or not, we just want to show that the answer does not really matter, so we don’t need to debate it. In other words, the status of the brain as a machine or not has no logical consequence for whether AI chatbots could be conscious.

The good news: even if we assume that only machines can think, consciousness remains a deep mystery for science and philosophy to explain—and for the arts and humanities to help humankind interpret, with all the wonders, paradoxes, and horrors of our modern predicament.

One interest of mine is understanding how our social consciousness works. Can we measure social or collective consciousness? How does social influence constrain or enable our personal and spiritual growth and happiness? How do people agree “in advance” to perform complementary roles to achieve shared tasks, even if they just met?

I list other ideas of new questions that appear under this perspective after unpacking what these AI systems actually are in the next section.

Now the question is, are today’s AI systems the kind of things that could be conscious in the way that humans are conscious? I believe the answer is no, because humans have vastly greater social and cognitive capacities than AI systems in terms of cognitive throughput. The physical existence of an AI chatbot is not localized anywhere—parts of its capacities are implemented by code running on your device or web browser, which communicates with computer clusters in various server farms via photons marching through fiber optic cables.

These AI systems boast powerful features: they have access to an unprecedented—and biased—aggregation of all of written history, stored in a compressed format that can be instantly searched, read, and recombined using computational pattern recognition techniques. The chatbot is just an interface that allows users to use familiar natural language to “program” the system.

All told, AI chatbots are extremely sensory- and cognition-poor compared to humans—we can fluidly interact with hundreds or thousands of people every day, all the while sensing, interpreting, and filtering what would amount to at least several petabytes of data every day, all at a fraction of the energy required to build, train, and maintain a chatbot system.

On top of all that, humans constantly perform “thought experiments” to predict the outcomes of our actions before we take them, and then we compare the results of these simulated outcomes! We imagine and remember all sorts of things, all the time. With AI, on the other hand, we pay twenty extra dollars per month just to analyze a few extra kilobytes of text data per day, maybe 100GB of video creation at the most in a month—all done much less efficiently than in the brain.

The difference is that AI systems have a massive capacity for storing, retrieving, and processing unprecedented amounts of data, instantly. They certainly have us beat there. But none of that has to do with consciousness.

Human consciousness is about synthesis at scale with a clear goal of adapting to changing conditions, alone or with others, instantly to catch prey or collective architecture projects carried out on timescales that exceed one human lifetime—show me an artificial entity with those capacities. Until one arises, I say let’s ignore the possibility that AI systems are conscious because there simply is no evidence to support the claim.

The debate about whether chatbots are conscious distracts us from learning to use them wisely. The idea that today’s AI systems could be conscious seems corrosive: human consciousness is then implicitly devalued, framed as something that could be created and controlled scientifically—not something God gave you. Casual belief in chatbot consciousness, then, could have devastating real-world consequences by undermining our trust in ourselves and each other.

To those who might say, “but you can’t prove AI isn’t conscious,” I would first say that you also can’t prove that your thermostat doesn’t have feelings. There is simply no evidence to support the belief that chatbots will ever be conscious. Whether the brain is a machine or not is irrelevant. Given that human cognition outperforms chatbot cognition on nearly every measure, and on every measure that has anything critical to do with human consciousness, there is every reason to conclude that AI chatbots and the like are not conscious.

I am curious to have a critical examination of the methods and philosophy of the cognitive science of consciousness. What, exactly, does it mean to “measure” consciousness? Does this fit with our folk understanding of consciousness? I know there’s research on this, but maybe someone out there more familiar with it than me that could explain that?

If we lack reliable, non-circular measures of consciousness even in humans—where we at least have first-person reports—what justifies confidence in measuring chatbot consciousness in silico?

More broadly, there may be a structural problem of self-reference at work when we attempt to pin down consciousness. When humans study consciousness, this is consciousness studying itself, using the very cognitive tools whose limits and capacities are under investigation. This raises the possibility of blind spots that the system either can’t detect, or can’t fix.

In this sense, Gödel’s incompleteness theorems serve as a suggestive analogy, even if they don’t rigorously apply: sufficiently rich systems may be unable to fully and consistently describe their own source, foundations, or causal structure. This means that the search for a complete, operational definition of consciousness may be constrained logically and empirically.

So, I say, bits are its. Man-made chatbots cannot have human consciousness, whether or not a human being qualifies as a machine. Chatbots are just a temporary mirage of dancing bits and photons worming through fiber-optic cables and splashing our faces in waves as they pour out of our screens.

Even if it were possible that something like consciousness emerged in silico, as a chatbot or whatever, it is up for debate how to measure consciousness in natural beings, so odds are we can’t reliably measure any form of consciousness.

Human beings are unlike anything else we know of in the Universe. Carl Sagan, a hero of mine, said, “We are a way for the Cosmos to understand itself.” This is true whether we human beings are just meat machines—or if we’re something more. AI systems can do some amazing things—but they can’t care about your life and your world like you do.