What is Intelligence? Is AI Intelligent?

If large language models have taught us one thing, it is the extraordinary extent to which language and encoded knowledge can simulate intelligence. Yet simulation should never be confounded with reproduction. Rich heirs of our conceptual kingdom, LLMs acquire skills as a result of passive knowledge absorption: wealth they inherit, not conquer.

Among the many definitions of intelligence used to evaluate AI systems, most fail to recognize this issue and start too late. They begin with language, knowledge, problem-solving, and test skills in established domains. But in humans, by the time these appear, intelligence has already been at work for a long time — in the process of inventing and acquiring languages, in the ability to learn and memorize efficiently, in the moments of insight that lead to the creation of the abstract structures of philosophy, mathematics and science as well as the symbolic dimensions of art and games.

Intelligence deals, above all, with concept acquisition and concept invention.

The failure to evaluate artificial intelligence is however producing progress. Recently, a very interesting new intelligence test has been proposed: ARC-AGI 3. Inspired by this recent development, our aim is to propose an operative definition of intelligence and then examine this fascinating new test.

If I succeed, the reader will perhaps form clearer ideas about what intelligence is; if I fail, many will at least have fun discovering the ARC-AGI 3 test, which is first of all an entertaining game that tests vision, reasoning, and agency.

Some social media philosophers claim that intelligence is impossible to define and we cannot even understand it. Although I have no reason to doubt their personal lack of understanding, I believe that our inability to construct intelligence should not be mistaken with inability to define it. Consider nuclear fusion, the grand objective of replicating, in a controlled way, the production of energy that occurs in the stars. We know very well that nuclear fusion propels atoms toward one another until their momentum overcomes the repulsive nuclear force. Yet one thing is to know what fusion is supposed to do, and quite another to do it efficiently.

Likewise, any lack of consensus on a single definition of intelligence should not be misinterpreted as an impossibility. Defining a concept is not a matter of capturing an absolute truth, magically existing outside of us: it is a matter of agreeing on a stipulation. It therefore makes no sense to claim that a definition is correct or wrong: what we seek to establish is whether it represents a useful concept. Nor does it make sense to insist that a definition of a word should be coherent with all the ways the word has been used so far: vague intuitions about a concept that for centuries we did not grasp completely should have no place in a logical theory.

The aim of a definition is clarity and usefulness. In case of intelligence, we wish to capture the human ability to navigate the world, to manipulate the environment to our advantage, to solve problems, to create knowledge and all sorts of intellectual products. A definition of intelligence should guide us toward its construction and should explain why intelligence appears in so many forms, across species and domains; why modern AI systems are intelligent in some respects but limited in others.

Intelligence begins with concepts.

Consider a dog recognizing its owner. The owner may look and smell different, wear different clothes, or appear in different places. Yet the dog treats all these situations as the same. In music, a melody remains recognizable when played in a different key or by a different voice or instrument. In physics, there is always a gravitational pull toward the ground, independent of the particular environment.

These example differ in domain, but share the same structure: something has been abstracted; an invariant has been detected. We call these invariances concepts.

Intelligent living beings do not live directly in the world; they inhabit conceptual worlds, and even their experience of the physical world is mediated by concepts. For animals, many concepts organize, and thus they directly relate to, perception.

Yann LeCun’s indeed emphasizes the notion of world model: intelligent systems learn internal representations of the external world that support prediction and action.

Yet, the philosopher Ernst Cassirer convincingly argued that humans do not simply live in a physical world, but in a symbolic universe composed of myth, language, art, games, science, and mathematics. Mathematics and art, for example, are self-consistent symbolic worlds whose truths depend on internal relations, not empirical correspondence.

Nelson Goodman later sharpened this idea by speaking of ways of worldmaking: systems of symbols and rules that define different versions of reality. Not only does intelligence model existing worlds, it also construct new ones.

For Peter Gärdenfors, the collection of all conceptual spaces is the basis of human thought.

Concepts do not float freely. They must be realized concretely. There are two kinds of realization.

Realizing Concepts: Internal realization

An internal realization is a structured representation within a cognitive system. In humans, this can be symbolic (language, diagrams) or non-symbolic (neural patterns, sensorimotor maps). In animals, it is mostly non-symbolic. In machines, it may take the form of latent representations in neural networks. Internal realizations allow a system to recognize, anticipate, and reason within a conceptual world. Reasoning and thinking are tools for manipulating our conceptual structures in order to exploit them in problem solving, creation, or prediction.

Realizing Concepts: External realization

Beyond detecting existing patterns, humans also shape reality according to new abstract concepts they create. An external realization is a concrete instantiation or enactment of a concept in the world. This includes:

• bodily actions,

• tools,

• machines,

• artworks.

Gymnastics and sport realize patterns and rules of movement that would serve no purpose in ordinary reality. A refrigerator is, in essence, an external realization of a conceptual map. A machine, in general, is not invented as a model of the world: on the contrary, it gives rise to a new physical reality that could not emerge autonomously. Games like chess are physical realizations that do not intend to model reality, but rather to instantiate a conceptual space.

To some extent, all animals possess the abilities we have described: they can understand the world and even realize concepts as tools or new movement patters. At this point, intelligence can be described without invoking language, consciousness, or even humans.

Intelligence is a capacity profile. It is the capacity to:

• acquire conceptual worlds through interaction,

• construct new conceptual worlds,

• manipulate concepts within them, producing reasoning and simulation,

• internally represent those concepts,

• and externally realize them through action, tools, or machines.

It also naturally includes emotional and motor intelligence, which operate in conceptual worlds of affect and movement rather than symbols.

We can now state the definition in its fullness.

Intelligence deals with conceptual worlds and their internal and external realizations. A concept is an invariant of a modeled domain; an internal realization is a structured representation within a cognitive system, while an external realization is a concrete instantiation or enactment of the concept.

Intelligence is a capacity profile that allows a cognitive system to efficiently acquire, construct, manipulate, and internally and externally realize conceptual worlds, with the aim of contemplation, causal description or explanation, reasoning, prediction, use, and action within those worlds and in the realities they relate to.

Intelligence therefore includes:

• memory, because conceptual worlds, to be built, must first be retained;

• language, which emerges as a rich, symbolic way to represent and manipulate our internal conceptual worlds, but it is not required for their internal realization: neural activations can represent concepts without symbolic words;

• reasoning, which manipulates concepts and uses logic to combine them.

Moreover, two dimensions are crucial: how efficiently a system acquires concepts, and whether it can invent significant, new conceptual domains.

We can now answer some important questions: Are animals intelligent? Is AI intelligent?

From the beginning we made clear that intelligence is not only a matter of kind, but also of degree. Some aspects of intelligence may be present to an extremely low degree, or not at all. Intelligence is therefore a continuum. When all aspects are present to a substantial degree, we speak of human-level intelligence, which is the most general form of intelligence we can conceive.

Animals possess most of the defining properties of intelligence; but while they can acquire concepts as structural invariances of the external world, their ability to construct and externally realize new conceptual worlds is limited. They can only harness the environment in primitive ways and imagine very few tools. They also have a restricted ability to internally realize conceptual models, as their representations are limited by the absence of language.

In contrast, large language models such as Gemini possess language and vast conceptual maps, far beyond what animals can achieve. In this respect, they are similar to us. And contrarily to common belief, they do extract conceptual world models that they internally realize as geometric representations in their latent space and hidden activations. This is evident in modern AI video generators, which can take arbitrary images as input and animate them realistically: this would not be possible if the models did not possess an internal model of what they see.

However, artificial intelligence models display a poor ability to acquire concepts, as they need vast troves of data where humans need little. AI also has a very limited ability to invent significant conceptual domains and its creative activity takes place mainly within the boundaries of conceptual spaces established by humans. It can master chess, but not invent it. It can solve mathematical problems, but not invent new mathematical theories. AI models are truly creative mainly in brute-forceable domains such as protein folding or games.

Artificial intelligence is thus, at present, inferior both in kind and in degree when compared with human intelligence.

It is not difficult to guess that intelligence, defined as above, is very difficult to test; in particular, creativity and concept invention is hard to formalize. I do not think this is a limitation of the theory: testing the highest levels of creativity and concept invention simply requires comparable levels of creativity and invention.

Moreover, as a consequence of our definition, any serious test of intelligence should measure, rather than stored knowledge, the ability to acquire and manipulate new conceptual structures under constraint.

ARC-AGI 3 has made significant progress on some of these aspects of intelligence: efficiently acquiring new concepts, devising plans, and navigating new environments with the aid of visual, logical and mathematical reasoning.

The idea of the ARC-AGI series is to measure how close artificial intelligence is to human intelligence. Although I have been critical of the first two iterations, now a much compelling philosophy has been proposed. The goal of ARC-AGI is not to verify that a system is an AGI: its aim is to falsify such a claim. If an ARC-AGI test is consistently solvable by humans but not by machines, this constitutes a refutation of the claim that AI is as general as human intelligence.

Indeed, Claude Mythos and similar hype notwithstanding, the highest score that current AI systems reach on ARC-AGI 3 is: 0%.

The capability of human intelligence that ARC-AGI 3 targets is the ability to autonomously extract concepts from arbitrary new environments — efficiently and effortlessly. Namely:

ARC-AGI-3 is an interactive reasoning benchmark which challenges AI agents to explore novel environments, acquire goals on the fly, build adaptable world models, and learn continuously.

A 100% score means AI agents can beat every game as efficiently as humans.

Instead of solving static puzzles, agents must learn from experience inside each environment—perceiving what matters, selecting actions, and adapting their strategy without relying on natural-language instructions.

So the AI systems undertaking the test must, in effect, play games. But the objective of each game is hidden and there is no explanation of the rules. The AI agent must recognize the main conceptual building blocks of each world, which continually change from game to game. There is no common pattern at all. Moreover, each game has several levels, and each level introduce a new concept, which the AI must be quick to figure out.

The reader might be wondering: “But AI is already good at games, what about Chess and Go? Hasn’t AI already blown away human competition? Where is the challenge?”

Here lies the clever twist of ARC-AGI 3. AI learns games such as Chess and Go by playing a disproportionately large number of times compared to humans. We learn from the thousands; AI from the millions. However, in ARC-AGI 3 each game allows only a very limited number of attempts, so the agent must pause, plan and analyze before acting: it must work out a strategy not only for succeeding, but also for probing the environment. Action in the real world is subject to the same constraints: how many attempts can a cat afford before learning how to catch a mouse?

Here is an example of the first level of a game. You are just shown this picture:

you can move up, down, left, or right. But, first who are you to begin with? What could possibly be your goal? What is that cross? And what about that yellow bar? Those are natural questions for a human, but not for a system that merely operates with statistics and previously encountered patterns, like current AI.

As soon as you grow accustomed to that environment — so that intelligence gives way to knowledge and intellectual comfort — you are cast into pure chaos again:

New patterns, new machinery, new rules: still few attempts.

This series of games taxes the ability of AI to discern new visual patterns without the possibility of seeing anything more than one instance. I confirm that these games are relatively easy to solve, but they are nonetheless tricky. Although I completed all the seven levels of the first three games on the first try, as one is supposed to do, I often had to pause and carefully plan my strategy. Exploring these environments with a limited number of moves requires sophisticated approaches. But it is also fun, and I invite the reader to try.

IS ARC-AGI 3 SUFFICIENT?

I believe that this test, if varied enough, will resist the attacks of current artificial intelligence technology. AI’s visual ability in novel environments is rather poor and their long-term planning flawed when there is no prior receipt to follow.

However, applying our definition of intelligence, we see that only the acquisition rather the invention of new concepts is challenged by ARG-AGI 3. Moreover, the memory system is taxed very little. The concepts learned across each game’s levels are very few and can be comfortably held within the AI agents’ context windows. Thus, the transfer of information from working to long-term memory is not required — a mechanism current missing in all AI systems, which prevents continual learning and adaptation. Without robust long-term memory, conceptual worlds cannot accumulate, and intelligence remains episodic. Probably each game should have hundreds of levels to assess the memory system in realistic scenarios.

Nevertheless, the aim of this test is to falsify that current AI has reached AGI level. And such a falsification has been obtained. How long will it last?

Federico Aschieri