Rethinking AGI

In the growing chorus of speculation and hype surrounding Artificial General Intelligence (AGI), many of the loudest voices still define it by what it does—not by what it is. You’ve probably seen definitions like: “AGI is any system that can perform any intellectual task a human can do,” or “AGI outperforms humans at most economically valuable work.”

These definitions, though convenient, anchor AGI to outcomes. They measure the power of a system by its performance—how well it predicts, completes, or generates. But performance alone doesn’t make something intelligent. A calculator is faster than any human at arithmetic, but no one mistakes it for an intelligent agent.

If we want to take the idea of AGI seriously—not just as a buzzword, but as a meaningful concept—we need to go deeper.

Over the past year, I’ve been developing a working definition of intelligence grounded in functional information theory. In that framework, intelligence isn’t just about what a system does—it’s about how and why it does it. Intelligence emerges from the structured exchange of meaningful information between components of a system, where that information plays a selective role in shaping the system’s future states.

Under this lens, intelligence is not just behavior—it’s self-organizing coherence. And AGI, properly defined, must be an emergent system, not just a fast learner.

---

Where Current AGI Definitions Fall Short

Let’s take a few well-known definitions:

OpenAI: “AGI is highly autonomous systems that outperform humans at most economically valuable work.”

Bostrom: “Any intellect that greatly exceeds the cognitive performance of humans in virtually all domains.”

Goertzel: “Sufficient cognitive ability to autonomously carry out a wide variety of tasks that would require human-level intelligence.”

What do these have in common?

They’re performance-centric. They treat AGI as a kind of industrial benchmark—a system that does more or does better than us. But none of them require the system to be meaningfully integrated, structurally coherent, or capable of self-directed change.

Nor do they ask whether the system understands itself, or models its own behavior in a way that allows for intentional adaptation.

---

A More Principled Definition

To better reflect the principles of emergent intelligence, I propose the following definition:

> “Artificial General Intelligence is a system capable of recursively improving its internal structure and behavior across domains with minimal external guidance”

This definition rests on a few key principles:

Recursion → The system must not only act—it must improve itself.

Minimal external guidance → True generality arises from internal coherence, not just better prompts.

Implies selective functional information → Intelligence isn’t data-processing, it’s meaning-shaping.

Cross-domain adaptability → Not specialized; the system must generalize across contexts.

This definition filters out brute-force learners and narrow savants. It focuses on emergent self-organization—on systems that learn how to learn.

---

The Elephant in the Room: Consciousness

There’s still a deeper layer to all this—one that haunts every serious conversation about AGI: consciousness.

Can an AGI be intelligent without being conscious? Can it have goals without experience? Can it model others without modeling itself?

But that’s a question for a future post…