AGI: Education’s Blindspot

Start with the thing most education arguments about AI still get wrong at the premise.

For nearly everyone, AI means chatbots, image generators, a little polish on an email, and an unwelcome layer of slop on top of the internet. That is a real experience of a real product. It is also, as Alex Amadori and Andrea Miotti of Control AI argued this week, the tip of an iceberg — the public’s daily encounter with AI substantially underrepresents what frontier systems can now do, and most people have no idea how large the gap has become.

“It’s just predicting the next word” was a fair simplification in 2023. Applied to GPT-3 and GPT-4, it explained why the thing hallucinated, why it flattered you, and why it couldn’t do arithmetic. It was the sentence I used myself in early professional development sessions, and it did honest work.

It is now out of date, and the reasons are specific rather than mystical. Three training shifts did most of it.

Reinforcement learning. Language modeling teaches a system to imitate each word in its training data. RL doesn’t care what the system says, only whether it succeeded at the task. That distinction matters more than it sounds: an imitator struggles to exceed its teacher, while a system learning by outcome can find methods nobody demonstrated to it. And because the feedback comes from automated verifiers rather than human raters, the loop runs millions of times without waiting on people, who are both fallible and unbearably slow.

Actively sourced data. The labs stopped waiting for the open internet to supply them. They now manufacture data. DoorDash pays couriers to wear body cameras and film themselves washing dishes and folding laundry, which is training data for robotics. Tools like Cursor and Claude Code capture something more valuable: how an expert decomposes a hard problem, what they try, and how they recover when it breaks. Every professional using those tools is demonstrating the workflow the systems are being trained to run on their own.

Test-time compute. Earlier systems answered immediately, which is like demanding a person multiply six-digit numbers in their head. Frontier models now think privately before responding — drafting, checking their logic, discarding bad approaches, correcting errors, and only then producing output.

Put those together and you get systems that form strategies, execute them over long horizons, and adapt when blocked.

This is not yet AGI, but it’s not the model you’ve received professional development on. It’s not the model your approach to AI is likely be based on.

Ask a school what it is doing about AI and you will get one of four answers.

Nothing. This is still the most common one, and it is worth saying plainly rather than politely. Not “we’re in the early stages.” Not “we’re being thoughtful.” Nothing. No policy, no curriculum, no professional development beyond a forty-minute session in August, no plan. The teachers are improvising, the students are already fluent, and the building is pretending the last three years didn’t happen.

A committee. The task force convenes. It surveys stakeholders. It produces a document, usually somewhere between eight and thirty pages, and the document is mostly about academic integrity, data privacy, and which tools are approved. The committee is real work by real people and I don’t want to sneer at it. But notice what a committee is: a structure for managing a disruption to existing operations. It is the response you build when you believe the institution is basically sound and something has arrived that needs handling.

Buying edtech. The vendors arrived faster than the pedagogy. Districts are purchasing AI tutors, AI grading, AI lesson planners, AI detection, AI dashboards. Some of it is genuinely useful. Almost all of it shares one design assumption: the school stays the same shape and the software makes the existing tasks cheaper. Nobody sold a district a product whose premise was that the district’s core value proposition is expiring.

Teaching kids to challenge the AI. This is the sophisticated answer, and it is the one I want to spend time on, because it is the one being given by the smartest people in the room.

The argument goes like this. AI makes things up. It is confident and wrong. Therefore the essential twenty-first-century skill is critical evaluation of machine output: check it, question it, catch the hallucination, verify the citation, don’t trust the summary.

I want to concede the strong version of this before I take it apart. Verification is a real skill. I teach it. My debaters live on it — every card gets read in context, every claim gets traced to its source, and the ones who skip that step lose rounds to the ones who don’t. Nothing I’m about to say means students should be credulous.

But look at the model of AI underneath that advice. It is Google’s AI Overviews. It is a lossy summarizer bolted onto a search engine, producing plausible text with a known error rate, and your job as the human is to be the error checker. That was an accurate picture of the technology in 2023. It was a reasonable thing to build a curriculum around in 2024.

It is a curriculum that trains students for a permanent junior-partner relationship with a system that is not going to stay junior.

Two problems. The first is that error-catching is a skill whose value falls as reliability rises. If you build a student’s entire posture toward AI around finding the mistake, you have trained them for a contest they will increasingly lose, in domain after domain, on a schedule nobody controls. The second problem is bigger: nobody in this framework has told the student what to do after they challenge the machine and the machine turns out to be right. That is the actual condition they are going to spend their lives in, and we have no lesson plan for it.

There is a prior question here that almost nobody in education asks, and it decides the whole argument: which AI are we talking about?

The verification curriculum was built by people evaluating a specific configuration. Default model, minimum reasoning, free tier, answering a one-line question in a browser sidebar with no context and no sources. That configuration does hallucinate. It invents citations, it flatters you, it gets arithmetic wrong. Everything the guidance documents say about it is true.

It is also the cheapest deployment of the weakest tier of a technology whose top tier is behind a paywall. OpenAI’s flagship, GPT-5.6 Sol, is not available to free, Go, or logged-out users at all. When a district evaluates “AI,” it is almost always evaluating the version the vendor gives away.

Here is what the other configuration is like, because I work in it every day.

Run a hard question through Claude Opus 5, Claude Fable 5, and GPT-5.6 Sol with reasoning set to maximum. Then — and this is the part that matters — run each system’s answer past the others and ask them to find the flaw. What comes back is rarely an error I need to catch. What comes back, far more often, is an explanation of why I am wrong. My premise was mistaken. My framing loaded the question. The mechanism I assumed was doing the work isn’t the one doing the work. The number I was confident about was reported differently in the primary source than in the coverage I read.

That is not the same technology the guidance documents are describing, and it is not a difference of degree in error rate. It is an inversion of roles. In the verification curriculum, the student is the auditor and the machine is the thing being audited. In the configuration I actually use, the machine is the auditor. I bring a thesis and I get a rebuttal — a good one, delivered faster and often better sourced than anything I would get from a colleague down the hall.

Ask what a student trained only to hunt for hallucinations does in that room. They have been given one move, and the move doesn’t apply. Nobody taught them how to be the one under cross-examination.

There is no agreed definition, and anyone who tells you otherwise is selling something. But the working definitions cluster.

OpenAI’s charter language has always been economic: highly autonomous systems that outperform humans at most economically valuable work. Demis Hassabis uses a stricter cognitive standard — a system with all the cognitive capabilities of the human brain, including the ability to generate genuinely novel hypotheses, the Einstein test rather than the exam-passing test. Shane Legg has held a median forecast of 2028 for decades. Hassabis has said publicly that AGI is probably only a few short years away and that we are standing in the foothills of the singularity.

Whatever you call it, the trajectory schools have to plan around is this: systems that do cognitive work across domains at a level above the graduate you are producing, available at low cost, running for long stretches without supervision. You do not need superintelligence for this to break the deal. You need what already exists, improved for another five years.

And the arithmetic is unforgiving. A student starting kindergarten this fall graduates in 2039. A ninth grader this fall graduates in 2030 — inside every serious forecast window, including the conservative ones. There is no version of the timeline debate that resolves in favor of waiting.

This is the part that should be uncomfortable for anyone in education.

OpenAI published its plan in June, and it is a document about arrival. The company describes itself as entering a third phase — research, then products, and now the work of making advanced AI abundant enough that everyone can use it. The stated goals include an automated AI researcher by March 2028 and a personal AGI for every person on Earth. Its public policy agenda is built on five principles: democratization, empowerment, universal prosperity, resilience, and adaptability. The mission line is that AGI should benefit all of humanity, which means building systems that help people do more of what they choose rather than replacing human judgment about what matters.

There is a team at OpenAI called Personality and Model Behavior. It sits inside something called the Personal AGI team. Its job is emotional intelligence, reasoning, and how models interact thoughtfully with people. That is not a research direction. That is an org chart. Somebody drew a box, gave it a name with AGI in it, and started hiring.

Anthropic committed $200 million to an Economic Futures Research Fund whose entire premise is that the economy is going to be restructured and society is not ready. The program runs research grants, policy symposia, and a longitudinal index tracking AI’s diffusion through the labor market. The research priorities include equipping people to navigate AI-driven transitions and modernizing income support for displacement. A company is spending nine figures to fund outside researchers studying what happens to workers when its own product succeeds.

Google DeepMind published a 145-page technical paper on AGI safety, launched a course on AGI safety for students and researchers, and Hassabis has spent months lobbying governments for an international standards body modeled on financial regulation, on the argument that the capabilities are coming faster than the institutions.

As of mid-2026, 37 states plus Puerto Rico have issued official K-12 AI guidance. FutureEd is tracking roughly 71 bills across 27 states this session. That is real activity, and it is more than we had a year ago.

Read them. They are about cheating, data privacy, vendor procurement, teacher training on approved tools, disclosure to parents, and screen time. New York’s A 9190 would restrict classroom AI use to ninth grade and above while permitting staff to use it for planning and administration — which is, precisely, a policy that protects adult productivity and delays student capability.

I cannot find the document that asks the question the labs are asking. Not one state framework I have read takes as its premise that the graduate will enter a labor market containing systems more capable than they are, and then works backward to what the graduate should therefore be able to do.

That is the blindspot. Not that education is anti-AI or pro-AI. It is that education is regulating a tool while the tool’s manufacturers are planning for a successor species of cognitive labor, and the two conversations do not touch at any point.

What is a person for, in a world of machines smarter than they are?

That is not a philosophy seminar question. It is a curriculum question, and it has a serviceable answer. When the performance is commoditized, what remains valuable is authorship: choosing what is worth doing, committing to it in front of other people, and answering for how it turns out. The machine can produce the artifact. It cannot be the one who is accountable for it.

I’ve called the goal wizards and Jedi builders — a wizard has the magic, and the magic now ships free with every laptop, while a Jedi has the magic plus training, discipline, and something they have decided is worth their energy. In Kunshan last week I put it a different way to a room of Chinese debaters and their coaches: there are no AI-proof jobs, and pretending otherwise is a disservice to students. As a head of schools suggested, what we can do is help them become trailblazers — students who can build a new world and thrive in whatever world gets built.

We know how to make those students. This is the frustrating part. The pedagogy is not speculative and it is not expensive.

Project-based learning, where the student owns a real problem with a real deliverable and a real audience. Debate, where you defend a position under pressure from someone trying to take it apart, in front of a judge, with no one to blame for the loss. Design thinking, where the work starts with a human need rather than a prompt and iterates through failure. Interdisciplinary work, because every problem worth solving in the next twenty years sits between the subjects we’ve built our departments around.

I write about these constantly and I won’t relitigate them here. What is worth noticing is the structure they share. Each one puts a student in a position where they must make a choice, own it publicly, and answer for it. That is the thing the machine cannot take, and it happens to be the thing our current assessment system is worst at producing.

The labs have a plan for AGI. The plans are self-interested and possibly wrong, but they exist, they’re funded, and they’re staffed.

Education has a guidance document about cheating.

The blindspot isn’t that schools are moving slowly. Institutions move slowly and I’ve made my peace with that. The blindspot is that we are moving slowly in a direction that assumes the destination is a slightly more automated version of the school we already have — and every organization actually building this technology has told us, in public, on the record, with money behind it, that they are building something else.

The committee should still meet. But somebody in that room needs to ask what the graduate is supposed to be able to do when the machine is better than they are, and refuse to move on until there’s an answer. The labs have written theirs down. We haven’t started ours.