The AI Advances I’m Hoping For in 2026

"The limits of my language mean the limits of my world." — Ludwig Wittgenstein

In 2025, Ilya pronounced the end of the era of scaling. And indeed the year’s biggest AI breakthroughs didn’t come from larger models but from a diverse bag of algorithmic improvements, the most impactful of which was the paradigm-changing RLVR (Reinforcement Learning from Verifiable Rewards).

By training models against problems with checkable answers (math puzzles, coding challenges, etc), we discovered that models spontaneously develop strategies that resemble thinking - strategies that become more powerful when given extended inference-time compute (i.e., chain-of-thought). Models learn how to break problems into steps, check their work, and backtrack when stuck. In the past I’d compared this to humans discovering writing. But it wasn’t RLVR by itself that made the biggest difference, it was how the reasoning steps could now drive external tool-usage more seamlessly - planning multiple steps in advance, and then revisiting earlier decisions based on the output of tools.

This led to some stunning accomplishments: AIs winning gold in International Math Olympiads, autonomous coding agents, browser use, Terence Tao et al using AI to solve Erdős Problem #1026, and so on. AI-assisted search (or more accurately, search-assisted AI) now just works, and is seamlessly integrated into the reasoning process.

But despite the increase in general capabilities1, as should be expected, the most tangible progress happened in domains where correctness is verifiable. AI performance spiked in domains where verifiable rewards exist (the VR in RLVR), and only modestly improved everywhere else, further exaggerating the jaggedness in the landscape of intelligence.

“The hardware scheduled to come online in the next few years makes the clusters of 2025 look like pocket calculators… If Moravec is right then we are not approaching a plateau. The octopus is done with the jar. It is now unscrewing the aquarium.” - Samuel Albanie

My hot take is that it’s not just pre-training scaling that’s hitting a wall, but that even current inference-time scaling will hit diminishing returns faster than people think. So the future, IMO, will be contingent on algorithmic research. Of course, Moravec’s insight still holds, just in different ways. The gobs of compute that are going to be available soon will enable all kinds of algorithmic and architectural innovations, which in turn will leverage all that available compute to give us new and useful applications.

Rather than predictions for 2026, here’s a wishlist - the algorithmic and technical advances in AI I’m hoping to see. These are the things I would be working on myself (and might, who knows):

1. Extending RLVR and tool usage into messier domains (for obvious reasons)

2. Continual learning that lets models keep learning after deployment

3. Latent reasoning that thinks in continuous meaning-space rather than discrete tokens

4. World models that more viscerally understand causality and physics

5. Computational creativity that goes beyond simple temperature-based stochasticity

Frontier AI research is guided by intuition. The state of the art is a bag of tricks found more through trial-and-error and tinkering rather than theory. I.e., more artisanal craft than science. Here are a few areas of interest where my intuition says we might see progress (or small but meaningful dents) in 2026.

This is the safest “prediction” on my wishlist because everyone is already working on this. The RLVR + tool advance of 2025 is only the tip of the iceberg. First, the capabilities overhang is vast, with years of application development possible just to catch up to the possibilities.

Second, we should be able to use the same magic on messier domains. Medicine, law, chemistry, scientific reasoning - places where answers aren’t binary right/wrong but where we can still define what “better” looks like.

Many paths seem promising: soft reward signals that provide graded feedback rather than binary right/wrong. Tool outputs as verification - when a model executes code or queries a database, the result becomes a training signal. World models that can simulate whether an action achieves its goal or provide hints as to some notion of rightness.

The dream is RLVR for anything where we can define success, even imperfectly. If we get there, the jagged capabilities that define current models (making them overachievers when it comes to benchmarks, but unreliable in practice) might start to smooth out.

There’s more to extract from tool usage too. Not just more and different kinds of tools but also clever little hacks and things like Anthropic’s skills.

Also, LLMs should be able to code up their own bespoke tools for one-time usage as one of the steps of a complex task. There's no reason an LLM couldn't write a Python script mid-task, execute it, and discard it - a kind of cognitive scaffolding that exists only for one problem. We're already seeing hints of this with code interpreter usage, where models write throwaway analysis scripts.

The intelligence of current AI models is frozen in time. Everything they know was fixed at training time. They can use new information you provide in context, but they can’t actually learn from it for future tasks. Also, while they can acquire declarative knowledge through search and RAG, they can’t easily update their procedural knowledge or their “intuitive” fast understanding over time. I.e., they can’t grow from experience. It’s becoming very clear that this is a non-trivial shortcoming impacting AI agents.

Solving this the “right way” or in a theoretically sound way is a truly challenging problem. Gradient Descent, which is the learning algorithm for updating the neural network’s weights based on new data is slow and brittle. Catastrophic forgetting, where learning new things erases old knowledge, has plagued neural networks for decades. But there are some promising approaches being tested right now. Google’s Nested Learning paper presented at NeurIPS 2025 shows one great example, where instead of one big update for the whole model, it treats a model as a hierarchy of nested loops. Inner loops (fast) handle immediate context and “working memory,” while outer loops (slow) consolidate that into long-term knowledge.

But we don’t need to solve this at a fundamental level to get some benefits. Several researchers, including Sholto Douglas at DeepMind, have hinted that clever combinations of structured external memory and context management might get us surprisingly far on continual learning without solving the fundamental problem. Think of it as giving the model a well-organized notebook it knows how to use. My sense is these kinds of clever hacks will come to light in 2026 and move the needle a fair bit.

When models think step by step, they generate discrete tokens, committing to a direction with each word, collapsing possibilities as they go. They then have to justify their words or direction they committed to in the next steps. Or they have to explicitly backtrack, “Wait I was wrong,” with the previous choice of words remaining as overhang or cognitive debris that impacts future generation.

Consider a thought that wants to be between two discrete concepts. Say we want a felt-concept that’s neither quite “cautious” nor “skeptical” but meaningfully in between until we can articulate it properly. The LLM has to pick one or the other when thinking in tokens. The next-token model has to choose a word and move on, even when the truer meaning lives in the gradient between them, thereby losing the implicit mental richness that exists before commitment.

By iterating on internal representations in continuous vector space (not discrete tokens, but an in-between location in the high-dimensional manifold or “meaning-space” that tokens point to), models can think without words. They can ruminate in their interior space of meaning. This enables the ability to capture more nuance and richness of thought for longer.

More importantly it enables exploring multiple paths simultaneously before committing to the most promising trajectory. The Coconut paper (Hao et al, 2024, 2025) showed models that loop through recurrent blocks before emitting each token naturally develop breadth-first exploration (because their internal thoughts can remain in a superposition of multiple potential reasoning paths simultaneously).

And perhaps for me, even more intriguing is how latent reasoning allows thinking at length about things that don’t easily map to language at all. For example: spatial intuition, physical prediction, and the kind of thinking that happens below the threshold of verbalization.

Unfortunately this is also a very hard problem to solve. The core difficulty is that model’s internal deliberation is illegible and opaque, and you can only supervise the final answer. This makes credit assignment difficult. If the model gets a problem wrong, which latent “thought” was the mistake? There's also no natural curriculum: with token-based reasoning, we can train on progressively harder problems with longer chains of thought, but latent reasoning offers no obvious analog - we don't know how to structure a progression from simple to complex latent deliberation.

While LLMs have a non-trivial verbalizable understanding of the world and the mechanics behind its operation, certainly enough to convincingly reason in textual form about it, their internal understanding can often be fairly shallow (and alien). Sure they may verbally be able to predict what happens when you knock a glass off a table and talk about it beautifully, but their understanding is not at the level of a physics model.

This is not dissimilar to a person who is blind and never actually seen a glass fall (or somehow never had the sensory experience of falling themselves), but understands it entirely though other people’s textual descriptions of said glass falling. In philosophy this problem is known as Symbol Grounding. It’s the idea that words are "floating" symbols until they are anchored (grounded) in physical experience.

World models aim to change this. Instead of predicting the next word, they predict the next state of the world given an action. We need models that learn causal structure, not just correlational patterns, from video and interaction. NVIDIA's Cosmos and Google's Genie are steps toward learning physics from observation, but the real prize is models that can answer counterfactuals - i.e., what would happen - and that likely requires embodied interaction or simulation, not just passive video.

What excites me is the convergence with RLVR. Physical tasks have verifiable outcomes. For instance, did the robot grasp the object? Did the car avoid the obstacle? World models could provide the verification environment that makes RLVR possible for embodied AI.

I suspect there is a multi-modal nested model possible (e.g. VLA-W or WorldVLA) which combine causal world modeling with the expressive richness of text. This also ties it back to latent reasoning, where the most important parts of "intelligence" (like the robot not dropping a glass) happen in the silent, non-verbal layers of the model.

Here’s my most speculative wish, and also a wish after my own heart. Current training optimizes for finding the most likely correct answer, which is convergent thinking. But creativity often requires the opposite, the divergent: exploring unlikely possibilities, making surprising connections, generating novelty that has value precisely because it’s unexpected.

Unfortunately we don’t have a good theory of what creativity actually is, computationally speaking. Is it recombination, i.e., novel arrangements of existing concepts? Is it intelligent navigation to low-probability regions of learned distributions? Compression and abstraction that finds new structural patterns? Something about modeling other minds and anticipating what would surprise them? And how do we measure and benchmark creativity?

Some mechanisms I find myself excited about include the deliberate perturbation of latent representations during reasoning, forcing exploration of adjacent regions of possibility space. Branching inference that maintains multiple hypotheses in parallel, pruning based on an “interestingness quotient” than raw probability. Multi-agent dynamics where models with different perspectives challenge each other, producing emergent ideas neither would generate alone.

I wonder if we can build approximate rewards for novelty, surprise, and aesthetic value? If someone cracks verifiable rewards for creativity, even if imperfect, it would open RLVR-style training for creative tasks. That would be genuinely transformative.

One real problem with AIs is that they’re inherently gullible. There is no principled way to distinguish between the prompt and the data, or the system prompt and the user prompt.

The moment an AI agent operates in the real world - browsing the web, reading emails, interacting with APIs - it encounters untrusted input. We have no reliable way to prevent that input from hijacking the agent’s behavior. A cleverly crafted email could redirect your AI assistant to do something entirely different from what you asked. Maybe it replies to the email with your credit card information, or sends all your sensitive and private data/credentials to some scammer, or deletes content on your drives, or turns off the power in your home. Imagine an AI agent in a nuclear power plant that has access to the internet (thereby exposing it to all kinds of suggestions) and can control things or recommend actions to a human user (who of course doesn’t bother checking the AI’s entire chain of reasoning).

True agent autonomy in enterprise settings - AI that can take consequential actions without human approval at every step - requires solving this problem. Unfortunately we’re not close. Which then means AI agents can never be given the ability to do anything consequential. Perhaps this the most important item on any wishlist but also the hardest to solve.

Unfortunately we humans are quite susceptible to what security researcher Johann Rehberger calls as the “Normalization of Deviance,” where repeated exposure to risky behavior without negative consequences leads people and organizations to accept that risky behavior as normal.

There may be some ways to mitigate the gullibility problem such as capability control, formal verification of agent boundaries, and architectural separation of "instruction" and "data" channels. Perhaps. But I’m not sure this is solvable in full any time soon, even if we get “AGI” (whatever that is) given how gullible we humans ourselves are. Just look at how many people fall for the simplest scams.

This is a bit of a wet blanket but it’s also the truth. The gullibility problem means deployed AI agents will remain toys relative to their actual capabilities for years, regardless of AI advances.