RSS Amplifier

A Strange Attractor · May 13, 2026

Alternative Milestones to AGI

Read inOriginalenesdefrptjaittr
0
Sign in to vote or save

This page did not load. You can still read it on the original site — the toolbar below keeps your place in the directory.

Being good at tasks is not the same as being good at learning tasks.

AGI is about human-level (or above) capabilities, but use of the concept often bundles together capabilities that could have different impacts and arrive at different times. This short post will tease apart the implications of different capabilities and propose some alternative milestones to AGI.

Following François Chollet,1 I’ll distinguish between two types of capabilities:

  • Task capabilities: Capabilities that allow accomplishing specific tasks, such as coding or solving math problems.2

  • Learning capabilities: Capabilities that allow AI systems to learn task capabilities.3 In-context learning, sample efficiency,4 and continual learning are examples of learning capabilities. Learning capabilities are related to, but distinct from, how fast or how much task capabilities improve. For example, a system with poor sample efficiency could still learn a task capability given sufficient compute and data.

Each type of capability matters independently:

  • Task capabilities themselves could have significant impacts, even without strong learning capabilities. For example, an AI system with bio capabilities could aid legitimate biological research or cause harm by helping synthesize a bioweapon.

  • Learning capabilities could have impacts independent of the task capabilities that an AI system starts off with.

    • Suppose there is an AI system that can perform every economically valuable task today, but is weak on sample efficiency and continual learning relative to humans. Such a system would struggle to pick up new valuable tasks as they emerged, and potentially permit human labour to remain important economically.

    • An AI system that is superhuman on sample efficiency could acquire superhuman task capabilities faster than humans can adapt to. For example, rapid improvements in the capability to evade monitors could undermine human oversight of AI behaviour.

    • More broadly, learning capabilities could tell us whether or which task capabilities might improve rapidly if compute or data bottlenecks are removed.

For task capabilities, AIs can vary in how many tasks they cover and how strong they are within each task. Some potential milestones are:

  • Narrow task expert: Matches or exceeds human performance on a narrow range of tasks. May or may not have strong learning capabilities.

  • General task expert: Matches or exceeds human performance across a wide range of tasks. May or may not have strong learning capabilities.

Learning capabilities can also vary by generality and level (e.g., superhuman sample efficiency in physics, subhuman in biology). Some potential milestones are:

  • Narrow human-level5 learner: Can learn a narrow set of task capabilities at least as well as humans can, without necessarily starting off with any task capabilities.

  • General human-level learner: Can learn a wide range of task capabilities at least as well as humans can, without necessarily starting off with any task capabilities.

Note that a (narrow or general) human-level learner can become a (narrow or general) task expert given access to sufficient compute and data.

As key takeaways:

  • Discussions of AI timelines and impacts should distinguish between progress towards task capabilities and progress towards learning capabilities.

  • Alternative milestones to AGI could be (narrow / general) task expert and (narrow / general) human-level learner.

  • We should work harder to measure learning capabilities. Some evaluations exist,6 but we need more to test language model agents across varied environments.

Acknowledgements

Thanks to the following people for feedback on this piece: Markus Anderljung, Zaheed Kara, Amelia Michael, Tom Reed, Elias Groll, and Helen Toner.

Thanks for reading A Strange Attractor! Subscribe for free to receive new posts.

1

François Chollet makes the distinction between “task-specific skills” and “general learning ability”, arguing that we should define intelligence as the latter. My goal here is not to define intelligence.

2

I’ll consider properties like reliability and robustness to fall within task capabilities because such properties are also important aspects of human performance on a task.

3

The distinction isn’t perfect because tasks can involve some degree of learning. For example, software engineering involves being able to work with new codebases. Task capabilities could also help to improve learning capabilities. For example, it’s likely easier to learn new program languages after one has learned at least one.

4

Sample efficiency can be a property of both an algorithm and a model. As an example of the latter, bigger models are more sample-efficient than smaller models when fine-tuning on downstream tasks.

5

“Narrow/general expert learner” is also possible, but I don’t think it sounds as nice.

6

For example, see ARC-AGI, The MiniPile Challenge for Data-Efficient Language Models, BabyLM Challenge, and MEAL: A Benchmark for Continual Multi-Agent Reinforcement Learning. Measurements of sample efficiency should try to distinguish between improvements to data quality and improvements to algorithmic quality.

Read on astrangeattractor.substack.com ↗

Comments

Nothing yet. Say the first thing.

    Sign in to join the conversation.