We know how to train AI, but what should we train it for?

AI models can be thought of as black boxes trained to generate desirable outputs from certain inputs. For example, language models are trained to take in user prompts (e.g., “what is the capital of France?”) and output useful responses (“The capital of France is Paris”). It follows that a big chunk of AI research is the study of how to train these models.

Research on how to train models has come a long way. A few decades ago, state-of-the-art models were neural networks trained to recognize handwritten digits. In 2017 we invented the transformer which is now the basis of modern language models. And more recently, we’ve trained some impressive models for robotics, and even begun to automate scientific discovery. Clearly, we know a lot about how to train an AI model.

What I don’t think enough people are talking about is what we should train the AI for, especially when it relates to how we make sense of the world. Training an AI to recognize handwritten digits or work as a coding assistant is not very hard to justify. But once the low-hanging fruit is gone, I bet we’ll run into the problem of figuring out what is worth training an AI for, which seems tied to how well we understand our desires, values, and sense of meaning.

For example, AI models better than 90% of human programmers have been around for ~1 year now, and I’d argue that most people’s output of valuable code is… relatively unchanged. The bottlenecks used to be how to code and what to code. The first is largely gone, but a good sense of what’s worth making is still missing for most people.

Another example: I’ve been thinking about automated scientific discovery and using Einstein’s discovery of general relativity as an intuition pump. As a reminder, what sparked his discovery was a thought experiment that exposed an inconsistency in the accepted theory at the time. Given the thought experiment, it seems plausible that an AI in the near future could derive general relativity. After all, we already have examples of AI models that can generate theories for things like neuroscience. However, I’m skeptical an AI would ever feel compelled to generate a thought experiment like Einstein did. It seems to me that the need for such a thought experiment could only come from a desire to understand the universe, which seems like a uniquely human thing.

We know how to train AI, but what should we train it for?

It feels like a sense for what is worth training an AI for is intimately tied to our (human) values. The impulse to create a clever thought experiment only makes sense if, like Einstein, you have a deep curiosity about how the universe works and what it all means. It is true that many of our values have been distilled into text, and partially understood by language models. But anyone who’s sat down to contemplate the infinite richness of life and sensory experience knows that language is “like a single hair in vast emptiness,” as Zen Master Deshan once said. And how we make sense of the richness of life—how we decide something is worth making sense of—is often not obvious to us, and definitely not obvious to the AI models.

I think it’s worth understanding where our sense of what matters comes from. At worst, it might buy us a few more years of relevance as machines get smarter. At best, it might help us cultivate the parts of ourselves that make us human and give life meaning.

Post 42 out of 100