Are LLMs the Wrong Kind of Intelligent?

Adam Mastroianni has a great article where he argues that:

• There are two kinds of intelligence: objective and subjective

• Objective is easier to measure, and hence is the one that usually gets measured in life in various ways: from standardized tests to IQ tests to the g-factor

• Often, when we say “intelligent” or “smart” we mean someone with high objective intelligence

• But, a lot of highly intelligent people are unable to convert their intelligence to social popularity, political success, or dating success (or a large number of other things that the world wants but aren’t directly correlated with objective intelligence)

• In short, the world needs a lot of other capabilities (let’s label them subjective intelligence)

• Having lots of objective intelligence only helps somewhat with subjective intelligence

And here’s his key point:

• LLMs have a lot of objective intelligence, but not much subjective intelligence

• So, the job impact of LLMs will be felt most in fields where the primary skill is objective intelligence, and far less in fields that value subjective intelligence

That is the high level argument, which is explained in detail with lots of examples in the article. Here are some interesting excerpts:

Some problems have clear boundaries and verifiable solutions, like “What’s the cube root of 38,126?”. These problems require objective intelligence. Other problems are vague and squishy and it’s not clear whether you’ve solved them, or whether they exist at all, like “How do I live a good life?”. These problems require subjective intelligence. Objective intelligence can be trained, reinforced, and validated. Subjective intelligence cannot.

It’s unfortunate that people use one word to refer to both of these capabilities, when in fact they have nothing to do with each other.

(Click on the link the the last line; that is itself another fascinating article, titled “Why Aren't Smart People Happier?—the reason has to do with the same point being made in this article.)

LLMs are mostly objectively intelligent, because that is what can be trained, reinforced, and validated by the AI companies. But, you can’t check what the subjective intelligence of the LLM is:

It’s hard to judge the subjective intelligence of a machine both because it’s hard to judge subjective intelligence in general, and because LLMs occupy such a small slice of existence. When you meet a human who can do quadratic equations in their head but can’t hold onto a job or a relationship, you know they’re missing something upstairs. But machines don’t have lives they can ruin

How do we know that objective intelligence does not help you with subjective intelligence?

How far can you get with objective intelligence alone?

I think we already have a decent answer to this question, because we’ve seen what happens to humans who are high on objective intelligence but low on subjective intelligence. We used to call these people nerds, and they were famous for getting their heads dunked in toilets.2

When I was growing up, this paradox was an endless source of sitcom plot lines—if you’re so smart, nerds, why don’t you figure out how to make yourselves popular?

[…]

The nerds I knew in high school—myself included—were always hatching harebrained schemes to increase our social status. They just didn’t work.

[…]

We couldn’t use our smarts to make ourselves popular because we had the wrong kind of smarts.

Nerds tend to do better after high school, but look around: our world is not run by people who won their statewide spelling bee. The nerds keep losing to charismatic know-nothings who, I bet, can’t even recite an impressive number of state capitals. If objective intelligence is all it takes to succeed, then Mensa should be the Illuminati, not a social club for people who know lots of digits of pi.

Forget politics and dating, objective intelligence might not be enough to succeed in science research. As soon as all scientists start using the best LLMs, the objective intelligence of LLMs will speed up the literature review, and idea generation, and data crunching, but we will discover that real science wasn’t blocked on those; the bottleneck was elsewhere:

For example, some people are hoping that AI will defibrillate sluggish areas of science and usher in scientific revolutions across the board. I would also like this to happen. But I am doubtful we’ll achieve it with an infusion of objective intelligence, because infusions of similar capabilities haven’t achieved it either.

When my PhD advisor was in grad school, he literally had to call people on the phone and ask them if they’d like to take part in a psychology study. If he could get 30 participants in a semester, he was cookin’. Participant pool management software like Sona made this process go twice as fast, and then Amazon Mechanical Turk made it go 1000x as fast. Meanwhile, Google Scholar turned a half-day spent in the library into a two-second search, and stats software like SPSS and R made data analysis go lickety-split.

All of this should have supercharged progress in psychology, but it didn’t. I think it’s questionable whether we’ve made much progress at all. So I’m not optimistic that adding another labor-saving technology to our repertoire is going to get us unstuck. People are already saying that LLMs can write a passable social science paper; unfortunately, our problem is not that we produce too few papers. Science is a strong link problem—what we need is new paradigms, not taller towers of journal articles.

The situation is different in other fields. If you’ve got your paradigm in place and all you’re missing is an army of research assistants, or an automated lab that can run 24/7, or an indefatigable grad student who can perform a billion regressions for you, you’re in luck. In those cases, unlimited objective intelligence ought to speed things up a lot, and indeed, it already has.

But the faster you go, the sooner you hit the wall. I have found myself facing all of those limitations at one time or another, and as soon as I overcame them, I was immediately stymied by some other obstacle. I think all of us suffer from this bottleneck blindness: we assume our current bottleneck is our only bottleneck. When you’re strapped for cash, you think all of your problems are cash problems. But once you’ve got some money in you pocket, you realize that what you really need is time. Free up some time, and you discover that you’re actually lacking motivation. Acquire some motivation, and you realize what you’re missing is ideas. Then you need direction, then you need discipline, then you need buy-in, and so on, forever.

Once objective intelligence is too cheap to meter, we’re going to run into all of the other bottlenecks that are still expensive and heavily metered.

In short:

As Montaigne put it back in 1580, “though we could become learned by other men’s learning, a man can never be wise but by his own wisdom”. What does it look like to have all the learning ever created, but no wisdom of your own? Well, “as a large language model...”

Of course, the argument above is rather simplified, and there are lots of objections and nuances. But I think there is a core of truth here: LLMs represent “legible” intelligence, and until now, the world handsomely rewarded legible intelligence, but LLMs will commoditize legible intelligence, and the value of illegible intelligence (aka subjective intelligence) will go up.