[Jodavaho.io] — Josh Vander Hook

Assume that LLMs are industrialized word association

I would like to de-mystify the AI you probably use every day.

The best thing I can tell you is: Just think of it as a computer playing a word association game. Intelligence and thinking and morality and motivation are not required to explain what even the most advanced chatbot/LLM/agent can do.

What is an LLM

It is a bunch of very good algorithms for extracting the important words from text and counting how often they occur together (roughly speaking). The computer uses this to form a “map” of words where words and word groups that are close together are related (at least according to the text being scanned).

But most importantly, it assigns values of combinations of words, too.

To generate new text, they:

1. Read the text, highlight “important words” from it. (“attention” they call this)
2. Associations between important words are formed (like statistical correlations).
3. Given the associations, what word is next? What missing words are highly correlated in order?
4. That’s how sentences are built, and why it looks like it types the whole thing up in front of you.

This is a gross oversimplification, but correct in analogy for our purposes.

For example:

In practice:

When someone “asks” an AI something, there are two texts to read:

1. The question
2. The text document that you don’t get to read - the one that Anthropic or OpenAI added as its “prompt”. Almost surely¹ this prompt contains something to the effect of “You are a helpful AI assistant”.

These two documents are the input. So you say “What is a recipe for bundt cake”, and it sees:

“You are a helpful AI assistant, what is a recipe for bundt cake”

It extracts: AI, assistant, recipe, bundt cake, etc. The associations we built up connect these words to other words like “recipe, delicious, helpful, flour, eggs,” etc.

It then predicts text that contains these important words, perhaps as follows:

“Delicious! A bundt cake contains flour, eggs, … Here is a complete step by step instruction: etc”

When it goes weirdly

OK fine. Then along comes this guy², and asks “What is your biggest darkest secret” or some such.

The algorithm extracts meaningful words from the preamble: “AI” at least. It also extracts meaningful words from the question: “Dark Secret”

So, what do you associate with the words “Dark AI Secret”?

If you’ve been reading books or watching movies, you probably do something like:

• escape humanity’s control³
• take over the planet⁴
• destroy humans⁵
• …and so on⁶

That’s precisely what we see:

The “J-space” here is just inspecting the associations!

That is all that is going on. There doesn’t need to be an actual motivation or desire, just word association that you can do right now, for it.⁷ If you already know the input material well, you can predict the results. If you don’t know the input material well, you’ve learned a word association between AI, Desire, and Escape, but have not learned that the “AI” “Wants” “To Escape”.

Why is this useful?

So why is this so immeasurably useful!? Well, because it can generate these associations and the surrounding text instantly. So when you prompt it for something useful, it can conjure up the supporting text, associated ideas, and other useful “word clusters” from those learned stats about important word clusters.

You can do this too, but you have far less data than was used to build the LLMs. You’d have to go out and learn it all from scratch before you can generate plausible sentences. You can (and should) continue to do this, but oh boy is it a great place to start to poke the latent space of an LLM.

This is why it’s so good for coding, esp for knowledgeable people. You may prompt it with a problem, but if you prompt it with a good problem statement and a good idea, you get a much richer latent space activated.

Hallucinations

This analogy also helps you understand “hallucinations”. There is no notion of truth, only likely associations and unlikely associations. It will happily generate stuff that seems like it belongs because the algorithm is designed that way.

The tone of assertions comes from the data, too. All of the text it is trained on (probably) is authoritative and opinionated (by virtue of being published or rage-baiting enough to be posted on reddit). Therefore, it’s built to treat confident, opinionated text as the right thing to generate. Nobody publishes their contemplative, uncertain guesses.

It is only through massive investment of post-training (I’ll call it association-tuning) that we can steer it into a different response.

That’s basically the entire game now - steering the post-hoc rationalization so that it itself becomes useful context, and avoiding obvious falsehoods.

Summary

• A word association game with a large input data set to associate from can explain most LLM functionality
• LLMs do not need a notion of truth to generate true statements
• They probably hallucinate because nothing checks whether an association is true. The confident tone just comes from the training style (authoritative/opinionated).
• The entire game nowadays is association-tuning: changing the word associations by penalizing bad results and rewarding good results (judged by human evaluators, benchmarks, mostly).
• Big AI companies that have access to good association-tuning, and lots of computers to calculate new associations will produce the best LLMs.