Before I started doing some fairly cursory reading into what LLMs actually do, I assumed it involved some form of world modelling.
It may seem an odd interest, but I created what I would term world models quite a lot when I was a teenager. When my group of friends and I were into role-playing games (RPGs) – around the time D&D was making us all satanists – I’d be the games master (GM, not DM – rather sweetly, we looked down on D&D), responsible for creating the normally more or less Tolkeinesque, medieval world for the games to take place in and, most interestingly for me, a set of rules governing what was possible in that world. This would determine the outcome of things like fights between the protagonists and whatever adversary got in their way, the use of magic and even social interactions. These rules formed a world model.
(Side note: it strikes me now how gruesome this world could be, with its measurable notions of strength and intelligence, and its obsession with things like race.)
I guess this need to formulate rules that represented fictional worlds was strong in me, as my other great hobby at the time was writing text adventures (now more commonly known as interactive fiction). In a text adventure, the game displays some text and invites the player to respond using natural language. Classic early text adventure Zork opens with a description of the player’s surroundings and a > prompt to type a response:
West of House.
You are standing in an open field west of
a white house, with a boarded front door.
There is a small mailbox here.
> OPEN MAILBOX
Opening the small mailbox reveals a leaflet.
> READ LEAFLET
(Taken)
"WELCOME TO ZORK!…
A text adventure does two things when responding to the player’s input. Both involve interacting with the invisible set of rules that govern the behaviour of things in the game world, i.e. the world model.
Firstly, it translates the player’s responses into a reduced set of words and simple relationships – normally verbs, prepositions and direct and indirect objects. This is similar to how LLMs tokenise prompts, converting them from sentences to a string of numbers. Through ignoring certain words such as articles, and allowing conjunctions and modifiers, a well-programmed text adventure can translate complex instructions into a computer-friendly set of data items to work with.
Once the game has parsed the player’s command, it attempts to match the set of data items and their relationships to its database of things and rules, and their current statuses. In other words, its world model.
In the above example, the world model is a hidden set of statements and rules that a computer can process:
- the mailbox is an openable container
- the mailbox can hold things with a mass of, say, 2 or less
- at the start of the game, the mailbox is closed
- the leaflet is a carryable object with a mass of 1
- the leaflet is a text
- at the start of the game the leaflet is in the mailbox
- the field is scenery
Because the player has used the verb to open with an openable direct object, rather than for example the verb to eat, and the mailbox is currently closed, opening it returns a successful message. The status of the mailbox changes to open, so trying to open it again would return a different message, along the lines of That’s already open. The world model is therefore dynamic.
Game writers could make their adventures seem adept at “understanding” quite complex language through writing lots of rules, and by subtly prompting the player to use the “correct” words in the right relationship to each other. Unfortunately, it was still quite easy for a player to shatter the illusion by going off piste, perhaps by choosing a word not in the game’s dictionary, or by trying some unaccounted for combination of allowed words, such as TOUCH FIELD.
The text adventure world model consists of a set of words and rules about how they can be used with each other. Combining them successfully creates an illusion of spontaneity and intelligence. It’s like an analogue machine with a set of knobs and levers that can and can’t be pressed and pulled according to how they interact behind the scenes. Manipulate the knobs and levers in the right order, and the machine will reward you with meaningful text. The game writer’s job is to subtly prompt the correct combinations, and make the process seem unmechanical.
AI works well with world models because computers are best when processing data according to a clear set of relationships and rules. A good example is chess, where there are a finite set of allowed moves (the queen can move in any direction any number of squares, pieces have to stay on the board, only the knight can jump over other pieces etc.) Because each move in chess can have literally billions of eventual outcomes that are governed by these rules, the more powerful a computer is, the better and quicker it can evaluate any move.
I had naively assumed LLMs worked with lots of world models that they somehow imbibed when fed lots of data – chess would be just one among thousands. I also assumed these were huge compared to a text adventure’s set of rules – which might explain their thirst for computing power.
No. One of the mind-blowing things about LLMs is that they are in fact less “conscious” of how things work than a 1980s text adventure, or a chess program. Which is why they have the queen jump over pieces. Instead of calculating the consequences of available legal moves before choosing the most effective, an LLM will draw on its huge store of historical games and positions and choose the most common. This works while there are lots of examples to analyse, but deteriorates as the game enters less chartered territory, and falls apart when there are no examples at all. If it knew the rules of chess, it would at least make its next move legal. It expends a vast amount of effort appearing to be a good chess player, despite being unaware of how a queen can move.
If we were to ask an LLM to create a text adventure, it would have no concept of a world model, or a set of rules about how words can be used and how they relate. There would be no updatable database. Instead, it would look at every published game and output text based on how often tokens appeared in relation to each other in their transcripts. While it might provide some entertainment, and even interest, the entropy would set in far more quickly than in a game of chess.
(Note – I have tried! But for whatever reason, when I enter a command I get an error message in red and then it tries to get me to create an account.)
Two things strike me. Firstly, how inefficient LLMs are. Relying on existing streams of words to produce apparently meaningful responses requires billions and billions of texts, and vast, city-eating reserves of computation. Zork ran with less than 100kb. Secondly, the idea that intelligence will somehow emerge from ingesting enough texts and analysing them with enough power is self-evidently absurd – pure marketing bullshit.