How LLMs Work

It is increasingly hard to avoid large language models. Google recently integrated AI into its Chrome web browser, while Microsoft has added its own AI tools into Word and Excel. But despite the technology’s proliferation, large language model’s inner workings are poorly understood. So how do they work?

An LLM begins life as an complex statistical function that takes in text and spits out predictions for the next likely sequence of words. Inside this function are billions or trillions of parameters, values that determine how input data is transformed into predictions. At the start, these parameters are randomly generated. 

This random function goes through pre-training. The pre-training process starts with scraping billions of words of text from the internet, which is then fed through the function. The LLM predicts what word should come next, and then compares its prediction against the data, using math to figure out how to change its algorithm to better match the correct answer. 

As the LLMs initial parameters are random, so too are its initial answers. But over time, the LLM automatically and incrementally adjusts its parameters (or “weights”) to reduce the discrepancy between its predictions and the actual text. By repeating this process with billions of samples, it iterates the function to one that can produce reasonable text. This gradual improvement is referred to as gradient descent or, more simply, learning. The result is the base model, an LLM that can produce walls of webpage-like text. 

Fine-tuning adapts that base model into the helpful chatbot assistant offering to draft our emails. Companies hire people to collect and label large datasets of higher-quality examples covering the particular tasks the LLM will perform. These task-specific data sets refine the LLM, specializing it for particular purposes. Additional fine-tuning steps have humans rate the LLM’s answers for additional feedback to improve its responses, a process known as reinforcement learning from human feedback (or RLHF). This stage also trains the LLM to refuse answering certain types of questions – such as “How do I build a bomb?” 

However, as LLMs get better, they are increasingly being used to create their own training data. This evolving process could herald a breakthrough in LLM capabilities, similarly to how Alpha Go raced past human players after it started playing itself. Such a leap would lead to groundbreaking advancements, as well as new challenges in oversight and control. 

If you want to go deeper on this topic, check out these additional resources on how LLMs work:

Trio of explainer videos from 3Blue1Brown (2017) on how neural networks (the structure underlying LLMs) work technically.

• But what is a neural network? 

• Gradient descent, how neural networks learn 

• What is backpropagation really doing? 

Intro to Large Language Models by Andrej Karpathy (2023) - A practical explanation of how LLMs are trained, such as what output from a pre-trained model looks like. 

How ChatGPT Works Technically by ByteByteGo (2023) - A high-level explanation of how ChatGPT was trained.