How LLMs like ChatGPT, Gemini, Claude & Grok Really Work

You’ve probably used one of these things already.
Maybe you asked ChatGPT to write an email you didn’t feel like typing.
Maybe you used Grok to craft a comeback on X that sounded dangerously smarter than you are.
Maybe you asked Gemini a deeply scientific question like, “How many ants can live inside one house?”
(Answer: more than you want to know.)

Whatever the case, we’ve gotten weirdly comfortable chatting with machines that answer like people. We don’t even pause to ask, “Wait… how is this thing doing that?” Maybe it’s time we do.

ChatGPT, Gemini, Claude, and Grok all belong to the same species of technology: LLMs, large language models.

The name sounds intimidating, but the idea is pretty straightforward. An LLM is a type of AI designed to understand and generate human language. Ask it something, and it replies in full sentences that look like they were written by a real person.

What makes them “large” isn’t their size in gigabytes. It’s the amount of text they were trained on and the insane number of internal connections they contain. We’re talking about models built on top of trillions of words and powered by hundreds of billions (sometimes trillions) of parameters.

In other words, they learned to talk by analyzing almost everything humanity has ever written.

And once you know that, the next step becomes obvious: to understand how these models work, you need to understand what they were fed, how they were trained, and how they turn language into math.

So let’s start at the beginning with the data diet that made them possible.

To understand how these models work, start with something simple: they learned to talk by reading everything. Not just a few textbooks. Not just Wikipedia. Everything. Imagine handing someone the entire internet: books, news archives, legal documents, medical journals, GitHub repositories, Reddit threads, product manuals, poetry, TikTok transcripts, memes. All of it.

That’s the diet these models were built on.

OpenAI trained GPT-4 using a blend of massive web crawls (Common Crawl), digitized books, academic research corpora, reputable news archives, and enormous code datasets.

The exact files are closely guarded, but OpenAI openly confirmed the scale: trillions of tokens, tiny chunks of text that together represent millions upon millions of documents.

Google followed the same recipe for Gemini, but added more variety. They trained not just on text, but also on YouTube transcripts, large image datasets, audio corpora, and multilingual web archives. Gemini isn’t just a language model, it is multimodal. It can answer questions about screenshots, diagrams, and even photos of your fridge.

Anthropic, the company behind Claude, went in a different direction. Instead of grabbing everything they could, they filtered aggressively. Their dataset focused on cleaner, more academic and professional writing to steer Claude toward safer, calmer behavior. It worked. Claude answers like a thoughtful professor instead of a meme dealer.

And then there’s Grok from xAI. Grok trained on standard internet data too, but with one major twist: it was fed real-time posts from X. That gives it a strange advantage. It understands live internet slang, trending topics, and the general chaos of social media. If GPT-4 learned from libraries, Grok learned from the streets.

The takeaway is simple. These models speak well because they learned from nearly everything humanity has ever written, from Shakespeare to StackOverflow.

It’s tempting to imagine that you could just dump the internet into a computer and it would wake up speaking perfect English. That isn’t how it works.

Training goes far deeper.

Instead of reading text like a human reads a book, these models practice prediction. They’re shown pieces of text with parts missing, and their task is to guess what comes next. Word after word. Token after token.

When they guess incorrectly, the system adjusts billions of internal values called parameters to make the model slightly better next time. When they guess correctly, those values are reinforced.

This loop happens billions of times across trillions of tokens. It runs for months on supercomputers the size of warehouses. Through that brutal repetition, patterns emerge: grammar, structure, logic, reasoning, humor, tone, memory, rhythm, cause-and-effect, and world knowledge.

They aren’t memorizing sentences. They are absorbing the hidden rules that shape language itself.

That’s how the internet turns into intelligence.

There’s a strange truth about these systems that’s easy to forget. ChatGPT, Gemini, Claude, and Grok don’t actually see words. Not like you do.

When you read the word “apple,” your brain jumps to the fruit, maybe even to taste and smell. When an LLM reads “apple,” it sees numbers.

After the dataset is collected, every piece of language gets converted into long lists of numbers called embeddings. Think of embeddings as coordinates on a map, except the map isn’t two dimensional like Google Maps, it’s hundreds or thousands of dimensions deep.

Words that mean similar things end up close to each other. Words with different meanings end up far apart.

Here’s a simple example.

Take the sentence: “The surgeon entered the hospital.”

Inside the model, “surgeon” lives extremely close to doctor, nurse, scalpel, and operating room. “Hospital” lives close to clinic, healthcare, patient, and ambulance.

So if you ask the model: “What job does someone in a hospital usually have?” it doesn’t search Google. It doesn’t remember a specific sentence. It simply looks at its internal map and sees that “doctor,” “nurse,” and “hospital” are neighbors.

Another example. If you ask an LLM to translate “dog” into Spanish, it doesn’t pull a translation from a dictionary. It knows “dog” sits close to “animal,” which sits near “perro,” which sits near Spanish-language embeddings. The translation emerges naturally from relationships, not lookup tables.

This is why embeddings matter. They convert human language into something the model can reason about mathematically.

Once language becomes numbers and words become locations in a giant meaning space, the model can do things computers were never able to do before. It can detect emotion, rewrite text in new styles, summarize long articles, spot hidden patterns, solve analogy puzzles, classify intent, and generate completely new sentences.

All because language turns into coordinates.

And once everything is numbers, something remarkable becomes possible: the model can decide which parts of your sentence matter most, the same way your brain focuses on certain words while ignoring others.

That leads directly to the next breakthrough.

Before transformers arrived, language models had a huge limitation. They basically read like goldfish. They processed text one word at a time and constantly forgot what came before.

You could give an older neural network a sentence like “The dog that chased the cat was brown,” and by the time it reached the word “brown,” it had already forgotten “dog,” so it didn’t know what was supposed to be brown.

Transformers solved that problem with something called self-attention. Self-attention lets the model look at every word in a sentence at once and decide which words are connected to which.

In the dog sentence, the transformer can attach “brown” to “dog,” “that chased” to “dog,” “cat” to the chase action, and “was” to the structure.

Because of this, the model understands long sentences, complex grammar, and relationships across distance.

Now zoom out. If the model can do this inside one sentence, it can do it across an entire conversation. This is why you can talk to ChatGPT for 20 messages and it still remembers what you asked at the start.

A real example you’ve probably used.

Ask ChatGPT: “Explain quantum physics like I’m five, then rewrite it professionally for a research paper,” and it will keep the same topic, switch tone, and preserve structure.

Self-attention makes that possible.

The transformer architecture, invented at Google in 2017, changed everything because it allowed models to process text in parallel, understand long-range relationships, and scale up to billions of parameters.

This is the reason modern AI exploded. Not hype. Not luck. Transformers are the foundation.

Every major model today, ChatGPT, Gemini, Claude, and Grok, is a transformer at its core.

Think of embeddings as the map and transformers as the GPS reading it.

Most people imagine ChatGPT has a library of pre-written answers and just picks the right one when you ask a question. That isn’t true.

When you ask something, the model generates the answer one tiny fragment at a time. These fragments are called tokens.

A token might be a full word, half a word, or even just punctuation. Think of tokens as the tiny building blocks the model uses to build sentences. For example, the word “gravity” might be split into “grav” and “ity,” while short words like “the” stay whole. Even commas and spaces become tokens. The model reads and predicts these tiny units one by one, then strings them together so smoothly that we don’t notice the pieces underneath.

Behind the scenes, the model predicts the next token, then the next, and the next, thousands of times in a single reply.

Here’s a simple example.

Ask ChatGPT to explain gravity in one sentence.

The model doesn’t search for a pre-written sentence. Instead, it predicts the most statistically likely next token after “gravity,” then predicts the token after “gravity is,” then after “gravity is a,” and so on until the sentence is complete.

This is why the same question can produce slightly different answers each time. The model is sampling probability, not pulling text from storage.

It isn’t retrieving facts or accessing memory.
It isn’t searching the web unless you give it that ability.
It isn’t conscious or thinking.

It is performing massive statistical predictions at incredible speed.

It explains all the weird behavior you’ve seen.

When the model starts confidently and ends in nonsense, it wandered off the probability path.
When it repeats itself, it got stuck in a loop.
When it invents sources, it chose tokens that looked likely rather than true.

There is no truth detector inside it.
Just math.

But because it predicts tiny pieces of language instead of retrieving whole sentences, it can create text no human has ever written before. It isn’t copying. It’s generating.

And that’s where I’m stopping this one.

At this point, you already understand the core of how these models work. They were trained on the internet, turned language into math, learned context through transformers, and generate answers one token at a time.

That alone puts you ahead of most people casually using AI every day.

And here’s the truth: we are only scratching the surface.

In the next posts, I’ll go into the parts that really blew my mind, like where the “knowledge” actually sits inside these models, why hallucinations happen, how billions of parameters translate to real intelligence, and what the next generation of LLMs will look like.

Until then, let’s do a quick recap:

LLMs like ChatGPT, Gemini, Claude, and Grok were trained on huge portions of the internet. They learned language by predicting missing words over and over across trillions of examples, turning sentences into numbers through embeddings. They use transformers to understand context and meaning across long stretches of text, and they generate answers by predicting one tiny token at a time. They don’t store sentences or search the web, they don’t “think” like humans, and they don’t memorize facts. They operate purely through probability, structure, and scale. The reason they sound intelligent is because we trained them on almost everything humans ever wrote, and the patterns of language encoded inside them are now so vast that they seem to reason.

That is the foundation. Everything else is layers built on top.

If this first breakdown helped you understand what’s going on under the hood, wait until you see the next layer.

This rabbit hole is just getting started. If you’re curious where AI is going next, subscribe and come with me.