Hallucination Causes: Why Language Models Fabricate Facts

Article links

Make inline references clickable

Why do language models say things that are not true? Not infrequently, and not only in edge cases: models hallucinate confidently on questions as simple as who wrote a specific book, what a company's revenue was last year, or when a scientific discovery was made. Understanding the answer matters enormously, because fixing a symptom without understanding its cause leads to superficial patches that break down whenever conditions shift.

As we established in the previous chapter on hallucination types, hallucinations span intrinsic contradictions, extrinsic fabrications, and factual errors of many varieties. What we need now is a mechanistic account: where do these failures originate in the training pipeline and generation process? The answer is not a single bug to be patched. Hallucination is structural: baked into how training data is collected, how the optimization objective is constructed, what knowledge the model can and cannot represent, and how the generation process converts probability distributions into words.

This chapter walks through each of these root causes in detail, building a diagnostic framework you can use when a model you are deploying starts confidently making things up. We will cover four major cause categories: training data issues, exposure bias during learning, structural knowledge gaps, and generation pressure at inference time. Each category deserves careful treatment because each points to a different class of remedies, and the most effective mitigation strategies address multiple causes simultaneously rather than one in isolation. We will return to mitigation in the next chapter. Here, the goal is to understand the failure modes deeply enough to reason about them precisely.

The simplest explanation for why models say false things is that they learned from false things. A language model is, in one sense, a compressed statistical summary of its training corpus. If that corpus contains errors, the model will reproduce them. But the problem runs deeper than occasional inaccuracies. The structure of web-scale training data introduces systematic biases that affect entire categories of knowledge, not just isolated facts.

Modern large language models are trained on enormous web crawls: CommonCrawl, C4, The Pile, and similar datasets containing hundreds of billions to trillions of tokens. Web scraping at this scale is unavoidable given the data requirements of large models, but the web is not a curated encyclopedia. It contains:

• Factual errors in forums and blogs that are stated confidently and never corrected
• Outdated information that was accurate when written but no longer reflects the world
• Duplicate and near-duplicate content that artificially overweights certain claims
• Spam, SEO content, and boilerplate with low informational density but high token volume
• Hallucinated content from prior AI systems, increasingly common as generated text propagates and gets indexed

When a model trains on this data, it learns to produce text in the statistical style of the corpus, errors included. The model has no mechanism for distinguishing "this website stated X confidently" from "X is true." Both contribute equally to the optimization objective.

The training objective is next-token prediction: for a corpus of tokens , the model maximizes:

where:

• : the total language modeling loss, summed over all token positions in the corpus
• : the token at position (the target token to predict)
• : all preceding tokens, forming the context the model conditions on
• : the model's predicted probability for token given its context
• : the total number of tokens in the corpus

The model is trained to maximize the log-probability of each correct next token given its context. Here, "correct" means "what appeared in the training corpus," not "what is factually true in the world."

This distinction is the root of the problem. The loss function contains no factual correctness term, no mechanism for cross-referencing claims across documents, and no reward for consistency with verified external sources. A factual error that appears consistently across thousands of web pages will be learned as a strong pattern exactly as a true fact appearing with the same frequency would be. The model encodes the statistical co-occurrence structure of tokens without any representation of the epistemic status of the claims those tokens compose.

There is also an amplification dynamic at play. The internet has a tendency to copy and redistribute content. A single article containing a factual error may be scraped, republished, summarized, and cited across dozens of derivative pages, each appearing as a separate training example. The model sees what looks like consensus (many sources agreeing) when in fact there is only duplication. This means the model may learn erroneous facts with higher confidence than the original single-source error would warrant, because the training signal artificially strengthens the incorrect claim through repetition.

Not all topics receive equal coverage on the internet. Widely documented phenomena (major historical events, famous figures, popular programming languages, capital cities) appear in billions of training tokens across diverse contexts. Obscure entities (small companies, local politicians, minor historical figures, niche scientific subfields) appear in tens or hundreds of tokens at best.

This creates a direct imbalance in the model's statistical knowledge. Well-represented entities have strong, consistent signal across many contexts, which allows the model to build a stable internal representation of the associated facts. Under-represented entities have weak or noisy signal, often from a small number of sources that may themselves be unreliable.

When asked about a tail entity, the model faces a harder inference problem: it cannot rely on many consistent exposures. Instead, it generalizes from surface-level patterns: "this looks like the type of question that should have a date/name/place answer." The model knows the form of the answer it should produce, and it knows the general vocabulary and discourse style associated with the topic domain, but the specific facts it generates are extrapolated from thin signal rather than reliably remembered.

The internet frequently disagrees with itself. For any given factual question, you can find confident conflicting answers across different sources. Consider a claim like "Person X received Award Y in year Z." Multiple sources might report different years. Multiple sources might attribute the same quote to different people. Dates for historical events sometimes vary by source, particularly when primary sources are rare.

The model sees all of these during training and has no oracle to adjudicate between them. What does it learn? It learns a weighted average of the conflicting signals, where the weights are proportional to how often each version of the claim appears. The most-cited version wins, which may or may not be the correct one.

For very common facts, the correct version is typically cited far more than any incorrect alternative, so the weighted average converges to something accurate. For uncommon or contested facts, the distribution of citations may not favor accuracy. The model's "consensus" answer may simply reflect which version of the claim was most frequently published rather than which was most carefully verified.

The model has no concept of source reliability. A well-edited Wikipedia article, a peer-reviewed journal paper, a tabloid blog post, and a social media thread all contribute to the training signal with equal weight per token. There is no quality weighting in the standard pretraining objective. Some training pipelines apply quality filters that upweight curated sources, but these filters are imperfect and cannot eliminate the fundamental equalization that the loss function performs.

This matters most for facts that are close to true. A completely fabricated claim may simply not appear in enough training data to produce a strong signal. A partially wrong claim, however, may appear in many sources (because most sources got most of it right while disagreeing on one detail), generating a muddled but confident-sounding mixture of correct and incorrect information.

Pretraining on web data is usually followed by supervised finetuning (SFT) on instruction-following datasets. These datasets are created by human annotators who are asked to write ideal responses to a given prompt. Annotators are human, and humans make mistakes, disagree, and have knowledge gaps of their own. More subtly, annotators writing responses for topics outside their expertise may produce fluent, authoritative-sounding text that is factually wrong.

The model trained on SFT data learns the style of confident, well-structured prose, because that is what annotators tended to produce when demonstrating good responses. The formatting and rhetorical pattern of confident knowledge-sharing is reinforced even when the underlying claims are uncertain or incorrect. This is not the annotators' fault: they are doing their best with the knowledge they have. But the structural consequence is that the model learns to sound more certain than it should.

The SFT problem also interacts with a selection bias: annotators are more likely to write detailed, authoritative responses for topics they know well, and shorter, more hedged responses for topics where they are less confident. This means the model receives many examples of detailed, confident prose for well-known topics and fewer examples of careful hedging for under-known topics, reinforcing the pattern of overconfidence precisely where it is least warranted.

Beyond individual annotator errors, SFT datasets also suffer from coverage gaps that parallel the pretraining data gaps. Instruction-following datasets are created with particular use cases in mind: question answering, summarization, code completion, and similar tasks. They may have thin coverage of rare query types where hallucination is most likely. This means the model gets very little practice with exactly the queries it is most likely to fail on, while getting abundant practice with the queries it would handle correctly regardless.

There is also a subtler issue with the temporal consistency of SFT annotations. Annotation projects often span weeks or months, and different annotators may have different levels of up-to-date knowledge about the same topics. A question about an evolving scientific field or a recent product may receive different answers depending on which annotator handled it and when. These inconsistencies enter the SFT dataset and train the model on conflicting "correct" responses for the same type of query, which can degrade calibration and produce inconsistent outputs at inference time.

The path from raw web data to training-ready tokens involves many processing steps, and each introduces its own opportunities for error propagation. Understanding this pipeline helps clarify why "better data" is a difficult challenge.

Raw text from web crawls is typically filtered using heuristics: minimum document length, minimum token entropy, language identification, perplexity filtering against a small reference model, and blacklist-based removal of known low-quality domains. Each filter catches some noise but introduces its own biases. Perplexity filtering against a language model, for example, tends to remove text that is unusual or non-standard in surface form, which can over-remove domain-specific technical content that is perfectly accurate but simply uses different vocabulary patterns than the reference model learned.

Deduplication is applied to remove duplicate and near-duplicate documents. Exact deduplication removes documents that appear multiple times verbatim. Fuzzy deduplication removes documents that are highly similar. Both are necessary to prevent overweighting of content that was widely republished. But deduplication also changes the effective frequency of entities: if many different web pages discuss the same fact using slightly different wording, deduplication may collapse them into fewer examples, reducing the training signal for that fact below what the raw token count suggests.

The result is that the model's training data is the output of a complex pipeline with many implicit choices, each of which shapes which facts get encoded reliably and which do not. The relationship between "how many times did fact X appear in raw web crawl data" and "how strongly is fact X encoded in the model" is mediated by all of these pipeline steps, making it difficult to predict or audit from the outside.

One of the most important and underappreciated causes of hallucination is a mismatch between how the model is trained and how it generates text. This mismatch is called exposure bias, and it emerges from a fundamental tension between training efficiency and inference realism.

During training, language models use a technique called teacher forcing. At every training step, when computing the probability of the next token, the model conditions on the ground-truth previous tokens from the training data, not on its own previous predictions.

For an input sequence where the asterisk denotes ground-truth tokens, training computes:

where:

• : the ground-truth token at position (the correct answer the model should predict)
• : the ground-truth tokens at all prior positions (the "teacher" signal that forces correct context)
• : the total sequence length
• : the model's predicted probability for the correct token, conditioned on perfect context

The superscript throughout indicates that these are ground-truth tokens taken from training data, not tokens that the model generated itself.

Teacher forcing is computationally efficient. Because all positions in the sequence can be computed in a single forward pass using attention masking, training is fast and parallelizable. The gradient signal is also clean: at every step, the model is evaluated against the correct answer given the correct context, producing a well-defined and unambiguous learning signal.

But this efficiency comes at a cost. The training condition (always given perfect context) never occurs at inference time.

At inference time, the model has no access to ground-truth previous tokens. It must condition on its own previously generated tokens:

where:

• : the token the model generates at step (using a hat to distinguish from ground-truth )
• : the model's own previously generated tokens, which form the inference context
• : the model's predicted probability for its next token, conditioned on its own prior outputs

The key contrast: during training, the conditioning context is always (perfect ground-truth). During inference, the conditioning context is (the model's own outputs, which may contain errors).

The distributions and are fundamentally different, because the conditioning context differs. The model was never trained to recover from its own mistakes. If it generates a slightly off-topic or inaccurate token early in a sequence, it must condition all future tokens on that error. Training never prepared it for this scenario, and the model has no learned error-correction behavior to draw on.

Exposure bias creates compounding errors. A small factual inaccuracy or semantic drift at position changes the conditioning context for position . The model, never trained on such contexts, generates the statistically most likely continuation given that erroneous context, which may compound the original error rather than correct it.

This is particularly visible in long-form generation. Ask a model to write a multi-paragraph technical explanation or a detailed biography, and early factual errors tend to cascade: the model continues to build on its own (incorrect) premises rather than reversing course. Reversal would require generating tokens that conflict with the highly probable completions that the erroneous context suggests, and nothing in the model's training incentivizes such self-correction.

Suppose the model commits an error at step . The log-probability of the correct answer at step given the error context is:

where:

• : the correct token steps after the error
• : correctly generated tokens before the error occurred
• : tokens generated after the error, each conditioned on the growing error context

Because the conditioning diverges increasingly from the correct prefix as grows, the model encounters an increasingly out-of-distribution sequence of contexts. Its probability estimates for these contexts are unreliable by construction, and the error compounds with each step.

We can quantify this divergence using KL divergence between the model's behavior in the correct-context case and the error-context case:

where:

• : the divergence at step between the distribution conditioned on correct context and the distribution conditioned on error context
• : the Kullback-Leibler divergence from distribution to distribution , measuring how much differs from
• : the model's next-token distribution conditioned on the correct prefix up to step
• : the model's next-token distribution conditioned on the error-containing prefix up to step

This divergence grows as the generated prefix diverges further from the true prefix. Since the model is never trained on diverged prefixes, is not bounded by the training objective and can grow arbitrarily with sequence length. This is why you tend to observe more hallucinations in long responses than in short ones: the opportunities for error accumulation multiply with sequence length.

Exposure bias is not unique to language models. The same issue arises in any sequence-to-sequence system trained with teacher forcing, including neural machine translation systems from the pre-transformer era. Various approaches have been proposed to reduce it: scheduled sampling, which randomly replaces ground-truth conditioning tokens with model-generated ones during training at increasing rates, is one. But none of these approaches fully close the training-inference gap, and most large language models are still trained with teacher forcing for practical efficiency reasons.

A practical observation that follows from exposure bias is that hallucination rates are not uniform across a response. Hallucinations tend to cluster later in long responses rather than appearing uniformly throughout. This matches the theoretical prediction: early in a response, the model is conditioned largely on the correct prompt (which falls within the training distribution), so its outputs are more reliable. As the model generates more of its own tokens, the conditioning context shifts increasingly toward generated content, moving further from the training distribution and accumulating more divergence. For practical deployments, this suggests that very long generated responses should be treated with more skepticism on their later sections than on their early ones, all else being equal.

Exposure bias also interacts with the specific model architecture in non-obvious ways. Autoregressive models that use full causal attention are more susceptible to long-range error accumulation than models with shorter effective context windows, because a model with a large effective attention range can be influenced by early errors across many subsequent steps. This is one reason why some researchers have explored recurrent or state-space architectures as alternatives: models with more bounded memory may sometimes be more resilient to error propagation, even if they sacrifice performance on tasks requiring long-range coherence.

Even a model trained on perfectly accurate data with no exposure bias would still hallucinate, because no training corpus is complete. Knowledge gaps are structural: there are things the model cannot know because no training data covered them, or because the coverage was insufficient to create reliable representations.

All large language models have a knowledge cutoff: a date after which no training data was collected. Events, discoveries, product releases, policy changes, and cultural shifts after this cutoff are invisible to the model. When asked about post-cutoff facts, the model has two choices: acknowledge ignorance, or extrapolate from prior patterns. Well-calibrated models do the former. Under-calibrated models do the latter, producing confident but outdated or entirely fabricated answers.

The problem is compounded by gradual knowledge representation near the cutoff. Events that occurred close to the training cutoff are under-represented even if they technically fall within the training window, simply because less content had been written about them by the time data collection ended. A major event two weeks before the cutoff may have thousands of times fewer training tokens than one from two years earlier, simply because the internet had not yet produced the full volume of analysis, commentary, and follow-up reporting that longer-established events accumulate.

The practical consequence is that the model's effective knowledge cutoff is "soft" rather than "hard": knowledge near the nominal cutoff date is less reliable than knowledge from earlier periods, and the model's confidence does not automatically adjust to reflect this gradient. You should treat model knowledge as degrading progressively as you approach the training cutoff, not as a sharp binary transition.

A useful mental model is to think of the training corpus as a snapshot of the internet taken over a particular window of time. The density of that snapshot is not uniform across time: older events have had more time to accumulate commentary, analysis, and cross-referencing, so they are more densely represented. Recent events, even within the training window, are sparsely represented. The model's knowledge is proportional to this density: reliable and detailed for well-established facts, thin and error-prone for recent or rapidly evolving situations.

This temporal thinning problem affects factual recall and the model's understanding of causal and temporal relationships. If an important development happened six months before the training cutoff and generated relatively little discussion by that time, the model may not have learned the full downstream implications of that development. It may know a fact occurred without understanding the context that would be obvious to anyone following events in real time. This produces a specific type of hallucination where the model acknowledges a fact but gets its significance or consequences wrong, because the contextual understanding that should surround the fact was never fully developed in the training data.

The training cutoff also interacts with temporal reasoning in complex ways. A model that was trained with data through a specific date does not automatically know that its knowledge stops there. It may not have a reliable internal representation of its own knowledge cutoff, and it may conflate the current date (which it does not know without being told) with the period it knows most about. This can produce errors where the model treats outdated information as current, failing to apply even the uncertainty it technically has access to.

Web-scale training data systematically under-represents specialized professional domains. Medical literature, legal case law, financial regulations, scientific preprints, proprietary engineering specifications, and internal business documents are less frequently crawled and less richly interconnected than general consumer content.

When a model encounters questions in these domains, it is often working from fewer, less consistent training signals. It may know the general vocabulary and discourse conventions of medicine or law without reliably encoding the specific facts. The result is text that sounds authoritative (correct register, correct structure, plausible terminology) while containing factually wrong claims about dosages, legal precedents, regulatory requirements, or technical specifications.

This is particularly dangerous because the surface quality of the output does not provide a reliable signal about its accuracy. A model that gets a basic biochemistry question wrong will often produce a response that sounds exactly as confident and well-structured as one that answers correctly. The fluency is consistent; the accuracy is not. Domain experts can detect the errors; non-experts may not be able to.

The under-representation of specialized domains is worsened by access restrictions. Much valuable scientific knowledge is behind paywalls, published in subscription journals that web crawlers cannot access. Legal databases, medical electronic health records, proprietary financial data, and internal organizational documents are not publicly accessible and thus not available as training data. This means that the most authoritative, carefully verified sources of specialized knowledge are often exactly the sources the model learned least from.

Language models do not have a symbolic world model. They do not represent "Paris is the capital of France" as a discrete logical fact accessible via a direct lookup. They represent the statistical co-occurrence of tokens like Paris, capital, and France across many billions of contexts, encoded in the weights of a neural network. For high-frequency, well-attested facts, this statistical representation is stable and generalizes reliably. For rare or domain-specific facts, the statistical signal is weak, and the model's representation of those facts is sparse, blurry, and susceptible to interference from similar but different facts.

This creates a phenomenon sometimes called the "stochastic parrot" effect: the model produces text that sounds like it contains knowledge because it has learned the surface form of knowledgeable discourse, without having a reliable underlying representation of the facts being asserted. This philosophical concern has practical consequences for how you should interpret model outputs.

When a model generates an authoritative-sounding answer about an obscure topic, there is no internal "confidence score" grounded in how much training data covered that topic. The fluency of the output is determined by the model's language modeling capability, which is a separate property from the accuracy of the facts it is asserting. A model can be extremely fluent about topics it knows nothing factual about, because fluency is a learned property of text generation, not a property of factual recall.

This distinction matters for hallucination diagnosis. A model that hallucinates "Paris" when asked about France's capital is exhibiting a different failure mode than a model that hallucinates the year a niche scientific paper was published. The former involves contradicting a strong, highly consistent statistical pattern (and is accordingly rare with modern models). The latter involves filling a gap in a weak pattern (and is accordingly common for tail knowledge). The surface behavior looks the same: a confident factual assertion. The underlying causes are completely different.

The absence of a structured world model also means that the model cannot systematically check its own answers for internal consistency. A symbolic system could, in principle, check whether the facts it is asserting about an entity are mutually consistent. A language model has no such mechanism. It generates text token by token based on local statistical dependencies, and it may produce a sequence that contains two mutually contradictory facts about the same entity without any internal signal that a contradiction has occurred.

Research in memorization and generalization has consistently shown that model accuracy drops sharply for low-frequency entities. Entities appearing fewer than roughly 100 times in training data are hallucinated at substantially higher rates than high-frequency entities. This follows naturally from the statistics: with limited signal, the model cannot reliably discriminate the correct facts associated with a rare entity from plausible-but-wrong alternatives drawn from similar, more common entities.

The model may, for example, know that academic researchers have PhDs, affiliations, publication records, and grant histories. When asked about an obscure researcher, it does not have reliable facts to report. But it knows what kinds of facts should be present in a researcher's profile, so it generates plausible-sounding facts of those types: a plausible affiliation, a plausible PhD institution, plausible-sounding paper titles. Each individual fact sounds reasonable for the type of entity being described; none of it is grounded in actual knowledge of that specific person.

This is a systematic asymmetry with real consequences: models are far more reliable when discussing well-known subjects than obscure ones, even when prompted with the same type of question. A question about Abraham Lincoln's presidency will be answered accurately. The same question structure applied to a minor historical figure will frequently produce fabricated specifics. The question format provides no signal about whether the model has the knowledge required to answer it reliably.

The tail entity problem is particularly acute for proper nouns. Common nouns (concepts, processes, definitions) tend to be explained consistently across many sources, even if any specific source is imperfect. Proper nouns (specific people, companies, places, events) have facts that are unique to that entity and must be memorized specifically. With insufficient training exposures, memorization fails and the model confabulates.

There is an interesting subtlety here regarding what "training frequency" measures. When researchers report that entities appearing fewer than some threshold number of times have high hallucination rates, they are typically counting the number of documents mentioning the entity by name. But the model's ability to learn reliable facts about an entity depends on how many times the entity is mentioned and how many times the specific facts being asked about are stated, verified, and contextualized. An entity can appear in many documents mostly as an incidental mention without any of those documents explaining the fact you are asking about. Frequency of mention and frequency of fact-stating are different quantities, and the latter is the more relevant predictor of hallucination risk.

This has a practical implication for prompt engineering and context provision. When you include relevant factual context about a tail entity in the prompt (the model's current context window), you are effectively providing the training-data-like signal that was absent during pretraining. Retrieval-augmented generation exploits this: by retrieving accurate documents about the query entity and including them in context, you give the model the factual grounding it needs to answer reliably even for tail entities. The model's parametric knowledge of the entity may be weak, but its in-context reasoning ability can use the retrieved facts if they are present. This is why retrieval augmentation is often the most effective single intervention for tail entity hallucination specifically.

Even if a model had perfectly accurate training data and no exposure bias, the generation process itself introduces pressures that favor fluent hallucination over honest uncertainty. The generation process is, fundamentally, a sequence of probability distributions: at each step, the model assigns a probability to every token in its vocabulary and selects one. Nothing in this process encodes "I don't know." The only question is which token the model generates; in the absence of reliable knowledge, it generates a plausible-sounding one.

Language models are trained to always continue a sequence. The training objective rewards producing probable continuations; there is no built-in mechanism for "stopping because I don't know" or for outputting a distinguished "uncertain" token. If a question implies that a factual answer exists, the model generates the most statistically plausible answer, not "I am not sure."

This is fundamentally different from a retrieval system, which can return zero results when no relevant document is found. A language model's probability distribution over vocabulary always has a mode, and the model will always produce the mode (or a sample near it). There is no probability mass on "abstain." The generation process is complete only when a termination token is produced or a length limit is reached. Abstention is not a natural outcome; it is a learned behavior that must be explicitly trained.

Instruction-tuning can teach models to express uncertainty with phrases like "I'm not certain, but..." or "I don't have reliable information about that." But this is learned as a surface pattern rather than a calibrated epistemic state. The model learns to prefix uncertain-sounding outputs with uncertainty qualifiers while still generating the most probable factual claim, which may or may not be accurate. The uncertainty language is decorative unless it is backed by a reliable mechanism for detecting when the model is operating outside its knowledge boundary.

This matters because the form of a question creates what can be called completion pressure: a strong signal about what type of answer is expected. A well-formed factual question creates an expectation of a factual answer. The model, having learned from millions of question-answer pairs where questions reliably receive detailed answers, is under structural pressure to produce a detailed answer even when no reliable answer is available. The path of least resistance is to generate a plausible-sounding response rather than to acknowledge ignorance, because the former produces lower perplexity under the distribution of question-answer pairs in training.

The web is largely written by people who are confident in what they say. Academic papers state conclusions directly. News articles report facts declaratively. Forum responses answer questions authoritatively. Technical documentation makes definitive claims. The model learns the rhetorical style of confident, fluent prose because that is what the training data predominantly contains.

This means the model learns a prior in favor of confident assertion. Hedged, uncertain language ("it is unclear whether," "some sources suggest," "I am not certain") appears far less frequently in training data than confident declarations. As a result, the model's default generation mode is confident, not hedged, independent of whether it has reliable knowledge to back up that confidence.

The effect is reinforced by the structure of instruction tuning and RLHF annotation. Human annotators rating responses tend to prefer responses that sound knowledgeable and direct over responses that sound uncertain and hedged, all else being equal. This annotator preference, encoded into the reward model used for RLHF, pushes the model further toward confident generation. The very feedback process meant to align the model with human preferences can inadvertently train it to be overconfident, because confidence is often mistaken for competence in evaluating text quality.

If a question's phrasing strongly implies that a specific type of answer exists, the model generates an answer of that type even without reliable knowledge. "Who invented X?" implies a person's name is the answer. "When was X established?" implies a year. "What is the capital of Y?" implies a city. The model is sensitive to these pragmatic implications because it has seen millions of question-answer pairs in training, and question-answer co-occurrence creates strong prompt-completion associations.

This means that a factual question about a subject the model does not know can still receive a confident, structured, plausible-sounding answer, because the question's format telegraphs exactly what kind of answer is expected. The model knows the answer's form; it does not know the answer's content; it fills the form with the most statistically plausible content given the question's subject.

The practical implication is that the model is better thought of as an extremely sophisticated pattern matcher than as a reliable oracle. For high-frequency questions with well-established answers, the pattern matching points to the right answer. For rare questions with uncertain or unknown answers, the pattern matching produces an answer of the right form but wrong content.

The specific decoding strategy used at inference time affects hallucination rates in measurable ways. Each strategy makes different tradeoffs between fluency, diversity, and factual reliability.

Greedy decoding (always selecting the argmax token) tends to produce locally optimal but globally inconsistent outputs. Each token is individually the most probable given context, but this greedy choice may commit to a factual direction that becomes increasingly inconsistent as the sequence grows. Greedy decoding also tends to be repetitive and can get stuck in loops where the most probable next token keeps reinforcing the same context.

Top- (nucleus) sampling selects the next token by sampling from the smallest vocabulary subset whose cumulative probability exceeds a threshold . Higher values allow more randomness, which increases creative diversity but also increases the probability of selecting factually deviant tokens. Lower values reduce diversity but push generation toward the most probable tokens, which may reinforce confident errors rather than preventing them.

Temperature scaling modifies the token probability distribution before sampling. For a vocabulary distribution with logits , temperature transforms the softmax as:

where:

• : the probability assigned to token after temperature scaling
• : the unnormalized logit for token from the model
• : the temperature parameter (positive real number)
• The sum in the denominator is over all tokens in the vocabulary

When , the distribution sharpens toward the most probable tokens, reducing diversity and sometimes reducing certain types of hallucination. When , the distribution flattens, increasing diversity at the cost of more random sampling that may deviate from factual content. Neither direction reliably eliminates hallucination: sharpening can reinforce confident factual errors, and flattening can introduce new ones.

Beam search maintains multiple candidate sequences in parallel, selecting the globally highest-probability continuation over the full beam. While this typically improves coherence compared to greedy decoding, it does not improve factual accuracy: it finds the most probable hallucinated story as effectively as the most probable factual one. Beam search may produce more coherent hallucinations, because the beam selection pressure favors internally consistent sequences over correct but locally less probable ones.

To make exposure bias concrete, consider a simplified example of how error accumulates across generation steps when a model is asked to produce a factual historical account.

Suppose the model is generating a paragraph about a specific historical event. At step 10, it generates an incorrect date: "1914" instead of the correct "1917." This incorrect date becomes part of the conditioning context for all subsequent tokens. The model now operates as if the event happened in 1914.

The correct year 1917 has high probability when conditioned on the correct prefix (the model has strong associations between the event's name and the year 1917 from training). But conditioned on a prefix that already states 1914, the model's distribution shifts: it may now amplify details that are more consistent with 1914 (other events from that year, the political climate of that period, figures who were prominent then) and suppress details associated with 1917. The hallucination cascades, not because the model lacks knowledge about 1917, but because the erroneous context now points in a different direction.

The formal probability divergence illustrates this. Let be the correct continuation and be the continuation after an error at step . We measure the divergence between the model's behavior in the two cases:

where:

• : the divergence at step between the distribution conditioned on correct context and the distribution conditioned on error context
• : the Kullback-Leibler divergence from distribution to distribution , measuring how much differs from
• : the model's next-token distribution conditioned on the correct prefix up to step
• : the model's next-token distribution conditioned on the error-containing prefix up to step
• and : shorthand for the full prefix sequences and respectively

This divergence grows as the generated prefix diverges further from the true prefix. Since the model is never trained on diverged prefixes during teacher-forced training, is not bounded by the training objective and can grow arbitrarily with sequence length. This is the mathematical reason why hallucination compounds in long-form generation.

The key insight from this worked example is that exposure bias does not require the model to "not know" the correct answer. The model may have reliable knowledge of the correct year in the abstract. The hallucination arises because the error changes the input distribution in a way that activates different, incorrect associations. This is why hallucination is not always fixed by giving the model more facts: sometimes the problem is not missing knowledge but corrupted context.

We will implement three demonstrations: a training data quality analyzer showing how factual accuracy varies across source domains, a visual simulation of exposure bias divergence showing how errors accumulate, and an entity frequency analysis illustrating the tail entity hallucination pattern.

Simulating Training Data Quality DistributionLink Copied

Visualizing Data Quality by DomainLink Copied

We simulate how a single early error propagates across a generation sequence, modeling the divergence between the true distribution and the model's conditioned distribution under different initial error magnitudes.

The following analysis models how entity frequency in training data relates to hallucination rates, based on patterns observed in memorization research.

We simulate how completion pressure interacts with knowledge availability, showing the choices a model faces when generating answers to different types of factual questions.

The key parameters that affect generation pressure in practice are:

• temperature: Scales the logit distribution before sampling. Higher values ( ) flatten the distribution, increasing randomness and hallucination risk. Lower values ( ) sharpen it toward the most probable tokens, which may reinforce confident errors.
• top_p (nucleus sampling threshold): Controls the fraction of probability mass considered at each step. Higher values increase diversity at the cost of factual consistency.
• top_k: Limits the number of candidate tokens at each step. Lower values reduce but do not eliminate hallucination risk.
• repetition_penalty: Penalizes tokens that appeared earlier in the sequence. Can prevent repetitive loops but may also discourage the model from correctly reusing technical terms.
• max_new_tokens: Controls sequence length. Longer generations face higher cumulative exposure bias divergence, increasing hallucination risk as the sequence grows.

In practice, these four causes do not operate in isolation. They interact and reinforce each other in ways that make the hallucination problem harder than any single cause would suggest.

Consider what happens when all four operate simultaneously: a model trained on noisy web data (training data issues) encounters a question about a tail entity (knowledge gap). It generates a confident-sounding response (completion pressure) in a long-form answer where an early error compounds (exposure bias). Each cause amplifies the others. The noisy training data means the model has unreliable signal for the tail entity. The knowledge gap means the model must extrapolate. The completion pressure means it will not abstain. The exposure bias means that whatever error it commits early in the response will cascade through the rest.

This interaction also explains why hallucination patterns vary systematically across model sizes and training configurations. Larger models trained on more data tend to hallucinate less on high-frequency facts: their stronger signal for well-attested entities reduces the training data issue and the knowledge gap problem. But they may not reduce hallucination on tail entities proportionally, because scale helps only where data is already sufficient. And larger models can be better at fluent, confident generation, which may paradoxically worsen completion pressure and overconfident priors.

The interaction between causes also explains why mitigation strategies that address only one cause produce partial results. Retrieval augmentation (adding external knowledge at inference time) addresses knowledge gaps but leaves exposure bias and completion pressure untouched. Careful data curation addresses training data issues but cannot retroactively fill knowledge gaps for entities that never appeared in sufficient quantity. Better uncertainty calibration through RLHF addresses the surface expression of completion pressure but does not change the underlying lack of a world model or the exposure bias structure. These interventions complement each other rather than substitute for each other.

Understanding the interaction of causes also changes how you should think about evaluation. A benchmark that tests only high-frequency factual questions will not reveal tail entity hallucination. A benchmark that tests only short responses will not reveal exposure bias accumulation. A benchmark that only measures whether answers are correct or incorrect will not reveal miscalibration in uncertainty expression. Complete hallucination evaluation requires explicitly probing each of the four causes, because a model can perform well on one dimension while failing badly on another.

The interaction of causes is also sensitive to model scale in ways that are not always intuitive. Scaling up model size and training data simultaneously tends to reduce hallucination on well-documented facts because larger models have greater capacity to memorize and accurately recall high-frequency information. But scaling also increases the fluency and coherence of generated text, which means that hallucinations, when they do occur, tend to be more convincing and harder to detect. A small model's hallucination may be obviously wrong or awkwardly phrased, making it easy to catch. A large model's hallucination may be perfectly fluent, internally consistent within the generated text, and superficially plausible, making it much more dangerous for users who cannot independently verify the claims. The very property that makes large models useful (fluent, coherent, confident generation) is also what makes their hallucinations more harmful.

This creates a critical calibration challenge as models improve. Progress in language modeling capability, as measured by perplexity or instruction-following quality, does not automatically translate into progress in hallucination reduction. A model that is much better at following instructions and producing fluent prose may hallucinate at similar rates as a simpler model on tail entities, while producing hallucinations that are far more convincing. Evaluating models on hallucination separately from general capability is therefore essential, and using hallucination metrics that account for the deceptiveness of the errors (not just their frequency) is important for capturing the practical risk.

Understanding hallucination causes also requires honesty about the limitations of the mechanistic account we have built.

Training data issues, exposure bias, knowledge gaps, and generation pressure are real phenomena, but quantifying their individual contributions to a specific hallucination is not straightforward. When a model generates an incorrect fact, we cannot run a controlled experiment to isolate which cause was responsible. The four mechanisms are entangled throughout training and inference, and their contributions overlap rather than being cleanly separable.

Empirical research has made progress on individual dimensions: studies measuring entity frequency and hallucination rates, analyses of knowledge cutoff effects, and ablations of decoding strategies. But a unified causal model that assigns blame proportionally does not exist. Practitioners must work with incomplete attribution, using the framework of four causes as a diagnostic lens rather than a precise measurement tool.

The measurement problem is particularly acute for training data issues. We generally do not have access to the exact training corpus of any commercial model, and even for open-weight models, corpus composition is often reported at a high level without the detailed provenance information that would allow precise analysis of how errors in specific data sources propagate to specific hallucinations.

Web-scale training means training on web-scale noise. Human curation at the scale of hundreds of billions of tokens is not feasible. Automated filtering can remove low-quality content (boilerplate, spam, obviously AI-generated text) but cannot reliably identify factual errors at scale. There is no practical filter that reads text and checks it against a source of truth for every factual claim.

The fundamental tension between data scale and data quality remains unresolved. Larger models trained on more data tend to hallucinate less on high-frequency facts while remaining vulnerable on tail entities, suggesting that scale helps only where data is already sufficient. For the long tail of entities and facts, more data does not help if the additional data contains similar noise levels to what was already present.

One partial response is to mix web-scraped data with high-quality curated sources (textbooks, encyclopedias, scientific literature) at ratios that give disproportionate weight to verified content relative to its raw volume in the web corpus. This helps but cannot overcome the fundamental problem that high-quality sources cover only a fraction of the world's facts.

Instruction tuning and reinforcement learning from human feedback (RLHF) significantly improve a model's ability to express appropriate uncertainty and abstain from answering when knowledge is insufficient. But they do not retroactively fill knowledge gaps or undo exposure bias at the level of the base model. A model finetuned to say "I'm not sure" more often may produce fewer overconfident hallucinations but still generates factually wrong answers when it does commit to a response.

The causes are encoded in the base model during pretraining. Finetuning modifies the model's response style around those causes without fixing them. This is an important asymmetry: finetuning can make the model appear better calibrated in terms of expressed confidence, but the underlying knowledge gaps and exposure bias patterns remain. If you evaluate only the model's tendency to express uncertainty, finetuning looks effective. If you evaluate whether its stated facts are correct when it does commit to an answer, the picture is less clear.

The practical impact of these causes is asymmetric by application domain. For entertainment and creative writing, hallucination is often acceptable or even desirable: a fictional narrative benefits from imagination, and factual accuracy is not the goal. For medical information, legal research, financial analysis, and factual journalism, the same failures are potentially harmful. A model that confidently fabricates a drug dosage or a legal precedent can cause real harm in ways that a fictional story cannot.

Understanding that hallucination is especially severe for tail entities and post-cutoff knowledge allows product designers to make principled deployment decisions. High-stakes factual queries can be routed to retrieval-augmented systems that ground answers in verified external sources. Fact-checking layers can be applied for sensitive domains. Query types where models are structurally most likely to be wrong (obscure entities, recent events, domain-specific regulations) can be explicitly excluded from model scope or handled with additional verification steps.

The causes framework also informs user interface design. If a deployment involves tasks with high hallucination risk (asking for specific historical facts about minor figures, asking for recent events, asking for detailed technical specifications), communicating appropriate uncertainty to users becomes essential. A user who understands that the model may be unreliable on these specific query types can apply appropriate skepticism; a user who treats all model outputs as equally reliable cannot.

Hallucination is a structural consequence of how language models are trained and how they generate text, not a random failure mode. The four root causes are:

• Training data issues: Web corpora contain factual errors, misinformation, and knowledge imbalances that the next-token prediction objective cannot distinguish from accurate content. The model learns errors with the same confidence as truths, weighted by their frequency in the corpus.
• Exposure bias: Teacher forcing creates a training-inference mismatch where the model is never trained to handle its own errors. Early mistakes in generation cascade across subsequent tokens because the model was only ever trained on perfect-context conditioning, not on error-containing contexts.
• Knowledge gaps: Training cutoffs, tail entity under-representation, restricted access to specialized domains, and the absence of a symbolic world model mean that many factual questions fall outside the model's reliable knowledge boundary, yet the model cannot reliably identify when it is operating outside that boundary.
• Generation pressure: Completion pressure from the always-generate objective, overconfident priors learned from confident web content, prompt-answer alignment bias, and decoding artifacts cause the model to generate confident assertions regardless of its actual knowledge state.

These causes interact and amplify each other. Addressing any one in isolation produces partial improvements. The most reliable approaches, which we will examine in the next chapter on hallucination mitigation, target multiple causes simultaneously: retrieval augmentation for knowledge gaps, better data curation for training data issues, scheduled sampling variants for exposure bias, and calibration training for generation pressure. Understanding the causes is the prerequisite for understanding which combination of mitigations is warranted for any specific deployment context.

QuizLink Copied