What is AI hallucination?
AI hallucination is when a model states something false as confidently as something true — an invented citation, a fake statistic, a court case that never existed. OpenAI researchers argued in September 2025 that it isn't a mysterious glitch: models hallucinate because training and grading reward guessing over admitting uncertainty.
Why — the first-principles explanation
The word "hallucination" is a bad metaphor that keeps people from understanding the problem. A hallucinating person sees something that isn't there and believes it. A language model has no beliefs and isn't perceiving anything. It's doing arithmetic: given the tokens so far, score every possible next token and pick a likely one. Nothing in that process contains a step where truth is checked. Fluency is the objective; accuracy is a side effect that usually comes along because true statements were common in the training text.
So the real question isn't why models hallucinate — it's why they're ever right. And once you frame it that way, the answer follows. A false statement and a true statement can be equally probable continuations. "The study was published in the Journal of Clinical Oncology in 2019" is a beautifully typical sentence whether or not that study exists. The model isn't reaching for a fact and missing. It's generating a well-shaped sentence, which is exactly what it was built to do. Kalai and colleagues at OpenAI put it precisely: hallucinations "originate simply as errors in binary classification" — if incorrect statements can't be distinguished from facts, hallucinations arise through natural statistical pressures.
Then comes the part that turns a bug into an epidemic: how models are graded. Most benchmarks score like a multiple-choice exam with no penalty for wrong answers. Say "I don't know" and you score zero. Guess, and you might score a point. So a model optimized to do well on tests learns that bluffing beats abstaining — every time, on average. The OpenAI researchers call models "good test-takers" and argue this is why hallucination persists even in state-of-the-art systems. Their proposed fix is notably not a better model: it's changing how existing benchmarks score uncertainty, a socio-technical fix rather than a technical one.
This is why hallucination shows up worst exactly where you'd least want it: obscure facts, specific numbers, citations, case law, dosages. Common knowledge is heavily represented in training text, so the probable answer is usually the true one. Rare specifics aren't — so the model falls back on producing something shaped like the right answer. The more precise and obscure your question, the more the output resembles a plausible guess wearing the costume of a fact.
An example that makes it click
Picture a student who has read an enormous number of books but is taking an oral exam with no notes. The examiner asks for the exact page where a quote appears. She doesn't know. But she also knows that saying "I don't know" scores zero, while a confident answer might land. So she says "page 214" — in the same steady voice she used for everything she actually knew.
That's a hallucination. She isn't lying; lying requires knowing the truth and hiding it. She's producing the shape of an answer because the shape is what's rewarded. And here's the cruel detail: her voice doesn't change. The made-up page number sounds exactly as certain as the real ones. If she stammered when guessing, you'd catch it instantly — and the whole problem would evaporate. She doesn't. That's why you have to check.
Key facts
- OpenAI researchers Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala and Edwin Zhang published "Why Language Models Hallucinate" on arXiv (2509.04664) on September 4, 2025.
- The paper argues hallucinations "originate simply as errors in binary classification" — if incorrect statements cannot be distinguished from facts, hallucinations arise through natural statistical pressures during pretraining.
- It attributes persistence to evaluation: models are "optimized to be good test-takers, and guessing when uncertain improves test performance," because most benchmarks penalize saying 'I don't know.'
- The authors' proposed remedy is socio-technical — modifying the scoring of existing dominant benchmarks — rather than adding new hallucination evaluations or building better models.
- Hallucinations persist in state-of-the-art systems as of 2026 and are not an engineering defect awaiting a patch.
- Retrieval-augmented generation reduces but does not eliminate hallucination: Lewis et al. (NeurIPS 2020) found RAG produced more factual language than parametric-only baselines, an improvement rather than a fix.
▶ The 60-second explainer (script)
AI hallucination is when a model states something false with exactly the same confidence it states something true. An invented citation. A fake statistic. A court case that never existed. And the word hallucination is a terrible metaphor, because it makes this sound mysterious. It isn't. A model has no beliefs and isn't perceiving anything. It does arithmetic: given the text so far, score every possible next chunk and pick a likely one. Nowhere in that process is there a step that checks whether something is true. So the real question isn't why models hallucinate. It's why they're ever right. And the answer is that true statements were common in the training text — so the probable answer is usually the true one. But a false sentence can be just as probable as a true one. The sentence 'the study was published in the Journal of Clinical Oncology in 2019' is a perfectly typical sentence whether or not that study exists. The model isn't reaching for a fact and missing. It's making a well-shaped sentence. That's the job. Now the part that makes it worse. OpenAI researchers argued in September 2025 that grading is the culprit. Most benchmarks work like a multiple-choice test with no penalty for guessing. Say 'I don't know' and score zero. Bluff, and you might score a point. So models learn that guessing beats admitting uncertainty. Their fix isn't a better model — it's changing how benchmarks score uncertainty. Which is why hallucination is worst exactly where it hurts most: obscure facts, specific numbers, citations, dosages.
What authoritative sources say
People also ask
Can hallucination be fixed completely?
No approach so far eliminates it. The OpenAI paper frames it as arising from statistical pressures inherent to the training pipeline, and proposes reducing incentives to guess rather than claiming a cure.
Does asking 'are you sure?' help?
Barely, and it can backfire. The model's stated confidence is generated text too, subject to the same pressures. It may cave and change a correct answer simply because being challenged makes agreement the probable next move.
Why does it invent citations specifically?
Because citations have a strong, learnable format — author, journal, year — and formats are exactly what next-token prediction is best at. Producing a citation-shaped string is easy; producing a real one requires knowledge it may not have.
Do newer models hallucinate less?
Generally yes on common facts, and the paper notes hallucinations persist even in state-of-the-art systems. Better models can also make it more dangerous by making wrong answers more polished and harder to spot.
Is the model lying to me?
No. Lying requires knowing the truth and choosing to hide it. The model has no separate store of truth to consult — it's generating probable text, and a false statement can be as probable as a true one.
The same question, asked other ways
- What are AI hallucinations?1,000/mo
- What is an AI hallucination?720/mo