What is AI hallucination?

Why — the first-principles explanation

Use hallucination as a practical label for an output that fails a factual or evidentiary check: a nonexistent citation, an invented number, a wrong person attributed to a quote, a fabricated API field or a confident answer to an unanswerable question. Not every bad result has the same cause. A stale source, a broken calculator, a malicious document, an ambiguous request and a model-generated claim can all produce an incorrect answer, so a useful investigation asks which component failed instead of blaming the AI as one black box. A language model generates a continuation from patterns learned during training and post-training. Fluency and truth are related in ordinary text but are not identical objectives, and the generation process does not automatically open a primary source or prove every sentence. Research on language-model hallucinations argues that training and evaluation can reward guessing when the model is uncertain; this helps explain why a polished answer can carry no reliable signal of truth. The model’s wording of confidence is also generated output, not an independent measurement. Grounding changes the workflow by supplying explicit evidence. Retrieval-augmented generation can give a model passages to use, and Google’s grounding documentation says search grounding can improve factual accuracy and provide inline citations. Neither makes the retrieved source correct, current or relevant. A citation can point to a page that does not support the sentence, and a model can misread or overextend a good source. Inspect the cited passage, check dates and identities, and keep an explicit unknown or cannot verify state. Reliable systems use layers. Define the claims and risk level, retrieve approved primary sources, ask for a constrained answer with evidence spans, validate numbers and tool outputs programmatically, evaluate known and unanswerable cases, and route consequential decisions to a person. OpenAI describes evals as tests against specified style and content criteria and recommends datasets with human-provided ground truth; NIST’s AI Risk Management Framework is intended to help manage AI risks across design, development, use and evaluation. These are operational controls, not a promise that a model has become infallible. For a low-stakes brainstorm, a plausible draft may be acceptable after a quick review. For medical, legal, financial, identity, safety, academic attribution or customer-facing claims, treat every material assertion as a claim to verify. Record the model, prompt, retrieved sources, tool calls, date and reviewer so you can reproduce an error, correct the page or roll back a workflow. The goal is not to make a model sound less confident; it is to make unsupported claims harder to publish and easier to detect.

An example that makes it click

How to do it

1. State the task, audience and decision risk. Separate brainstorming from claims that will affect health, law, money, identity, safety, school, customers or public reputation.
2. List the claims the answer must establish and the cases where the correct response is unknown, unavailable or requires a human. This creates an abstention target instead of rewarding a guess.
3. Prefer current primary sources, approved internal documents or authoritative databases. Record the URL, owner, version and date, and do not treat a search snippet as evidence.
4. Ground the request in those sources and ask the model to attach each material claim to a supporting passage. Require it to distinguish quoted facts, calculations, assumptions and inferences.
5. Constrain the output with a schema or checklist: answer, evidence, source date, uncertainty, missing information and next verification step. A format alone does not make the content true, so validate it.
6. Run deterministic checks for arithmetic, dates, identifiers, URLs, permissions, API fields and required policy language. Open citations and compare the quoted passage with the claim.
7. Evaluate a representative test set containing ordinary questions, rare facts, ambiguous prompts, stale documents, adversarial instructions and deliberately unanswerable cases. Include human ground truth.
8. Set a risk-based review threshold. Automatically publish only low-risk, well-grounded outputs; require human approval or a safe refusal for consequential claims and novel edge cases.
9. Log prompt, model or mode, retrieved context, tool calls, citations, output, reviewer, date and final correction. Monitor drift after a model, source, prompt or product change.
10. When an error appears, correct the source or workflow, add a regression case, notify affected users and roll back if necessary. Do not hide a recurring failure behind more confident wording.

Key facts

• An AI hallucination is a false, unsupported or misattributed generated claim that can sound plausible; the label describes an output failure, not a human-like perception.
• A factual error can arise from model generation, stale or irrelevant retrieval, a faulty tool, an ambiguous request or a malicious source; diagnose the failed component before choosing a fix.
• Research on language-model hallucinations argues that training and evaluation can reward guessing over acknowledging uncertainty, so fluent confidence is not a truth signal.
• Retrieval-augmented generation adds explicit source passages and can improve factuality, but retrieval quality, source authority, date and synthesis still need checking.
• Google’s grounding documentation says search grounding can reduce hallucinations and return inline citations; a citation still needs to be opened and matched to the claim.
• OpenAI’s evals guidance treats evaluations as tests against specified criteria and supports datasets with human-provided ground-truth outputs; a demo or benchmark score is not a universal hallucination rate.
• NIST’s AI Risk Management Framework is a voluntary framework for managing trustworthiness and AI risks across design, development, use and evaluation; it is not a guarantee of model accuracy.
• Asking “are you sure?” or lowering randomness may change wording or sampling but does not replace evidence, abstention tests, tool validation or human review.
• A high-stakes workflow should expose uncertainty, source dates, failure paths and a human owner instead of treating a model’s confidence language as a calibrated probability.
• The strongest measurement is domain-specific: grounded claim accuracy, citation support, refusal or abstention quality, harmful error rate, latency, cost and correction effort on representative cases.

What authoritative sources say

People also ask

The same question, asked other ways

This page answers one intent expressed in 3 phrasings. How the index is organized →

• What are AI hallucinations?
• What is an AI hallucination?

Related questions