NIST’s Generative AI Profile uses the term confabulation for “confidently stated but erroneous or false content”, including output that contradicts the prompt or the model’s own earlier statements, and notes that it is known colloquially as hallucination or fabrication. Research surveys use hallucination for generated text that is unintended or unfaithful to its source, across summarization, dialogue, question answering and translation.
encyclopedia
Hallucination
Published Updated
Reviewed by Coy Robison, Founder, on
Because large language models generate text by predicting likely continuations rather than retrieving stored facts, they can produce plausible, confident and wrong answers. NIST calls this confabulation and treats it as a core risk of generative AI, especially where people act on the output. Hallucinations cannot currently be eliminated, but they can be made much rarer and easier to catch by grounding answers in supplied sources, letting the model admit uncertainty, requiring citations and measuring the error rate.
Definition
Instances where an AI model generates false, nonsensical, or unverified information, presenting it confidently as fact.
Need this applied?
We help teams go from definitions to deployed workflows—safely and fast.
Generative models approximate the statistical distribution of their training data; the same next-token prediction that often yields accurate text can also yield text that is factually wrong or inconsistent. OpenAI’s research adds an incentive problem: standard training and accuracy-only evaluations reward a lucky guess over an honest “I don’t know”.
- • No grounding: the model answers from its weights instead of a supplied source.
- • Missing or niche knowledge: the model fills the gap with something plausible.
- • Ambiguous prompts: the model guesses at what was meant.
- • Scoring that rewards guessing: models learn to answer rather than abstain.
Where it does the most damage
NISTThe risk comes from people acting on false content, often because of how confident it sounds. NIST highlights open-ended, long-form answers and domains needing specialist context, gives healthcare as an example, and flags consequential decision-making as the place to monitor most closely. Models can also produce confabulated reasoning or citations that make a wrong answer look justified.
The most effective controls change what the model answers from and what it is allowed to say.
- • Ground answers in retrieved or supplied documents rather than the model’s memory.
- • Explicitly allow the model to say it does not know.
- • For long documents, have the model extract direct quotes first and answer from them.
- • Require a citation for each claim and retract claims without one.
- • Restrict the model to the provided material for factual questions.
Accuracy alone hides the problem, because a model that always guesses can score well. OpenAI recommends scoring that penalizes confident errors more than abstentions. For grounded applications, groundedness checks compare each answer with its source material and flag statements the sources do not support.
FAQ
Can hallucinations be completely eliminated?
Currently, no. Grounding, permission to express uncertainty and citation checks reduce them significantly, but vendor guidance is explicit that they do not eliminate them, so critical information still needs validation.
Is a hallucination a bug that can be patched?
Not in the usual sense. NIST describes confabulation as a natural result of how generative models are designed, so it is managed with system design, guardrails and evaluation rather than fixed once.
Why does the model sound so confident when it is wrong?
Fluent, confident phrasing is what the model learned to produce, and many evaluations reward answering over abstaining. Confidence in the wording therefore says little about whether the content is correct.
Does retrieval-augmented generation stop hallucinations?
It makes them less frequent, because the model can answer from relevant source text rather than memory. The model can still misread the sources or answer when retrieval found nothing useful, so groundedness checks remain worthwhile.
Email this summary + checklist
Get a copy of “Hallucination” and an AI readiness checklist in your inbox.