Hallucination
An AI hallucination is when a language or generative model produces content that is plausible-sounding but factually incorrect, fabricated, or unfaithful to its sources — the single biggest failure mode of LLMs in regulated and high-stakes domains.
Full definition
Hallucinations arise because LLMs are trained to produce probable text, not true text. They have no built-in fact-check, no internal "I don't know" signal, and they will often produce confident, well-formed answers about subjects they have no reliable information on. Mitigations include retrieval-augmented generation (RAG), grounding with verified sources, fine-tuning on domain data, output validation, and using reasoning models that can flag uncertainty.
Why it matters
Hallucination is the #1 reason AI projects fail in regulated industries. RAG plus citation-grounded answers typically reduces hallucination rates by 30-60%. Any production AI system serving customers, patients, or regulated decisions needs an explicit hallucination-mitigation strategy and continuous evaluation.
Example
In 2023 a US lawyer submitted a brief citing 6 court cases that ChatGPT had hallucinated. The cases did not exist; the lawyer was sanctioned. The case (Mata v. Avianca) is now a standard cautionary example in legal AI policy.
Related terms
- Retrieval-Augmented Generation (RAG)Retrieval-Augmented Generation (RAG) is a technique where a language model retrieves relevant documents from an external knowledge base before generating a response, reducing hallucination by 30-60% and enabling citation of sources.
- Large Language Model (LLM)A Large Language Model (LLM) is a deep neural network — almost always a transformer — trained on hundreds of billions to trillions of words to predict the next token, and to generate, summarize, translate, or reason over text.
- Generative AIGenerative AI is a class of AI systems that produce new content — text, images, code, audio, or video — by learning the distribution of their training data and sampling from it, rather than classifying or predicting from existing inputs.
- Explainability (XAI)Explainability (XAI) is the property of an AI system whose decisions can be understood by humans — through model-level documentation, prediction-level attributions, and counterfactual explanations — and a regulatory expectation in finance, healthcare, hiring, and other high-stakes domains.
- AI GovernanceAI governance is the framework of policies, roles, controls, and processes an organization uses to ensure its AI systems are lawful, safe, fair, accountable, and aligned with business intent — across the full lifecycle from problem framing to retirement.
Source & further reading
Primary source: Ji et al. — "Survey of Hallucination in Natural Language Generation" (ACM Computing Surveys) (2023).
Citation policy: this entry is part of the AIDOLS AI Implementation Glossary and may be quoted for research, journalism, and education with attribution to aidolsgroup.com/no/glossary/hallucination/.