What is AI hallucination?
An AI hallucination is output in which a language model presents information, a source or a figure that does not exist, fluently and confidently, as if it were true. It cannot be eliminated entirely, but RAG, source citation, systematic testing and human approval at critical steps can bring it down to an acceptable level for business use.
Definition
An AI hallucination is output in which a language model presents information, a source or a figure that does not exist, fluently and confidently, as if it were true.
How do hallucinations arise?
A large language model generates its answer token by token based on probability; it does not look facts up in a knowledge base. Topics that are rare in training data, very recent events, questions that demand exact figures or sources, and vague instructions all raise the risk. Typical examples: a citation to a paper that does not exist, a wrong article number in a law, an invented product feature.
How can you reduce them?
- Provide context. Ground the model in company documents with RAG and ask it to cite sources.
- Allow it not to know. State explicitly in the prompt that it should say “I don’t know” when the sources do not cover a question.
- Test. Measure every change against an evaluation set of real questions.
- Add human approval. Have a person approve output in steps involving money, contracts or customer communication.
Local law and less-covered languages
On topics such as Turkish law, tax rules or local procedures, models may have seen far less training data than for English-language subjects, which means more risk. Model output on regulation, KVKK or e-invoicing rules should not be used without checking official sources. Who performs those checks, and how, belongs in your AI governance policy.
Related terms
- LLM (Large Language Model)An LLM (large language model) is an AI model trained on very large amounts of text that answers questions, summarises, translates and writes by predicting what text should come next.
- RAG (Retrieval-Augmented Generation)RAG (Retrieval-Augmented Generation) is an AI architecture that lets a language model retrieve relevant passages from a chosen document collection, such as a company's own documents, and use them as context before it writes an answer.
- Prompt engineeringPrompt engineering is the practice of writing, testing and refining the instructions, examples and context given to a language model so that it produces consistent, accurate results.
- AI governanceAI governance is the set of policies, roles and processes that determine how an organisation selects, uses, oversees and manages the risks of its AI systems.
Frequently asked questions
Why do hallucinations happen?
Language models do not check facts; they produce the continuation that looks most plausible in context. When a model lacks knowledge on a topic or the question is ambiguous, it can produce an answer that sounds reasonable but is wrong.
Does a more capable model solve hallucination?
It can reduce it, but it does not solve it. In business use, feeding the model the right context, asking it to cite sources and checking critical outputs matter more than the choice of model.
Can hallucinations have legal consequences?
Yes. Wrong information given to a customer, an inaccurate contract summary or a reference to a regulation that does not exist can create liability. Human review is advisable for anything sent to customers or authorities.
Let’s talk about your transformation
Tell us which work repeats every week. We will tell you honestly whether it is worth automating.
How we start
- A free first callWe pin down the problem and the goal together.
- A 2-week discovery sprintWe look at the data, the process and the tools, and decide where to start.
- A written proposalScope, architecture and timeline, in writing.