On this page
An AI hallucination is not a lie: the model does not know it is wrong. That follows from how a large language model works. It predicts the most likely next piece of text. When it does not know the answer, it writes what sounds likely. Ask for the price of a bathroom and you get a figure that fits bathrooms, not the one in your price list. OpenAI wrote in September 2025 that the way models are trained and tested rewards guessing over admitting uncertainty.
The cases that made it expensive
Researchers used the word around 2018 for translation systems that added sentences missing from the source. ChatGPT made it common vocabulary, and two cases made it concrete for business. In June 2023 a New York court fined two lawyers $5,000 for filing a brief with case law ChatGPT had invented. In February 2024 Air Canada had to compensate a passenger because its website chatbot had described a bereavement fare refund its policy did not offer. The airline argued the chatbot was responsible for its own words; British Columbia's Civil Resolution Tribunal disagreed.
One customer question, answered with and without sources
CustomerWhat does a new zinc gutter cost per metre, and how long is the guarantee?
Without sources
A zinc gutter costs 85 euros per metremade up, with a ten-year guaranteemade up on the work.
From your own sources
A zinc gutter costs 62 euros per metre, fitting included, with a five-year guarantee.
Worked example with a fictional roofer. On the right, every part of the answer sits next to where it came from.
Four kinds of error in a business
| Kind | Example | What stops it |
|---|---|---|
| Invented fact | Opening hours or a price that is wrong | Answers drawn from your own sources |
| Invented source | A regulation or study that does not exist | Showing the source, with a link, and checking it |
| Wrong number | A calculation or date that is slightly off | Doing the maths in ordinary code, not the model |
| Invented policy | A discount or guarantee you never offered | Hard limits on what it may promise |
How often it happens varies widely by model and task. Summarising a text you supply rarely goes wrong. Asking for a fact that is nowhere in the context goes wrong far more often.
Three safeguards we build into every AI system
- Answers from your own sources. For each question the model receives the relevant passages from your price list, manual or policy, and the instruction to answer only from them. If it finds nothing, it says so. That is retrieval-augmented generation.
- The source on screen. Every answer points to the document or page it came from, so anyone in doubt can check, and you see at once where an error started.
- A person approves what goes out. Quotes, customer emails and anything with a price or promise is drafted by AI and approved by a person. Internal work can move faster.
These cost little next to building the system. Sources must be in order and searchable; showing them is a design choice; approval is one click per message, against the minutes it would take to write. What a full assistant costs is set out in AI chatbot cost.
Three habits when you use ChatGPT yourself
Give the model the text you are asking about instead of asking what it knows. Ask for the source and open it. Never let it do arithmetic or set dates unchecked. Which assistant suits you is compared in ChatGPT vs Claude. If you deploy AI with customers, the EU AI Act adds transparency duties; see the EU AI Act guide. And if an assistant describes your own company wrongly, clear facts on your own site are the fix, the work called GEO.
What companies ask about invented answers
Can hallucinations be prevented entirely?
Not entirely, but they can be reduced sharply: answers from your own sources, sources on screen, a model allowed to say it does not know, and a person checking what goes out.
Am I liable for what my chatbot says?
In the Air Canada case the company was. Assume customers treat your chatbot as your company, and build limits into it.
Do newer models still hallucinate?
Less, but never zero. Each generation improves on facts it was trained on; a fact about your own business is something no model was trained on.