How do you stop AI hallucinations?

A hallucination is when an AI states something false or made up with the same confident tone as something true: an invented statistic, a nonexistent court case, a wrong date, a quote nobody said. See the glossary entry.

Why it happens

Language models produce plausible next words from patterns, not by looking facts up in a database. When the right information is missing, the pattern-completing behavior can fill the gap with something that sounds right. A 2025 paper argues that language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty. Microsoft’s Copilot documentation says its responses are not guaranteed to be 100% factual. Microsoft also says users should still use their judgment when reviewing the output before sending them to others.

Prevention: reduce the chance

1. Provide the source. “Using only the attached report, answer…” is much safer than asking from general knowledge. Retrieval-style setups that first look up your documents are built around this idea (see RAG).

2. Allow “I don’t know”: Anthropic guidance says to explicitly give Claude permission to admit uncertainty. Add: “If the document doesn’t say, write ‘not stated.’ Do not guess.”

3. Ask for evidence: “Quote the sentence that supports each claim, with page number.” Anthropic suggests making a response auditable by having Claude cite quotes and sources for each of its claims. For long documents, Anthropic suggests asking Claude to extract word-for-word quotes first.

4. Narrow the task. Broad questions invite invention. Specific extraction tasks are safer than open-ended recall.

5. Use tools with search when facts matter. Tools that browse and cite sources let you open the source; but check that the source actually says it.

6. Break the task up. Extract facts first, then analyze them in a second step you can review.

Detection: catch it after

1. Verify anything checkable: names, dates, figures, citations, URLs. Open every link.
2. Ask the model to audit itself: “List any claims in your answer that you are not confident in.” It is not perfect, but it can surface weak spots.
3. Ask a second time, differently: if answers conflict, something is unreliable. Anthropic says that if Claude gives inconsistent outputs when run several times on the same prompt, that could indicate hallucinations.
4. Cross-check with a second source for anything important.
5. Recompute numbers in a spreadsheet instead of trusting mental arithmetic.

Why retrieval alone does not fix it

Tools built on retrieval are often sold as hallucination-free. An academic evaluation found that Lexis+ AI from LexisNexis and the Westlaw research and Ask Practical Law AI tools from Thomson Reuters each hallucinate between 17% and 33% of the time. The same study found that the LexisNexis and Thomson Reuters tools hallucinate less than general-purpose chatbots such as GPT-4. Anthropic says its techniques significantly reduce hallucinations in Claude but do not eliminate them entirely. Plan for checking even when the tool cites sources.

High-risk situations

• Legal, medical and financial claims.
• Citations to cases, studies or statistics.
• Anything going to a client, regulator or the public.
• Facts about recent events or niche topics.

In these, treat AI as a drafter and researcher whose work is always verified. See whether AI can review contracts.

A simple rule

Match your checking effort to the cost of being wrong. A brainstorm needs no checking; a board figure needs source verification.

Build the habit into workflows

When you automate something, add a check that does not rely on the AI, like recalculating totals. See how to automate weekly reports.

Day two of Learn AI Fast teaches how to catch hallucinated data before it costs you, as part of a session on risks, privacy and policy. For better instructions in general, see how to write better prompts.

Sources

• Kalai, Nachum, Vempala and Zhang, Why Language Models Hallucinate (arXiv 2509.04664): “sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty”
• Microsoft Learn, Data, Privacy, and Security for Microsoft Copilot: “users should still use their judgment when reviewing the output before sending them to others”
• Anthropic, Reduce hallucinations (Claude Platform Docs): “Explicitly give Claude permission to admit uncertainty”
• Anthropic, Reduce hallucinations (Claude Platform Docs): “having it cite quotes and sources for each of its claims”
• Anthropic, Reduce hallucinations (Claude Platform Docs): “ask Claude to extract word-for-word quotes first”
• Anthropic, Reduce hallucinations (Claude Platform Docs): “Inconsistencies across outputs could indicate hallucinations”
• Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (arXiv 2405.20362): “Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time”
• Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools (arXiv 2405.20362): “hallucinations are reduced relative to general-purpose chatbots”
• Anthropic, Reduce hallucinations (Claude Platform Docs): “these techniques significantly reduce hallucinations”