How do you reduce LLM hallucinations with prompts?
Ground the model in reference text and instruct it to answer only from that text, explicitly allowing 'I don't know.' Ask for direct quotes or citations, break the task into steps, and use a low temperature. Providing trusted source material in the prompt (retrieval/RAG) is the single most effective prompt-level fix for hallucinations.
Why — the first-principles explanation
LLMs are next-token predictors trained to produce plausible text, not to look up facts. They have no built-in database of truth; their 'knowledge' is statistical patterns baked into their weights. When a prompt asks about something those patterns don't pin down, the model still produces a fluent, confident answer. That mismatch between fluency and accuracy is a hallucination.
The fix follows directly from the cause. If the model invents facts because it lacks them, put the facts in the prompt. Pasting a trusted document, or retrieving one automatically (RAG), lets the model copy from real text instead of guessing. Then add a rule: 'answer only using the text above; if it's not there, say you don't know.' Removing the pressure to always answer removes the incentive to fabricate.
Two more levers help. Asking for quotes or citations forces the model to point at supporting words, which it cannot do for invented claims. And giving it room to reason step by step, or splitting a big question into smaller checked steps, reduces the shortcut-guessing that produces errors. A low temperature makes output less random. None of these make a model perfectly truthful, but together they cut hallucinations sharply.
An example that makes it click
Imagine asking a student a history question with a closed textbook. To avoid looking dumb, they might confidently make up a date, that's a hallucination. Now hand them the open textbook and say, 'Only answer from these two pages, and if it's not there, just say I don't know.' Suddenly the made-up dates stop, because they can read the real answer and they're allowed to admit when it's missing.
Adding 'point to the exact sentence you used' makes it even harder to bluff. Prompting an LLM works the same way: give it the source, let it say 'I don't know,' and ask it to show its evidence.
How to do it
- Paste trusted reference text into the prompt (or use retrieval/RAG to fetch it) so the model has real facts to draw from.
- Add an explicit instruction: 'Answer only using the provided text; if the answer isn't there, say I don't know.'
- Ask the model to quote or cite the exact sentences it based its answer on.
- Break complex questions into smaller steps and let the model reason step by step before the final answer.
- Set a low temperature (around 0 to 0.3) for factual tasks to reduce random, made-up output.
- Verify high-stakes answers against the source yourself, prompting reduces but never fully eliminates hallucinations.
Key facts
- A hallucination is fluent, confident text that is factually wrong; it stems from LLMs predicting likely tokens, not retrieving verified facts.
- Grounding the prompt in provided reference text (RAG) is the most effective prompt-level way to cut hallucinations, matching OpenAI's advice to 'provide reference text.'
- Explicitly allowing the model to answer 'I don't know' reduces fabricated answers by removing pressure to always respond.
- Asking for citations or direct quotes forces the model to ground claims in the supplied source.
- Lower temperature (near 0) reduces randomness for factual tasks; prompting alone cannot make a model 100% reliable.
▶ The 60-second explainer (script)
How do you reduce LLM hallucinations with prompts? The most powerful trick is to give the model the facts instead of making it guess. Language models don't look things up, they predict likely words, so when they don't know something they'll still answer confidently and sometimes make it up. That's a hallucination. The fix flows from the cause. First, paste trusted reference text into your prompt, or use retrieval, so the model has real material to work from. Second, add a clear rule: answer only from this text, and if the answer isn't here, say 'I don't know.' That single line stops a lot of bluffing. Third, ask it to quote the exact sentences it used, made-up claims have no sentence to point to. For factual tasks, keep the temperature low, and break hard questions into smaller steps. None of this makes a model perfect, so always double-check high-stakes answers. But grounding plus permission to say 'I don't know' cuts hallucinations dramatically.
What authoritative sources say
People also ask
What is an LLM hallucination?
It's when a model produces fluent, confident text that is factually false. It happens because LLMs predict likely words rather than retrieving verified facts.
Does telling the model 'don't make things up' work?
A little, but it's weak on its own. Pairing it with provided reference text and permission to say 'I don't know' works far better.
Can prompting eliminate hallucinations completely?
No. Prompting sharply reduces them, but no prompt makes a model 100% reliable. Always verify high-stakes facts against the source.
What is RAG and how does it help?
Retrieval-augmented generation automatically fetches relevant documents and inserts them into the prompt, so the model answers from real sources instead of guessing.