What is RAG in AI?
RAG means retrieval-augmented generation: the AI searches a document collection first, then writes its answer using what it found. Introduced by Patrick Lewis and 11 co-authors at NeurIPS 2020, it pairs the model's trained-in memory with a searchable outside index — so you can update facts and show sources without retraining the model.
Why — the first-principles explanation
A plain language model keeps everything it knows inside its own weights. The 2020 RAG paper calls this parametric memory — knowledge baked into billions of numbers during training. It works surprisingly well, but it has three structural problems the authors named directly: the model's ability to "access and precisely manipulate knowledge" is limited, it can't show you where an answer came from, and updating what it knows means retraining it.
RAG's move is to stop treating memory as something that must live in the weights. It adds a second kind of memory the paper calls non-parametric — in the original experiments, a dense vector index of Wikipedia sitting outside the model, reachable through a trained neural retriever. Your question gets converted into a vector, the retriever finds passages whose vectors sit nearby, those passages get dropped into the model's context, and the model writes its answer while reading them.
That one change fixes all three problems at once, and the reason is economic, not magical. Retraining a large model costs millions of dollars and takes weeks. Re-indexing a document costs a fraction of a cent and takes seconds. So facts that live in the index can change hourly. Because you know exactly which passages were retrieved, you can cite them — provenance comes free. And the model is now reading rather than recalling, which is a much easier job: getting a phone number right from text in front of you beats dredging it out of compressed memory.
What RAG does not do is make a model truthful. Retrieval can pull the wrong passages, or nothing useful at all, and the model may still write around gaps or contradict what it retrieved. Lewis and colleagues reported RAG produced "more specific, diverse and factual language" than a parametric-only baseline — better, which is not the same as solved. RAG narrows the gap between what the model knows and what it says. It doesn't close it.
An example that makes it click
Two students take the same history exam. One studied for months and must answer from memory alone. She's good, but on the date of an obscure treaty she half-remembers something and writes down a confident guess — she can't tell you which textbook it came from, and if the textbook was revised last year, she'd never know.
The second student takes it open-book. Same brain, same training — but she can flip to page 214, read the treaty date, and copy it correctly. She can also tell you it came from page 214. That's RAG. The model didn't get smarter; it got a library card. And note the catch that survives: if she grabs the wrong book, or the index sends her to the wrong page, she'll write a confident wrong answer anyway — now with a citation attached.
Key facts
- RAG was introduced in "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks," submitted to arXiv on May 22, 2020 and published at NeurIPS 2020 by 12 authors led by Patrick Lewis.
- The original system combined a pre-trained seq2seq model (parametric memory) with a dense vector index of Wikipedia (non-parametric memory) accessed by a pre-trained neural retriever.
- The paper compared two formulations: one conditioning on the same retrieved passages across the whole generated sequence, and one able to use different passages for each token.
- RAG models set state-of-the-art results on three open-domain question-answering tasks, beating both parametric seq2seq models and task-specific retrieve-and-extract architectures.
- The authors found RAG generated "more specific, diverse and factual language" than a state-of-the-art parametric-only baseline — an improvement, not an elimination of errors.
- Retrieved passages must fit inside the model's context window, which is measured in tokens and capped for every model.
▶ The 60-second explainer (script)
RAG stands for retrieval-augmented generation. It means the AI looks things up first, then answers using what it found. Here's why that matters. A normal language model keeps everything it knows inside its own weights — knowledge baked in during training. That creates three problems. It can't reliably pin down precise facts. It can't tell you where an answer came from. And updating it means retraining the whole thing. RAG fixes all three with one idea: stop forcing memory to live in the weights. Put the documents in a searchable index outside the model. Your question gets turned into a vector, a retriever finds the closest passages, and the model writes its answer while reading them. The economics are the whole story. Retraining a big model costs millions and takes weeks. Re-indexing a document costs a fraction of a cent. So facts can change hourly. And because you know which passages came back, citations are free. Patrick Lewis and his co-authors published this at NeurIPS in 2020, using a vector index of Wikipedia. But be clear on the limit. RAG does not make a model truthful. Retrieval can grab the wrong passage, and the model can still write around the gaps. It's an open-book exam. Open book still beats closed book — but only if you grab the right book.
What authoritative sources say
People also ask
Does RAG stop hallucinations?
It reduces them, and does not stop them. If retrieval returns the wrong passages or none, the model can still produce a confident wrong answer — sometimes with a citation attached, which makes it look more trustworthy than it is.
Is RAG the same as fine-tuning?
No. Fine-tuning changes the model's weights to shift style or skills. RAG leaves the weights alone and hands the model documents at question time. Use fine-tuning to change how it behaves, RAG to change what it knows.
Why not just paste documents into the prompt?
That is RAG, minus the retrieval step — and it only works while your documents fit in the context window. Retrieval exists because you may have millions of pages and room for a handful.
What is a vector index?
A database of passages stored as lists of numbers that capture meaning, so text with similar meaning sits nearby. It lets the retriever find relevant passages even when they share no keywords with your question.
Does RAG make answers current?
Only as current as the index. RAG moves the staleness problem from the model's training cutoff to your re-indexing schedule, which is a far cheaper problem — but it is still a problem you own.
The same question, asked other ways
- What does RAG stand for in AI?720/mo