What is RAG (retrieval-augmented generation)?

What it means at work

Instead of relying only on what a model learned in training, a RAG system retrieves relevant passages from your files, policies or knowledge base and hands them to the model with your question. This keeps answers tied to your material and lets them show sources. Many ‘chat with your documents’ and internal help-desk tools work this way.

What the sources say

• Where the term comes from. The paper by Lewis et al. introduced RAG as models which combine pre-trained parametric and non-parametric memory. In their setup the second memory was a dense vector index of Wikipedia.
• What problem it targets. The paper says that providing provenance for decisions and updating world knowledge remain open research problems for models alone.
• Reported result. Lewis et al. found RAG models generate more specific and factual language than a parametric-only baseline.
• How it runs. The Anthropic glossary says the data is retrieved at runtime when a query is sent to the model.
• Lewis et al. report RAG models set the state-of-the-art on three open domain QA tasks.
• Liu et al. found performance is often highest when relevant information occurs at the beginning or end of the input context, which affects how retrieved passages are ordered.
• Where it helps. The Anthropic glossary says RAG can be particularly useful for tasks that require up-to-date information.

An example

An HR team loads the employee handbook into a RAG-based assistant. A new hire asks about parental leave; the assistant finds the leave section, answers from it and links to the page. If the handbook does not cover the question, a well-built system says so.

A common misconception

RAG reduces made-up answers but does not eliminate them. In the same vein, the Anthropic glossary states that the effectiveness of RAG depends on the quality and relevance of the external knowledge base. If the search pulls the wrong passage, or the documents are outdated or contradict each other, the answer can still be wrong. Good results depend on clean, current source material and on checking cited passages.

Sources

• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (arXiv): “models which combine pre-trained parametric and non-parametric memory for language generation”
• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (arXiv): “a dense vector index of Wikipedia”
• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (arXiv): “providing provenance for their decisions and updating their world knowledge remain open research problems”
• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (arXiv): “generate more specific … factual language than a state-of-the-art parametric-only seq2seq baseline”
• Anthropic, Claude Platform Docs glossary: “The data is retrieved at runtime when a query is sent to the model”
• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” (arXiv): “set the state-of-the-art on three open domain QA tasks”
• Liu et al., “Lost in the Middle: How Language Models Use Long Contexts” (arXiv): “performance is often highest when relevant information occurs at the beginning or end of the input context”
• Anthropic, Claude Platform Docs glossary: “RAG can be particularly useful for tasks that require up-to-date information”
• Anthropic, Claude Platform Docs glossary: “the effectiveness of RAG depends on the quality and relevance of the external knowledge base”