Retrieval-Augmented Generation Overview

Retrieval-Augmented Generation (RAG) overview

RAG lets a language model look up external knowledge at answer time instead of relying only on what is in its weights. The term comes from Lewis et al. (NeurIPS 2020), which paired a seq2seq generator with dense retrieval over Wikipedia and set state of the art on three open-domain QA benchmarks. Today the word covers a whole family of pipelines around any LLM, not one specific model.

The baseline pipeline

1. Ingest: parse, chunk, optionally enrich chunks (Anthropic’s contextual retrieval prepends a short generated context to each chunk; it reported top-20 retrieval failures falling from 5.7% to 2.9% with contextual embeddings plus BM25, and to 1.9% with reranking added).
2. Index: dense vectors in an ANN index (ann-index-algorithms) plus a lexical index.
3. Retrieve: often hybrid, fused with RRF (hybrid-search-and-rank-fusion).
4. Rerank: a cross-encoder or late-interaction model re-scores the top candidates (rerankers-and-cross-encoders, exploring-colbert-with-ragatouille).
5. Generate with the retrieved context and citations.

Retrieval quality is capped by the query, so rewriting, HyDE and decomposition help (query-understanding).

Where the field is in 2026

• Agentic RAG: an agent decides whether, when and what to retrieve, reflects on results and iterates, using planning, tool use and sometimes several agents (survey: arXiv 2501.09136; earlier building blocks include Self-RAG and Corrective RAG). See _agentic-moc.
• Long context vs RAG: long-context models often beat chunk-based RAG on QA benchmarks, while RAG keeps advantages on dialogue-style and general questions and on cost (Li et al., arXiv 2501.01880). Anthropic’s guidance: below roughly 200k tokens you can put the whole knowledge base in the prompt, with prompt caching. Above that, or when knowledge changes or needs access control, retrieval remains necessary.
• GraphRAG: knowledge-graph indexes for corpus-wide “sensemaking” questions, see graphrag, LightRAG, Neo4j.
• Managed RAG: hosted stores such as gemini-file-search remove the vector DB from your stack.
• Memory for agents is the stateful sibling of RAG: agent-memory-systems.
• Evaluation is first-class: score retrieval (ir-ranking-metrics) and answers (rag-evaluation) separately.

Related

_rag-relevance-moc · embedding-models · context-engineering

Sources

• Lewis et al., RAG (2020): https://arxiv.org/abs/2005.11401 (accessed 2026-09-30)
• Anthropic, Contextual Retrieval (2024-09-19): https://www.anthropic.com/news/contextual-retrieval (accessed 2026-09-30)
• Agentic RAG survey: https://arxiv.org/abs/2501.09136 (accessed 2026-09-30)
• Self-RAG: https://arxiv.org/abs/2310.11511 ; CRAG: https://arxiv.org/abs/2401.15884 (accessed 2026-09-30)
• Long Context vs. RAG: https://arxiv.org/abs/2501.01880 (accessed 2026-09-30)