Retrieval-augmented generation (RAG): from naive to agentic
Retrieval-augmented generation, or RAG, lets a language model look up documents at question time and answer from them instead of relying only on what it memorized. The idea was introduced by Lewis and colleagues in 2020 and is now the standard way to give a language model fresh, private, or specialist knowledge.
The basic pipeline
- Chunk the documents into passages.
- Embed each passage as a vector and store it in an index.
- Retrieve the passages closest to the question.
- Generate an answer from the question plus the retrieved passages.
This is naive RAG. It works for simple questions and fails in predictable ways: a chunk cuts the answer in half, the relevant passage is not among the top results, or the model ignores the context and answers from memory.
Advanced RAG
Advanced RAG adds steps before and after retrieval: rewriting the query, combining keyword and vector search, filtering by metadata, and re-scoring the shortlist with a cross-encoder reranker. Contextual retrieval prepends a short description of the surrounding document to each chunk before it is embedded, which reduces the number of chunks that lose their meaning when they are cut out.
Modular and graph RAG
Modular RAG treats retrieval, routing, and generation as parts that can be rearranged for each task. GraphRAG builds a knowledge graph from the corpus and answers questions that span many documents by reading entities, relations, and community summaries.
Agentic RAG
Agentic RAG lets a model plan. It decides whether to retrieve, which source to use, whether the result is good enough, and whether to search again. It handles multi-step questions better, at the cost of more calls, more latency, and more ways to fail.
Evaluating a RAG system
Evaluate retrieval and generation separately. For retrieval, ask whether the relevant passage was found and whether irrelevant passages crowded it out. For generation, ask whether the answer is faithful to the retrieved text and whether it answers the question. RAGAS packages metrics such as faithfulness, answer relevancy, context precision, and context recall. As with any benchmark, read the protocol before the number.
RAG and tables
Much of the knowledge a business needs sits in tables, and chunking a table into text loses its structure. Two routes help: keep tables as tables and retrieve them with their schema, or answer from them with a model that reads tables natively. A Data Language Model is built for the second route.
Frequently asked questions
What is RAG?
A technique where a language model retrieves relevant documents first and writes its answer from them, which reduces made-up facts and keeps answers current.
What is the difference between naive and advanced RAG?
Naive RAG retrieves once and reads. Advanced RAG adds query rewriting, hybrid search, and reranking around retrieval.
When should I use agentic RAG?
When questions need several steps or several sources. For simple lookups, a well-tuned advanced pipeline is cheaper and more predictable.
How do I measure a RAG system?
Measure retrieval quality and answer faithfulness separately, for example with RAGAS, and test on questions from your own data.