Özer Cem KELAHMETKELAHMET
Topics · Generative AI and LLMs

RAG and grounding: giving language models the right documents

Retrieval-augmented generation puts relevant documents into the prompt so the model answers from them. This page covers each part of a RAG system and the variants that fix its weak spots.

Overview

A language model's own knowledge is frozen at training time and can be wrong. RAG fetches passages from your documents at question time and asks the model to answer from them, with citations. It is the most common way to build assistants over private or changing information.

The pipeline

Step Choice to make Read more Try it
Chunking Size and boundaries of passages Simulation
Embedding Model that turns text into vectors Embeddings Simulation
Index Vector database and index type HNSW, pgvector, Qdrant Simulation
Retrieval Vector, keyword or both Hybrid search, BM25
Re-ranking Reorder the top results precisely Cross-encoder reranking
Generation Prompt with passages and citation rules Prompt engineering
Evaluation Faithfulness, relevance, recall RAGAS Simulation

Variants that fix common failures

Failure Fix Read more
The question does not match document wording Rewrite or expand the query, or embed a hypothetical answer Query rewriting, HyDE
Chunks lose their context Prepend document context to each chunk Contextual retrieval
Wrong passages retrieved Check relevance and retry or search the web Advanced RAG
Questions need several hops Let an agent plan searches Agentic RAG
Answers need relations across documents Build a knowledge graph GraphRAG

The RAG types simulation and agentic RAG simulation compare them on the same questions; the GraphRAG simulation shows graph traversal.

RAG or fine-tuning?

Use RAG for knowledge that changes or must be cited; use fine-tuning to change style, format or behaviour. Many systems use both. The article RAG types explained covers the trade-offs in depth.

Frequently asked questions

How big should chunks be?

Commonly 200 to 800 tokens with some overlap, tuned on your own evaluation set; structure-aware chunking by headings usually beats fixed sizes.

Do long context windows make RAG unnecessary?

No. Retrieval keeps cost and latency down, scales beyond any window, and gives precise citations.

Summary

Chunk well, embed and index, retrieve with hybrid search, re-rank, generate with citations, and measure. Add query rewriting, agents or graphs only where evaluation shows the simple pipeline fails.

Back to the topic

This page is part of Generative AI and LLMs, which maps every concept, tool, simulation and article on the subject.

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open