Özer Cem KELAHMETKELAHMET
AI Simulations · Retrieval (RAG)

RAG types compared: naive, hybrid, rerank, HyDE, corrective, agentic

Retrieval-augmented generation is a family of techniques. Run seven of them on one small corpus and see which ones find the evidence when the question uses different words from the documents.

Interactive simulation. It needs JavaScript; the text below explains what it shows.

What you are looking at

The corpus has eight short policy passages; the relevant ones are marked. Scores are computed in your browser with simple word vectors and BM25, so each strategy's ranking is real for this corpus. The right side scores context recall and precision and compares all strategies on the chosen question.

Things to try

  • Ask the 'money back' question: naive search misses, rewriting and HyDE recover.
  • Compare hybrid search with naive search on the data question.
  • Open Agentic RAG to see how splitting the question into sub-queries helps.

How it works in real systems

Production RAG systems usually combine hybrid search with a cross-encoder reranker, add a grading step for important questions and route hard multi-hop questions to an agent. The RAG types guide covers each in depth.

Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

Step 1 · Index the knowledge base

Install the libraries and index a small knowledge base with a sentence-embedding model. Normalised embeddings make the dot product equal to cosine similarity.

pip install sentence-transformers rank-bm25 numpy openai
import numpy as np
from sentence_transformers import SentenceTransformer
chunks = [
    "Most items can be returned within 30 days of delivery for a full refund.",
    "Damaged or defective items qualify for a full refund or a free replacement; send a photo of the damage to support first.",
    "Refunds are issued within five business days after the returned item reaches our warehouse.",
    "Parcels damaged in transit must be reported to the carrier within 48 hours of delivery.",
    "The Ridge 2 tent weighs 1.9 kg and uses Aurora ripstop fabric for its fly sheet.",
    "Aurora ripstop fabric is woven by Lumen Textiles at its mill in Porto, Portugal.",
    "Customer complaints in Q3 were mostly about late deliveries during the September peak.",
]
emb = SentenceTransformer("all-MiniLM-L6-v2")                 # 384 dimensions, small and fast
E = emb.encode(chunks, normalize_embeddings=True)
def dense_search(q, k=3):
    s = E @ emb.encode([q], normalize_embeddings=True)[0]
    top = np.argsort(-s)[:k]
    return [(int(i), float(s[i])) for i in top]

Step 2 · Three retrievers side by side

Vector search, BM25 and their fusion, on the same questions. Each returns chunk indices.

from rank_bm25 import BM25Okapi
tok = lambda s: s.lower().replace(",", " ").replace(".", " ").split()
bm25 = BM25Okapi([tok(c) for c in chunks])
def bm25_search(q, k=3):
    s = bm25.get_scores(tok(q)); return [int(i) for i in np.argsort(-s)[:k]]
def rrf(*lists, k=60):
    score = {}
    for l in lists:
        for r, i in enumerate(l): score[i] = score.get(i, 0) + 1 / (k + r + 1)
    return sorted(score, key=score.get, reverse=True)
for q in ["Can I get my money back if my tent arrived torn?", "Who makes the fabric used in the Ridge 2?"]:
    d = [i for i, _ in dense_search(q, 5)]; b = bm25_search(q, 5)
    print(q, "\n  dense", d[:3], "\n  bm25 ", b[:3], "\n  fused", rrf(d, b)[:3])

Step 3 · Generate the answer

A small helper to call a chat model (any OpenAI-compatible endpoint works, including a local server):

from openai import OpenAI
client = OpenAI()                                              # reads OPENAI_API_KEY
def ask(prompt, model="gpt-4o-mini"):
    r = client.chat.completions.create(model=model, messages=[{"role": "user", "content": prompt}], temperature=0)
    return r.choices[0].message.content

Put the retrieved chunks in the prompt, numbered, and ask for an answer that cites them and admits when the context is not enough.

def answer(question, ids):
    ctx = "\n".join(f"[{n+1}] {chunks[i]}" for n, i in enumerate(ids))
    return ask(f"Answer using only the context. Cite sources like [1]. If the context is not enough, say you don't know.\n\nContext:\n{ctx}\n\nQuestion: {question}")

Step 4 · Measure before you switch strategy

Label a few questions with the chunks that answer them and compute recall@3 for each retriever. Change one thing at a time.

labelled = {"Can I get my money back if my tent arrived torn?": {1, 3, 0}, "Who makes the fabric used in the Ridge 2?": {4, 5}}
def recall_at_k(search, k=3):
    return np.mean([len(set(search(q)[:k]) & rel) / len(rel) for q, rel in labelled.items()])
print("dense", recall_at_k(lambda q: [i for i, _ in dense_search(q, 5)]))
print("bm25 ", recall_at_k(bm25_search))
print("fused", recall_at_k(lambda q: rrf([i for i, _ in dense_search(q, 10)], bm25_search(q, 10))))

More simulations

← Sampling · All AI Simulations · Agent run →

Last updated Nov 17, 2026
Type to search.
    ↑↓ navigate↵ open