# Self-RAG: the model decides when to retrieve and critiques itself

Instead of a fixed pipeline, a Self-RAG model is trained to judge its own retrieval and output with special tokens, and to keep only the segments that pass its own critique.

<div data-sim="ragx" data-v="selfrag"><p>Interactive simulation. It needs JavaScript; the text below explains what it shows.</p></div>

## What you are looking at

Step 1 is the retrieve decision, step 2 shows a candidate per passage with its IsRel, IsSup and IsUse critiques, step 3 keeps the best-critiqued segments. All RAG simulations share one knowledge base of 18 chunks from 11 documents of a fictional outdoor-gear shop, and three questions: one phrased differently from the documents, one that needs two documents, and one about the whole corpus.

## Things to try

- Find the passage the model marks relevant but not supporting.
- See how the usefulness stars change the final order.
- Compare with corrective RAG, which uses a separate grader instead of the model itself.

## How it works in real systems

Self-RAG (Asai et al., 2023) fine-tunes the generator to produce reflection tokens. Most teams approximate it with prompted self-checks in an agent loop.

## Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

<div class="cl-start"></div>

### Step 1 · Index the knowledge base

Install the libraries and index a small knowledge base with a sentence-embedding model. Normalised embeddings make the dot product equal to cosine similarity.

```bash
pip install sentence-transformers rank-bm25 numpy openai
```

```python
import numpy as np
from sentence_transformers import SentenceTransformer
chunks = [
    "Most items can be returned within 30 days of delivery for a full refund.",
    "Damaged or defective items qualify for a full refund or a free replacement; send a photo of the damage to support first.",
    "Refunds are issued within five business days after the returned item reaches our warehouse.",
    "Parcels damaged in transit must be reported to the carrier within 48 hours of delivery.",
    "The Ridge 2 tent weighs 1.9 kg and uses Aurora ripstop fabric for its fly sheet.",
    "Aurora ripstop fabric is woven by Lumen Textiles at its mill in Porto, Portugal.",
    "Customer complaints in Q3 were mostly about late deliveries during the September peak.",
]
emb = SentenceTransformer("all-MiniLM-L6-v2")                 # 384 dimensions, small and fast
E = emb.encode(chunks, normalize_embeddings=True)
def dense_search(q, k=3):
    s = E @ emb.encode([q], normalize_embeddings=True)[0]
    top = np.argsort(-s)[:k]
    return [(int(i), float(s[i])) for i in top]
```

### Step 2 · Reflection as prompts

A small helper to call a chat model (any OpenAI-compatible endpoint works, including a local server):

```python
from openai import OpenAI
client = OpenAI()                                              # reads OPENAI_API_KEY
def ask(prompt, model="gpt-4o-mini"):
    r = client.chat.completions.create(model=model, messages=[{"role": "user", "content": prompt}], temperature=0)
    return r.choices[0].message.content
```

Self-RAG fine-tunes a model to emit reflection tokens. Without that model you can approximate each token with a short judgement call:

```python
def yes(prompt): return ask(prompt + "\nAnswer yes or no.").strip().lower().startswith("y")
def need_retrieval(q): return yes(f"Does answering this need facts from the shop's documents? {q}")
def is_relevant(q, p): return yes(f"Is this passage relevant to the question?\nQ: {q}\nP: {p}")
def is_supported(seg, p): return yes(f"Is every claim in the answer supported by the passage?\nAnswer: {seg}\nPassage: {p}")
def usefulness(q, seg): return int(ask(f"Rate 1-5 how useful this answer is for the question. Reply with the digit only.\nQ: {q}\nA: {seg}")[:1])
```

### Step 3 · Generate per passage, keep the best-critiqued

```python
def self_rag(q, k=4):
    if not need_retrieval(q): return ask(q)
    best = []
    for i, _ in dense_search(q, k):
        p = chunks[i]
        if not is_relevant(q, p): continue
        seg = ask(f"Using only this passage, answer briefly: {q}\nPassage: {p}")
        sup = is_supported(seg, p)
        best.append((sup, usefulness(q, seg), seg))
    best.sort(reverse=True)
    return best[0][2] if best else "I don't know."
print(self_rag("Can I get my money back if my tent arrived torn?"))
```

This costs several model calls per question; cache judgements and use a small model for them.

<div class="cl-end"></div>

## More simulations

[← Corrective RAG](/ai-simulations/rag-corrective) · [All AI Simulations](/ai-simulations) · [Agentic RAG →](/ai-simulations/rag-agentic)
