Naive RAG: embed, search, take the top three
Every RAG system starts here. Watch the question become a vector, every chunk get a similarity score, and the top three go into the prompt, then see exactly where this simple pipeline fails.
Interactive simulation. It needs JavaScript; the text below explains what it shows.
What you are looking at
The left column runs the pipeline one step at a time and shows what each step produces; the right column shows the context the model would receive, retrieval quality against the known relevant chunks, and an answer quoted from that context. All RAG simulations share one knowledge base of 18 chunks from 11 documents of a fictional outdoor-gear shop, and three questions: one phrased differently from the documents, one that needs two documents, and one about the whole corpus.
Things to try
- Run the torn-tent question: the word 'torn' never appears in the documents, so it relies on synonyms.
- Run the fabric question: one chunk names the fabric, another names the supplier; does the top three hold both?
- Run the complaints question: no single chunk answers it.
How it works in real systems
Production systems use a trained embedding model (for example a sentence transformer or an API embedding with 384 to 3,072 dimensions) and a vector index such as HNSW in pgvector, Qdrant or FAISS. Naive RAG is the right first version because it gives every later change a number to beat.
Code lab
The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.
Step 1 · Index the knowledge base
Install the libraries and index a small knowledge base with a sentence-embedding model. Normalised embeddings make the dot product equal to cosine similarity.
pip install sentence-transformers rank-bm25 numpy openai
import numpy as np
from sentence_transformers import SentenceTransformer
chunks = [
"Most items can be returned within 30 days of delivery for a full refund.",
"Damaged or defective items qualify for a full refund or a free replacement; send a photo of the damage to support first.",
"Refunds are issued within five business days after the returned item reaches our warehouse.",
"Parcels damaged in transit must be reported to the carrier within 48 hours of delivery.",
"The Ridge 2 tent weighs 1.9 kg and uses Aurora ripstop fabric for its fly sheet.",
"Aurora ripstop fabric is woven by Lumen Textiles at its mill in Porto, Portugal.",
"Customer complaints in Q3 were mostly about late deliveries during the September peak.",
]
emb = SentenceTransformer("all-MiniLM-L6-v2") # 384 dimensions, small and fast
E = emb.encode(chunks, normalize_embeddings=True)
def dense_search(q, k=3):
s = E @ emb.encode([q], normalize_embeddings=True)[0]
top = np.argsort(-s)[:k]
return [(int(i), float(s[i])) for i in top]
Step 2 · Retrieve the top three
q = "Can I get my money back if my tent arrived torn?"
for i, s in dense_search(q):
print(f"{s:.3f} {chunks[i]}")
Note what is missing: the 48-hour carrier rule may rank below unrelated chunks, because the question never says "transit" or "carrier".
Step 3 · Generate a grounded answer
A small helper to call a chat model (any OpenAI-compatible endpoint works, including a local server):
from openai import OpenAI
client = OpenAI() # reads OPENAI_API_KEY
def ask(prompt, model="gpt-4o-mini"):
r = client.chat.completions.create(model=model, messages=[{"role": "user", "content": prompt}], temperature=0)
return r.choices[0].message.content
Put the retrieved chunks in the prompt, numbered, and ask for an answer that cites them and admits when the context is not enough.
def answer(question, ids):
ctx = "\n".join(f"[{n+1}] {chunks[i]}" for n, i in enumerate(ids))
return ask(f"Answer using only the context. Cite sources like [1]. If the context is not enough, say you don't know.\n\nContext:\n{ctx}\n\nQuestion: {question}")
print(answer(q, [i for i, _ in dense_search(q)]))
Step 4 · Store it in Postgres with pgvector
For more than a toy corpus, keep the vectors in a database with an approximate-nearest-neighbour index.
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (id bigserial PRIMARY KEY, doc text, body text, embedding vector(384));
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
-- query: $1 is the question embedding
SELECT id, body, 1 - (embedding <=> $1) AS cosine FROM chunks ORDER BY embedding <=> $1 LIMIT 3;