Özer Cem KELAHMETKELAHMET
AI Simulations · Retrieval (RAG)

Chunking strategies: size, overlap and boundaries

How you cut documents decides what retrieval can find. Change the strategy, size and overlap and watch the answer sentence stay whole, get cut in two, or get diluted.

Interactive simulation. It needs JavaScript; the text below explains what it shows.

What you are looking at

The document is shown split into chunks; the highlighted text is the sentence that answers the selected question. The right side ranks the chunks for the question and says whether the top chunk holds the whole answer.

Things to try

  • Fixed size 120, overlap 0: the answer is often cut in half.
  • Switch to sentence chunking: boundaries follow sentences.
  • Make chunks very large: the answer is whole but buried among unrelated sentences.

How it works in real systems

Common defaults are 200 to 500 tokens with 10 to 20 percent overlap, split on headings, paragraphs and sentences (LangChain's RecursiveCharacterTextSplitter). Semantic chunking splits where the topic changes. Always measure retrieval on real questions.

Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

Step 1 · Three splitters

import re
doc = open("returns_policy.txt").read() if False else "Returns policy. Most items can be returned within 30 days. Items must be unused. Damaged items qualify for a full refund. Refunds are issued within five business days after the item reaches our warehouse. Sale items can be exchanged but not refunded."
def fixed(text, size=120, overlap=20):
    return [text[i:i + size] for i in range(0, len(text), size - overlap)]
def by_sentence(text, size=180, overlap_sentences=1):
    s = re.split(r"(?<=[.!?])\s+", text); out, cur = [], []
    for sent in s:
        if cur and len(" ".join(cur + [sent])) > size:
            out.append(" ".join(cur)); cur = cur[-overlap_sentences:] if overlap_sentences else []
        cur.append(sent)
    return out + [" ".join(cur)] if cur else out
for f in (fixed, by_sentence): print(f.__name__, f(doc))

Step 2 · The standard recursive splitter

# pip install langchain-text-splitters
from langchain_text_splitters import RecursiveCharacterTextSplitter
sp = RecursiveCharacterTextSplitter(chunk_size=180, chunk_overlap=30, separators=["\n\n", "\n", ". ", " ", ""])
print(sp.split_text(doc))

It tries paragraph breaks first, then lines, then sentences, then words, so chunks end at natural boundaries whenever possible.

Step 3 · Pick the size with an experiment

Index the same documents with several sizes and compare recall@k on a labelled question set (see the RAG evaluation code lab). Typical winners are 200–500 tokens with 10–20% overlap, but it depends on your documents and questions.

More simulations

← GraphRAG · All AI Simulations · Embeddings →

Last updated Nov 17, 2026
Type to search.
    ↑↓ navigate↵ open