Özer Cem KELAHMETKELAHMET
AI Simulations · Retrieval (RAG)

Embedding space: where your query lands among the documents

Semantic search is geometry. Each chunk is a point, the query is a point, and the answer is whatever lies closest. Type your own query and watch it move.

Interactive simulation. It needs JavaScript; the text below explains what it shows.

What you are looking at

The map is a two-dimensional PCA projection of the chunk vectors, coloured by document; the query is the glowing point, with dashed lines to its three nearest chunks. The list uses the true cosine in the full space.

Things to try

  • Type 'who weaves the fabric' and watch the query jump to the suppliers.
  • Turn off synonym folding: 'torn' and 'damaged' stop matching.
  • Notice that points close on the map are not always closest in the full space.

How it works in real systems

Real embedding models (sentence transformers, OpenAI, Cohere, Voyage) learn hundreds or thousands of dimensions from billions of sentence pairs, so that paraphrases land together. Tools such as Nomic Atlas or the TensorBoard projector draw maps like this one with UMAP or t-SNE.

Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

Step 1 · Embed and compare

from sentence_transformers import SentenceTransformer
import numpy as np
m = SentenceTransformer("all-MiniLM-L6-v2")
texts = ["my tent arrived torn", "damaged items qualify for a refund", "delivery takes three days", "the fabric is woven in Porto"]
E = m.encode(texts, normalize_embeddings=True)
print(np.round(E @ E.T, 2))          # cosine similarity matrix: "torn" is close to "damaged" without sharing a word

Step 2 · Draw the map

from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
xy = PCA(2).fit_transform(E)
plt.scatter(*xy.T)
for (x, y), t in zip(xy, texts): plt.annotate(t, (x, y))
plt.show()

PCA keeps global distances roughly; UMAP and t-SNE keep local neighbourhoods and make clusters clearer. In both cases the 2-D map is an approximation: search with the full vectors.

Step 3 · Choose a model

Compare candidates on your own queries, not only on leaderboards (MTEB). Check the dimension (storage), the maximum input length, the languages and whether the model expects prefixes such as "query: " and "passage: ".

More simulations

← Chunking · All AI Simulations · HNSW search →

Last updated Nov 17, 2026
Type to search.
    ↑↓ navigate↵ open