Özer Cem KELAHMETKELAHMET
AI Simulations · End-to-end projects

The meaning map: this whole site as vectors, and your text among them

Search engines, RAG systems and recommendation all rest on one idea: turn text into a list of numbers so that texts with similar meaning get similar numbers. This map shows that idea on real data: every page of this site, embedded with the same model the site's search uses, and your own text placed among them.

Level IntermediateAbout 12 minRead first: Embeddings, RAG types1 challenge in the Lab

Interactive simulation. It needs JavaScript; the text below explains what it shows.

What you are looking at

Each dot is a page of this site. A sentence-embedding model (all-MiniLM-L6-v2, the model behind the site's search) turned every section of every page into 384 numbers; a page's vector is the average of its sections'. The map squeezes those 384 dimensions into two with principal component analysis, so pages about similar things sit near each other. Colours show the part of the site each page belongs to.

When you place a text, the same model embeds it on the server, the browser projects it with the same two directions, and the lines join it to its six closest pages. Closeness in that list is measured in all 384 dimensions, so it is more exact than the picture.

The arithmetic box adds and subtracts vectors: it starts from the first word, removes the second and adds the third, then lists the closest of about two hundred words embedded with the same model.

Things to try

  • Place "cheap way to run an LLM": its closest pages are LLM routing and serving LLMs efficiently, although neither title uses the word cheap.
  • Place two phrasings of the same idea one after the other and watch them land on almost the same spot.
  • Try king − man + woman, paris − france + italy and walking − walk + swim. Some work and some do not: sentence models were not trained for word analogies.
  • Zoom into a cluster of grey guide pages and hover the dots: neighbours share a topic even when their titles do not share a word.

How it works in real systems

Production retrieval works exactly like this list of closest pages: documents are split into chunks, each chunk is embedded once and stored in a vector index such as HNSW, and each query is embedded at search time and compared by cosine similarity. Larger embedding models use 768 to 3,072 dimensions and are trained with contrastive learning on hundreds of millions of text pairs. Maps such as this one use PCA, t-SNE or UMAP; they are for looking, never for ranking, because two dimensions keep only a small part of the information.

Key formulas

$$\cos(\mathbf a,\mathbf b)=\frac{\mathbf a\cdot\mathbf b}{\lVert\mathbf a\rVert\,\lVert\mathbf b\rVert}$$

Similarity of meaning: 1 for the same direction, 0 for unrelated. Search ranks by it.

$$\mathbf z=W_{\mathrm{PCA}}^\top(\mathbf x-\boldsymbol\mu),\qquad W_{\mathrm{PCA}}\in\mathbb R^{384\times 2}$$

The map: the two directions along which the site's pages differ most. Your text is projected with the same two directions.

$$\mathbf v=\mathbf e_{\text{king}}-\mathbf e_{\text{man}}+\mathbf e_{\text{woman}}$$

Vector arithmetic: move along the man→woman direction, starting from king.

Common mistakes

  • Reading distances on a 2-D map as exact: the projection keeps only part of the variance; the list of closest pages uses all 384 dimensions.
  • Expecting sentence embeddings to solve word analogies like word2vec: they are trained for sentences, so analogies are hit and miss.
  • Comparing vectors from two different embedding models: their spaces are unrelated.

Real-world numbers

  • all-MiniLM-L6-v2 maps any text to 384 numbers; it has 22.7 million parameters.
  • The map shows every page of this site; its two axes keep only a few percent of the total variance.
  • word2vec (2013) made king − man + woman ≈ queen famous.

Further reading

Code lab

The same pipeline in a few lines of Python, with the same model.

Step 1 · Embed texts

from sentence_transformers import SentenceTransformer
import numpy as np
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
docs = ["Hybrid search combines BM25 with vector search.",
        "A KV cache stores keys and values of earlier tokens.",
        "Grounding answers in retrieved documents reduces hallucination.",
        "LoRA fine-tunes a model by training small low-rank matrices."]
E = model.encode(docs, normalize_embeddings=True)
print(E.shape)          # (4, 384)

Step 2 · Find the closest texts to a question

q = model.encode(["how do I stop a chatbot from making things up"], normalize_embeddings=True)[0]
scores = E @ q          # cosine similarity, because the vectors have length 1
for i in np.argsort(-scores):
    print(f"{scores[i]:.3f}  {docs[i]}")

Step 3 · Draw a map with PCA

mu = E.mean(0)
U, S, Vt = np.linalg.svd(E - mu, full_matrices=False)
xy = (E - mu) @ Vt[:2].T                       # two coordinates per text
print("variance kept:", (S[:2] ** 2).sum() / (S ** 2).sum())
q_xy = (q - mu) @ Vt[:2].T                     # a new text goes on the same map

Step 4 · Vector arithmetic

words = ["king", "queen", "man", "woman", "prince", "princess", "boy", "girl"]
W = model.encode(words, normalize_embeddings=True)
v = W[0] - W[2] + W[3]; v /= np.linalg.norm(v)
for i in np.argsort(-(W @ v)):
    if words[i] not in ("king", "man", "woman"):
        print(words[i], round(float(W[i] @ v), 3))

More simulations

All AI Simulations · From 0 to 100 → · RAG types →

Last updated Nov 20, 2026
Type to search.
    ↑↓ navigate↵ open