k-means clustering, step by step
Not all learning needs labels. k-means finds groups on its own with two alternating steps. Step through them, change k, and compare a careless start with k-means++.
Interactive simulation. It needs JavaScript; the text below explains what it shows.
What you are looking at
The crosses are cluster centres; the shaded regions show which centre is nearest everywhere. Each assign step colours every point by its nearest centre; each update step moves every centre to the mean of its points. The chart tracks inertia, the total squared distance from points to their centre, which never goes up.
Things to try
- Run to convergence with k = 4: the four blobs are found.
- Set k = 2 or 6 and see how the algorithm splits or merges real groups.
- Switch to random starts and press New data a few times: sometimes two centres fight over one blob, a local optimum.
- Count the iterations: k-means usually converges in a handful.
How it works in real systems
k-means is used to segment customers, compress colours, build the coarse level of vector indexes (IVF in FAISS) and initialise other models. In practice it runs several times with k-means++ starts and keeps the run with the lowest inertia.
Code lab
The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.
Step 1 · Data
Three blobs in two dimensions.
import numpy as np
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=450, centers=3, cluster_std=0.9, random_state=4)
Step 2 · k-means from scratch
Two steps repeated until nothing changes: assign every point to its nearest centre, move every centre to the mean of its points. k-means++ seeding picks spread-out starting centres.
def kmeans(X, k, iters=100, seed=0):
rng = np.random.default_rng(seed)
C = [X[rng.integers(len(X))]]
for _ in range(1, k): # k-means++ seeding
d2 = np.min(((X[:, None] - np.array(C)[None]) ** 2).sum(-1), axis=1)
C.append(X[rng.choice(len(X), p=d2 / d2.sum())])
C = np.array(C)
for _ in range(iters):
labels = np.argmin(((X[:, None] - C[None]) ** 2).sum(-1), axis=1) # assign
newC = np.array([X[labels == j].mean(0) if np.any(labels == j) else C[j] for j in range(k)])
if np.allclose(newC, C): break
C = newC # update
inertia = ((X - C[labels]) ** 2).sum()
return C, labels, inertia
C, labels, inertia = kmeans(X, 3)
print(C, inertia)
Step 3 · Compare with scikit-learn and choose k
n_init runs several seedings and keeps the best, because k-means can stop in a poor local minimum. The elbow of inertia and the silhouette score help choose k.
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
for k in range(2, 7):
km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
print(k, round(km.inertia_, 1), round(silhouette_score(X, km.labels_), 3))