# k-means clustering, step by step

Not all learning needs labels. k-means finds groups on its own with two alternating steps. Step through them, change k, and compare a careless start with k-means++.

<div data-sim="kmeans"><p>Interactive simulation. It needs JavaScript; the text below explains what it shows.</p></div>

## What you are looking at

The crosses are cluster centres; the shaded regions show which centre is nearest everywhere. Each assign step colours every point by its nearest centre; each update step moves every centre to the mean of its points. The chart tracks inertia, the total squared distance from points to their centre, which never goes up.

## Things to try

- Run to convergence with k = 4: the four blobs are found.
- Set k = 2 or 6 and see how the algorithm splits or merges real groups.
- Switch to random starts and press New data a few times: sometimes two centres fight over one blob, a local optimum.
- Count the iterations: k-means usually converges in a handful.

## How it works in real systems

k-means is used to segment customers, compress colours, build the coarse level of vector indexes (IVF in FAISS) and initialise other models. In practice it runs several times with k-means++ starts and keeps the run with the lowest inertia.

## Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

<div class="cl-start"></div>

### Step 1 · Data

Three blobs in two dimensions.

```python
import numpy as np
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=450, centers=3, cluster_std=0.9, random_state=4)
```

### Step 2 · k-means from scratch

Two steps repeated until nothing changes: assign every point to its nearest centre, move every centre to the mean of its points. k-means++ seeding picks spread-out starting centres.

```python
def kmeans(X, k, iters=100, seed=0):
    rng = np.random.default_rng(seed)
    C = [X[rng.integers(len(X))]]
    for _ in range(1, k):                                   # k-means++ seeding
        d2 = np.min(((X[:, None] - np.array(C)[None]) ** 2).sum(-1), axis=1)
        C.append(X[rng.choice(len(X), p=d2 / d2.sum())])
    C = np.array(C)
    for _ in range(iters):
        labels = np.argmin(((X[:, None] - C[None]) ** 2).sum(-1), axis=1)      # assign
        newC = np.array([X[labels == j].mean(0) if np.any(labels == j) else C[j] for j in range(k)])
        if np.allclose(newC, C): break
        C = newC                                                                 # update
    inertia = ((X - C[labels]) ** 2).sum()
    return C, labels, inertia

C, labels, inertia = kmeans(X, 3)
print(C, inertia)
```

### Step 3 · Compare with scikit-learn and choose k

`n_init` runs several seedings and keeps the best, because k-means can stop in a poor local minimum. The elbow of inertia and the silhouette score help choose k.

```python
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
for k in range(2, 7):
    km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
    print(k, round(km.inertia_, 1), round(silhouette_score(X, km.labels_), 3))
```

<div class="cl-end"></div>

## More simulations

[← Overfitting](/ai-simulations/overfitting-and-regularisation) · [All AI Simulations](/ai-simulations) · [One neuron →](/ai-simulations/one-neuron)
