# Recommender systems: matrix factorisation of a ratings table

Most of a ratings table is empty: each user has seen a handful of films. Matrix factorisation fills it in by giving every user and every film a short vector of learned numbers, so that a rating is predicted by their dot product. It won the Netflix Prize era and is still the backbone of many recommenders and of the idea of embeddings.

<div data-sim="mf"><p>Interactive simulation. It needs JavaScript; the text below explains what it shows.</p></div>

## What you are looking at

Rows are 24 users and columns 16 films from four genres; coloured cells are known ratings, from 1 (faint) to 5 (bright). Users have hidden tastes for genres, but the model is never told the genres. Press Train and stochastic gradient descent learns k numbers per user and per film plus a bias for each. Switch the view to predictions or to validation errors, and pick a user to see their top recommendations among films they have not rated. The scatter shows each film's first two learned factors.

## Things to try

- Train with k = 2: films of the same genre gather together in the embedding plot, although genres were never given.
- Use k = 8 with λ = 0 on sparse data: training error keeps falling while validation error stops improving. That is overfitting.
- Lower the known ratings to 10%: with so little data the model barely beats predicting the average.
- Set k = 1: every user and film sits on one line, and the genres collapse.
- Pick different users and compare their top picks with the genres of the films they rated highly.

## How it works in real systems

Large recommenders combine this idea with much more data: implicit feedback such as clicks and watch time, user and item features, and neural two-tower models that learn the user and item vectors from them. The learned item vectors serve candidate retrieval through approximate nearest-neighbour search, and a separate ranking model orders the candidates.














<!-- simx:start -->
## Key formulas

$$\hat r_{ui}=\mu+b_u+b_i+p_u^{\top}q_i$$

Predicted rating: global mean, user and item biases and a dot product of k factors.

$$\min\sum_{(u,i)\in\mathcal K}\big(r_{ui}-\hat r_{ui}\big)^2+\lambda\big(\lVert p_u\rVert^2+\lVert q_i\rVert^2+b_u^2+b_i^2\big)$$

## Common mistakes

- Treating missing ratings as zeros: a film nobody rated is not a film everybody disliked.
- Evaluating with random splits when the real task is predicting future ratings: time-based splits are harder and more honest.
- Leaving out user and item biases: much of the signal is that some users rate everything high and some films please everyone.

## Real-world numbers

- The Netflix Prize (2006–2009) paid $1 million for a 10% improvement in RMSE over Cinematch.
- Matrix factorisation with biases was central to the winning BellKor solution.
- The Netflix data set had about 100 million ratings from 480,000 users on 17,770 films.

## Further reading

- [Matrix Factorization Techniques for Recommender Systems (Koren, Bell, Volinsky)](https://doi.org/10.1109/MC.2009.263)
- [Collaborative Filtering for Implicit Feedback Datasets](https://doi.org/10.1109/ICDM.2008.22)

<!-- simx:end -->

## Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

<div class="cl-start"></div>

### Step 1 · Matrix factorisation with SGD

Biases plus k latent factors, trained on the known ratings only.

```bash
pip install numpy
```

```python
import numpy as np
rng = np.random.default_rng(0)
n_users, n_items, k_true = 200, 100, 3
R = np.clip(np.round(3 + rng.normal(size=(n_users, k_true)) @ rng.normal(size=(k_true, n_items)) * 0.6 + rng.normal(0, 0.3, (n_users, n_items))), 1, 5)
mask = rng.random(R.shape) < 0.3                        # 30% of ratings are known
cells = np.argwhere(mask); rng.shuffle(cells)
train, val = cells[: int(0.8 * len(cells))], cells[int(0.8 * len(cells)):]
```

### Step 2 · Train and evaluate

Stochastic gradient descent with L2 regularisation; RMSE on held-out ratings.

```python
k, lam, lr = 3, 0.1, 0.02
P = rng.normal(0, 0.1, (n_users, k)); Q = rng.normal(0, 0.1, (n_items, k))
bu = np.zeros(n_users); bi = np.zeros(n_items); mu = R[tuple(train.T)].mean()
predict = lambda u, i: mu + bu[u] + bi[i] + P[u] @ Q[i]
rmse = lambda cells: np.sqrt(np.mean([(R[u, i] - predict(u, i)) ** 2 for u, i in cells]))
for epoch in range(30):
    for u, i in train[rng.permutation(len(train))]:
        e = R[u, i] - predict(u, i)
        bu[u] += lr * (e - lam * bu[u]); bi[i] += lr * (e - lam * bi[i])
        P[u], Q[i] = P[u] + lr * (e * Q[i] - lam * P[u]), Q[i] + lr * (e * P[u] - lam * Q[i])
    if epoch % 5 == 4:
        print(epoch + 1, 'train', rmse(train).round(3), 'validation', rmse(val).round(3))
print('baseline (global mean):', np.sqrt(np.mean([(R[u, i] - mu) ** 2 for u, i in val])).round(3))
```

The validation RMSE should end well below the global-mean baseline. Raise k with lam = 0 to watch the two curves separate.

<div class="cl-end"></div>

## More simulations

[← Bayesian optimisation](/ai-simulations/bayesian-optimisation) · [All AI Simulations](/ai-simulations) · [Time-series forecasting →](/ai-simulations/time-series-forecasting)
