Recommender systems: matrix factorisation of a ratings table
Most of a ratings table is empty: each user has seen a handful of films. Matrix factorisation fills it in by giving every user and every film a short vector of learned numbers, so that a rating is predicted by their dot product. It won the Netflix Prize era and is still the backbone of many recommenders and of the idea of embeddings.
Level IntermediateAbout 12 minRead first: Embeddings1 challenge in the Lab
Interactive simulation. It needs JavaScript; the text below explains what it shows.
What you are looking at
Rows are 24 users and columns 16 films from four genres; coloured cells are known ratings, from 1 (faint) to 5 (bright). Users have hidden tastes for genres, but the model is never told the genres. Press Train and stochastic gradient descent learns k numbers per user and per film plus a bias for each. Switch the view to predictions or to validation errors, and pick a user to see their top recommendations among films they have not rated. The scatter shows each film's first two learned factors.
Things to try
- Train with k = 2: films of the same genre gather together in the embedding plot, although genres were never given.
- Use k = 8 with λ = 0 on sparse data: training error keeps falling while validation error stops improving. That is overfitting.
- Lower the known ratings to 10%: with so little data the model barely beats predicting the average.
- Set k = 1: every user and film sits on one line, and the genres collapse.
- Pick different users and compare their top picks with the genres of the films they rated highly.
How it works in real systems
Large recommenders combine this idea with much more data: implicit feedback such as clicks and watch time, user and item features, and neural two-tower models that learn the user and item vectors from them. The learned item vectors serve candidate retrieval through approximate nearest-neighbour search, and a separate ranking model orders the candidates.
Key formulas
$$\hat r_{ui}=\mu+b_u+b_i+p_u^{\top}q_i$$
Predicted rating: global mean, user and item biases and a dot product of k factors.
$$\min\sum_{(u,i)\in\mathcal K}\big(r_{ui}-\hat r_{ui}\big)^2+\lambda\big(\lVert p_u\rVert^2+\lVert q_i\rVert^2+b_u^2+b_i^2\big)$$
Common mistakes
- Treating missing ratings as zeros: a film nobody rated is not a film everybody disliked.
- Evaluating with random splits when the real task is predicting future ratings: time-based splits are harder and more honest.
- Leaving out user and item biases: much of the signal is that some users rate everything high and some films please everyone.
Real-world numbers
- The Netflix Prize (2006–2009) paid $1 million for a 10% improvement in RMSE over Cinematch.
- Matrix factorisation with biases was central to the winning BellKor solution.
- The Netflix data set had about 100 million ratings from 480,000 users on 17,770 films.
Further reading
- Matrix Factorization Techniques for Recommender Systems (Koren, Bell, Volinsky)
- Collaborative Filtering for Implicit Feedback Datasets
Code lab
The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.
Step 1 · Matrix factorisation with SGD
Biases plus k latent factors, trained on the known ratings only.
pip install numpy
import numpy as np
rng = np.random.default_rng(0)
n_users, n_items, k_true = 200, 100, 3
R = np.clip(np.round(3 + rng.normal(size=(n_users, k_true)) @ rng.normal(size=(k_true, n_items)) * 0.6 + rng.normal(0, 0.3, (n_users, n_items))), 1, 5)
mask = rng.random(R.shape) < 0.3 # 30% of ratings are known
cells = np.argwhere(mask); rng.shuffle(cells)
train, val = cells[: int(0.8 * len(cells))], cells[int(0.8 * len(cells)):]
Step 2 · Train and evaluate
Stochastic gradient descent with L2 regularisation; RMSE on held-out ratings.
k, lam, lr = 3, 0.1, 0.02
P = rng.normal(0, 0.1, (n_users, k)); Q = rng.normal(0, 0.1, (n_items, k))
bu = np.zeros(n_users); bi = np.zeros(n_items); mu = R[tuple(train.T)].mean()
predict = lambda u, i: mu + bu[u] + bi[i] + P[u] @ Q[i]
rmse = lambda cells: np.sqrt(np.mean([(R[u, i] - predict(u, i)) ** 2 for u, i in cells]))
for epoch in range(30):
for u, i in train[rng.permutation(len(train))]:
e = R[u, i] - predict(u, i)
bu[u] += lr * (e - lam * bu[u]); bi[i] += lr * (e - lam * bi[i])
P[u], Q[i] = P[u] + lr * (e * Q[i] - lam * P[u]), Q[i] + lr * (e * P[u] - lam * Q[i])
if epoch % 5 == 4:
print(epoch + 1, 'train', rmse(train).round(3), 'validation', rmse(val).round(3))
print('baseline (global mean):', np.sqrt(np.mean([(R[u, i] - mu) ** 2 for u, i in val])).round(3))
The validation RMSE should end well below the global-mean baseline. Raise k with lam = 0 to watch the two curves separate.
More simulations
← Bayesian optimisation · All AI Simulations · Time-series forecasting →