# PCA: principal components from the covariance matrix

PCA finds the directions in which data varies most. It centres the points, computes their covariance and takes its eigenvectors. Change the correlation, add your own points and watch the axes turn.

<div data-sim="pca"><p>Interactive simulation. It needs JavaScript; the text below explains what it shows.</p></div>

## What you are looking at

The plot shows the centred data, the two principal axes scaled by the standard deviation along each, the two-sigma ellipse and the projection of every point onto PC1. The panel shows the covariance matrix, the eigenvalues and the share of variance kept.

## Things to try

- Set the correlation near 0 and the spread ratio to 1: no direction is special and PC1 keeps half.
- Set the correlation to 0.95: almost everything lies on one line.
- Add a few far-away points and watch PC1 swing towards them: PCA is sensitive to outliers.

## How it works in real systems

PCA is used to compress features, decorrelate inputs, visualise embeddings and denoise. scikit-learn computes it with the singular value decomposition; for huge data, randomised SVD or incremental PCA.

## Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

<div class="cl-start"></div>

### Step 1 · PCA by hand: centre, covariance, eigenvectors

```python
import numpy as np
rng = np.random.default_rng(0)
X = rng.multivariate_normal([0, 0], [[1.0, 0.8], [0.8, 1.0]], size=300)

Xc = X - X.mean(0)                              # 1. centre
C = Xc.T @ Xc / (len(X) - 1)                    # 2. covariance matrix
vals, vecs = np.linalg.eigh(C)                  # 3. eigen-decomposition (symmetric matrix)
order = np.argsort(vals)[::-1]; vals, vecs = vals[order], vecs[:, order]
print("explained variance ratio:", vals / vals.sum())
Z = Xc @ vecs[:, :1]                            # 4. project onto PC1: one number per point
X_back = Z @ vecs[:, :1].T + X.mean(0)          # reconstruction from one component
print("reconstruction MSE:", ((X - X_back) ** 2).mean())
```

### Step 2 · The same with scikit-learn, via SVD

scikit-learn computes PCA with the singular value decomposition of the centred data, which is more accurate than forming the covariance matrix.

```python
from sklearn.decomposition import PCA
p = PCA(n_components=2).fit(X)
print(p.components_, p.explained_variance_ratio_)
```

### Step 3 · Standardise when units differ, and choose the number of components

```python
from sklearn.datasets import load_wine
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
Xw, _ = load_wine(return_X_y=True)
raw = PCA().fit(Xw).explained_variance_ratio_[:3]
std = make_pipeline(StandardScaler(), PCA()).fit(Xw)[-1].explained_variance_ratio_
print("raw:", raw.round(3))                     # one feature (proline, in the hundreds) dominates
print("standardised, components for 90%:", np.searchsorted(np.cumsum(std), 0.9) + 1)
```

<div class="cl-end"></div>

## More simulations

[← Decision tree](/ai-simulations/decision-tree) · [All AI Simulations](/ai-simulations) · [Init, BatchNorm, dropout →](/ai-simulations/normalisation-and-dropout)
