# Information theory

Entropy, mutual information and KL divergence, which underlie many training losses.

## What it is used for

Loss design, contrastive objectives and compression arguments.