What is Self-supervised learning? Definition and meaning
Creates its own training signal by masking, corrupting or transforming the input and asking the model to recover or relate it.
Definition
Self-supervised learning: Creates its own training signal by masking, corrupting or transforming the input and asking the model to recover or relate it.
In short: Learning from the data itself by hiding parts and predicting them.
When it is used
Pre-training foundation models on unlabelled text, images and tables, before any task-specific labels are used.
Where it fits
Self-supervised learning belongs to Learning paradigms in the Machine learning topic. These are the other terms in the same group:
- Supervised learning: A learning setup where every training example comes with the correct answer, and the model learns the mapping from inputs to that answer.
- Unsupervised learning: Learning from data without target labels, by discovering clusters, directions of variation, or density.
- Semi-supervised learning: Training with a small labelled set and a much larger unlabelled set, so that the unlabelled data shapes the decision boundary.
- Reinforcement learning: An agent acts in an environment, receives rewards, and learns a policy that maximizes long-term reward.
- Transfer learning: Starts from weights learned on a large source task and adapts them to a smaller target task.
- Few-shot learning: Adapts a model to a new task using only a few labelled examples, often through in-context examples or small updates.
- Zero-shot learning: The model solves a task it was never explicitly trained on, relying on general knowledge and a task description.
- Meta-learning: Trains across many tasks so the model can adapt to a new one with very little data or few gradient steps.
Learn more
The full guide to Self-supervised learning explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: scikit-learn · Next term: Semantic sector heads