Glossary · Generative AI and LLMs
What is Fine-tuning? Definition and meaning
Further training of a pretrained model on new data so it performs a specific task, domain or style better.
Definition
Fine-tuning: Further training of a pretrained model on new data so it performs a specific task, domain or style better.
In short: Teach a trained model new habits.
When it is used
Adapting language, vision and tabular models to company data, formats, tone, tools and preferences.
Where it fits
Fine-tuning belongs to Fine-tuning in the Generative AI and LLMs topic. These are the other terms in the same group:
- Full fine-tuning: Continues training with all model parameters trainable, rather than a small add-on.
- SFT: Trains a model on curated prompts and ideal responses so it follows instructions in the desired style.
- LoRA: Freezes the model and learns small low-rank update matrices for selected layers, so only a small fraction of parameters change.
- QLoRA: Combines 4-bit quantization of the frozen base with LoRA adapters, which cuts memory enough to fine-tune large models on one GPU.
- PEFT: A library that implements LoRA, prefix tuning, prompt tuning and related methods on top of Hugging Face models.
- DPO: Optimizes a model directly on preferred versus rejected answers, without training a separate reward model or running reinforcement learning.
- RLHF: Trains a reward model from human preference comparisons and then optimizes the language model against it with reinforcement learning.
Learn more
The full guide to Fine-tuning explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: Few-shot learning · Next term: FinTech