Glossary · Generative AI and LLMs
What is Full fine-tuning? Definition and meaning
Continues training with all model parameters trainable, rather than a small add-on.
Definition
Full fine-tuning: Continues training with all model parameters trainable, rather than a small add-on.
In short: Updating every weight of a model on new data.
When it is used
Maximum adaptation when data and compute are plentiful, at the cost of memory and storage.
Where it fits
Full fine-tuning belongs to Fine-tuning in the Generative AI and LLMs topic. These are the other terms in the same group:
- Fine-tuning: Further training of a pretrained model on new data so it performs a specific task, domain or style better.
- SFT: Trains a model on curated prompts and ideal responses so it follows instructions in the desired style.
- LoRA: Freezes the model and learns small low-rank update matrices for selected layers, so only a small fraction of parameters change.
- QLoRA: Combines 4-bit quantization of the frozen base with LoRA adapters, which cuts memory enough to fine-tune large models on one GPU.
- PEFT: A library that implements LoRA, prefix tuning, prompt tuning and related methods on top of Hugging Face models.
- DPO: Optimizes a model directly on preferred versus rejected answers, without training a separate reward model or running reinforcement learning.
- RLHF: Trains a reward model from human preference comparisons and then optimizes the language model against it with reinforcement learning.
Learn more
The full guide to Full fine-tuning explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.
Previous term: FT-Transformer · Next term: Function calling