Özer Cem KELAHMETKELAHMET
Glossary · Generative AI and LLMs

What is LoRA? Definition and meaning

Freezes the model and learns small low-rank update matrices for selected layers, so only a small fraction of parameters change.

Definition

LoRA: Freezes the model and learns small low-rank update matrices for selected layers, so only a small fraction of parameters change.

In short: Low-rank adapters that train a tiny fraction of weights.

When it is used

Cheap fine-tuning of large models, and keeping many task adapters over a single base.

Where it fits

LoRA belongs to Fine-tuning in the Generative AI and LLMs topic. These are the other terms in the same group:

  • Fine-tuning: Further training of a pretrained model on new data so it performs a specific task, domain or style better.
  • Full fine-tuning: Continues training with all model parameters trainable, rather than a small add-on.
  • SFT: Trains a model on curated prompts and ideal responses so it follows instructions in the desired style.
  • QLoRA: Combines 4-bit quantization of the frozen base with LoRA adapters, which cuts memory enough to fine-tune large models on one GPU.
  • PEFT: A library that implements LoRA, prefix tuning, prompt tuning and related methods on top of Hugging Face models.
  • DPO: Optimizes a model directly on preferred versus rejected answers, without training a separate reward model or running reinforcement learning.
  • RLHF: Trains a reward model from human preference comparisons and then optimizes the language model against it with reinforcement learning.

Learn more

The full guide to LoRA explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: Long-running tasks · Next term: Loss functions

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open