# ORPO

Merges supervised fine-tuning and preference optimization into one stage with no reference model.

## What it is used for

Cheaper single-stage alignment.