Özer Cem KELAHMETKELAHMET
Glossary · Agentic AI

What is LLM-as-judge? Definition and meaning

A model scores or compares responses against a rubric.

Definition

LLM-as-judge: A model scores or compares responses against a rubric.

In short: Using a language model to grade outputs.

When it is used

Scalable evaluation and the quality gate inside agent loops.

Where it fits

LLM-as-judge belongs to Patterns in the Agentic AI topic. These are the other terms in the same group:

  • ReAct loop: An agent pattern in which the model writes a thought, takes an action with a tool, observes the result and repeats.
  • Plan-and-execute: A pattern where one step produces a plan and later steps execute and revise it.
  • Reflection: A pattern where the model reviews its output, finds mistakes and revises it before finishing.
  • Tree-of-thoughts: A search over multiple partial reasoning paths with evaluation and backtracking.
  • Orchestrator-workers: One model breaks down a task dynamically and hands subtasks to workers, then combines their results.
  • Multi-agent systems: A design in which specialized agents with separate contexts divide work and exchange results.
  • Human-in-the-loop: Places human review at defined points of an automated workflow, especially before consequential actions.
  • Guardrails: Checks on inputs, outputs and actions that block unsafe, off-topic or malformed behaviour.

Learn more

The full guide to LLM-as-judge explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: LLM routing · Next term: lm-evaluation-harness

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open