LLM-as-judge
LLM-as-judge uses a language model to grade AI outputs against a rubric. Learn grading modes, known biases, how to calibrate judges against humans and when to trust them.
Overview
LLM-as-judge is the practice of using a language model to evaluate the output of another model or agent. Instead of a person reading every answer and marking it good or bad, a judge model reads the question, the answer and a rubric, and returns a score or a verdict with a reason.
Think of a large exam. One senior teacher cannot mark ten thousand essays. So the school writes a clear marking scheme, trains assistant markers, and checks a sample of their marks against the senior teacher's. If the assistants agree with the senior teacher often enough, their marks are trusted for the rest. LLM-as-judge works the same way: the rubric is the marking scheme, the judge model is the assistant marker, and a human-labelled sample is the check.
It has become a standard part of building AI systems, because open-ended outputs such as summaries, answers and agent actions cannot be checked with simple exact-match tests.
What it is
A judge setup has four parts:
- The item: the input and the output to grade, sometimes with reference material or a reference answer.
- The rubric: criteria, such as correctness, groundedness, completeness, tone or policy compliance, with a clear description of each score level.
- The judge model: a capable model prompted to apply the rubric.
- The verdict: a score, a label or a choice, plus a short justification.
| Grading mode | What the judge does | Typical use |
|---|---|---|
| Pointwise | Scores one output on a scale or pass/fail | Monitoring production quality |
| Pairwise | Chooses the better of two outputs | Comparing prompts or models |
| Reference-based | Compares the output with a known good answer | Question answering with ground truth |
| Groundedness check | Checks each claim against provided sources | RAG and citation quality |
| Trajectory review | Reviews an agent's sequence of actions | Agent evaluation and safety |
Where it came from
Automatic metrics such as BLEU and ROUGE, which count word overlap, were the norm for evaluating generated text for many years. They correlate poorly with human judgement for open-ended answers. As models became strong enough to follow detailed instructions, researchers began using them as graders. A widely cited 2023 study on MT-Bench and Chatbot Arena by Lianmin Zheng and colleagues measured how often a strong model's judgements agreed with human preferences and documented specific biases, such as favouring the first answer shown or favouring longer answers.
Since then, judge models have become part of every major evaluation toolkit, both open-source and commercial, and of production monitoring for chat, RAG and agent systems.
How it works
Writing the rubric
The rubric is the most important part. Each criterion should be specific and observable. "Is the answer good?" is weak. "Does every factual claim appear in the provided documents? Answer yes if all claims are supported, no if any claim is unsupported" is strong. Binary or small scales, such as pass/fail or 1 to 3, are usually more reliable than 1 to 10.
Asking for reasons first
Asking the judge to explain its reasoning before giving the score tends to improve consistency, and the reasons make disagreements easy to review.
Calibrating against humans
Before trusting a judge, collect a set of items labelled by people who understand the task. Run the judge on the same items and measure agreement, for example accuracy, Cohen's kappa or correlation. Inspect disagreements, refine the rubric and repeat until agreement is acceptable for the decision being made.
Controlling bias
Known biases include position bias (preferring the first or second option), verbosity bias (preferring longer answers), self-preference (favouring outputs from the same model family) and leniency. Common controls are swapping the order in pairwise tests and averaging, instructing the judge to ignore length, using a judge from a different model family, and spot-checking with humans.
Running at scale
Judges run in batch over test sets and continuously over samples of production traffic. Scores are tracked over time, so a drop after a prompt or model change is caught quickly.
Key concepts and terms
| Term | Meaning |
|---|---|
| Rubric | Criteria and score definitions the judge applies. |
| Pointwise grading | Scoring a single output. |
| Pairwise grading | Choosing between two outputs. |
| Reference answer | A known good answer used for comparison. |
| Groundedness | Whether claims are supported by provided sources. |
| Agreement rate | How often the judge matches human labels. |
| Cohen's kappa | An agreement measure corrected for chance. |
| Position bias | Preference for an option because of its order. |
| Verbosity bias | Preference for longer answers. |
| Golden set | A curated set of items with trusted labels. |
A worked example
A team runs a RAG assistant over internal policy documents. They want to know how often answers contain unsupported claims.
- They write a groundedness rubric: split the answer into claims; for each claim, mark supported if the retrieved passages state it, unsupported otherwise; the answer passes only if all claims are supported.
- Two policy experts label 200 real question-answer pairs.
- The judge model labels the same 200 items. It agrees with the experts on 182 of them, 91 percent. Most disagreements involve claims that paraphrase the source loosely.
- The team adds an instruction: a paraphrase counts as supported only if it keeps the same numbers, conditions and exceptions. Agreement rises to 95 percent on a fresh set of 100 items.
- The judge now grades a daily sample of 500 production answers. When the pass rate drops after a retriever update, the team rolls back the update the same day.
Strengths and limitations
Strengths
- Scale. Thousands of outputs can be graded quickly and cheaply.
- Nuance. Judges can assess qualities that word-overlap metrics miss.
- Speed of iteration. Teams can compare prompts and models in hours instead of weeks.
- Explanations. Written reasons make results reviewable.
Limitations
- Not ground truth. A judge is a proxy that must be calibrated against people.
- Biases. Position, length and self-preference effects distort results if not controlled.
- Shared blind spots. A judge may miss the same errors as the model it grades, especially on facts it does not know.
- Gaming. Systems optimised against a judge can learn to please the judge rather than users.
- Cost at scale. Strong judge models are not free; sampling is often necessary.
How it compares with alternatives
| Method | Strength | Weakness |
|---|---|---|
| Human review | Highest validity | Slow and expensive |
| Exact match and unit tests | Objective and cheap | Only for tasks with one correct output |
| Overlap metrics | Cheap and fast | Weak correlation with quality |
| Trained reward or classifier models | Fast at scale | Needs training data and maintenance |
| LLM-as-judge | Flexible and nuanced | Needs calibration and bias control |
The best evaluation stacks combine them: deterministic checks wherever possible, judges for open-ended quality, and humans for calibration and high-stakes cases.
Practical guidance and best practices
- Use deterministic checks first. If a result can be verified with code, do that.
- Write narrow criteria. One criterion per judge call is easier to calibrate than one combined score.
- Prefer small scales. Pass/fail or three levels are more stable than ten.
- Ask for reasons, then the verdict. Store both.
- Calibrate and recalibrate. Measure agreement with human labels and repeat after rubric or model changes.
- Control order and length. Swap positions in pairwise tests and tell the judge to ignore length.
- Use a different model family when possible. This reduces self-preference.
- Track scores over time. Treat judge results as monitoring signals with alerts.
Common mistakes
- Trusting an uncalibrated judge. Without a human baseline, scores are opinions.
- Vague rubrics. "Rate helpfulness 1 to 10" produces noisy numbers.
- One giant prompt for everything. Mixing many criteria hides which one failed.
- Optimising against the judge only. Real user outcomes must stay in the loop.
Frequently asked questions
Is LLM-as-judge reliable?
It can be, for well-defined criteria, after calibration against human labels and with bias controls. It is not reliable by default and should not be the only signal for high-stakes decisions.
Should the judge be a stronger model than the one being graded?
Usually a strong model works best, and a different model family reduces self-preference. For simple binary checks, smaller judges can be enough once calibrated.
What is position bias?
The tendency of a judge to prefer an answer because it appears first or second. Swapping the order and averaging the results reduces it.
How many human labels do I need for calibration?
Often between one and a few hundred items per criterion is enough to estimate agreement and find rubric problems. Use more for high-stakes decisions.
Can LLM-as-judge evaluate agents?
Yes. Judges can review trajectories, checking whether tool choices were sensible, whether actions stayed within policy and whether the final result met the goal.
Glossary
- Calibration: aligning judge outputs with trusted human labels.
- Evals: systematic tests of model or agent behaviour.
- Golden set: curated examples with trusted labels.
- RAG: retrieval-augmented generation.
- Reward model: a model trained to score outputs, often for training.
- Trajectory: the sequence of steps an agent took.
Summary
LLM-as-judge lets a model grade outputs against a rubric, which makes evaluation of open-ended answers and agent behaviour fast enough to run on every change. It works when criteria are narrow and concrete, reasons are recorded, results are calibrated against human labels, and biases such as order and length are controlled. Combined with deterministic checks and human review, it is one of the most practical tools for keeping AI systems honest.
Related
It belongs to Patterns in my stack, under Agentic AI. Others in the same group: