# What is lm-evaluation-harness? Definition and meaning

EleutherAI's framework that runs a large library of academic benchmarks against a model with a unified interface.

## Definition

**lm-evaluation-harness**: EleutherAI's framework that runs a large library of academic benchmarks against a model with a unified interface.

In short: Standard framework for benchmarking language models.

## When it is used

Reproducible benchmark scores for base and fine-tuned models.

## Where it fits

lm-evaluation-harness belongs to **LLM observability and evaluation** in the [Generative AI and LLMs](/topics/generative-ai-llms) topic. These are the other terms in the same group:

- **[LangSmith](/glossary/langsmith)**: A platform from LangChain for tracing runs, building datasets and evaluating prompts and agents.
- **[Langfuse](/glossary/langfuse)**: A platform for tracing, prompt management, scoring and cost tracking of LLM applications.
- **[Phoenix](/glossary/phoenix)**: A tool for tracing LLM applications with OpenTelemetry and running evaluations on the traces.
- **[Braintrust](/glossary/braintrust)**: A platform for running experiments, scoring outputs and tracking quality over time.
- **[PromptFoo](/glossary/promptfoo)**: A command-line tool that runs test cases and comparisons across prompts and models, including security tests.

## Learn more

The full guide to **[lm-evaluation-harness](/tools/lm-evaluation-harness)** explains how it works, when to use it and how it compares with the alternatives, with examples. The [glossary](/glossary) lists every term on the site from A to Z.

Previous term: [LLM-as-judge](/glossary/llm-as-judge) · Next term: [Logistic regression](/glossary/logistic-regression)
