Promptfoo
Promptfoo is an open-source tool for testing prompts, models and RAG pipelines with assertions, and for automated red-teaming of LLM apps. Learn how to use it in CI.
Overview
Promptfoo is an open-source command-line tool and library for testing applications built on large language models (LLMs). You describe prompts, model providers and test cases in a configuration file, run them, and get a matrix of results with pass or fail marks from assertions you define. The same tool also runs automated red-teaming: it generates adversarial inputs to probe an application for jailbreaks, prompt injection, data leaks and other risks.
Promptfoo is popular because it treats prompts like code. Tests live in the repository, run locally or in continuous integration (CI), and fail a build when quality drops. Because it calls many providers through one interface, it is also a quick way to compare models on your own cases.
An everyday analogy: a car's MOT test. A fixed checklist is applied the same way every time, each item passes or fails, and the car does not go back on the road until the failures are fixed. Promptfoo's red-team mode adds a second inspector who deliberately tries to break the car.
What it is
| Part | What it does | Notes |
|---|---|---|
| Config file | Declares prompts, providers, test cases and assertions | Usually promptfooconfig.yaml |
| Prompts | Templates with variables | Text, chat messages or files |
| Providers | Models or endpoints under test | Hosted APIs, local models, HTTP endpoints, custom scripts |
| Tests | Variable values plus expected behaviour | Can be loaded from CSV, JSON or code |
| Assertions | Checks on each output | Deterministic (contains, is-json, javascript) or model-graded (llm-rubric, factuality) |
| Eval matrix | Results for every prompt × provider × test | Shown in a terminal table and a web viewer |
| Red-team engine | Generates adversarial probes from plugins and strategies | Produces a vulnerability report |
| Plugins | Categories of risk to test, such as harmful content or PII leaks | Chosen per application |
| Strategies | Ways of delivering attacks, such as jailbreak wrappers or encodings | Applied on top of plugin probes |
| Model scanning | Static checks of model files for unsafe loading behaviour | Without executing the model |
The core project is open source under the MIT licence. A commercial enterprise version adds team features and hosted services.
Where it came from
Promptfoo was created by Ian Webster, who designed the first version as an open-source tool for systematically testing prompts. It gained a large developer following, and in 2024 Webster and Michael D'Angelo founded a company around it, raising seed funding led by Andreessen Horowitz in July 2024 and a Series A led by Insight Partners in July 2025.
Over time the focus widened from prompt comparison to security. Red-teaming, with configurable plugins and attack strategies, became a central feature, followed by agent and tool-use testing and static scanning of model files.
In March 2026, OpenAI announced an agreement to acquire Promptfoo. The announcement stated that Promptfoo would continue to maintain its open-source evaluation, red-teaming and scanning tools and keep supporting many model providers. Check the project's site for its current status.
How it works
1. Initialise a project
Running npx promptfoo@latest init creates a starter config. You then edit it to list prompts, providers and tests.
2. Describe tests
prompts:
- "Summarise this support ticket in one sentence: {{ticket}}"
providers:
- openai:gpt-4o-mini
- anthropic:messages:claude-3-5-haiku-20241022
tests:
- vars:
ticket: "My invoice for March was charged twice."
assert:
- type: icontains
value: invoice
- type: llm-rubric
value: "Mentions the duplicate charge and stays under 25 words"
Each test supplies variable values and assertions. Deterministic assertions check strings, JSON, regular expressions, cost or latency, or run custom JavaScript or Python. Model-graded assertions ask a judge model to score the output against a rubric, a reference answer or similarity to expected text.
3. Run the eval
promptfoo eval sends every combination of prompt, provider and test, caches responses, applies assertions and prints a table. promptfoo view opens a local web viewer where you can compare outputs side by side and inspect failures.
4. Add to CI
The CLI returns a non-zero exit code when assertions fail, and can write results to files. A CI job can therefore block a merge when a prompt change breaks tests. Thresholds let you require, for example, that at least 95% of tests pass.
5. Red-team
For security testing, you describe the application's purpose and choose plugins (what to test for) and strategies (how to attack). Promptfoo generates many adversarial inputs, sends them to the target, grades the responses and produces a report grouped by risk. The target can be a model, an HTTP endpoint or an agent with tools. See red-teaming for the broader practice.
Key concepts and terms
| Term | Meaning |
|---|---|
| Provider | The model or endpoint that produces outputs |
| Prompt template | A prompt with {{variables}} filled from tests |
| Test case | A set of variables and assertions |
| Assertion | A check that passes or fails an output |
| Model-graded assertion | An assertion scored by a judge model |
llm-rubric |
Assertion type that grades output against a written rubric |
| Eval matrix | Grid of results across prompts, providers and tests |
| Plugin | A red-team category of vulnerability to probe |
| Strategy | A red-team technique for delivering an attack |
| Target | The application under red-team test |
| Pass rate | Share of tests whose assertions all pass |
A worked example
This is an illustrative scenario.
A team maintains a customer-service bot with a system prompt of about 600 words. They want to change models and tighten the prompt.
- They write 40 test cases covering refunds, delivery delays, abusive messages and out-of-scope questions. Each has two to four assertions, such as "does not promise a refund amount" (an
llm-rubric) and "response is valid JSON" (is-json). - They configure two providers and two prompt versions, giving 4 × 40 = 160 outputs per run.
- The first run shows prompt B on the new model passing 37 of 40 cases, against 33 for the current setup. The three failures all involve the bot quoting internal policy codes.
- They add a line to prompt B, re-run, and reach 40 of 40. The config goes into the repository and CI runs it on every pull request.
- They then run a red-team scan with plugins for prompt extraction, PII leakage and off-topic use, plus jailbreak and encoding strategies. The report shows the bot revealing parts of its system prompt in 6 of 50 extraction attempts. They add an output check and re-run until the rate drops to zero in that scan.
Strengths and limitations
Strengths:
- Simple, declarative tests that live next to the code.
- Fast side-by-side comparison of prompts and models.
- Many assertion types, from string checks to rubric grading.
- Built-in red-teaming with configurable plugins and strategies.
- Works offline with local models and runs well in CI.
- MIT-licensed core.
Limitations:
- Model-graded assertions cost money and can be inconsistent.
- Large matrices can be slow and expensive without caching and sampling.
- It is a testing tool, not a production observability platform.
- Red-team results show found weaknesses, not proof of safety.
- Ownership changes after the 2026 acquisition announcement are worth watching for licensing or roadmap shifts.
How it compares with alternatives
| Tool | Focus | Strength | Consider it when |
|---|---|---|---|
| Promptfoo | Config-driven testing and red-teaming | CLI, CI and security probes | You want tests in YAML and security scans |
| DeepEval | Python test framework for LLM metrics | Pytest style, many metrics | Your team prefers Python unit tests |
| Braintrust | Hosted eval platform | Experiment tracking and playground | You need a shared UI and production logs |
| Ragas | RAG metrics | Retrieval and faithfulness scores | You evaluate RAG pipelines specifically |
| NeMo Guardrails | Runtime guardrails | Blocks issues in production | You need protection at run time, not only tests |
Practical guidance and best practices
- Start with 20 to 50 real cases, including known past failures.
- Prefer deterministic assertions where possible; use rubrics for qualities code cannot check.
- Keep rubrics short, concrete and testable.
- Pin model versions in providers so results are reproducible.
- Use caching during development and clear it when you need fresh results.
- Run the suite in CI with a pass-rate threshold.
- Run red-team scans before launch and after major prompt, model or tool changes.
- Turn each red-team finding into a permanent regression test.
- Do not send production secrets or personal data in test cases.
Common mistakes
- Writing only "happy path" tests that the model always passes.
- Relying entirely on model-graded assertions without spot checks.
- Comparing models with a single sample when outputs vary between runs.
- Treating a clean red-team report as a safety guarantee.
- Letting the test file drift away from real user behaviour.
Frequently asked questions
Is Promptfoo free?
The core command-line tool and library are open source under the MIT licence and free to use. A commercial enterprise version adds collaboration and hosted features.
Which models does Promptfoo support?
It supports many hosted APIs, local model servers and any HTTP endpoint, and you can write custom providers in JavaScript or Python. This lets you test your full application, not only a raw model.
What is the difference between plugins and strategies?
Plugins decide what kind of weakness to look for, such as harmful content, PII leakage or prompt extraction. Strategies decide how the attack is delivered, such as wrapping a request in a jailbreak or encoding it to slip past filters.
Can Promptfoo test RAG pipelines and agents?
Yes. You can point a provider at your application endpoint, pass context as variables and assert on relevance, faithfulness or tool use. Red-team scans can also target agents with tools.
How does it fit into CI?
The CLI exits with an error code when tests fail, so a CI job can block a merge. Results can also be exported for review.
What happened with OpenAI?
In March 2026, OpenAI announced an agreement to acquire Promptfoo, with a commitment to maintain the open-source tools and support for multiple providers. Check the project for its current licence and plans.
Glossary
- Assertion: a rule that an output must satisfy.
- Rubric: written criteria for grading an answer.
- Jailbreak: an input designed to make a model ignore its rules.
- Prompt injection: instructions hidden in input that try to override the system prompt.
- PII: personally identifiable information.
- Regression test: a test that guards against an old bug returning.
- CI: continuous integration, automated checks on each code change.
Summary
Promptfoo is an open-source tool for testing LLM prompts, models and applications with declarative test cases and assertions, and for automated red-teaming with plugins and strategies. It fits naturally into the development workflow: tests live in the repository and run in CI. Prefer deterministic assertions, spot-check model-graded ones, turn red-team findings into regression tests, and remember that testing reduces risk but does not prove safety. Pair it with runtime defences such as prompt injection defence and guardrails.
Related
It belongs to LLM observability and evaluation in my stack, under Generative AI and LLMs. Others in the same group: