# DSPy

DSPy replaces hand-written prompts with signatures, modules and optimizers that tune LLM pipelines against a metric. Learn how it works and when it pays off.

## Overview

DSPy is an open-source Python framework for building language model programs without hand-writing long prompts. You describe what each step should do, in terms of inputs and outputs, and compose those steps into a program. Then you give DSPy some examples and a metric that says what a good answer looks like. An optimizer tries different instructions and examples for each step and keeps the versions that score best.

The idea is that prompts are a bit like the weights of a model: they should be learned from data, not guessed by hand. When you change the model, the data or the pipeline, you re-run the optimizer instead of rewriting every prompt. DSPy started as research at Stanford and its motto is "programming, not prompting".

An everyday comparison is tuning a recipe. One cook keeps rewriting the instructions by gut feel: "a bit more salt next time". Another cook writes down what "good" means (a tasting score), tries a few variations, scores each one and keeps the best. DSPy is the second cook. The recipe steps are your modules, the tasting score is your metric, and the trial-and-score process is the optimizer.

## What it is

DSPy is a Python library, published under the MIT licence and developed in the open on GitHub. It replaces one long prompt string with a small set of concepts.

| Part | What it does |
|---|---|
| Signature | A declaration of a step's inputs and outputs, such as `question -> answer` |
| Module | A building block that uses a signature with a strategy, such as `Predict` or `ChainOfThought` |
| Program | Your own class that combines modules in ordinary Python code |
| Language model client | The connection to the model, configured once and used by all modules |
| Adapter | The layer that turns signatures into actual prompts and parses replies, for example as chat or JSON |
| Example | A training or test item with inputs and, optionally, expected outputs |
| Metric | A function that scores a program's output |
| Optimizer | An algorithm that improves the program's instructions, examples or weights against the metric |
| Evaluate | A utility that runs a program over a dataset and reports the metric |

Common modules include `Predict` (a direct answer), `ChainOfThought` (reasoning before the answer), `ReAct` (a tool-using agent loop; see [the ReAct loop](/tools/react-loop)) and modules that write and run code.

## Where it came from

DSPy came from the Stanford NLP group, led by Omar Khattab with colleagues including Christopher Potts and Matei Zaharia. An earlier framework, Demonstrate-Search-Predict, was published in late 2022 and focused on retrieval pipelines. The paper "DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines" appeared in October 2023 and set out the core idea of signatures, modules and optimizers, which it then called teleprompters.

New optimizers followed. MIPROv2 (2024) jointly searches for instructions and few-shot examples. GEPA, described in a July 2025 paper titled "Reflective Prompt Evolution Can Outperform Reinforcement Learning", has a model read its own execution traces, reflect in plain language on what went wrong, and propose improved prompts. DSPy 3.0, released in August 2025, added GEPA and SIMBA as optimizers, reinforcement-learning-based training, multimodal inputs, MCP tool support, async execution and MLflow tracing.

## How it works

### Step 1: Configure the model

You create a language model client and set it as the default. DSPy can talk to many providers and to local models.

### Step 2: Write signatures

A signature names inputs and outputs. The short form is a string like `"question -> answer"` or `"context, question -> answer: str, confidence: float"`. For more control, you write a class with typed fields and short descriptions. The signature says what to do, not how to phrase the prompt.

```python
import dspy

dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))
classify = dspy.ChainOfThought("ticket_text -> category, urgency: int")
result = classify(ticket_text="The invoice total is wrong and payment is due today.")
print(result.category, result.urgency)
```

### Step 3: Compose a program

You combine modules in a Python class, much like layers in a neural network. A RAG program might call a retriever, then a `ChainOfThought` module that answers from the retrieved text. A multi-hop program might generate a search query, retrieve, generate a second query, retrieve again and then answer.

### Step 4: Define a metric and data

A metric is a Python function that takes an example and a prediction and returns a score. It can be exact match, a numeric score, a set of checks, or a judgement by another model; see [LLM as judge](/tools/llm-as-judge). You prepare a training set and a separate development set, often only tens to a few hundred examples.

### Step 5: Optimize

You choose an optimizer and "compile" the program. Depending on the optimizer, it may:

- run the program on training examples and keep successful traces as few-shot demonstrations (bootstrapping);
- propose new instructions for each module and search for the best combination (as in MIPROv2);
- reflect on failures in natural language and evolve better prompts (as in GEPA);
- fine-tune the model's weights on successful traces.

The result is the same program with better instructions, examples or weights.

### Step 6: Evaluate, save and deploy

You score the optimized program on the held-out development set and compare it with the starting version. You can save the optimized program and load it in production. When you change models, you re-run the optimizer rather than rewriting prompts.

## Key concepts and terms

| Term | Meaning |
|---|---|
| Signature | Declared inputs and outputs of a step |
| Module | A reusable step that applies a strategy to a signature |
| `Predict` | The basic module that maps inputs to outputs |
| `ChainOfThought` | A module that adds a reasoning field before outputs |
| Program | User code that composes modules |
| Metric | A scoring function for outputs |
| Optimizer | An algorithm that improves prompts, examples or weights |
| Bootstrapping | Generating demonstrations from the program's own successful runs |
| MIPROv2 | An optimizer that searches over instructions and demonstrations together |
| GEPA | An optimizer that uses reflection on traces to evolve prompts |
| Adapter | The layer that renders signatures into prompts and parses outputs |
| Compile | Running an optimizer on a program to produce an improved version |

## A worked example

This scenario is illustrative. A support team wants to route incoming tickets into 9 categories with an urgency score.

1. They write a signature: ticket text in, category and urgency out. They wrap it in a `ChainOfThought` module.
2. They label 300 past tickets: 200 for training and 100 for development.
3. The metric gives 1 point for the correct category and 0.5 extra if urgency is within one point of the human label, scaled to 0 to 1.
4. The unoptimized program scores 0.71 on the development set with a small, cheap model.
5. They run MIPROv2. It proposes several instruction variants and selects four demonstrations. The development score rises to 0.84.
6. They try GEPA with the same budget and reach 0.86, with a shorter prompt.
7. Months later they switch to a different provider. Instead of rewriting prompts, they re-run the optimizer overnight, check the development score again, and deploy.

These numbers are for illustration only; real gains depend on the task, data and model.

## Strengths and limitations

**Strengths:**

- Replaces fragile prompt strings with clear, modular code.
- Optimizers improve quality against a metric you define, often with small datasets.
- Makes switching models much cheaper: re-optimize instead of re-writing.
- Encourages measurement through metrics and held-out sets.
- Supports retrieval, agents, multi-step pipelines and, in some setups, fine-tuning.

**Limitations:**

- You need a meaningful metric and labelled examples; without them, optimization has nothing to aim at.
- Optimization runs cost model calls and time.
- The abstraction feels unfamiliar to people used to editing prompts directly.
- Optimized prompts can overfit a small training set.
- It is a programming and tuning framework, not a full application platform with hosting, user interfaces or durable workflows.

## How it compares with alternatives

| Option | Strength | Trade-off |
|---|---|---|
| DSPy | Learns prompts and examples against a metric | Needs data and a metric |
| Manual [prompt engineering](/tools/prompt-engineering) | Fast to start, full control | Fragile, hard to transfer to new models |
| [Pydantic AI](/tools/pydantic-ai) | Type-safe agent apps | No automatic prompt optimization |
| [LlamaIndex](/tools/llamaindex) | Data ingestion and retrieval | Prompts tuned by hand |
| [Supervised fine-tuning](/tools/sft) | Changes model weights for strong gains | Needs more data, compute and hosting |

## Practical guidance and best practices

1. Write the metric first; if you cannot score outputs, DSPy cannot help much.
2. Start with 50 to 200 good examples and keep a separate development set.
3. Begin with simple modules and a baseline score before optimizing.
4. Use typed output fields so results are easy to check.
5. Try a light optimizer first, then heavier ones if gains justify the cost.
6. Watch for overfitting by comparing training and development scores.
7. Save optimized programs with version numbers and the model they were tuned for.
8. Re-optimize when you change models, data or pipeline structure, and re-run [evals](/tools/evals).

## Common mistakes

- Using a vague metric such as "looks good", which optimizers cannot use well.
- Evaluating on the same examples used for optimization.
- Expecting optimization to fix a badly designed pipeline.
- Copying optimized prompts to a different model without re-optimizing.
- Running very expensive optimizers on tasks where a simple module already meets the target.

## Frequently asked questions

### What is DSPy?

DSPy is an open-source Python framework from Stanford that lets you build language model pipelines from signatures and modules, then automatically improve their prompts, examples or weights against a metric.

### What does "programming, not prompting" mean?

It means you write code that declares what each step does and how to measure success, and let optimizers work out the actual prompt wording, rather than crafting prompts by hand.

### What is a signature?

A signature is a short declaration of a step's inputs and outputs, for example `question -> answer`. DSPy turns it into a prompt through an adapter.

### What do DSPy optimizers change?

Depending on the optimizer, they change instructions, add few-shot demonstrations chosen from successful runs, evolve prompts through reflection, or fine-tune the model's weights.

### Do I need a lot of data?

Not usually. Many tasks see gains with tens to a few hundred examples, but you do need a clear metric and a held-out set to check results.

### Can DSPy build agents?

Yes. The `ReAct` module builds tool-using agents, and the whole agent can be optimized against a metric like any other program.

## Glossary

- Signature: declared inputs and outputs.
- Module: a reusable step with a strategy.
- Program: composed modules in code.
- Metric: a function that scores outputs.
- Optimizer: an algorithm that improves a program.
- Demonstration: an example shown to the model in the prompt.
- Bootstrapping: creating demonstrations from successful runs.
- Adapter: the prompt rendering and parsing layer.
- Development set: held-out data for checking results.

## Summary

DSPy treats prompts as parameters to be learned, not strings to be guessed. You declare signatures, compose modules into a program, define a metric and let optimizers such as MIPROv2 or GEPA improve instructions, examples or weights. It came from Stanford research in 2023 and reached version 3.0 in August 2025. It pays off when you have a clear metric and some labelled data, and especially when you expect to change models over time.