# Structured outputs

Structured outputs make a language model return data that matches a JSON Schema. Learn constrained decoding, schema design, validation and when to use it.

## Overview

Structured outputs are a way to make a language model return data in an exact, machine-readable shape, usually JSON that matches a JSON Schema you supply. Instead of hoping the model writes valid JSON and then parsing whatever comes back, you tell the model, or the serving system, which fields, types and allowed values the answer must have. The best implementations enforce this during generation, so the output cannot break the schema.

This matters because most real applications feed model output into code. A missing field, a misspelled key, an extra sentence before the JSON or a number written as text can crash a pipeline or, worse, slip through silently. Structured outputs turn a model's reply into something closer to the return value of a typed function.

An everyday comparison is a paper form with boxes versus a blank sheet. Ask someone to "write down your details" on a blank sheet and you get every possible layout. Give them a form with boxes for name, date of birth and postcode, some of which only accept digits, and the result is easy to process. Structured outputs give the model a form instead of a blank sheet. Constrained decoding goes further: the pen simply cannot write outside the boxes.

## What it is

Structured output is a family of techniques, from simple prompting to strict, enforced decoding. They differ in how strongly they guarantee the shape.

| Technique | How it works | Guarantee |
|---|---|---|
| Prompt-only formatting | Instructions and examples ask for JSON | None; output may be invalid |
| JSON mode | The model is constrained to produce valid JSON | Valid JSON, but not a specific schema |
| Schema-constrained output | A JSON Schema is compiled into rules that restrict each generated token | Output matches the schema, within supported features |
| Strict tool use | Tool call arguments are constrained to the tool's schema | Tool names and inputs match the schema |
| Grammar-constrained decoding | A formal grammar, such as a regular expression or context-free grammar, restricts tokens | Output matches the grammar |
| Parse, validate and retry | Validate output in code and ask the model to fix errors | Eventually valid or a clear failure |

Constrained decoding is what makes the strong guarantee possible. At each step, a model proposes probabilities for the next token. The decoder masks out every token that would make the output break the schema, so the model can only choose among valid continuations.

## Where it came from

Early users of large language models asked for JSON in the prompt and wrote fragile parsers. When [function calling](/tools/function-calling) arrived in 2023, models were trained to produce arguments for declared functions, which improved reliability but did not guarantee it. OpenAI introduced a JSON mode in November 2023 that guaranteed valid JSON but not a particular schema.

Open-source projects showed how to enforce structure during generation. Libraries such as Outlines and Microsoft's Guidance, and grammar support in llama.cpp, used the idea of masking invalid tokens. Inference servers such as [vLLM](/tools/vllm) and [SGLang](/tools/sglang) added guided decoding using these methods.

In August 2024 OpenAI launched Structured Outputs in its API, with a `strict` option that constrains output to a supplied JSON Schema. Google's Gemini API supports response schemas. Anthropic released structured outputs for the Claude API in beta in November 2025, with JSON outputs and strict tool use, and the feature later became generally available without a beta header. By 2026 schema-constrained output is a standard feature of major model APIs and open serving stacks, each supporting a slightly different subset of JSON Schema.

## How it works

### Step 1: Define the schema

You describe the output as a JSON Schema: an object with properties, types, required fields and allowed values. Many developers write the schema as a Pydantic model in Python or a Zod schema in TypeScript, and let the SDK convert it.

```json
{
  "type": "object",
  "properties": {
    "invoice_number": {"type": "string"},
    "total": {"type": "number"},
    "currency": {"type": "string", "enum": ["EUR", "GBP", "USD"]},
    "due_date": {"type": "string", "format": "date"}
  },
  "required": ["invoice_number", "total", "currency", "due_date"],
  "additionalProperties": false
}
```

### Step 2: Compile the schema

The provider or inference server turns the schema into a grammar or a finite-state machine. This can take noticeable time for a new schema, so providers usually cache compiled schemas. Some features, such as recursive schemas or numeric ranges, may not be supported for constrained decoding, and the API will reject them or move them into descriptions.

### Step 3: Constrain each token

During generation, the decoder tracks where it is in the schema. If the next valid thing is the key `"currency"`, only tokens that continue that key are allowed. If the value must be one of three currency codes, only those can be produced. The model still chooses the content; the decoder only stops it from breaking the structure.

### Step 4: Validate in your code

Even with constrained decoding, validate the result. Some rules cannot be expressed or enforced in the supported schema subset, such as "the due date must be after the invoice date" or "total must be positive". There are also edge cases: an output cut off by a token limit can be incomplete, and a model may refuse a request instead of answering. Libraries such as Pydantic make validation easy.

### Step 5: Handle failures

If validation fails, you can retry, send the error back to the model to fix, fall back to another path, or flag the item for a person. Track how often this happens; a rising rate is a signal that the schema or prompt needs work.

## Key concepts and terms

| Term | Meaning |
|---|---|
| JSON Schema | A standard vocabulary for describing the shape of JSON data |
| JSON mode | A setting that guarantees valid JSON but not a particular schema |
| Constrained decoding | Restricting each generated token to those that keep the output valid |
| Grammar | A formal description of valid strings used to constrain output |
| Strict mode | A setting that enforces exact schema adherence |
| Strict tool use | Constraining tool call arguments to the tool's schema |
| Enum | A fixed list of allowed values for a field |
| `additionalProperties: false` | A rule forbidding keys not listed in the schema |
| Validation | Checking data against a schema and business rules in code |
| Refusal | A reply in which the model declines rather than filling the schema |
| Schema subset | The parts of JSON Schema a provider supports for constrained decoding |

## A worked example

This scenario is illustrative. An accounts payable team processes 3,000 supplier invoices a month that arrive as PDFs and emails.

1. They define the schema above, plus fields for supplier name, VAT number and a list of line items, each with description, quantity and unit price.
2. They use schema-constrained output. Every reply is valid JSON with all required fields.
3. Their code then checks business rules: line items must add up to the total within 0.01, the VAT number must match a known supplier, and the due date must be after the invoice date.
4. In the first month, 2,910 invoices pass all checks. 64 fail the sum check, mostly because shipping costs appear outside the line items. 26 fail the supplier check.
5. They add an optional `shipping` field and an instruction about it. The next month, sum failures fall to 11.
6. Every failed item goes to a clerk's queue rather than being dropped; see [human-in-the-loop](/tools/human-in-the-loop).

The schema guaranteed the shape; validation caught the meaning.

## Strengths and limitations

**Strengths:**

- Removes parsing errors and missing fields when decoding is constrained.
- Makes model output safe to pass directly to typed code and databases.
- Enums keep values to a known set, which simplifies downstream logic.
- Reduces the need for retries and custom repair code.
- Works for both final answers and tool call arguments.

**Limitations:**

- A valid shape does not mean correct content; models can still fill fields with wrong facts.
- Providers support only a subset of JSON Schema for strict decoding.
- New schemas may add latency while they compile.
- Very complex or deeply nested schemas can hurt quality or hit provider limits.
- Truncation and refusals can still produce non-conforming replies.
- Forcing a rigid format can sometimes reduce reasoning quality if there is no room to think.

## How it compares with alternatives

| Option | Guarantee | Best for |
|---|---|---|
| Structured outputs (strict schema) | Shape matches schema | Extraction, classification, data for code |
| [Function calling](/tools/function-calling) without strict mode | Usually well formed | Tool use where occasional repair is acceptable |
| Prompt-only JSON | None | Quick prototypes |
| Free text plus parsing | None | Human-facing answers |
| [Pydantic AI](/tools/pydantic-ai) validation and retry | Valid after retries or a clear error | Frameworks that combine several methods |

## Practical guidance and best practices

1. Use the strongest enforcement available, then validate in code anyway.
2. Keep schemas flat and small; split very large extractions into several calls.
3. Use enums for any field with a fixed set of values.
4. Give every field a clear description; the model reads them.
5. Make fields optional or nullable when the information may be missing, so the model does not invent values.
6. Add a short "reasoning" or "notes" field before key fields if the task needs thinking, or reason in a separate step.
7. Check business rules in code, not only in the schema.
8. Measure field-level accuracy on labelled examples as part of your [evals](/tools/evals).

## Common mistakes

- Assuming schema-valid output is factually correct.
- Making every field required, which pushes the model to invent values.
- Using schema features the provider does not support and being surprised by errors.
- Ignoring `max_tokens` truncation and refusals in error handling.
- Designing one giant schema for a whole document instead of smaller steps.
- Letting enum values differ only in letter case.

## Frequently asked questions

### What are structured outputs?

Structured outputs are features and techniques that make a language model return data matching a defined structure, usually a JSON Schema, so programs can use the result directly.

### What is the difference between JSON mode and structured outputs?

JSON mode guarantees that the reply is valid JSON, but not that it has the fields and types you need. Structured outputs with a strict schema guarantee the specific shape.

### How is the schema enforced?

Through constrained decoding. The schema is compiled into rules, and at each generation step the decoder only allows tokens that keep the output valid.

### Do I still need validation?

Yes. Schemas cannot express every business rule, and truncated replies or refusals can still break the shape. Validate in code and handle failures.

### Can open-source models do this?

Yes. Libraries such as Outlines and inference servers such as vLLM and SGLang support guided or constrained decoding for open models.

### Does constraining output hurt quality?

It can, if a rigid format leaves no room for reasoning or the schema is confusing. Clear field descriptions, a reasoning field or a separate reasoning step usually avoid this.

## Glossary

- Schema: a formal description of data shape.
- JSON Schema: the standard schema language for JSON.
- Constrained decoding: token-level enforcement of a format.
- Grammar: rules defining valid output strings.
- Enum: a closed list of allowed values.
- Strict mode: exact schema adherence.
- Validation: checking data in code.
- Refusal: a model declining to answer.
- Truncation: output cut off by a token limit.

## Summary

Structured outputs make language models return data in a defined shape, typically JSON matching a schema. The strongest form uses constrained decoding, which masks invalid tokens so the output cannot break the schema. OpenAI added it in August 2024, Anthropic in November 2025, and open serving stacks support it too. Use it whenever code consumes model output, keep schemas simple, use enums and optional fields, and always validate the content as well as the shape.