Özer Cem KELAHMETKELAHMET
Glossary · Agentic AI

What is Cost and latency budgets? Definition and meaning

Explicit ceilings on tokens, tool calls and seconds for an agent run.

Definition

Cost and latency budgets: Explicit ceilings on tokens, tool calls and seconds for an agent run.

In short: Limits on spend and time per task.

When it is used

Keeping agents economical and responsive.

Where it fits

Cost and latency budgets belongs to Operations in the Agentic AI topic. These are the other terms in the same group:

  • Tracing: Captures prompts, tool calls, inputs and outputs of every step as a trace.
  • Evals: Test sets and graders that measure quality of a prompt, model or agent.
  • Regression boards: A standing set of benchmark rows that is re-run in full after each change so that a new fix cannot silently break an old result.
  • Sandboxing: Executes code and tools inside a restricted container or virtual machine.
  • Prompt-injection defense: Techniques that stop text in web pages, files or tool results from hijacking the agent.

Learn more

The full guide to Cost and latency budgets explains how it works, when to use it and how it compares with the alternatives, with examples. The glossary lists every term on the site from A to Z.

Previous term: ConvNeXt · Next term: CQRS

Last updated Nov 18, 2026
Type to search.
    ↑↓ navigate↵ open