OpenTelemetry

Overview

OpenTelemetry, often shortened to OTel, is an open standard and a set of tools for producing and collecting telemetry from software: traces, metrics and logs. It defines common APIs that developers use to instrument code, SDKs that process the data, a wire protocol called OTLP for sending it, and a Collector that receives, transforms and forwards it. It does not store or display data itself. Instead it feeds any back end you choose, such as Prometheus, Jaeger, Tempo, Grafana or a commercial observability service.

An everyday analogy is the standard shipping container. Before containers, every port handled cargo differently, and switching shipping lines meant repacking goods. Once boxes had a common size and fittings, any crane, ship, train or lorry could move them. OpenTelemetry is the shipping container for telemetry: instrument once in a standard shape, and the data can travel to whichever tool you use today or switch to tomorrow.

For AI systems this matters because a single user request may pass through a gateway, a retrieval service, a vector database, a model server and a tool call. Without consistent tracing, nobody can see where time or errors come from. OpenTelemetry also has semantic conventions for generative AI, so model calls can be recorded with common attribute names, which LLM observability tools such as Langfuse and Phoenix can ingest. See tracing for the agent-specific view.

What it is

OpenTelemetry is a specification plus many implementations.

The main signals are traces, metrics and logs, all of which have reached general availability. Profiling has been added as a newer signal and is less mature.

Where it came from

OpenTelemetry was formed in May 2019 by merging two earlier open-source projects with overlapping goals: OpenTracing, a tracing API standard hosted by the Cloud Native Computing Foundation (CNCF), and OpenCensus, a set of instrumentation libraries that originated at Google. Having two competing standards was confusing for users and library authors, so the communities combined them under the CNCF.

The project moved to incubating status in the CNCF in 2021. The tracing specification stabilised first, followed by metrics and then logs. OpenTracing was archived in January 2022 and OpenCensus in July 2023. OpenTelemetry graduated in the CNCF in May 2026. It is one of the CNCF's most active projects by contributor count and is supported by most observability vendors.

How it works

1. Instrument

Developers add the OpenTelemetry API to their code, or rely on instrumentation libraries and automatic agents. Each incoming request starts a span, a timed operation with a name, attributes and status. Nested work creates child spans. The same API records metrics such as counters and histograms and attaches trace context to logs.

2. Propagate context

When a service calls another, the SDK injects trace context into the outgoing request, typically as the W3C Trace Context traceparent header. The receiving service extracts it and continues the same trace. This is what links spans across processes into one end-to-end trace.

3. Process in the SDK

The SDK decides which traces to keep through sampling, adds resource attributes such as service.name and deployment environment, batches data and hands it to an exporter.

4. Export over OTLP

Exporters send data, usually over OTLP, to a Collector or directly to a back end. OTLP is now accepted natively by many systems, including Prometheus 3.0 for metrics.

5. Collect and route

The Collector receives telemetry through receivers (OTLP, Prometheus scraping, log files and others), runs processors (batching, filtering, redacting attributes, tail-based sampling) and sends it through exporters to one or more back ends. It can run as an agent next to each application or as a central gateway.

6. Analyse in a back end

Storage and analysis happen in tools you choose. Because the data follows shared conventions, changing back ends usually means changing Collector configuration, not application code.

A minimal Python example of manual tracing:

from opentelemetry import trace

tracer = trace.get_tracer("scoring-service")

def score(request):
    with tracer.start_as_current_span("score") as span:
        span.set_attribute("model.version", "2026-11")
        return run_model(request)

Key concepts and terms

A worked example

This scenario is illustrative. A team runs a question-answering assistant made of four services: an API gateway, a retrieval service that queries a vector database, a reranker, and a model gateway that calls a hosted language model. Users report that some answers take more than ten seconds.

The team adds automatic instrumentation to the gateway and retrieval service, and manual spans around the reranker and model calls. They set service.name on each and use generative AI semantic conventions for the model span, recording the model name and input and output token counts. All services export OTLP to a Collector running in the cluster. The Collector batches data, removes an attribute that contained raw user email addresses, keeps 10% of normal traces but 100% of traces slower than 5 seconds through tail sampling, and exports traces to Tempo and metrics to Prometheus.

A day later the slow traces show a clear pattern. In 70% of them, the retrieval span lasts over 6 seconds, and inside it a child span shows the vector query running against an index without a filter, scanning far more vectors than needed. The model call itself averages 1.8 seconds. Fixing the filter brings p95 end-to-end latency from 9.4 to 3.2 seconds, and the team can prove it from the same traces.

Strengths and limitations

Strengths:

• Vendor-neutral: instrument once, send anywhere.
• One framework for traces, metrics and logs with shared context.
• Wide language support and a large set of instrumentation libraries.
• The Collector centralises filtering, redaction and routing without code changes.
• Semantic conventions, including for generative AI, make data comparable across tools.
• Broad industry support and CNCF graduated status.

Limitations:

• Many moving parts; configuration of SDKs and Collectors has a learning curve.
• Maturity varies by language and by signal; check the status for your stack.
• Generative AI conventions are still evolving, so attribute names can change.
• High trace volumes need careful sampling to control cost.
• It collects and transports data only; storage and analysis need other tools.

How it compares with alternatives

Practical guidance and best practices

1. Set service.name and deployment environment on every service; untagged telemetry is hard to use.
2. Start with automatic instrumentation for frameworks, then add manual spans for business-relevant steps.
3. Run a Collector rather than exporting straight from every service to a vendor.
4. Redact sensitive data, including prompts with personal data, in the SDK or Collector before export.
5. Use tail-based sampling to keep slow and failed traces while reducing volume.
6. Follow semantic conventions instead of inventing attribute names.
7. Link logs to traces by including trace and span IDs in log records.
8. Pin and track the stability status of conventions you depend on, especially for generative AI.

Common mistakes

• Breaking context propagation at a message queue or custom HTTP client, so traces split into fragments.
• Putting high-cardinality values, such as user IDs, into metric attributes.
• Recording full prompts and responses with personal data and sending them to third parties.
• Sampling only at the head and losing the rare slow requests that matter most.
• Mixing several instrumentation agents in one process and producing duplicate spans.
• Treating OpenTelemetry as a back end and expecting it to store or chart data.

Frequently asked questions

Is OpenTelemetry a monitoring tool?

No. It is a standard and toolkit for creating and moving telemetry. You still need a back end to store, query and visualise it.

Does OpenTelemetry replace Prometheus?

Not necessarily. They work together. OpenTelemetry can collect metrics and send them to Prometheus, which accepts OTLP natively since version 3.0, and the Collector can scrape Prometheus endpoints.

Which signals are stable?

Traces, metrics and logs have reached general availability in the specification. Profiling is a newer signal. Stability of individual language SDKs and conventions varies, so check the project's status pages.

Can OpenTelemetry trace LLM calls?

Yes. Generative AI semantic conventions define attributes for model requests, such as model name and token usage, and several libraries instrument popular model SDKs automatically.

What is the Collector for?

It decouples applications from back ends. It receives telemetry, applies processing such as batching, sampling and redaction, and exports it to one or more destinations.

Is OpenTelemetry graduated in the CNCF?

Yes. It graduated in May 2026, after forming in 2019 and incubating from 2021.

Glossary

• OTel: common short name for OpenTelemetry.
• Span: one timed step within a trace.
• Trace ID: the identifier shared by all spans in one trace.
• W3C Trace Context: the standard HTTP headers for propagating trace identifiers.
• Receiver: the Collector component that accepts incoming data.
• Processor: the Collector component that modifies data in transit.
• Exporter: the component that sends data onwards.
• Tail sampling: keeping or dropping traces after seeing them complete.
• Resource attributes: metadata describing the source of telemetry.
• GenAI semantic conventions: standard attributes for generative AI operations.

Summary

OpenTelemetry is the vendor-neutral standard for instrumenting software and moving traces, metrics and logs to any back end. Formed in 2019 from OpenTracing and OpenCensus, it graduated in the CNCF in May 2026, with traces, metrics and logs generally available and conventions for generative AI still evolving. It works best with consistent service naming, a Collector in the middle, redaction of sensitive data, tail-based sampling and careful context propagation across every hop.

Related

It belongs to Platform in my stack, under MLOps and AI infrastructure. Others in the same group: