Langfuse

Overview

Langfuse is an open-source platform for observing, evaluating and improving applications built on large language models (LLMs). It records every step of a request as a trace, stores and versions prompts, runs evaluations and tracks cost, latency and quality over time. Teams can use the managed cloud service or run it on their own servers.

The open-source angle is its main draw. Many organisations cannot send prompts and responses, which may contain customer data, to a third-party service. Langfuse's core is MIT-licensed and can be self-hosted, so the data stays inside the organisation's own infrastructure while still giving engineers modern tracing and evaluation tools.

An everyday analogy: a restaurant kitchen's order tickets and tasting log. Each ticket shows what was ordered, which station cooked each part and how long it took. The tasting log records how dishes scored when the chef checked them. Langfuse keeps both for an LLM application, and you can keep the binder in your own kitchen rather than at head office.

What it is

The repository is MIT-licensed except for folders marked as enterprise edition, which hold some commercial features.

Where it came from

Langfuse was founded in Berlin by Marc Klingen, Max Deichmann and Clemens Rawert, and went through Y Combinator's Winter 2023 batch. The team started by building LLM applications themselves, found debugging and evaluating them hard, and turned their internal tooling into an open-source product in 2023.

The project grew a large set of integrations, including the OpenAI SDK, LangChain, LlamaIndex, LiteLLM, the Vercel AI SDK and many agent frameworks. Later major versions moved the analytics storage onto ClickHouse to handle high trace volumes, and rebuilt the SDKs on OpenTelemetry.

In January 2026, Langfuse joined ClickHouse, the company behind the open-source analytical database of the same name. The project remains open source and continues to offer both cloud and self-hosted options. See ClickHouse for the database.

How it works

1. Instrument the application

You add an SDK and API keys. There are three common routes:

• Use the observe decorator to trace your own functions.
• Use a drop-in integration, such as Langfuse's wrapper for the OpenAI client or its LangChain callback handler.
• Send OpenTelemetry spans from any instrumented framework to Langfuse's endpoint.

from langfuse import observe
from langfuse.openai import openai  # drop-in wrapper

@observe()
def answer(question: str) -> str:
    r = openai.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": question}])
    return r.choices[0].message.content

2. Ingest and store

The SDKs batch events and send them in the background, so tracing adds little latency. On the server, events are queued and written to an analytical store. A self-hosted deployment typically runs the web and worker containers with PostgreSQL for transactional data, ClickHouse for traces and observations, Redis for queues and caching, and object storage such as S3 for raw events and large payloads.

3. Explore traces

In the interface, each trace shows a tree of observations: spans for steps, generations for model calls and events for single points. Each generation records the model, parameters, tokens and calculated cost. You can filter by user, session, tag, release, latency or score.

4. Manage prompts

Prompts live in Langfuse with versions and labels such as production or staging. The application fetches the prompt by name and label, and the SDK caches it. Traces link back to the prompt version used, so you can compare versions by their scores.

5. Evaluate

Scores attach to traces or observations. They can come from users (thumbs up or down), from annotators working through a review queue, from code checks or from LLM judges configured in the interface to run on new traces. Datasets hold test inputs and expected outputs, and experiment runs score each application version against them.

6. Monitor

Dashboards summarise cost, latency, volume and scores by model, user, prompt version or release.

Key concepts and terms

A worked example

This is an illustrative scenario.

A legal-tech team runs a contract summariser. Data must not leave their cloud account, so they self-host Langfuse with Helm on their Kubernetes cluster.

1. They wrap their summariser with observe and the OpenAI wrapper. Each request now shows retrieval of clauses, the summary call and a formatting step.
2. The cost dashboard shows an average of 6,200 input tokens per request. Traces reveal that the retriever passes every clause of long contracts.
3. They move the summary prompt into Langfuse as version 3, labelled production, and create version 4 that asks for clause numbers in citations.
4. They build a dataset of 120 contracts with lawyer-written summaries and run both versions. An LLM judge scores faithfulness from 1 to 5, and a code check confirms that cited clause numbers exist.
5. Version 4 raises mean faithfulness from 3.9 to 4.3 and cuts invalid citations from 14 to 3. They move the production label to version 4 without redeploying.
6. They also cap retrieval to the top 20 clauses, and average input tokens fall to about 3,500.

Strengths and limitations

Strengths:

• Open-source core under MIT, with full self-hosting.
• Tracing, prompts, evaluation and cost tracking in one tool.
• Many integrations and OpenTelemetry support, so it is framework-neutral.
• Prompt labels allow changes without code deploys.
• Designed for high trace volumes using an analytical database.

Limitations:

• Self-hosting means running several services: databases, cache, storage and workers.
• Some features sit in enterprise-licensed folders.
• Deep evaluation workflows may need custom code around the SDK.
• LLM-judge scores need checking against human labels.
• Traces contain prompts and outputs, so data protection still needs planning.

How it compares with alternatives

Practical guidance and best practices

1. Decide early between cloud and self-hosting, based on data rules and operations capacity.
2. Add user ids, session ids, tags and release versions to traces from the start.
3. Move prompts into prompt management and use labels for promotion between environments.
4. Track cost per generation and set alerts on unusual spikes.
5. Build datasets from real failing traces, then add edge cases.
6. Combine code checks with LLM judges, and review a sample by hand.
7. For self-hosting, follow the Helm or Terraform templates and back up PostgreSQL, ClickHouse and object storage.
8. Mask sensitive data in the SDK before it is sent.

Common mistakes

• Tracing only model calls, so retrieval and tool problems stay invisible.
• Keeping prompts in code and losing the link between prompt version and score.
• Running self-hosted Langfuse without log rotation or backups.
• Treating LLM-judge scores as ground truth.
• Comparing experiment runs that used different datasets.

Frequently asked questions

Is Langfuse really open source?

Yes. The repository is MIT-licensed, except for folders marked as enterprise edition. The core tracing, prompt management, evaluation and datasets features can be self-hosted.

Can I use Langfuse without LangChain?

Yes. It has native Python and JavaScript/TypeScript SDKs, wrappers for popular model clients, and accepts OpenTelemetry data, so it works with almost any framework or plain code.

What does a self-hosted deployment need?

A typical production setup runs the Langfuse web and worker containers with PostgreSQL, ClickHouse, Redis and S3-compatible object storage. Docker Compose suits trials; Kubernetes with Helm is recommended for production.

Does tracing slow down my application?

The SDKs send data asynchronously in batches, so the added latency is usually small. In short-lived processes, flush the client before exit so events are not lost.

What changed when Langfuse joined ClickHouse?

Langfuse announced in January 2026 that it had become part of ClickHouse. The project has continued as open source with cloud and self-hosted options. Check the official site for current plans and pricing.

Glossary

• LLMOps: practices for running LLM applications reliably in production.
• Span: a timed unit of work inside a trace.
• Token: a piece of text counted by a model for input or output.
• Flush: sending any events still waiting in a buffer.
• Helm: a package manager for Kubernetes applications.
• Analytical database: a database built for fast queries over large volumes of records.
• Drop-in wrapper: a replacement client that adds tracing without changing calls.

Summary

Langfuse is an open-source LLM engineering platform that combines tracing, prompt management, evaluation, datasets and cost monitoring. Its MIT-licensed core and self-hosting option make it a strong choice when data must stay in-house, and its SDKs and OpenTelemetry support make it framework-neutral. Since January 2026 it is part of ClickHouse. Instrument every step, link prompt versions to scores, check automated judges against people, and treat self-hosting as a real service to operate. See evals and model monitoring for the wider practice.

Related

It belongs to LLM observability and evaluation in my stack, under Generative AI and LLMs. Others in the same group: