Özer Cem KELAHMETKELAHMET
AI Simulations · Agents

An agent run: LangGraph loop, Langfuse trace, DeepEval and RAGAS scores

An agent is a loop: the model decides, a tool acts, the result goes back to the model. Watch one run through the graph, its trace and its evaluation, including a run that has to correct a bad first search.

Interactive simulation. It needs JavaScript; the text below explains what it shows.

What you are looking at

The left side is the LangGraph state machine: the dot moves from the agent to the tools and back until the model answers. The right side shows what the other tools record: the message state, the trace as a waterfall of spans with time, tokens and cost, and the DeepEval and RAGAS scores at the end. The runs are scripted with typical values.

Things to try

  • Run the calculator question: one tool call, one answer.
  • Run the bad-retrieval question: the grader notices and the agent searches again.
  • Use Step to pause between nodes and read the state.

How it works in real systems

The agent guide builds this exact system in code: a LangGraph graph, Langfuse tracing, DeepEval tests in CI and RAGAS retrieval scores.

This simulation replays a scripted run so that every step is the same each time; the structure is real, the timings and scores are typical values.

Code lab

The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.

Step 1 · A tool-calling agent in LangGraph

# pip install langgraph langchain-openai
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, MessagesState, START
from langgraph.prebuilt import ToolNode, tools_condition

@tool
def get_order_status(order_id: str) -> str:
    """Look up the delivery status of an order."""
    return {"A1001": "shipped, arrives Thursday"}.get(order_id, "not found")

tools = [get_order_status]
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools(tools)
def agent(state: MessagesState): return {"messages": [llm.invoke(state["messages"])]}
g = StateGraph(MessagesState)
g.add_node("agent", agent); g.add_node("tools", ToolNode(tools))
g.add_edge(START, "agent")
g.add_conditional_edges("agent", tools_condition)        # tool call → "tools", otherwise → END
g.add_edge("tools", "agent")
app = g.compile()

Step 2 · Trace it with Langfuse

# pip install langfuse ; set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST
from langfuse.langchain import CallbackHandler          # Langfuse v3
out = app.invoke({"messages": [("user", "Where is my order A1001?")]}, config={"callbacks": [CallbackHandler()]})
print(out["messages"][-1].content)

Every model call and tool call becomes a span with its latency, tokens and cost, which is the trace panel in the simulation.

Step 3 · Evaluate it

from deepeval.test_case import LLMTestCase, ToolCall
from deepeval.metrics import ToolCorrectnessMetric, AnswerRelevancyMetric
from deepeval import evaluate
called = [ToolCall(name=m.tool_calls[0]["name"]) for m in out["messages"] if getattr(m, "tool_calls", None)]
case = LLMTestCase(input="Where is my order A1001?", actual_output=out["messages"][-1].content,
                   tools_called=called, expected_tools=[ToolCall(name="get_order_status")])
evaluate(test_cases=[case], metrics=[ToolCorrectnessMetric(), AnswerRelevancyMetric()])

More simulations

← RAG types · All AI Simulations · Agentic patterns →

Last updated Nov 17, 2026
Type to search.
    ↑↓ navigate↵ open