An agent run: LangGraph loop, Langfuse trace, DeepEval and RAGAS scores
An agent is a loop: the model decides, a tool acts, the result goes back to the model. Watch one run through the graph, its trace and its evaluation, including a run that has to correct a bad first search.
Interactive simulation. It needs JavaScript; the text below explains what it shows.
What you are looking at
The left side is the LangGraph state machine: the dot moves from the agent to the tools and back until the model answers. The right side shows what the other tools record: the message state, the trace as a waterfall of spans with time, tokens and cost, and the DeepEval and RAGAS scores at the end. The runs are scripted with typical values.
Things to try
- Run the calculator question: one tool call, one answer.
- Run the bad-retrieval question: the grader notices and the agent searches again.
- Use Step to pause between nodes and read the state.
How it works in real systems
The agent guide builds this exact system in code: a LangGraph graph, Langfuse tracing, DeepEval tests in CI and RAGAS retrieval scores.
This simulation replays a scripted run so that every step is the same each time; the structure is real, the timings and scores are typical values.
Code lab
The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.
Step 1 · A tool-calling agent in LangGraph
# pip install langgraph langchain-openai
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, MessagesState, START
from langgraph.prebuilt import ToolNode, tools_condition
@tool
def get_order_status(order_id: str) -> str:
"""Look up the delivery status of an order."""
return {"A1001": "shipped, arrives Thursday"}.get(order_id, "not found")
tools = [get_order_status]
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0).bind_tools(tools)
def agent(state: MessagesState): return {"messages": [llm.invoke(state["messages"])]}
g = StateGraph(MessagesState)
g.add_node("agent", agent); g.add_node("tools", ToolNode(tools))
g.add_edge(START, "agent")
g.add_conditional_edges("agent", tools_condition) # tool call → "tools", otherwise → END
g.add_edge("tools", "agent")
app = g.compile()
Step 2 · Trace it with Langfuse
# pip install langfuse ; set LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_HOST
from langfuse.langchain import CallbackHandler # Langfuse v3
out = app.invoke({"messages": [("user", "Where is my order A1001?")]}, config={"callbacks": [CallbackHandler()]})
print(out["messages"][-1].content)
Every model call and tool call becomes a span with its latency, tokens and cost, which is the trace panel in the simulation.
Step 3 · Evaluate it
from deepeval.test_case import LLMTestCase, ToolCall
from deepeval.metrics import ToolCorrectnessMetric, AnswerRelevancyMetric
from deepeval import evaluate
called = [ToolCall(name=m.tool_calls[0]["name"]) for m in out["messages"] if getattr(m, "tool_calls", None)]
case = LLMTestCase(input="Where is my order A1001?", actual_output=out["messages"][-1].content,
tools_called=called, expected_tools=[ToolCall(name="get_order_status")])
evaluate(test_cases=[case], metrics=[ToolCorrectnessMetric(), AnswerRelevancyMetric()])