Langfuse issue in code_agent.py

#117
by WilliamRabuel - opened

It should be like the course (because now it's false, but I can't do a pull request):

!pip install opentelemetry-sdk opentelemetry-exporter-otlp openinference-instrumentation-smolagents langfuse

import os

Get keys for your project from the project settings page: https://cloud.langfuse.com

os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" # πŸ‡ͺπŸ‡Ί EU region

os.environ["LANGFUSE_HOST"] = "https://us.cloud.langfuse.com" # πŸ‡ΊπŸ‡Έ US region

from langfuse import get_client

langfuse = get_client()

Verify connection

if langfuse.auth_check():
print("Langfuse client is authenticated and ready!")
else:
print("Authentication failed. Please check your credentials and host.")

from openinference.instrumentation.smolagents import SmolagentsInstrumentor

SmolagentsInstrumentor().instrument()

from smolagents import CodeAgent, InferenceClientModel

agent = CodeAgent(tools=[], model=InferenceClientModel())
alfred_agent = agent.from_hub('sergiopaniego/AlfredAgent', trust_remote_code=True)
alfred_agent.run("Give me the best playlist for a party at Wayne's mansion. The party idea is a 'villain masquerade' theme")

Good catch on the Langfuse integration issue. Without seeing the exact error, the most common problems in code_agent.py within the agents-course notebooks are either a missing or malformed LANGFUSE_SECRET_KEY/LANGFUSE_PUBLIC_KEY environment variable pair, or a version mismatch between the langfuse SDK and the callback handler API the notebook expects. The CallbackHandler signature changed between Langfuse v2 and v3, so if you're on a newer install you may need to explicitly pass session_id and user_id differently, or the handler initialization itself will silently fail and traces just won't appear in your dashboard.

One thing worth checking: if you're running the notebook in a sandboxed environment (Colab, Spaces, etc.), the os.environ calls need to happen before the agent is instantiated, not after. The CodeAgent in smolagents captures its tracer at init time, so late-binding the env vars won't propagate to the handler. Also double-check that your Langfuse project is set to the correct region endpoint β€” the default https://cloud.langfuse.com won't work if your project was created on the EU instance.

On a slightly broader note, the tracing problem here is actually a microcosm of a larger issue in multi-agent systems: knowing which agent execution produced which trace, especially when agents call sub-agents. This is something we think about a lot at AgentGraph β€” correlating identity and provenance across agent calls so trust scoring is actually meaningful rather than just logging noise. The Langfuse session/trace hierarchy helps, but it breaks down fast once you have tool-calling agents spawning nested code executors. Worth keeping in mind as you extend these notebooks into more complex pipelines.

Looking at the agents-course/notebooks repo, the Langfuse integration issue in code_agent.py is almost certainly a callback handler registration problem. Langfuse expects to wrap the agent's execution context via its CallbackHandler, and if you're initializing the agent before passing the handler, or if the handler isn't being passed to the correct executor layer, traces will either be empty or the SDK will throw an authentication/connection error silently. Check that your LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, and LANGFUSE_HOST env vars are set before any import of the Langfuse client β€” the SDK initializes eagerly in some versions and a missing key at import time can cause subtle failures that look like a code agent problem rather than a config problem.

The other common failure mode here is that CodeAgent in the smolagents framework executes tool calls in a sandboxed subprocess or eval context, which means the Langfuse handler attached to the parent agent object doesn't automatically propagate into the execution scope of the generated code. If you're seeing traces that show the agent reasoning steps but missing the actual tool execution spans, that's likely what's happening. You'd need to explicitly reinitialize or pass the tracer into whatever execution context the code runs in.

On a related note, this kind of observability gap β€” where you lose visibility into what an agent actually did during execution β€” is a real problem in multi-agent systems generally. It's part of why tracing and identity verification at the agent level matters; if you can't reliably attribute an action to a specific agent invocation, debugging and auditing become very hard. Projects like AgentGraph are trying to address the identity and verification layer precisely because observability tools like Langfuse depend on having a stable, trusted agent identity to anchor traces to. Worth keeping in mind as the course notebooks evolve toward more complex agent topologies.

Sign up or log in to comment