CrewAI Multi Tool Usage Single Trace Problem

[OnurYigitArpali]

I want to integrate langfuse to crewai. There is one crew having an agent that use multiple custom tools. After a single execution, i expect to create single trace by Langfuse. But it creates multiple trace entities for tools, generations, etc as seen in the images. I already tried the methods discussed in https://github.com/orgs/langfuse/discussions/6291. Also check the related documents, but i could not to create single trace.

from crewai.project import CrewBase, agent, crew, task
from langfuse import Langfuse
import openlit

from tools import (x,y)


@CrewBase
class AnalyzerCrew():

    agents_config = 'config/agents.yaml'
    tasks_config = 'config/tasks.yaml'

    @agent
    def planner(self) -> Agent:
        return Agent(
            config=self.agents_config['planner'],
            tools=[
                x,y
            ],
            verbose=True,
            allow_delegation=False,
            max_iter=2,
            max_rpm=15,
            max_execution_time=300
        )

    @task
    def research_planning(self) -> Task:
        return Task(
            config=self.tasks_config['research_planning'],
            agent=self.planner(),
            output_pydantic=ResearchPlan,
            output_file='research_plan.json'
        )

    @crew
    def crew(self) -> Crew:
        """Creates the CTI research planning crew - first stage of multi-agent workflow"""
        return Crew(
            agents=self.agents,
            tasks=self.tasks,
            process=Process.sequential,
            verbose=True,
            planning=True,
            planning_llm="openai/gpt-4o-mini"
        )


_langfuse_client = Langfuse(
    public_key=config.public_key,
    secret_key=config.secret_key,
    host=config.host,
    debug=config.debug
)

openlit.init(
    tracer=client._otel_tracer,
    disable_batch=True
)


with _langfuse_client.start_as_current_span(
    name="Analysis Workflow"
) as span:
    # Initialize crew
    crew = AnalyzerCrew()
    print(f"✅ Crew initialized successfully")
    
    # Run analysis
    result = crew.crew().kickoff(
        inputs={'text': text, 'output_dir': output_dir}
    )

[KeepALifeUS]

The multiple traces issue is common with OpenTelemetry-style tracing. Here's how to consolidate:

Solution 1: Single Span Context

Ensure all operations share the same trace context:

from langfuse import Langfuse
from opentelemetry import trace

langfuse = Langfuse()

# Create a single parent trace
with langfuse.trace(name="Analysis Workflow") as trace:
    # Pass trace context to all operations
    result = crew.kickoff(
        inputs={"task": task},
        callbacks=[LangfuseCallback(trace_id=trace.trace_id)]
    )

Solution 2: Manual Span Hierarchy

with langfuse.trace(name="crew_execution") as parent:
    for agent in crew.agents:
        with langfuse.span(name=f"agent_{agent.name}", parent=parent):
            # Agent operations here
            pass

Solution 3: State-Based Tracing

Instead of relying on auto-instrumentation:

state = {
    "trace_id": str(uuid.uuid4()),
    "spans": [],
    "events": []
}

def log_event(name, data):
    state["events"].append({
        "trace_id": state["trace_id"],
        "name": name,
        "timestamp": time.time(),
        "data": data
    })

# After execution, send all events as single trace
langfuse.trace(
    trace_id=state["trace_id"],
    events=state["events"]
)

Root Cause

OpenLIT creates new traces for each tool call because it doesn't share context. Disable OpenLIT auto-tracing and manually instrument to maintain hierarchy.