• September 29, 2026
• 6 min read

MLflow — GenAI Tracing¶

MLflow Tracing records what an LLM app or agent did on one request: every LLM call, tool call, retrieval and custom step as a span, with inputs, outputs, token usage and latency. Traces are stored in the tracking server next to runs, are OpenTelemetry-compatible, and are what evaluation scorers read.

Trace Model¶

flowchart TD
  T["Trace tr-46d8...<br/>state OK, 162 ms<br/>tags, metadata, assessments"] --> A["answer_question<br/>AGENT"]
  A --> R["retrieve<br/>RETRIEVER"]
  A --> P["build_prompt<br/>UNKNOWN"]
  A --> L["Completions<br/>CHAT_MODEL<br/>tokens in/out"]

Span types (mlflow.entities.SpanType): LLM, CHAT_MODEL, AGENT, CHAIN, TOOL, RETRIEVER, EMBEDDING, RERANKER, PARSER, MEMORY, GUARDRAIL, EVALUATOR, WORKFLOW, TASK, UNKNOWN. Built-in scorers depend on them — retrieval judges need RETRIEVER spans, tool-call judges need TOOL spans.

Autologging¶

One call per framework instruments every call of that library:

import mlflow

mlflow.set_tracking_uri("http://localhost:5000")
mlflow.set_experiment("support-bot")
mlflow.openai.autolog()          # call before the client is used

Enable only the flavors the app uses. mlflow.autolog() without a flavor turns on every installed integration — more spans than you need when one library calls another under the hood.

Tracing Your Own Code¶

import mlflow
from mlflow.entities import SpanType
from openai import OpenAI

mlflow.openai.autolog()
client = OpenAI()


@mlflow.trace(span_type=SpanType.RETRIEVER)
def retrieve(question: str) -> list[str]:
    return ["France: capital Paris"]                       # inputs and return value are captured


@mlflow.trace(name="answer_question", span_type=SpanType.AGENT)
def answer(question: str, session_id: str) -> str:
    mlflow.update_current_trace(
        tags={"test.run.id": "ci-1234"},
        metadata={"mlflow.trace.session": session_id, "mlflow.trace.user": "qa-bot"},
    )
    docs = retrieve(question)

    with mlflow.start_span(name="build_prompt") as span:  # a block, not a function
        span.set_inputs({"docs": docs, "question": question})
        prompt = f"Context: {docs}\nQuestion: {question}"
        span.set_outputs({"prompt": prompt})
        span.set_attribute("prompt.chars", len(prompt))

    reply = client.chat.completions.create(               # CHAT_MODEL span from autolog
        model="gpt-4.1-mini",
        messages=[{"role": "user", "content": prompt}],
    )
    return reply.choices[0].message.content

• @mlflow.trace captures arguments and the return value; mlflow.start_span() captures nothing unless you call set_inputs() / set_outputs().
• An exception inside a span marks the span and the trace as ERROR and records the exception as a span event.
• mlflow.trace.session and mlflow.trace.user are the standard metadata keys for multi-turn chats; the UI groups traces into sessions by them.
• Do not decorate functions that autolog already traces — you get two spans for one call.

Where Traces Go¶

Linking traces to an app version lets you compare quality and latency between releases:

active = mlflow.set_active_model(name=f"support-bot-{git_sha}")   # created on first use
# ... serve requests / run tests ...
traces = mlflow.search_traces(model_id=active.model_id)

Reading Traces Back¶

import mlflow
from mlflow.entities import SpanType

answer("What is the capital of France?", session_id="s-1")

mlflow.flush_trace_async_logging()                 # traces are exported in the background
trace = mlflow.get_trace(mlflow.get_last_active_trace_id())

print(trace.info.state, trace.info.execution_duration, trace.info.token_usage)
# OK 162 {'input_tokens': 10, 'output_tokens': 7, 'total_tokens': 17}
for span in trace.data.spans:
    print(span.name, span.span_type, span.inputs, span.outputs)
llm_spans = trace.search_spans(span_type=SpanType.CHAT_MODEL)

df = mlflow.search_traces(                          # pandas.DataFrame
    filter_string="tags.`test.run.id` = 'ci-1234'",
    max_results=100,
)
failed = mlflow.search_traces(filter_string="trace.status = 'ERROR'", return_type="list")

CLI equivalent: mlflow traces search --experiment-id 1 --max-results 10, mlflow traces get --trace-id tr-....

Feedback and Expectations¶

Human ratings and ground truth are attached to traces as assessments — the same place LLM judges write to, so they can be compared later:

import mlflow
from mlflow.entities import AssessmentSource, AssessmentSourceType

trace_id = mlflow.get_last_active_trace_id()       # or the id returned to the client with the answer
mlflow.log_feedback(
    trace_id=trace_id,
    name="user_thumbs_up",
    value=False,
    rationale="Answered with the wrong city",
    source=AssessmentSource(source_type=AssessmentSourceType.HUMAN, source_id="qa@example.com"),
)
mlflow.log_expectation(trace_id=trace_id, name="expected_response", value="Paris")

Traces with negative feedback plus an expectation are ready-made regression cases for an evaluation dataset.

OpenTelemetry Compatibility¶

Sending OTel spans to MLflow¶

The server accepts OTLP over HTTP on /v1/traces. The target experiment comes from the x-mlflow-experiment-id header:

# uv add opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

provider = TracerProvider(resource=Resource.create({"service.name": "rag-service"}))
provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter(
    endpoint="http://localhost:5000/v1/traces",
    headers={"x-mlflow-experiment-id": "3"},
)))

Or with env vars only: OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:5000/v1/traces and OTEL_EXPORTER_OTLP_TRACES_HEADERS=x-mlflow-experiment-id=3. This works for services in any language and for OTel Collectors (otlphttp exporter).

Sending MLflow traces to an OTel backend¶

• The default protocol is grpc (needs opentelemetry-exporter-otlp-proto-grpc and port 4317); for http/protobuf install opentelemetry-exporter-otlp-proto-http and use 4318 with the /v1/traces path.
• OTEL_SERVICE_NAME sets the service name shown in Jaeger.
• To pass the trace context to a downstream service in the same trace, see the Collector and propagation patterns in OpenTelemetry — Collector & Backends.

Production Settings¶

Tracing Checklist¶

• One autolog() per framework in use, called before the first client call
• Custom steps (retrieval, tools, routing) traced with the right span_type
• Session and user metadata set for chat flows
• Test traffic tagged (test.run.id, git_sha) or sent to a separate experiment
• flush_trace_async_logging() before reading traces in tests and at the end of scripts
• Sampling ratio and PII masking decided before production traffic
• OTLP export configured on purpose — setting OTEL_EXPORTER_OTLP_TRACES_ENDPOINT alone moves traces away from MLflow

See also¶

• MLflow — Experiment Tracking, LLM Tracing & Evaluation
• MLflow — Evaluation & Prompts
• OpenTelemetry — Tracing
• OpenTelemetry — Collector & Backends
• Jaeger — Distributed Tracing for OpenTelemetry
• Phoenix — Tracing & Instrumentation
• Langfuse — Tracing with the Python SDK
• LangGraph — Observability & Deployment