The domino.agents package provides tracing, evaluation, and search capabilities for agentic systems. This is part of the python-domino library.
The canonical import path is from domino.agents.tracing import …
Some documentation may reference domino.aisystems.tracing. This is an older alias. Both paths resolve to the same module.
add_tracing
Decorator that starts an MLflow span for the decorated function. If an existing trace is in progress, appends a span to it; otherwise creates a new trace.
domino.agents.tracing.add_tracing(
name: str,
autolog_frameworks: list[str] | None = [],
evaluator: Callable | None = None,
trace_evaluator: Callable | None = None,
eagerly_evaluate_streamed_results: bool = True,
allow_tracing_evaluator: bool = False
)
| Parameter | Type | Description | | | |
|---|
name | str | Name of the span (or new trace). | | | |
autolog_frameworks | `list[str] | None` | MLflow-supported frameworks to autolog (for instance, ["langchain"], ["openai"]). | | |
evaluator | `Callable | None` | Function that receives the span and returns evaluation results as `dict[str, int | float | str]`. Runs on every invocation. |
trace_evaluator | `Callable | None` | Function that receives the complete trace. Only fires when the trace was started and finished by this decorator. Useful for end-to-end quality checks. | | |
eagerly_evaluate_streamed_results | bool | When True (default), aggregates all yielded values into a single span for evaluation. When False, each yielded value creates a separate span with a shared group_id. | | | |
allow_tracing_evaluator | bool | Default False. When True, inline evaluators will also be traced by MLflow autolog. | | | |
init_tracing
Initialize MLflow autologging and set the active experiment to enable tracing. Used to initialize logging in both development and production modes.
domino.agents.tracing.init_tracing(
autolog_frameworks: list[str] | None = None
)
-
In production mode, the environment variables
DOMINO_AGENT_IS_PROD and DOMINO_APP_ID must be set.
-
Call
init_tracing() before your app starts serving requests.
search_traces
Search for traces in a development-mode evaluation run. Returns a paginated response of trace summaries.
domino.agents.tracing.search_traces(
run_id: str,
trace_name: str | None = None,
start_time: datetime | None = None,
end_time: datetime | None = None,
page_token: str | None = None,
max_results: int | None = None
) → SearchTracesResponse
search_agent_traces
Search for traces from a deployed production agent. Filter by agent version, trace name, and time range. If agent_version is not provided, searches across all versions.
domino.agents.tracing.search_agent_traces(
agent_id: str,
agent_version: str | None = None,
trace_name: str | None = None,
start_time: datetime | None = None,
end_time: datetime | None = None,
page_token: str | None = None,
max_results: int | None = None
) → SearchTracesResponse
Data classes
EvaluationResult
class domino.agents.tracing.EvaluationResult(
name: str,
value: float | str
)
| Parameter | Type | Description | |
|---|
name | str | Name of the evaluation. | |
value | `float | str` | Evaluation result value. |
TraceSummary
class domino.agents.tracing.TraceSummary(
name: str,
id: str,
spans: list[SpanSummary],
evaluation_results: list[EvaluationResult]
)
| Parameter | Type | Description |
|---|
name | str | Name of the trace. |
id | str | MLflow trace ID. |
spans | list[SpanSummary] | Child spans of this trace. |
evaluation_results | list[EvaluationResult] | Evaluation results for this trace. |
SpanSummary
class domino.agents.tracing.SpanSummary(
id: str,
name: str,
trace_id: str,
inputs: Any,
outputs: Any
)
| Parameter | Type | Description |
|---|
id | str | MLflow span ID. |
name | str | Span name. |
trace_id | str | Parent trace ID. |
inputs | Any | Inputs to the function that created the span. |
outputs | Any | Outputs of the function. |
SearchTracesResponse
class domino.agents.tracing.SearchTracesResponse(
data: list[TraceSummary],
page_token: str | None
)
| Parameter | Type | Description | |
|---|
data | list[TraceSummary] | List of trace summaries. | |
page_token | `str | None` | Token for the next page of results. |
log_evaluation
Log an evaluation result against an existing trace. Used for ad-hoc or production evaluations where you evaluate traces after they’ve been collected such as in a scheduled Job.
domino.agents.logging.log_evaluation(
trace_id: str,
name: str,
value: float | str
)
| Parameter | Type | Description | |
|---|
trace_id | str | The MLflow trace ID to attach the evaluation to. | |
name | str | Name of the evaluation metric. | |
value | `float | str` | Evaluation result value (numeric score or string label). |
DominoAgentContext
The DominoAgentContext wrapper from domino.agents.logging creates an MLflow agent version that stores traces and logged parameters.
from domino.agents.logging import DominoAgentContext
with DominoAgentContext(
agent_config_path="config.yaml",
aggregated_metrics=[("toxicity", "mean"), ("bleu", "median")]
) as run:
run_agent(...)
| Parameter | Description |
|---|
agent_config_path | Path to a YAML config file. Logged as parameters in the Experiment Manager. |
aggregated_metrics | List of (metric_name, aggregation) tuples. Aggregation types: mean, median, stdev, min, max. Defaults to mean for all metrics. |
Environment variables
| Variable | Required for | Description |
|---|
MLFLOW_TRACKING_URI | All modes | Set automatically in Domino executions. For local testing, use http://localhost:5000. |
DOMINO_AGENT_IS_PROD | Production | Set to indicate production mode for init_tracing(). |
DOMINO_APP_ID | Production | The ID of the deployed agent app. |
