Skip to main content
The Experiment Manager is where you compare agent configurations, review traces, and deploy the best version to production. Each time you run an evaluation script as a Domino Job, it creates an agent version containing traces and evaluation scores for that configuration.

​
How agent versions are created

Agent versions are created when you run your evaluation script as a Domino Job, not from a workspace. Each run captures:
  • The code commit used for the Job.
  • The YAML agent configuration (model, prompts, parameters) logged via DominoAgentContext(agent_config_path=…​).
  • All traces generated during execution, with inline evaluation scores.
  • Aggregated metrics across all traces in the run.
Each agent version is a complete, versioned snapshot of an agent configuration and its performance. Develop agentic systems has details on how to set up the instrumentation and evaluation code.

​
View traces in an agent version

View and analyze traces from any agent version:
  1. In your project, click Experiments in the left navigation.
  2. Find and click the experiment you want to review.
  3. Open an agent version and select the Traces tab.
View traces and add labels When reviewing traces you can:
  • See evaluation scores logged by your inline evaluators.
  • Add metrics (float values) or labels (string values) by clicking Add Metric/Label for manual annotation.
  • Click any metric or label cell to open the detailed trace view with the full span tree.

​
Compare agent versions

Compare different agent configurations (runs) to see how changes to prompts, models, or parameters affect quality:
  1. From the experiment view, select two to four runs and click Compare.
  2. The comparison view shows aggregated metrics side by side for each run.
  3. Open the Traces comparison view to compare individual traces across runs. This shows you exactly where one configuration outperforms another on the same inputs.
Compare runs

​
Deploy from an agent version

Once you’ve identified a winning configuration, you can deploy it to production directly from the Experiment Manager. Only agent versions that originated from a Job are eligible for deployment. Domino uses the Job’s code commit and agent config to maintain full lineage from experiment to production. Deploy agentic systems has the step-by-step walkthrough.

​
Manual annotations

Subject matter experts can manually annotate traces beyond automated evaluation scores:
  • Metrics: Add numeric scores (float values) to any trace by clicking Add Metric/Label.
  • Labels: Add string labels such as “approved”, “needs review”, or “hallucination” to categorize traces.
Manual annotations appear alongside automated evaluation scores in the comparison views.

​
Programmatic trace analysis

You can pull trace data programmatically using the MLflow APIs for custom aggregation and analysis. This is useful when you want to do your own analytical work with the trace data beyond the built-in dashboards. Examples include breaking down token usage, latency, and tool calls by span type, or building custom comparison visualizations in a notebook. The rag-agent-demo repository has a notebook example that demonstrates pulling span-level metrics and visualizing them.

​
Export results

Export data in two formats:
  • CSV: Open the experiment and click the three dots in the upper right.
  • PDF report: Select two to four runs to compare and click the three dots in the upper right.

​
Next steps