Experiment Tracking

---
name: domino-experiment-tracking
description: Track traditional ML experiments in Domino using the MLflow-based Experiment Manager. Covers experiment setup, auto-logging for sklearn/TensorFlow/PyTorch, manual logging, artifact storage, run comparison, and model registration. Use when training ML models, logging metrics and parameters, comparing model runs, or registering models.
---

# Domino Experiment Tracking Skill

This skill provides comprehensive knowledge for tracking ML experiments in Domino Data Lab using the built-in MLflow-based Experiment Manager.

## Key Concepts

### Experiment Manager Overview

Domino's Experiment Manager is built on MLflow and provides:
- Automatic and manual logging of parameters, metrics, and artifacts
- Run comparison and visualization
- Model versioning and registry
- Integration with Domino projects and jobs

### Critical Configuration

**Experiment names must be unique across the entire Domino deployment.** Always append username or project name to ensure uniqueness.

## Related Documentation

- [MLFLOW-BASICS.md](./MLFLOW-BASICS.md) - Auto-logging, manual logging
- [COMPARING-RUNS.md](./COMPARING-RUNS.md) - Run comparison, export
- [MODEL-REGISTRY.md](./MODEL-REGISTRY.md) - Model registration & stages

## Quick Start

```python
import mlflow
import os

# CRITICAL: Experiment names must be unique across Domino deployment
username = os.environ.get('DOMINO_STARTING_USERNAME', 'unknown')
experiment_name = f"my-experiment-{username}"

# Set the experiment
mlflow.set_experiment(experiment_name)

# Enable auto-logging (easiest approach)
mlflow.autolog()

# Run training
with mlflow.start_run(run_name="my-first-run"):
    model.fit(X_train, y_train)

    # Optional: manually log additional items
    mlflow.log_param("custom_param", "value")
    mlflow.log_metric("custom_metric", 0.95)
```

## Supported Frameworks

| Framework | Auto-log Command |
|-----------|------------------|
| Scikit-learn | `mlflow.sklearn.autolog()` |
| TensorFlow/Keras | `mlflow.tensorflow.autolog()` |
| PyTorch | `mlflow.pytorch.autolog()` |
| XGBoost | `mlflow.xgboost.autolog()` |
| LightGBM | `mlflow.lightgbm.autolog()` |
| All at once | `mlflow.autolog()` |

## Environment Variables

Domino automatically configures MLflow to use the built-in tracking server. These variables are pre-set:

| Variable | Description |
|----------|-------------|
| `MLFLOW_TRACKING_URI` | Domino's MLflow server URL |
| `DOMINO_STARTING_USERNAME` | User running the experiment |
| `DOMINO_PROJECT_NAME` | Current project name |
| `DOMINO_RUN_ID` | Domino job run ID |

## Documentation Links

- Domino Experiment Tracking: https://docs.dominodatalab.com/en/latest/user_guide/da707d/track-and-monitor-experiments/
- Domino Model Registry: https://docs.dominodatalab.com/en/latest/user_guide/3b6ae5/manage-models-with-model-registry/