Domainnet Eval

---
name: domainnet-eval
description: Evaluates multi-source domain adaptation methods on image classification tasks across multiple domains with varying visual styles and categories. Use when the user wants to benchmark on DomainNet, or asks about evaluating this task. Reports average accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1812.01754
  bibtex_key: peng2018momentmatching
  confidence: high
---

# domainnet-eval

> Moment Matching for Multi-Source Domain Adaptation — Peng et al. (2018) (arXiv:1812.01754, 2018)

## What this evaluates

Evaluates multi-source domain adaptation methods on image classification tasks across multiple domains with varying visual styles and categories.

## Datasets

- **DomainNet** — total 600000; splits: train (-1), test (-1)

## Metrics

- `average accuracy` **(primary)** — range: percent
  - Percentage of correctly classified images in the target domain test set, averaged across all source-to-target domain transfer combinations.

## Input / output format

**Input**: RGB images from multiple source domains and a target domain, with category labels.

**Output**: Predicted class labels for each image in the target domain test set.

## Scoring recipe

```python
correct = 0
total = 0
for src, tgt in source_to_target_pairs:
    preds = model.predict(test_imgs[tgt])
    correct += sum(p == g for p, g in zip(preds, gold[tgt]))
    total += len(gold[tgt])
return (correct / total) * 100
```

## Common pitfalls

- Negative transfer can degrade performance on specific domain pairs (e.g., MNIST-M).
- The 70/30 train/test split is applied independently per domain, not globally across the dataset.
- Performance is averaged over all source-to-target transfer directions, masking domain-specific failures.

## Evidence (verbatim from paper)

> Our model M³SDA achieves an 86.13% average accuracy, and M³SDA-β boosts the performance to 87.65%, outperforming other baselines by a large margin.

## Citation

```bibtex
@misc{peng2018momentmatching,
  title={Moment Matching for Multi-Source Domain Adaptation},
  author={Peng et al. (2018)},
  year={2018},
  note={arXiv:1812.01754}
}
```

- arXiv: 1812.01754