Histopath Domain Gen Eval

---
name: histopath-domain-gen-eval
description: Evaluates a model's ability to generalize to out-of-distribution domains (different hospitals or staining protocols) in histopathology image classification. It measures classification accuracy on held-out OOD validation and test splits, alongside the reconstruction quality of self-supervised generative augmentation. Use when the user wants to benchmark on CAMELYON17-WILDS, Epithelium-Stroma, or asks about evaluating this task. Reports Accuracy (%).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2407.02900
  bibtex_key: doerrich2024selfsupervised
  confidence: high
---

# histopath-domain-gen-eval

> Self-supervised Vision Transformer are Scalable Generative Models for Domain Generalization — Doerrich et al. (2024) (arXiv:2407.02900, 2024)

## What this evaluates

Evaluates a model's ability to generalize to out-of-distribution domains (different hospitals or staining protocols) in histopathology image classification. It measures classification accuracy on held-out OOD validation and test splits, alongside the reconstruction quality of self-supervised generative augmentation.

## Datasets

- **CAMELYON17-WILDS** — total 422394; splits: train (302436), val (34904), test (85054)
- **Epithelium-Stroma** — total 15633; splits: train (-1), val (-1), test (1376)

## Metrics

- `Accuracy (%)` **(primary)** — range: percent
  - Percentage of correctly classified samples out of the total samples in the validation or test set.
- `PSNR (dB)` — range: other
  - Peak Signal-to-Noise Ratio measured in decibels to quantify the reconstruction quality of the self-supervised image synthesizer.

## Input / output format

**Input**: 224x224 image patches standardized via bicubic interpolation, with binary class labels (tumor presence or epithelium/stroma).

**Output**: Binary class prediction for classification; reconstructed or synthetically mixed images for qualitative assessment.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return (correct / len(gold_labels)) * 100

def compute_psnr(original, reconstructed):
    mse = np.mean((original - reconstructed) ** 2)
    if mse == 0: return 100
    return 10 * np.log10(255**2 / mse)
```

## Common pitfalls

- Using in-distribution data for final evaluation instead of the specified OOD validation/test splits.
- Confusing the generative augmentation phase (training-time mixing of anatomy and characteristics) with the direct classification evaluation phase.
- Failing to standardize all input patches to 224x224 before feeding them to the ViT encoder, which breaks the model's expected input dimensions.

## Evidence (verbatim from paper)

> We evaluate our method on the class-balanced CAMELYON17-WILDS validation and test sets against the top-performing methods from the WILDS leaderboard, which utilize the same classifier. The results shown in Table 1 reveal our method's superior accuracy on both sets, setting a new state-of-the-art standard.

## Citation

```bibtex
@misc{doerrich2024selfsupervised,
  title={Self-supervised Vision Transformer are Scalable Generative Models for Domain Generalization},
  author={Doerrich et al. (2024)},
  year={2024},
  note={arXiv:2407.02900}
}
```

- arXiv: 2407.02900