Midog Eval

---
name: midog-eval
description: Evaluates deep learning models for detecting mitotic figures in histopathology whole-slide images, specifically probing their ability to generalize across different scanner-induced domain shifts such as color distribution, contrast, and depth-of-field variations. Use when the user wants to benchmark on MIDOG, or asks about evaluating this task. Reports F_1 score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2204.03742
  bibtex_key: aubreville2022midog
  confidence: high
---

# midog-eval

> Mitosis domain generalization in histopathology images -- The MIDOG challenge — Aubreville et al. (2022) (arXiv:2204.03742, 2022)

## What this evaluates

Evaluates deep learning models for detecting mitotic figures in histopathology whole-slide images, specifically probing their ability to generalize across different scanner-induced domain shifts such as color distribution, contrast, and depth-of-field variations.

## Datasets

- **MIDOG** — total ?; splits: train (-1), test (-1); repo https://github.com/DeepPathology/MIDOG_evaluation_docker

## Metrics

- `F_1 score` **(primary)** — range: [0, 1]
  - Harmonic mean of precision and recall: F_1 = 2 * (precision * recall) / (precision + recall). Precision is the ratio of true positive detections to all positive predictions, and recall is the ratio of true positives to all ground truth mitotic figures.

## Input / output format

**Input**: Whole-slide histopathology images (WSIs) from various scanners.

**Output**: Predicted bounding boxes or coordinates for mitotic figures.

## Scoring recipe

```python
tp = count of predicted boxes overlapping ground truth mitoses (IoU > threshold)
fp = count of predicted boxes with no matching ground truth
fn = count of ground truth mitoses with no matching prediction
precision = tp / (tp + fp) if (tp + fp) > 0 else 0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
return f1
```

## Common pitfalls

- Scanner-specific optical variations (color distribution, contrast, depth-of-field) cause severe domain shift, making models trained on one scanner fail on others.
- Borderline or atypical mitotic figures (e.g., late telophase, unclear boundaries) are often inconsistently labeled by experts, leading to ambiguous ground truth and high false-negative rates.
- Ensembling or test-time augmentation significantly boosts performance but may not reflect single-model generalization capability.

## Evidence (verbatim from paper)

> The domain-adversarial reference method yielded a competitive $F_{1}$ score of 0.718 on the test set and was outperformed by only four approaches (see Table 2). With an $F_{1}$ score of 0.748, the overall best performance was reached by Yang et al., utilizing segmentation and Fourier-domain mixing as augmentation.

## Citation

```bibtex
@misc{aubreville2022midog,
  title={Mitosis domain generalization in histopathology images -- The MIDOG challenge},
  author={Aubreville et al. (2022)},
  year={2022},
  note={arXiv:2204.03742}
}
```

- arXiv: 2204.03742