Mammography Mass Detection Eval

---
name: mammography-mass-detection-eval
description: Evaluates the ability of deep learning models to detect breast masses in digital mammography images across multiple clinical domains with varying scanner manufacturers and imaging protocols. It specifically probes domain generalization capabilities by measuring detection robustness on unseen data distributions. Use when the user wants to benchmark on OPTIMAM Hologic, OPTIMAM Siemens, OPTIMAM GE, OPTIMAM Philips, INbreast, BCDR, or asks about evaluating this task. Reports TPR at 0.75 FPPI.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2201.11620
  bibtex_key: garrucho2022mammographydg
  confidence: high
---

# mammography-mass-detection-eval

> Domain generalization in deep learning-based mass detection in mammography: A large-scale multi-center study — Garrucho et al. (2022) (arXiv:2201.11620, 2022)

## What this evaluates

Evaluates the ability of deep learning models to detect breast masses in digital mammography images across multiple clinical domains with varying scanner manufacturers and imaging protocols. It specifically probes domain generalization capabilities by measuring detection robustness on unseen data distributions.

## Datasets

- **OPTIMAM Hologic** — total ?; splits: train (-1)
- **OPTIMAM Siemens** — total ?; splits: test (-1)
- **OPTIMAM GE** — total ?; splits: test (-1)
- **OPTIMAM Philips** — total ?; splits: test (-1)
- **INbreast** — total ?; splits: test (-1)
- **BCDR** — total ?; splits: test (-1)

## Metrics

- `TPR at 0.75 FPPI` **(primary)** — range: [0, 1]
  - True Positive Rate calculated at a fixed threshold of 0.75 False Positives Per Image. TPR = TP / (TP + FN).
- `AUC (FROC)` — range: [0, 1]
  - Area Under the Free-response Receiver Operating Characteristic curve, plotting TPR against FPPI across varying detection thresholds.

## Input / output format

**Input**: Digital mammography images (2D X-ray scans).

**Output**: Bounding boxes (coordinates) indicating the location and extent of detected breast masses.

## Scoring recipe

```python
def evaluate(predictions, gold_boxes, n_images):
    tp = count_true_positives(predictions, gold_boxes)
    fp = count_false_positives(predictions, gold_boxes)
    tpr = tp / len(gold_boxes)
    fppi = fp / n_images
    auc = compute_auc(tpr_values, fppi_values)
    return tpr, fppi, auc
```

## Common pitfalls

- FPPI (false positives per image) normalizes by the number of images, not patches or regions, which differs from standard FPI metrics.
- Class imbalance heavily skews overall TPR; benign masses are rare in source domains (e.g., 9% in OPTIMAM Hologic) but dominate in others (e.g., 55% in BCDR), requiring per-class analysis.
- Domain shift is primarily driven by scanner manufacturer and acquisition protocol rather than pure covariate shift, making cross-domain generalization the core challenge.

## Evidence (verbatim from paper)

> Table 4 shows, for each domain, the performance of the methods in terms of TPR at 0.75 FPPI, 95% confidence intervals, and AUC. Fig. 4 contains the FROC curves for the six different domains.

## Citation

```bibtex
@misc{garrucho2022mammographydg,
  title={Domain generalization in deep learning-based mass detection in mammography: A large-scale multi-center study},
  author={Garrucho et al. (2022)},
  year={2022},
  note={arXiv:2201.11620}
}
```

- arXiv: 2201.11620