Optimam Mammography Eval

---
name: optimam-mammography-eval
description: Binary classification of high-resolution mammography images to detect malignant breast tissue. It probes the model's ability to distinguish between malignant and non-malignant cases using both localized patches and full-resolution inputs. Use when the user wants to benchmark on OPTIMAM, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2208.06066
  bibtex_key: taha2022deepisluxury
  confidence: high
---

# optimam-mammography-eval

> Deep is a Luxury We Don't Have — Ahmed Taha et al. (arXiv:2208.06066, 2022)

## What this evaluates

Binary classification of high-resolution mammography images to detect malignant breast tissue. It probes the model's ability to distinguish between malignant and non-malignant cases using both localized patches and full-resolution inputs.

## Datasets

- **OPTIMAM** — total 148448; splits: train (58376), val (7266), test (7358)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area under the receiver operating characteristic curve. Computed as the integral of the true positive rate versus false positive rate across all classification thresholds. 95% confidence intervals are estimated via percentile bootstrapping with 10,000 iterations.

## Input / output format

**Input**: High-resolution mammography images (3328x2560) or cropped patches (512x512). Positive patches are centered on malignant bounding boxes; negative patches are randomly sampled from the breast area in non-malignant images.

**Output**: Binary classification label: malignant (positive) or non-malignant (negative).

## Scoring recipe

```python
def compute_auc(predictions, labels):
    fpr, tpr, _ = roc_curve(labels, predictions)
    auc_value = auc(fpr, tpr)
    boot_aucs = []
    for _ in range(10000):
        idx = np.random.choice(len(labels), size=len(labels), replace=True)
        boot_aucs.append(auc(fpr[idx], tpr[idx]))
    ci_low, ci_high = np.percentile(boot_aucs, [2.5, 97.5])
    return auc_value, (ci_low, ci_high)
```

## Common pitfalls

- Patient-level splitting is required to prevent data leakage, as multiple images can come from the same patient.
- Patch-based training uses positively centered patches but randomly sampled negative patches, which may create an artificial distribution shift compared to full-image inference.
- Performance varies significantly between patch-level and full-image evaluation modes; results should not be conflated.

## Evidence (verbatim from paper)

> Performance is reported using AUC and their 95% confidence intervals (CI) are in square brackets. We evaluate both the small patch and the full image models. The CI is constructed via percentile bootstrapping with 10,000 bootstrap iterations.

## Citation

```bibtex
@misc{taha2022deepisluxury,
  title={Deep is a Luxury We Don't Have},
  author={Ahmed Taha et al.},
  year={2022},
  note={arXiv:2208.06066}
}
```

- arXiv: 2208.06066