Lumina Eval

---
name: lumina-eval
description: Evaluates deep learning models on multi-vendor full-field digital mammography for breast cancer diagnosis, BI-RADS classification, and breast density prediction. It specifically probes the model's robustness to domain shifts induced by different imaging vendors and X-ray energies. Use when the user wants to benchmark on LUMINA, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.14644
  bibtex_key: pan2026lumina
  confidence: high
---

# lumina-eval

> LUMINA: A Multi-Vendor Mammography Benchmark with Energy Harmonization Protocol — Pan et al. (2026) (arXiv:2603.14644, 2026)

## What this evaluates

Evaluates deep learning models on multi-vendor full-field digital mammography for breast cancer diagnosis, BI-RADS classification, and breast density prediction. It specifically probes the model's robustness to domain shifts induced by different imaging vendors and X-ray energies.

## Datasets

- **LUMINA** — total ?; splits: 5-fold cross-validation (-1); repo https://github.com/NubagciLab/LUMINA

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve. For multi-class tasks (BI-RADS, density), macro-AUC is reported.
- `ACC` — range: [0, 1]
  - Accuracy, calculated as the proportion of correctly classified instances out of the total.
- `F1-score` — range: [0, 1]
  - Harmonic mean of precision and recall. Macro-F1 is used for multi-class tasks.
- `sensitivity` — range: [0, 1]
  - Recall, calculated as true positives divided by the sum of true positives and false negatives.
- `precision` — range: [0, 1]
  - Proportion of true positive predictions among all positive predictions.
- `specificity` — range: [0, 1]
  - Proportion of true negative predictions among all actual negatives.

## Input / output format

**Input**: Mammogram images (single CC or MLO view, or paired CC+MLO views), resized to 224x224 or 512x512 pixels, grayscale replicated to 3 channels. Optionally preprocessed with foreground-only pixel-space CDF alignment (histogram matching).

**Output**: Class probabilities for binary (benign/malignant) or multi-class (BI-RADS, density) tasks, from which hard predictions are derived using a threshold (typically 0.5).

## Scoring recipe

```python
def score(y_true, y_pred, y_prob, multi_class=False):
    acc = sum(y_true == y_pred) / len(y_true)
    auc = roc_auc_score(y_true, y_prob, multi_class='ovr' if multi_class else 'binary')
    f1 = f1_score(y_true, y_pred, average='macro' if multi_class else 'binary')
    recall = sum((y_true == 1) & (y_pred == 1)) / max(sum(y_true == 1), 1)
    precision = sum((y_true == 1) & (y_pred == 1)) / max(sum(y_pred == 1), 1)
    specificity = sum((y_true == 0) & (y_pred == 0)) / max(sum(y_true == 0), 1)
    return {'ACC': acc, 'AUC': auc, 'F1': f1, 'Sensitivity': recall, 'Precision': precision, 'Specificity': specificity}
```

## Common pitfalls

- AUC is explicitly stated as the primary clinical metric, not accuracy or F1.
- Two-view models (CC+MLO) consistently outperform single-view models, so input configuration must be reported.
- Results vary significantly between raw and histogram-harmonized inputs; the preprocessing step must be specified.

## Evidence (verbatim from paper)

> We reported accuracy (ACC), area under the ROC curve (AUC), F1-score, sensitivity (recall), precision, and specificity for breast cancer diagnosis, and ACC, AUC, and F1 for the three-class BI-RADS and density classification (macro-AUC and macro-F1 for multi-class). AUC is considered the primary metric from a clinical perspective. Results were reported as mean±std over 5 folds.

## Citation

```bibtex
@misc{pan2026lumina,
  title={LUMINA: A Multi-Vendor Mammography Benchmark with Energy Harmonization Protocol},
  author={Pan et al. (2026)},
  year={2026},
  note={arXiv:2603.14644}
}
```

- arXiv: 2603.14644