Dosrecmc Mammography Eval

---
name: dosrecmc-mammography-eval
description: Evaluates cross-domain generalization of mammography classification models under domain shift, specifically testing resilience to variations in pixel intensity distributions across different imaging devices and datasets. Use when the user wants to benchmark on NYU, HCTP, VinDr, CSAW, or asks about evaluating this task. Reports PR-AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.15452
  bibtex_key: akyuz2025dosrecmc
  confidence: high
---

# dosrecmc-mammography-eval

> DoSReMC: Domain Shift Resilient Mammography Classification using Batch Normalization Adaptation — Akyüz et al. (2025) (arXiv:2508.15452, 2025)

## What this evaluates

Evaluates cross-domain generalization of mammography classification models under domain shift, specifically testing resilience to variations in pixel intensity distributions across different imaging devices and datasets.

## Datasets

- **NYU** — total ?; splits: train (-1)
- **HCTP** — total ?; splits: test (-1)
- **VinDr** — total ?; splits: test (-1)
- **CSAW** — total ?; splits: test (-1)

## Metrics

- `PR-AUC` **(primary)** — range: [0, 1]
  - Area under the Precision-Recall curve computed specifically on the malignant class predictions. It emphasizes the model's ability to minimize false positives and false negatives for malignant cases.
- `ROC-AUC` — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve computed on the malignant class predictions. It measures the trade-off between the true positive rate and false positive rate across classification thresholds.

## Input / output format

**Input**: Mammography images (CC and MLO views) processed into 256×256 patches. Training batches are composed of benign, malignant, and negative samples in a 1:1:2 ratio.

**Output**: Classification probabilities for benign, malignant, and negative classes. Final prediction values are obtained by averaging the outputs for CC and MLO images of the same breast.

## Scoring recipe

```python
def compute_pr_auc(predictions_cc, predictions_mlo, labels):
    # Average predictions for paired CC and MLO views
    avg_pred = [(c + m) / 2 for c, m in zip(predictions_cc, predictions_mlo)]
    # Extract malignant class scores and true labels
    y_true = [1 if label == 'malignant' else 0 for label in labels]
    y_score = [p[1] for p in avg_pred] # assuming index 1 is malignant
    # Compute PR-AUC
    precision, recall, _ = precision_recall_curve(y_true, y_score)
    return auc(recall, precision)
```

## Common pitfalls

- Failing to average the CC and MLO view predictions for the same breast before computing metrics.
- Evaluating on all classes equally instead of focusing specifically on the malignant class for PR-AUC.
- Using full model fine-tuning instead of the specified BN and FC layer adaptation protocol.

## Evidence (verbatim from paper)

> In order to evaluate the model’s performance, we use the Receiver Operator Characteristic - Area Under Curve (ROC-AUC) and PR-AUC. While evaluating the model, we average the predictions corresponding to CC and MLO images of a breast in the same study, and obtain final prediction values. Our primary goal is to achieve low false-positive and false-negative rates for malignant cases, and we focus on the model’s outputs for malignant cases during the analysis phase.

## Citation

```bibtex
@misc{akyuz2025dosrecmc,
  title={DoSReMC: Domain Shift Resilient Mammography Classification using Batch Normalization Adaptation},
  author={Akyüz et al. (2025)},
  year={2025},
  note={arXiv:2508.15452}
}
```

- arXiv: 2508.15452