Breast Cancer Classification Eval

---
name: breast-cancer-classification-eval
description: Evaluates a vision model's ability to classify mammograms as benign or malignant, focusing on fine-grained discrimination of localized malignancies and handling high-resolution medical images. Use when the user wants to benchmark on Public mammogram dataset(s), or asks about evaluating this task. Reports F1 score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.19350
  bibtex_key: sanghvi2026attend
  confidence: high
---

# breast-cancer-classification-eval

> Attend what matters: Leveraging vision foundational models for breast cancer classification using mammograms — Sanghvi et al. (2026) (arXiv:2604.19350, 2026)

## What this evaluates

Evaluates a vision model's ability to classify mammograms as benign or malignant, focusing on fine-grained discrimination of localized malignancies and handling high-resolution medical images.

## Datasets

- **Public mammogram dataset(s)** — total ?; splits: (unstated)

## Metrics

- `F1 score` **(primary)** — range: [0, 1]
  - Harmonic mean of precision and recall for the binary classification task. Computed without averaging across classes.
- `AUC` — range: [0, 1]
  - Area under the Receiver Operating Characteristic curve, measuring the model's ability to distinguish between classes across all classification thresholds.

## Input / output format

**Input**: High-resolution mammogram images (processed at 1024x1024 or via extracted Regions of Interest).

**Output**: Binary classification label (malignant vs. non-malignant).

## Scoring recipe

```python
def compute_metrics(preds, labels):
    tp = sum(p == 1 and l == 1 for p, l in zip(preds, labels))
    fp = sum(p == 1 and l == 0 for p, l in zip(preds, labels))
    fn = sum(p == 0 and l == 1 for p, l in zip(preds, labels))
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    auc = roc_auc_score(labels, preds)
    return {'F1': f1, 'AUC': auc}
```

## Common pitfalls

- Metrics are computed strictly under a binary classification setting without averaging across classes.
- Baseline comparisons may be biased if multimodal models are evaluated with empty text strings for non-malignant cases due to missing findings in the reference dataset.
- Reproducing prior SOTA is difficult as the previous best model (Mammo-CLIP) was trained on a private dataset.

## Evidence (verbatim from paper)

> All scores here are computed under the binary classification setting, without averaging across classes. ... our scores demonstrate a 4% improvement on the F1 score and a 1% gain on the AUC over this SOTA

## Citation

```bibtex
@misc{sanghvi2026attend,
  title={Attend what matters: Leveraging vision foundational models for breast cancer classification using mammograms},
  author={Sanghvi et al. (2026)},
  year={2026},
  note={arXiv:2604.19350}
}
```

- arXiv: 2604.19350