Vikl Mammography Eval

---
name: vikl-mammography-eval
description: Evaluates a model's ability to extract robust visual features from single-channel mammography images for binary classification of malignant versus benign breast lumps. It tests cross-dataset generalization and the effectiveness of multimodal contrastive pretraining on pathological classification tasks. Use when the user wants to benchmark on MVKL, CBIS-DDSM, INbreast, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2409.15744
  bibtex_key: wei2024vikl
  confidence: high
---

# vikl-mammography-eval

> ViKL: A Mammography Interpretation Framework via Multimodal Aggregation of Visual-knowledge-linguistic Features — Wei et al. (2024) (arXiv:2409.15744, 2024)

## What this evaluates

Evaluates a model's ability to extract robust visual features from single-channel mammography images for binary classification of malignant versus benign breast lumps. It tests cross-dataset generalization and the effectiveness of multimodal contrastive pretraining on pathological classification tasks.

## Datasets

- **MVKL** — total 2764; splits: train (1946), val (276), test (542)
- **CBIS-DDSM** — total 3568; splits: train (2577), val (287), test (704)
- **INbreast** — total 410; splits: train (287), val (41), test (82)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area under the receiver operating characteristic curve. It represents the probability that a randomly chosen positive instance (malignant) will be ranked higher than a randomly chosen negative instance (benign) by the classifier.

## Input / output format

**Input**: Single-channel mammography images.

**Output**: Binary classification prediction (malignant or benign).

## Scoring recipe

```python
def compute_auc(y_true, y_scores):
    # y_true: binary ground truth (0=benign, 1=malignant)
    # y_scores: predicted probability of malignancy
    fpr, tpr, _ = roc_curve(y_true, y_scores)
    auc_value = auc(fpr, tpr)
    return auc_value
```

## Common pitfalls

- Ensure strict patient-level splitting: images from the same patient must not appear in both train and test sets to prevent data leakage.
- AUC is threshold-independent; do not report accuracy or F1 as the primary metric, as the paper explicitly avoids thresholding complexities.
- Cross-dataset evaluation faces significant domain shifts due to variations in collection years and imaging equipment, which can cause performance drops compared to in-dataset results.

## Evidence (verbatim from paper)

> To ensure consistent and clear comparisons between models, we exclusively use the area under the receiver operating characteristic (AUC). This metric provides a straightforward measure of performance without the complexities introduced by thresholding.

## Citation

```bibtex
@misc{wei2024vikl,
  title={ViKL: A Mammography Interpretation Framework via Multimodal Aggregation of Visual-knowledge-linguistic Features},
  author={Wei et al. (2024)},
  year={2024},
  note={arXiv:2409.15744}
}
```

- arXiv: 2409.15744