Midam Mil Auc Eval

---
name: midam-mil-auc-eval
description: Evaluates multi-instance learning models for AUC maximization across tabular, histopathological, and medical image datasets. It probes the model's ability to handle large bags of instances via stochastic pooling while optimizing a min-max margin AUC loss. Use when the user wants to benchmark on MUSK1, MUSK2, Elephant, Fox, Tiger, Breast Cancer, Colon Ade., PDGM, OCT, or asks about evaluating this task. Reports testing AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2305.08040
  bibtex_key: zhu2023midam
  confidence: high
---

# midam-mil-auc-eval

> Provable Multi-instance Deep AUC Maximization with Stochastic Pooling — Zhu et al. (2023) (arXiv:2305.08040, 2023)

## What this evaluates

Evaluates multi-instance learning models for AUC maximization across tabular, histopathological, and medical image datasets. It probes the model's ability to handle large bags of instances via stochastic pooling while optimizing a min-max margin AUC loss.

## Datasets

- **MUSK1** — total 92; splits: train (-1), test (-1)
- **MUSK2** — total 102; splits: train (-1), test (-1)
- **Elephant** — total 200; splits: train (-1), test (-1)
- **Fox** — total 200; splits: train (-1), test (-1)
- **Tiger** — total 200; splits: train (-1), test (-1)
- **Breast Cancer** — total 58; splits: train (-1), test (-1)
- **Colon Ade.** — total 1100; splits: train (-1), test (-1)
- **PDGM** — total 458; splits: train (-1), test (-1)
- **OCT** — total 2682; splits: train (-1), test (-1)

## Metrics

- `testing AUC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve. Computed as the fraction of positive-negative score pairs where the positive score is greater than the negative score.

## Input / output format

**Input**: Bags of feature vectors or image patches, each bag associated with a single binary label (positive/negative).

**Output**: A single scalar prediction score per bag, passed through a sigmoid function.

## Scoring recipe

```python
def compute_auc(predictions, labels):
    pos_scores = [p for p, l in zip(predictions, labels) if l == 1]
    neg_scores = [p for p, l in zip(predictions, labels) if l == 0]
    auc = 0.0
    for ps in pos_scores:
        for ns in neg_scores:
            if ps > ns: auc += 1.0
            elif ps == ns: auc += 0.5
    return auc / (len(pos_scores) * len(neg_scores))
```

## Common pitfalls

- Model selection uses the checkpoint with the largest validation AUC, not the final epoch.
- Deterministic pooling on high-resolution medical images causes GPU OOM; stochastic sampling is mandatory for baselines.
- Bag-level labels are used for evaluation, not instance-level ground truth.

## Evidence (verbatim from paper)

> We report the testing AUC based on a model with the largest validation AUC value. We uniformly randomly split the data with 0.9/0.1 train/test ratio and run 5-fold-cross-validation experiments with 3 different random seeds (totally 15 different trials).

## Citation

```bibtex
@misc{zhu2023midam,
  title={Provable Multi-instance Deep AUC Maximization with Stochastic Pooling},
  author={Zhu et al. (2023)},
  year={2023},
  note={arXiv:2305.08040}
}
```

- arXiv: 2305.08040