Tcga Histopathology Eval

---
name: tcga-histopathology-eval
description: Evaluates the representation quality and generalization of self-supervised histopathology models across diverse patch-level diagnostic tasks and weakly supervised slide-level tasks using linear probing and fine-tuning on TCGA whole slide images. Use when the user wants to benchmark on TCGA Histopathology, or asks about evaluating this task. Reports average AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2310.13259
  bibtex_key: lai2023domain
  confidence: high
---

# tcga-histopathology-eval

> Domain-specific optimization and diverse evaluation of self-supervised models for histopathology — Lai et al. (2023) (arXiv:2310.13259, 2023)

## What this evaluates

Evaluates the representation quality and generalization of self-supervised histopathology models across diverse patch-level diagnostic tasks and weakly supervised slide-level tasks using linear probing and fine-tuning on TCGA whole slide images.

## Datasets

- **TCGA Histopathology** — total ?; splits: test (-1), tune (-1)

## Metrics

- `average AUC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve, averaged across all linear probe tasks. Reported with 95% confidence intervals.

## Input / output format

**Input**: Image patches extracted from whole slide images (WSIs) at varying magnifications. For weakly supervised slide-level tasks, 1000 patches per slide are averaged to form a single slide embedding.

**Output**: Linear probe predictions (probabilities or logits) for each diagnostic task (e.g., cancer subtype, biomarker status, mitotic figure presence).

## Scoring recipe

```python
def compute_average_auc(predictions, labels):
    aucs = []
    for pred, lab in zip(predictions, labels):
        fpr, tpr, _ = roc_curve(lab, pred)
        aucs.append(auc(fpr, tpr))
    return np.mean(aucs)
# predictions: list of arrays, each shape (n_samples,)
# labels: list of arrays, each shape (n_samples,)
```

## Common pitfalls

- Center embeddings significantly outperform class token embeddings for mitotic figure detection, but this is task-specific and not always reflected in overall linear probe averages.
- Ablation variations that improve performance on the tune set may not generalize to held-out test tasks, requiring careful validation on independent diagnostic tasks.

## Evidence (verbatim from paper)

> Baseline models as well as SimCLR-best and MSN-best (combining individual optimizations) were evaluated on the test sets for all linear probe tasks. See Table 2 for linear probe metric performance for all models. For the optimized models, average AUC was 93.20% [95% CI 92.71% - 93.72%] for SimCLR-best and 93.43% [95% CI 92.88% - 93.84%] for MSN-best.

## Citation

```bibtex
@misc{lai2023domain,
  title={Domain-specific optimization and diverse evaluation of self-supervised models for histopathology},
  author={Lai et al. (2023)},
  year={2023},
  note={arXiv:2310.13259}
}
```

- arXiv: 2310.13259