Ovarian Cancer Subtype Eval

---
name: ovarian-cancer-subtype-eval
description: Evaluates histopathology foundation models and ImageNet-pretrained encoders on classifying ovarian cancer subtypes from whole slide images. It probes the ability of vision models to extract diagnostically relevant features from medical histology slides for multi-class classification. Use when the user wants to benchmark on Ovarian Cancer WSI Dataset, or asks about evaluating this task. Reports balanced accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2405.09990
  bibtex_key: breen2024ovarian
  confidence: high
---

# ovarian-cancer-subtype-eval

> A Comprehensive Evaluation of Histopathology Foundation Models for Ovarian Cancer Subtype Classification — Breen et al. (2024) (arXiv:2405.09990, 2024)

## What this evaluates

Evaluates histopathology foundation models and ImageNet-pretrained encoders on classifying ovarian cancer subtypes from whole slide images. It probes the ability of vision models to extract diagnostically relevant features from medical histology slides for multi-class classification.

## Datasets

- **Ovarian Cancer WSI Dataset** — total 1864; splits: cross-validation (-1), hold-out (-1), external_transcanadian (-1), external_ocean (-1); repo https://github.com/scjjb/Ovarian_Features

## Metrics

- `balanced accuracy` **(primary)** — range: percent
  - The unweighted average of recall across all classes, calculated as the mean of true positive rates per class to handle class imbalance.
- `AUROC` — range: [0, 1]
  - Area under the receiver operating characteristic curve, measuring the model's ability to discriminate between classes across all classification thresholds.
- `F1 Score` — range: [0, 1]
  - The harmonic mean of precision and recall, calculated per class and then macro-averaged across all classes.

## Input / output format

**Input**: Whole slide histopathology images (WSIs) from ovarian cancer tissue samples.

**Output**: Predicted ovarian cancer subtype label (multi-class classification).

## Scoring recipe

```python
def compute_metrics(y_true, y_pred, y_prob, num_classes):
    from sklearn.metrics import balanced_accuracy_score, roc_auc_score, f1_score
    bal_acc = balanced_accuracy_score(y_true, y_pred)
    auroc = roc_auc_score(y_true, y_prob, multi_class='ovr')
    f1 = f1_score(y_true, y_pred, average='macro')
    return bal_acc, auroc, f1
```

## Common pitfalls

- Performance is averaged across four distinct validation settings (CV, hold-out, Transcanadian, OCEAN), which may obscure dataset-specific variance or overfitting to specific cohorts.
- RN18-Histo is the only foundation model that underperforms ImageNet-pretrained baselines, contrary to the general trend of histopathology models.
- 95% confidence intervals are derived from 10,000 bootstrap iterations rather than standard cross-validation variance, requiring careful interpretation of statistical significance.

## Evidence (verbatim from paper)

> The H-optimus-0 model achieved the greatest averaged performance across all validations (Table 3), with 83.0% average balanced accuracy, 0.965 average AUROC, and 0.822 average F1 score.

## Citation

```bibtex
@misc{breen2024ovarian,
  title={A Comprehensive Evaluation of Histopathology Foundation Models for Ovarian Cancer Subtype Classification},
  author={Breen et al. (2024)},
  year={2024},
  note={arXiv:2405.09990}
}
```

- arXiv: 2405.09990