Ssl Pathology Benchmark Eval

---
name: ssl-pathology-benchmark-eval
description: Evaluates the transfer learning capability of self-supervised learning (SSL) pre-trained models on diverse histopathology datasets. It probes domain-specific representation learning by measuring performance on image classification and nuclei instance segmentation tasks under linear probing and fine-tuning protocols. Use when the user wants to benchmark on BACH, CRC, PCam, MHIST, CoNSeP, or asks about evaluating this task. Reports top-1 accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2212.04690
  bibtex_key: kang2022benchmarking
  confidence: high
---

# ssl-pathology-benchmark-eval

> Benchmarking Self-Supervised Learning on Diverse Pathology Datasets — Kang et al. (2022) (arXiv:2212.04690, 2022)

## What this evaluates

Evaluates the transfer learning capability of self-supervised learning (SSL) pre-trained models on diverse histopathology datasets. It probes domain-specific representation learning by measuring performance on image classification and nuclei instance segmentation tasks under linear probing and fine-tuning protocols.

## Datasets

- **BACH** — total ?; splits: (unstated)
- **CRC** — total ?; splits: (unstated)
- **PCam** — total ?; splits: (unstated)
- **MHIST** — total ?; splits: (unstated)
- **CoNSeP** — total ?; splits: (unstated)

## Metrics

- `top-1 accuracy` **(primary)** — range: [0, 1]
  - Fraction of correctly classified instances out of the total number of instances in the test set.
- `panoptic quality (PQ)` — range: [0, 1]
  - PQ = (Σ IoU(p,g) for TP pairs) / (|TP| + 0.5|FP| + 0.5|FN|). Pairs with IoU > 0.5 are counted as True Positives. Multi-class PQ (mPQ) is used to measure performance simultaneously across classes.

## Input / output format

**Input**: Image patches extracted from whole slide images (WSIs) at various resolutions (e.g., 2048×1536, 224×224, 96×96, 1000×1000 pixels), labeled with cancer type or nuclei instance masks.

**Output**: For classification: predicted class label. For segmentation: predicted instance masks with class labels.

## Scoring recipe

```python
# Classification
accuracy = sum(pred == gold for pred, gold in zip(predictions, gold_labels)) / len(gold_labels)

# Segmentation (PQ)
tp_iou_sum = 0
tp_count = 0
fp_count = 0
for pred in predictions:
    matched = False
    for gold in gold_masks:
        iou = intersection(pred, gold) / union(pred, gold)
        if iou > 0.5:
            tp_iou_sum += iou
            tp_count += 1
            matched = True
            break
    if not matched:
        fp_count += 1
fn_count = len(gold_masks) - tp_count
pq = tp_iou_sum / (tp_count + 0.5 * fp_count + 0.5 * fn_count)
```

## Common pitfalls

- The paper uses 'ImageNet epoch' for training duration, which differs from standard epoch counting based on dataset size.
- Pre-training is primarily conducted on TCGA only, despite mentioning TULIP, which may limit claims about generalization across all sources.
- PQ calculation strictly counts pairs with IoU > 0.5 as TP, a convention that differs from some other instance segmentation metrics.

## Evidence (verbatim from paper)

> For image classification, we report top-1 accuracy, while using panoptic quality (PQ) [38] for nuclei instance segmentation. PQ is a standard metric for assessing the performance of nuclear instance segmentation [29] that accounts for both detection and segmentation quality with respect to each instance. The PQ metric is defined as, $$ PQ = \frac {\sum_ {(p , g) \in T P} \operatorname {I o U} (p , g)}{| T P | + \frac {1}{2} | F P | + \frac {1}{2} | F N |}, $$ where p denotes a predicted mask for each nuclei class and g denotes a corresponding ground truth.

## Citation

```bibtex
@misc{kang2022benchmarking,
  title={Benchmarking Self-Supervised Learning on Diverse Pathology Datasets},
  author={Kang et al. (2022)},
  year={2022},
  note={arXiv:2212.04690}
}
```

- arXiv: 2212.04690