Puma Challenge Eval

---
name: puma-challenge-eval
description: Evaluates pixel-level segmentation capability for distinguishing five histopathological tissue classes (tumour, stroma, necrosis, blood vessels, epidermis) in melanoma H&E images. Use when the user wants to benchmark on PUMA Challenge dataset, or asks about evaluating this task. Reports Dice score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2507.13974
  bibtex_key: lv2025leveraging
  confidence: high
---

# puma-challenge-eval

> Leveraging Pathology Foundation Models for Panoptic Segmentation of Melanoma in H&E Images — Jiaqi Lv et al. (2025) (arXiv:2507.13974, 2025)

## What this evaluates

Evaluates pixel-level segmentation capability for distinguishing five histopathological tissue classes (tumour, stroma, necrosis, blood vessels, epidermis) in melanoma H&E images.

## Datasets

- **PUMA Challenge dataset** — total ?; splits: train (-1), preliminary_test (10), final_test (-1)

## Metrics

- `Dice score` **(primary)** — range: [0, 1]
  - Computes the overlap between predicted and ground truth segmentation masks. Calculated as 2 * |A ∩ B| / (|A| + |B|), reported as micro-average across all classes and per-class.

## Input / output format

**Input**: RGB H&E histopathology image patches.

**Output**: 5-channel pixel-level segmentation mask corresponding to tumour, stroma, necrosis, blood vessels, and epidermis.

## Scoring recipe

```python
def compute_dice(pred, gt, num_classes=5):
    dice_scores = []
    for c in range(num_classes):
        p = pred[:, c]
        g = gt[:, c]
        intersection = np.sum(p * g)
        dice = (2.0 * intersection) / (np.sum(p) + np.sum(g) + 1e-6)
        dice_scores.append(dice)
    micro_dice = np.mean(dice_scores)
    return micro_dice, dice_scores
```

## Common pitfalls

- Class imbalance causes high variance in Dice scores for rare classes like necrosis and blood vessels.
- The preliminary test set contains only 10 images, making it unrepresentative of overall performance.
- Micro-average Dice can mask poor performance on minority classes if not examined per-class.

## Evidence (verbatim from paper)

> Our method achieved the highest average Dice score  $(68.23\%)$ . We found that incorporating the dual-stage loss strategy led to an improvement in segmentation performance, from  $66.84\%$  to  $68.23\%$ . However, we also observed a large standard deviation in Dice scores for the rarer tissue classes, notably epidermis, blood vessels, and especially necrosis.

## Citation

```bibtex
@misc{lv2025leveraging,
  title={Leveraging Pathology Foundation Models for Panoptic Segmentation of Melanoma in H&E Images},
  author={Jiaqi Lv et al. (2025)},
  year={2025},
  note={arXiv:2507.13974}
}
```

- arXiv: 2507.13974