Cell Instance Segmentation Eval

---
name: cell-instance-segmentation-eval
description: This evaluation probes a model's ability to perform precise instance segmentation on challenging biomedical microscopy images. It specifically tests handling of overlapping, irregularly shaped cells across varying contrast modalities (brightfield, phase-contrast, fluorescence) and object densities. Use when the user wants to benchmark on LIVECell, EVICAN2, ISBI2014, Revvity-25, or asks about evaluating this task. Reports AP (Average Precision).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.01928
  bibtex_key: prytula2025iaunet
  confidence: high
---

# cell-instance-segmentation-eval

> IAUNet: Instance-Aware U-Net — Prytula et al. (2025) (arXiv:2508.01928, 2025)

## What this evaluates

This evaluation probes a model's ability to perform precise instance segmentation on challenging biomedical microscopy images. It specifically tests handling of overlapping, irregularly shaped cells across varying contrast modalities (brightfield, phase-contrast, fluorescence) and object densities.

## Datasets

- **LIVECell** — total 5239; splits: train (-1), val (-1), test (-1)
- **EVICAN2** — total 5237; splits: train/val (4640), test (98)
- **ISBI2014** — total 961; splits: train (45), val (90), test (810)
- **Revvity-25** — total 110; splits: train (55), test (55)

## Metrics

- `AP (Average Precision)` **(primary)** — range: percent
  - Standard COCO-style instance segmentation metric. Computes average precision across IoU thresholds from 0.50 to 0.95 in steps of 0.05, averaged over object sizes (small, medium, large) and classes. Reported as a percentage.

## Input / output format

**Input**: Raw microscopy images (phase-contrast, brightfield, or fluorescence) resized to 512×512 pixels during training and inference, with aspect ratio preserved via longest-side resizing.

**Output**: A list of predicted instance masks (or bounding boxes/polygons) with associated confidence scores and class labels (cell or nucleus), limited to a maximum of 100 queries per image.

## Scoring recipe

```python
def compute_ap(predictions, ground_truth, iou_thresholds=np.arange(0.50, 0.96, 0.05)):
    ap_scores = []
    for iou_thr in iou_thresholds:
        tp, fp = 0, 0
        for pred in predictions:
            best_iou = max(iou(pred.mask, gt.mask) for gt in ground_truth)
            if best_iou >= iou_thr:
                tp += 1
            else:
                fp += 1
        precision = tp / (tp + fp) if (tp + fp) > 0 else 0
        ap_scores.append(precision)
    return np.mean(ap_scores) * 100
```

## Common pitfalls

- Models like CellPose require training separate instances per class and averaging results, which can artificially skew performance compared to multi-class native models.
- Datasets with low object counts (e.g., ISBI2014) may cause query-based models to predict duplicate instances, unfairly penalizing AP scores.
- Significant shifts in object size distribution between training and testing splits can break diameter-dependent post-processing heuristics used by older segmentation models.

## Evidence (verbatim from paper)

> In models with convolution-based backbones, IAUNet with ResNet-50 achieves an AP of 45.3 and AP50 of 75.3 on LiveCell. It outperforms Mask R-CNN, PointRend, Mask2Former, and MaskDINO while using fewer parameters (39M) and lower FLOPs (49G).

## Citation

```bibtex
@misc{prytula2025iaunet,
  title={IAUNet: Instance-Aware U-Net},
  author={Prytula et al. (2025)},
  year={2025},
  note={arXiv:2508.01928}
}
```

- arXiv: 2508.01928