Pancreas Segmentation Pruning Eval

---
name: pancreas-segmentation-pruning-eval
description: This protocol evaluates the effectiveness of data pruning strategies for 3D medical image segmentation. It measures how well a pruned training subset preserves model performance on pancreas segmentation tasks compared to using the full dataset or random subsets, specifically testing whether early-training dynamics can guide efficient sample selection without accuracy loss. Use when the user wants to benchmark on MSD-Pancreas, WORD, NIH-Pancreas, or asks about evaluating this task. Reports DSC score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2308.01189
  bibtex_key: he2023datacentricdiet
  confidence: high
---

# pancreas-segmentation-pruning-eval

> Data-Centric Diet: Effective Multi-center Dataset Pruning for Medical Image Segmentation — He et al. (2023) (arXiv:2308.01189, 2023)

## What this evaluates

This protocol evaluates the effectiveness of data pruning strategies for 3D medical image segmentation. It measures how well a pruned training subset preserves model performance on pancreas segmentation tasks compared to using the full dataset or random subsets, specifically testing whether early-training dynamics can guide efficient sample selection without accuracy loss.

## Datasets

- **MSD-Pancreas** — total 100; splits: train (80), test (20)
- **WORD** — total 100; splits: train (80), test (20); repo https://github.com/HiLab-git/WORD
- **NIH-Pancreas** — total 82; splits: train (60), test (22)

## Metrics

- `DSC score` **(primary)** — range: [0, 1]
  - Dice Similarity Coefficient, calculated as 2 * |prediction ∩ ground_truth| / (|prediction| + |ground_truth|). Measures voxel-wise overlap between the predicted segmentation mask and the manual annotation.

## Input / output format

**Input**: 3D CT scan patches of size 64×64×64 with corresponding binary segmentation masks for the pancreas.

**Output**: Voxel-wise probability map or binary segmentation mask indicating the predicted pancreas region.

## Scoring recipe

```python
def compute_dsc_score(pred_mask, gold_mask):
    pred = (pred_mask > 0.5).astype(np.int32)
    gold = (gold_mask > 0).astype(np.int32)
    intersection = np.sum(pred * gold)
    union = np.sum(pred) + np.sum(gold)
    if union == 0:
        return 0.0
    return 2.0 * intersection / union
```

## Common pitfalls

- Applying standard gradient-based importance metrics (e.g., VOG, EL2N) directly to dense medical labels fails because background voxels dominate the gradient signal.
- Assuming that retaining only the hardest-to-learn samples improves pruning; the protocol demonstrates that ambiguous samples yield better performance on small datasets.
- Computing pruning scores after full model convergence; sample difficulty rankings change dynamically during training, so early-epoch computation is recommended for efficiency.

## Evidence (verbatim from paper)

> Table 1: Comparison of DSC score for V-Net models trained on different regions selected by DAD score. The results show that models trained on those ambiguous samples performed better.

## Citation

```bibtex
@misc{he2023datacentricdiet,
  title={Data-Centric Diet: Effective Multi-center Dataset Pruning for Medical Image Segmentation},
  author={He et al. (2023)},
  year={2023},
  note={arXiv:2308.01189}
}
```

- arXiv: 2308.01189