Lung Tumor Segmentation Eval

---
name: lung-tumor-segmentation-eval
description: This benchmark evaluates 3D lung tumor segmentation accuracy in CT imaging, comparing traditional CNN architectures against foundation models under standard, few-shot, and prompt-based inference regimes. It probes model robustness to varying training data sizes and input prompting strategies in a medical imaging context. Use when the user wants to benchmark on NSCLC-Radiomics (Lung1), Task06 (Medical Segmentation Decathlon), or asks about evaluating this task. Reports Dice Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.01239
  bibtex_key: [REDACTED]
  confidence: high
---

# lung-tumor-segmentation-eval

> Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imaging — Mulero Ayllón et al. (2025) (arXiv:2505.01239, 2025)

## What this evaluates

This benchmark evaluates 3D lung tumor segmentation accuracy in CT imaging, comparing traditional CNN architectures against foundation models under standard, few-shot, and prompt-based inference regimes. It probes model robustness to varying training data sizes and input prompting strategies in a medical imaging context.

## Datasets

- **NSCLC-Radiomics (Lung1)** — total 304; splits: train (246), test (58)
- **Task06 (Medical Segmentation Decathlon)** — total 63; splits: train (51), test (12)

## Metrics

- `IoU` — range: [0, 1]
  - Intersection over Union. Calculated as the ratio of the intersection of the predicted and ground truth regions to the union of those regions: IoU = |A ∩ B| / |A ∪ B|.
- `Dice Score` **(primary)** — range: [0, 1]
  - Dice Similarity Coefficient. Measures similarity between predicted and ground truth regions: Dice Score = 2|A ∩ B| / (|A| + |B|).

## Input / output format

**Input**: 3D CT scan volumes preprocessed via Hounsfield unit conversion, resampled to 1x1x3 mm spacing, clipped to [-1000, 1000], and normalized to [0, 1]. Masks are combined into a single-channel image with distinct intensity values for lung and tumor regions. MedSAM 2 additionally accepts bounding box or click prompts.

**Output**: Predicted 3D segmentation mask indicating the tumor region, aligned with the input volume dimensions.

## Scoring recipe

```python
def compute_metrics(pred, gt):
    intersection = np.sum(pred & gt)
    union = np.sum(pred | gt)
    iou = intersection / union if union > 0 else 0.0
    dice = (2 * intersection) / (np.sum(pred) + np.sum(gt)) if (np.sum(pred) + np.sum(gt)) > 0 else 0.0
    return {'IoU': iou, 'Dice Score': dice}
```

## Common pitfalls

- Strict preprocessing pipeline is required: Hounsfield conversion, resampling to 1x1x3mm, clipping to [-1000, 1000], and normalization to [0,1] must be applied identically across datasets.
- Splits are fixed per dataset (Lung1: 246/58, Task06: 51/12) to prevent data leakage; altering these splits breaks comparability.
- Prompting strategies (bounding box vs. click) were only evaluated for MedSAM 2, not the baseline models, so results are not directly transferable to other architectures.

## Evidence (verbatim from paper)

> The performance of the segmentation models was evaluated using two widely recognized metrics: the Intersection over Union (IoU) [23] and the Dice Similarity Coefficient (Dice Score) [24]. Both metrics are commonly used in medical image segmentation tasks and provide complementary insights into the accuracy of the predicted segmentation masks relative to the ground truth. In the following equations, A represents the predicted segmentation, B represents the ground truth, and |A ∩ B| is the area of overlap between the predicted and true regions, while |A ∪ B| is the total area covered by either the predicted or the ground truth region: IoU: it quantifies the overlap between the predicted segmentation and the ground truth [23]. It is calculated as the ratio of the intersection of the predicted and ground truth regions to the union of those regions. Dice Score: it measures the similarity between the predicted and ground truth regions [24]. It is calculated as twice the intersection of the predicted and ground truth regions divided by the sum of their areas.

## Citation

```bibtex
@misc{muleroayllon2025lungtumor,
  title={Can Foundation Models Really Segment Tumors? A Benchmarking Odyssey in Lung CT Imaging},
  author={Mulero Ayllón et al. (2025)},
  year={2025},
  note={arXiv:2505.01239}
}
```

- arXiv: 2505.01239