Radioactive Eval

---
name: radioactive-eval
description: Evaluates interactive 3D medical image segmentation by measuring how well models segment target structures under varying human-in-the-loop prompting strategies (points, boxes, scribbles) and iterative refinement protocols. It specifically probes the trade-off between interaction effort and segmentation accuracy across 2D and 3D architectures. Use when the user wants to benchmark on RadioActive, or asks about evaluating this task. Reports Dice.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2411.07885
  bibtex_key: ulrich2024radioactive
  confidence: high
---

# radioactive-eval

> RadioActive: 3D Radiological Interactive Segmentation Benchmark — Ulrich et al. (2024) (arXiv:2411.07885, 2024)

## What this evaluates

Evaluates interactive 3D medical image segmentation by measuring how well models segment target structures under varying human-in-the-loop prompting strategies (points, boxes, scribbles) and iterative refinement protocols. It specifically probes the trade-off between interaction effort and segmentation accuracy across 2D and 3D architectures.

## Datasets

- **RadioActive** — total ?; splits: (unstated); repo https://github.com/MIC-DKFZ/radioactive

## Metrics

- `Dice` **(primary)** — range: [0, 1]
  - Computes the overlap between the predicted segmentation mask and the ground truth mask, calculated as 2 * |A ∩ B| / (|A| + |B|). Reported as mean or average across cases/slices.

## Input / output format

**Input**: 3D radiological volumes with interactive prompts provided as points, 2D/3D bounding boxes, or scribbles. Prompts are applied either slice-by-slice, per-volume, or via interpolation/propagation between initial prompts.

**Output**: 3D segmentation mask corresponding to the prompted target structure.

## Scoring recipe

```python
def dice_score(pred, gt):
    intersection = np.logical_and(pred, gt).sum()
    union = pred.sum() + gt.sum()
    return 2.0 * intersection / union if union > 0 else 0.0
mean_dice = np.mean([dice_score(p, g) for p, g in zip(predictions, ground_truths)])
```

## Common pitfalls

- Slice-by-slice prompting for 2D models is unrealistic and significantly inflates performance compared to human-in-the-loop scenarios.
- Propagation-based prompting can terminate early or cause severe oversegmentation if an initial prompt fails, unlike interpolation methods.
- Forgetting to re-provide initial prompts during iterative refinement steps leads to performance degradation.

## Evidence (verbatim from paper)

> Figure 5 shows that models employing box prompts achieved significantly higher average Dice scores, with SAM2 demonstrating the strongest performance across all models. Conversely, point-based prompts performed poorly, particularly for small target regions, such as small MS lesions in dataset D1 (see [Tab. 2] in the appendix).

## Citation

```bibtex
@misc{ulrich2024radioactive,
  title={RadioActive: 3D Radiological Interactive Segmentation Benchmark},
  author={Ulrich et al. (2024)},
  year={2024},
  note={arXiv:2411.07885}
}
```

- arXiv: 2411.07885