Seeds Superpixel Eval

---
name: seeds-superpixel-eval
description: Evaluates the quality of superpixel segmentation algorithms by measuring how well superpixel boundaries align with ground-truth object boundaries and how accurately superpixels can be used as indivisible units for downstream segmentation tasks. Use when the user wants to benchmark on Berkeley Segmentation Dataset (BSD), or asks about evaluating this task. Reports under-segmentation error (UE).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1309.3848
  bibtex_key: vandenberg2013seeds
  confidence: high
---

# seeds-superpixel-eval

> SEEDS: Superpixels Extracted via Energy-Driven Sampling — Van den Bergh et al. (2013) (arXiv:1309.3848, 2013)

## What this evaluates

Evaluates the quality of superpixel segmentation algorithms by measuring how well superpixel boundaries align with ground-truth object boundaries and how accurately superpixels can be used as indivisible units for downstream segmentation tasks.

## Datasets

- **Berkeley Segmentation Dataset (BSD)** — total 500; splits: train (200), val (100), test (200)

## Metrics

- `under-segmentation error (UE)` **(primary)** — range: [0, 1]
  - Sum of pixels in superpixels that do not overlap with their corresponding ground-truth segment, divided by total ground-truth area. Lower is better.
- `corrected under-segmentation error (CUE)` — range: [0, 1]
  - Each superpixel is matched to the ground-truth segment with the largest overlap. The number of pixels outside this matched segment is summed and divided by total ground-truth area. Lower is better.
- `boundary recall (BR)` — range: [0, 1]
  - Percentage of ground-truth boundary pixels that have at least one superpixel boundary pixel within a tolerance of epsilon=2 pixels. Higher is better.
- `achievable segmentation accuracy (ASA)` — range: [0, 1]
  - Maximum possible segmentation accuracy if superpixels are treated as indivisible units. Computed by assigning each superpixel to its most overlapping ground-truth label and dividing correctly labeled pixels by total area. Higher is better.

## Input / output format

**Input**: RGB or LAB color image and corresponding ground-truth segmentation map.

**Output**: Superpixel segmentation map where each pixel is assigned a superpixel ID.

## Scoring recipe

```python
def compute_ue(sp, gt):
    total_gt = sum(len(g) for g in gt)
    err = sum(len(s - g) for s in sp for g in gt if s & g)
    return err / total_gt

def compute_cue(sp, gt):
    total_gt = sum(len(g) for g in gt)
    err = sum(len(s - max(gt, key=lambda g: s & g)) for s in sp)
    return err / total_gt

def compute_br(sp, gt, eps=2):
    gt_b = get_boundaries(gt)
    sp_b = get_boundaries(sp)
    matches = sum(1 for p in gt_b if min_dist(p, sp_b) < eps)
    return matches / len(gt_b)

def compute_asa(sp, gt):
    total_gt = sum(len(g) for g in gt)
    correct = sum(max(len(s & g) for g in gt) for s in sp)
    return correct / total_gt
```

## Common pitfalls

- Tolerance handling for boundary pixels varies across papers (e.g., 5% margin vs. removing borders).
- UE penalizes boundary pixels equally on both sides, which can unfairly penalize grid-based initializations.
- BR uses a fixed tolerance epsilon=2 pixels, which may not match other implementations.

## Evidence (verbatim from paper)

> We compute the standard metrics used to evaluate the performance of superpixel algorithms, which are under-segmentation error (UE), boundary recall (BR) and achievable segmentation accuracy (ASA). Additionally, we introduce a new metric, which is a corrected under-segmentation error (CUE).

## Citation

```bibtex
@misc{vandenberg2013seeds,
  title={SEEDS: Superpixels Extracted via Energy-Driven Sampling},
  author={Van den Bergh et al. (2013)},
  year={2013},
  note={arXiv:1309.3848}
}
```

- arXiv: 1309.3848