Rgb Event Segmentation Eval

---
name: rgb-event-segmentation-eval
description: Evaluates the capability of RGB-Event fusion models to perform accurate per-pixel semantic segmentation under challenging conditions such as fast motion, varying lighting, and spatiotemporal misalignment between asynchronous modalities. Use when the user wants to benchmark on DDD17, DSEC, DELIVER, M3ED, or asks about evaluating this task. Reports mIoU.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.01548
  bibtex_key: yao2025learning
  confidence: high
---

# rgb-event-segmentation-eval

> Learning Flow-Guided Registration for RGB-Event Semantic Segmentation — Yao et al. (2025) (arXiv:2505.01548, 2025)

## What this evaluates

Evaluates the capability of RGB-Event fusion models to perform accurate per-pixel semantic segmentation under challenging conditions such as fast motion, varying lighting, and spatiotemporal misalignment between asynchronous modalities.

## Datasets

- **DDD17** — total 19840; splits: train (15950), test (3890)
- **DSEC** — total 10891; splits: test (10891)
- **DELIVER** — total ?; splits: test (-1)
- **M3ED** — total 7997; splits: train (5516), test (2481)

## Metrics

- `mIoU` **(primary)** — range: percent
  - Mean Intersection over Union across all semantic classes, computed as the average of IoU per class.
- `Accuracy` — range: percent
  - Pixel accuracy, calculated as the ratio of correctly classified pixels to the total number of pixels.

## Input / output format

**Input**: Paired RGB images and asynchronous event data (represented as Motion-Enhanced Event Tensor, voxel grids, or other event representations).

**Output**: Per-pixel semantic segmentation mask assigning one of the dataset-specific class labels to each pixel.

## Scoring recipe

```python
def compute_miou(preds, gold, num_classes):
    ious = []
    for c in range(num_classes):
        tp = np.sum((preds == c) & (gold == c))
        fp = np.sum((preds == c) & (gold != c))
        fn = np.sum((preds != c) & (gold == c))
        iou = tp / (tp + fp + fn + 1e-6)
        ious.append(iou)
    return np.mean(ious) * 100
```

## Common pitfalls

- Comparisons must use the same backbone (MiT-B2) to be fair, as performance heavily depends on it.
- Event representation format (e.g., MET vs. Voxel Grid) significantly impacts results and must be explicitly controlled or reported.
- M3ED dataset splits are manually curated from the Urban Day subset, not a standard public split, requiring careful reproduction.

## Evidence (verbatim from paper)

> As shown in Table [2], BRENet achieves 78.56% mIoU and 96.61% accuracy on DDD17 and 74.94% mIoU and 95.85% accuracy on DSEC using the MiT-B2 backbone. ... The table indicates that BRENet achieves 63.13% and 67.28% mIoU on DELIVER and M3ED.

## Citation

```bibtex
@misc{yao2025learning,
  title={Learning Flow-Guided Registration for RGB-Event Semantic Segmentation},
  author={Yao et al. (2025)},
  year={2025},
  note={arXiv:2505.01548}
}
```

- arXiv: 2505.01548