Robustspring Eval

---
name: robustspring-eval
description: Evaluates the robustness of dense correspondence models (optical flow, scene flow, stereo) to 20 types of image corruptions by measuring the divergence between predictions on clean images and predictions on corrupted images. Use when the user wants to benchmark on Spring, or asks about evaluating this task. Reports R^c_EPE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.09368
  bibtex_key: schmalfuss2025robustspring
  confidence: high
---

# robustspring-eval

> RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereo — Schmalfuss et al. (2025) (arXiv:2505.09368, 2025)

## What this evaluates

Evaluates the robustness of dense correspondence models (optical flow, scene flow, stereo) to 20 types of image corruptions by measuring the divergence between predictions on clean images and predictions on corrupted images.

## Datasets

- **Spring** — total ?; splits: test (-1)

## Metrics

- `R^c_EPE` **(primary)** — range: other
  - End-Point Error between the model's prediction on clean images and its prediction on corrupted images: $R^c_{\text{EPE}} = \frac{1}{N} \sum_{i} \| \hat{y}_{\text{clean}, i} - \hat{y}_{\text{corrupted}, i} \|_2$. Lower values indicate higher robustness. Variants include $R^c_{\text{1px}}$, $R^c_{\text{Abs}}$, $R^c_{\text{D1}}$, $R^c_{\text{D2}}$, and $R^c_{\text{Fl}}$ for different error thresholds.

## Input / output format

**Input**: Corrupted image pairs (or video frames) with applied corruptions (noise, blur, weather, quality, color) and ground truth dense correspondences/disparities.

**Output**: Dense flow vectors, scene flow vectors, or disparity maps.

## Scoring recipe

```python
def compute_rc_epe(pred_clean, pred_corrupted):
    # pred_clean and pred_corrupted are dense flow/disparity maps
    error = np.linalg.norm(pred_clean - pred_corrupted, axis=-1)
    return np.mean(error)
```

## Common pitfalls

- Ranking models using only the Average $R^c_{\text{EPE}}$ can misrepresent robustness compared to Median or Schulze rankings, as averages are highly sensitive to outlier corruptions like rain.
- Subsampling must exclude full-resolution 'Hero-frames'; otherwise, the 0.05% subsample will not align with full-dataset results.
- Weather and noise corruptions dominate the error scores, often masking performance differences on milder corruptions.

## Evidence (verbatim from paper)

> corruption robustness $R^{c}_{\text{EPE}}$ between clean and corrupted predictions. Low values indicate robust models

## Citation

```bibtex
@misc{schmalfuss2025robustspring,
  title={RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereo},
  author={Schmalfuss et al. (2025)},
  year={2025},
  note={arXiv:2505.09368}
}
```

- arXiv: 2505.09368