Optical Flow Eval

---
name: optical-flow-eval
description: Evaluates dense optical flow estimation by predicting pixel-wise displacement vectors between consecutive frames. It probes robustness to large motions, occlusions, blur, and atmospheric effects across synthetic and real-world driving scenes. Use when the user wants to benchmark on MPI Sintel, KITTI, Middlebury, or asks about evaluating this task. Reports AEE (Average Endpoint Error).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1704.03217
  bibtex_key: li2017pyramidal
  confidence: high
---

# optical-flow-eval

> Pyramidal Gradient Matching for Optical Flow Estimation — Li (2017) (arXiv:1704.03217, 2017)

## What this evaluates

Evaluates dense optical flow estimation by predicting pixel-wise displacement vectors between consecutive frames. It probes robustness to large motions, occlusions, blur, and atmospheric effects across synthetic and real-world driving scenes.

## Datasets

- **MPI Sintel** — total ?; splits: train (2082), test (-1)
- **KITTI** — total ?; splits: train (194), test (-1)
- **Middlebury** — total ?; splits: test (12)

## Metrics

- `AEE (Average Endpoint Error)` **(primary)** — range: pixels
  - Average Euclidean distance between predicted and ground truth flow vectors across specified pixel sets (all, non-occluded, or occluded).
- `Out-Noc / Out-All` — range: percent
  - Percentage of pixels where the endpoint error exceeds 3 pixels, computed over non-occluded (Out-Noc) or all (Out-All) regions.

## Input / output format

**Input**: Pairs of consecutive images (grayscale or color) from a video sequence or stereo rig.

**Output**: Dense 2-channel flow field representing horizontal (u) and vertical (v) displacement vectors for every pixel.

## Scoring recipe

```python
def compute_metrics(pred_flow, gt_flow, mask_noc=None):
    error = np.sqrt((pred_flow[:,:,0] - gt_flow[:,:,0])**2 + (pred_flow[:,:,1] - gt_flow[:,:,1])**2)
    if mask_noc is not None:
        aee = np.mean(error[mask_noc])
        out = np.mean(error[mask_noc] > 3) * 100
    else:
        aee = np.mean(error)
        out = np.mean(error > 3) * 100
    return aee, out
```

## Common pitfalls

- AEE is reported separately for non-occluded and all areas; comparing only one can misrepresent robustness.
- MPI Sintel provides 'clean' and 'final' passes with different GT generation; results are not directly comparable across passes.
- Processing time is highly hardware-dependent; the paper notes results are not normalized for CPU/GPU or parallelization.

## Evidence (verbatim from paper)

> AEE on each pass is computed with all 1041 image pairs of corresponding pass. Out-Noc (ALL) is the percentage of erroneous pixels (>3) in non-occluded (all) areas.

## Citation

```bibtex
@misc{li2017pyramidal,
  title={Pyramidal Gradient Matching for Optical Flow Estimation},
  author={Li (2017)},
  year={2017},
  note={arXiv:1704.03217}
}
```

- arXiv: 1704.03217