Sintel Kitti Flow Eval

---
name: sintel-kitti-flow-eval
description: Evaluates a neural network's ability to interpolate sparse optical flow matches into dense flow maps. It probes the model's capacity to handle missing pixels, occlusions, and motion boundaries while preserving flow accuracy across diverse scenes. Use when the user wants to benchmark on MPI Sintel, KITTI 2012, KITTI 2015, or asks about evaluating this task. Reports EPE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1611.09803
  bibtex_key: [REDACTED]
  confidence: high
---

# sintel-kitti-flow-eval

> InterpoNet, A brain inspired neural network for optical flow dense interpolation — Zweig et al. (2016) (arXiv:1611.09803, 2016)

## What this evaluates

Evaluates a neural network's ability to interpolate sparse optical flow matches into dense flow maps. It probes the model's capacity to handle missing pixels, occlusions, and motion boundaries while preserving flow accuracy across diverse scenes.

## Datasets

- **MPI Sintel** — total 1041; splits: train (1041), val (-1), test (-1)
- **KITTI 2012** — total 194; splits: train (194)
- **KITTI 2015** — total 200; splits: train (200)

## Metrics

- `EPE` **(primary)** — range: pixels
  - Mean Euclidean distance between predicted and ground truth flow vectors across all pixels: EPE = (1/N) * Σ ||v_pred - v_gt||_2. Lower values indicate better interpolation accuracy.
- `EPE-noc` — range: pixels
  - EPE computed only on non-occluded pixels.
- `EPE-occ` — range: pixels
  - EPE computed only on occluded pixels.

## Input / output format

**Input**: Four-channel tensor: two channels for sparse flow vectors (output from a matching algorithm), one binary mask channel indicating missing pixels, and one edge map channel. All inputs are downsampled by 8x.

**Output**: Dense optical flow map (H/8 × W × 2 channels) representing the interpolated flow field.

## Scoring recipe

```python
def compute_epe(pred_flow, gt_flow):
    # pred_flow and gt_flow are (H, W, 2) numpy arrays
    diff = pred_flow - gt_flow
    epe_map = np.sqrt(diff[..., 0]**2 + diff[..., 1]**2)
    return np.mean(epe_map)

# For occluded/non-occluded splits, apply a boolean mask before mean:
# epe_masked = np.mean(epe_map[mask])
```

## Common pitfalls

- The model does not take raw image pairs as input; it requires pre-computed sparse flow from a specific matching algorithm (e.g., FlowFields, CPM-Flow, DeepMatching).
- Performance is highly sensitive to fine-tuning; the paper explicitly notes that out-of-the-box pre-trained results are lower and fine-tuning on the target dataset/matching algorithm is required for reported benchmarks.
- Benchmarks are evaluated on the Sintel 'final' pass, not the 'clean' pass, which must be noted when comparing with other methods.

## Evidence (verbatim from paper)

> For Sintel (Table 4), we achieve state of the art results using FlowFields as the matching algorithm. For all the matching algorithms used, we achieve better results compared to EpicFlow improving the EPE by an average of 0.3px.

## Citation

```bibtex
@misc{zweig2016interpnet,
  title={InterpoNet, A brain inspired neural network for optical flow dense interpolation},
  author={Zweig et al. (2016)},
  year={2016},
  note={arXiv:1611.09803}
}
```

- arXiv: 1611.09803