Kitti Lidar Flow Eval

---
name: kitti-lidar-flow-eval
description: This evaluation probes a model's ability to estimate dense optical flow directly from sparse, noisy LiDAR range scans without using RGB images. It measures prediction accuracy against real-world ground truth flow maps and evaluates robustness to occlusions and foreground/background motion. Use when the user wants to benchmark on KITTI Tracking & Flow 2015, or asks about evaluating this task. Reports EPE (End-Point-Error).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1808.10542
  bibtex_key: vaquero2018hallucinating
  confidence: high
---

# kitti-lidar-flow-eval

> Hallucinating Dense Optical Flow from Sparse Lidar for Autonomous Vehicles — Victor Vaquero, Alberto Sanfeliu, Francesc Moreno-Noguer (2018) (arXiv:1808.10542, 2018)

## What this evaluates

This evaluation probes a model's ability to estimate dense optical flow directly from sparse, noisy LiDAR range scans without using RGB images. It measures prediction accuracy against real-world ground truth flow maps and evaluates robustness to occlusions and foreground/background motion.

## Datasets

- **KITTI Tracking & Flow 2015** — total 19045; splits: train (17500), val (1455), test (90)

## Metrics

- `EPE (End-Point-Error)` **(primary)** — range: pixels
  - Average Euclidean distance between predicted and ground-truth optical flow vectors across all pixels.
- `Outlier Percentage` — range: percent
  - Percentage of pixels where the EPE is less than 3 pixels or less than 5% of the ground-truth flow magnitude.

## Input / output format

**Input**: Consecutive pairs of sparse LiDAR scans (Velodyne HDL-64) containing range and reflectivity values, formatted as 64x384 grids.

**Output**: Dense optical flow map with 2D displacement vectors per pixel, resolution 256x1224.

## Scoring recipe

```python
def compute_metrics(pred_flow, gt_flow):
    # pred_flow, gt_flow: (H, W, 2) arrays
    diff = pred_flow - gt_flow
    epe = np.mean(np.sqrt(np.sum(diff**2, axis=-1)))
    gt_mag = np.sqrt(np.sum(gt_flow**2, axis=-1))
    outlier = (np.sqrt(np.sum(diff**2, axis=-1)) < 3) | \
              (np.sqrt(np.sum(diff**2, axis=-1)) / (gt_mag + 1e-6) < 0.05)
    return epe, np.mean(outlier) * 100
```

## Common pitfalls

- The test set is extremely small (90 pairs) because it requires matching RGB frames from KITTI Flow 2015 with LiDAR frames from the Tracking benchmark.
- Training uses pseudo-ground-truth flow generated by FlowNet2 on RGB images, but evaluation against real ground-truth is required for benchmark comparison.
- The outlier threshold uses an OR condition (<3px OR <5%), not an AND condition, which significantly changes the reported percentage.

## Evidence (verbatim from paper)

> A pixel is considered to be correctly estimated if the End-Point-Error (EPE) calculated as the averaged Euclidean distance between the prediction and the real ground-truth $G_{Test}$ is $<3$px or $<5$%. These measurements are averaged over background regions only, over foreground regions only, and over all ground truth pixels, which respectively are denoted in Table I as “Fl-BG”, “Fl-FG” and “All”.

## Citation

```bibtex
@misc{vaquero2018hallucinating,
  title={Hallucinating Dense Optical Flow from Sparse Lidar for Autonomous Vehicles},
  author={Victor Vaquero, Alberto Sanfeliu, Francesc Moreno-Noguer (2018)},
  year={2018},
  note={arXiv:1808.10542}
}
```

- arXiv: 1808.10542