Voldor Eval

---
name: voldor-eval
description: Evaluates monocular visual odometry accuracy and depth estimation quality on urban/highway driving sequences and indoor environments. Probes robustness to non-Gaussian optical flow noise and scale ambiguity without relying on hand-crafted features or loop closure. Use when the user wants to benchmark on KITTI odometry benchmark, KITTI stereo benchmark, TUM RGB-D dataset, or asks about evaluating this task. Reports Trans. error (%), Rot. error (deg/m).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2104.06789
  bibtex_key: min2021voldor
  confidence: high
---

# voldor-eval

> VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residuals — Min et al. (2021) (arXiv:2104.06789, 2021)

## What this evaluates

Evaluates monocular visual odometry accuracy and depth estimation quality on urban/highway driving sequences and indoor environments. Probes robustness to non-Gaussian optical flow noise and scale ambiguity without relying on hand-crafted features or loop closure.

## Datasets

- **KITTI odometry benchmark** — total ?; splits: train (-1), test (-1)
- **KITTI stereo benchmark** — total ?; splits: train (-1), test (-1)
- **TUM RGB-D dataset** — total ?; splits: test (-1)

## Metrics

- `Trans. error (%)` **(primary)** — range: percent
  - Percentage of total translation distance traveled incorrectly, averaged over sub-sequences of 100–800 m length.
- `Rot. error (deg/m)` **(primary)** — range: other
  - Total rotation angle in degrees divided by total translation distance in meters, averaged over sub-sequences.
- `EPE (px)` — range: other
  - Mean End-Point Error between predicted and ground-truth disparity/depth in pixels.
- `bg-outlier (%)` — range: percent
  - Percentage of background pixels where disparity error exceeds 3 px or 5%.
- `Translation RMSE (m)` — range: other
  - Root mean square error of the translation trajectory after rigid alignment to ground truth over 6-frame segments.

## Input / output format

**Input**: Sequential RGB image pairs and externally estimated dense optical flow fields (e.g., from PWC-Net).

**Output**: 6-DoF camera pose transformations, per-pixel depth maps, and pixel-wise rigidness/confidence probabilities.

## Scoring recipe

```python
def evaluate_kitti(pred_poses, gt_poses):
    trans_err, rot_err = 0.0, 0.0
    for seq in sequences:
        for subseq in sliding_window(seq, 100, 100):
            trans_err += trajectory_error(pred_poses[subseq], gt_poses[subseq])
            rot_err += rotation_error(pred_poses[subseq], gt_poses[subseq])
    return trans_err / N, rot_err / N

def evaluate_tum(pred_poses, gt_poses):
    rmse_vals = []
    for seg in sliding_window(dataset, 6):
        aligned_gt = align_to_gt(gt_poses[seg])
        rmse_vals.append(translation_rmse(pred_poses[seg], aligned_gt))
    return mean(rmse_vals)
```

## Common pitfalls

- Scale ambiguity: Monocular output requires explicit ground-plane height estimation or known camera height; unaligned scales invalidate error metrics.
- Rigidness threshold sensitivity: Depth map density and accuracy trade off sharply based on the W^j threshold, affecting EPE and outlier rates.
- Optical flow dependency: Performance is tightly coupled to the external flow estimator (e.g., PWC-Net); results degrade with noisier flow inputs.

## Evidence (verbatim from paper)

> The translation and rotation errors are averaged over all sub-sequences of length from 100 meters to 800 meters with 100 meter steps. ... In all instances, we rigidly align trajectories to groundtruth for segments with 6 frames and estimate mean translation RMSE of all segments.

## Citation

```bibtex
@misc{min2021voldor,
  title={VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residuals},
  author={Min et al. (2021)},
  year={2021},
  note={arXiv:2104.06789}
}
```

- arXiv: 2104.06789