Kitti Subt Vo Depth Eval

---
name: kitti-subt-vo-depth-eval
description: Evaluates unsupervised monocular visual odometry and depth estimation methods on challenging driving and subterranean environments. Probes the model's ability to predict consistent 6-DoF ego-motion and recover accurate depth maps without ground-truth supervision. Use when the user wants to benchmark on KITTI, DARPA Subterranean Challenge, or asks about evaluating this task. Reports relative translation error ($t_{err}$), relative rotation error ($r_{err}$).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2011.00341
  bibtex_key: almalioglu2020unsupervised
  confidence: high
---

# kitti-subt-vo-depth-eval

> Unsupervised Deep Persistent Monocular Visual Odometry and Depth Estimation in Extreme Environments — Almalioglu et al. (2020) (arXiv:2011.00341, 2020)

## What this evaluates

Evaluates unsupervised monocular visual odometry and depth estimation methods on challenging driving and subterranean environments. Probes the model's ability to predict consistent 6-DoF ego-motion and recover accurate depth maps without ground-truth supervision.

## Datasets

- **KITTI** — total ?; splits: test (-1)
- **DARPA Subterranean Challenge** — total ?; splits: test (-1)

## Metrics

- `relative translation error ($t_{err}$)` **(primary)** — range: percent
  - Relative translation error divided by distance travelled, averaged over trajectory segments of lengths {7, 14, 21, 28, 35} m. Predictions are aligned to ground truth via 7-DoF optimization minimizing ATE.
- `relative rotation error ($r_{err}$)` **(primary)** — range: percent
  - Relative rotation error divided by distance travelled, averaged over trajectory segments of lengths {7, 14, 21, 28, 35} m. Predictions are aligned to ground truth via 7-DoF optimization minimizing ATE.
- `AbsRel` — range: other
  - Mean absolute relative difference between predicted and ground truth depth: mean(|pred - gt| / gt).
- `SqRel` — range: other
  - Mean squared relative difference: mean((pred - gt)^2 / gt).
- `RMS` — range: other
  - Root mean square error between predicted and ground truth depth.
- `RMSlog` — range: other
  - Root mean square error between log-predicted and log-ground truth depth.
- `δ<1.25` — range: percent
  - Percentage of pixels where max(pred/gt, gt/pred) < 1.25.
- `δ<1.25²` — range: percent
  - Percentage of pixels where max(pred/gt, gt/pred) < 1.25².
- `δ<1.25³` — range: percent
  - Percentage of pixels where max(pred/gt, gt/pred) < 1.25³.

## Input / output format

**Input**: Sequential RGB images resized to 416×256 pixels.

**Output**: Per-frame 6-DoF ego-motion (translation and rotation) and a monocular depth map.

## Scoring recipe

```python
# Pose evaluation
t_errs, r_errs = [], []
for seg_len in [7, 14, 21, 28, 35]:
    for seg in trajectory_segments(length=seg_len):
        aligned_pred = align_7dof(seg.predicted_poses, seg.gt_poses)
        rel_err = compute_relative_error(aligned_pred, seg.gt_poses)
        t_errs.append(rel_err.t_err)
        r_errs.append(rel_err.r_err)
t_err = mean(t_errs)
r_err = mean(r_errs)

# Depth evaluation (Eigen split)
absrel = mean(abs(pred_depth - gt_depth) / gt_depth)
sqrel = mean((pred_depth - gt_depth)**2 / gt_depth)
rms = sqrt(mean((pred_depth - gt_depth)**2))
rmslog = sqrt(mean((log(pred_depth) - log(gt_depth))**2))
acc1 = mean((pred_depth / gt_depth < 1.25) & (gt_depth / pred_depth < 1.25))
acc2 = mean((pred_depth / gt_depth < 1.25**2) & (gt_depth / pred_depth < 1.25**2))
acc3 = mean((pred_depth / gt_depth < 1.25**3) & (gt_depth / pred_depth < 1.25**3))
```

## Common pitfalls

- Monocular pose predictions must be aligned to ground truth via 7-DoF optimization (minimizing ATE) before computing relative errors, as they lack real-world scale.
- Depth metrics are computed on the Eigen et al. crop/split of KITTI, not full-resolution images, which drastically changes absolute metric values.
- Relative pose errors are averaged across multiple segment lengths (7–35 m), not reported for a single fixed distance.

## Evidence (verbatim from paper)

> Thus, we show statistics for the relative translation and rotation error, divided by the distance travelled and averaged over the trajectory segments of lengths ${7,14,21,28,35}$ m over all sequences based on the shortest sequence. Here, the depth is evaluated on the Eigen et al. split of the raw KITTI dataset following the previous works.

## Citation

```bibtex
@misc{almalioglu2020unsupervised,
  title={Unsupervised Deep Persistent Monocular Visual Odometry and Depth Estimation in Extreme Environments},
  author={Almalioglu et al. (2020)},
  year={2020},
  note={arXiv:2011.00341}
}
```

- arXiv: 2011.00341