Dynamic Urban View Synthesis Eval

---
name: dynamic-urban-view-synthesis-eval
description: Evaluates novel view synthesis and image reconstruction quality in dynamic urban environments containing fast-moving objects and varying environmental conditions. It measures how well a model can render unseen viewpoints and reconstruct training views while handling dynamic geometry and pose drift. Use when the user wants to benchmark on Argoverse 2, KITTI, VKITTI2, or asks about evaluating this task. Reports PSNR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2404.00168
  bibtex_key: fischer2024multilevelneuralscenegraphs
  confidence: high
---

# dynamic-urban-view-synthesis-eval

> Multi-Level Neural Scene Graphs for Dynamic Urban Environments — Tobias Fischer et al. (arXiv:2404.00168, 2024)

## What this evaluates

Evaluates novel view synthesis and image reconstruction quality in dynamic urban environments containing fast-moving objects and varying environmental conditions. It measures how well a model can render unseen viewpoints and reconstruct training views while handling dynamic geometry and pose drift.

## Datasets

- **Argoverse 2** — total ?; splits: test (-1)
- **KITTI** — total ?; splits: test (-1)
- **VKITTI2** — total ?; splits: test (-1)

## Metrics

- `PSNR` **(primary)** — range: dB
  - Peak Signal-to-Noise Ratio in decibels. Computed as 10·log₁₀(MAX²/MSE), where MAX is the maximum possible pixel value (255 for 8-bit) and MSE is the mean squared error between predicted and ground truth images.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index. Measures perceived change in structural information, luminance, and contrast between two images. Values range from -1 to 1, with 1 indicating identical images.
- `LPIPS (AlexNet)` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity. Computes the L2 distance between deep features (AlexNet) of the reference and predicted images. Lower values indicate higher perceptual similarity.

## Input / output format

**Input**: Multi-view image sequences (e.g., 7 ring-camera images per time step) with ground-truth camera poses and 3D bounding boxes for dynamic objects.

**Output**: Synthesized novel-view RGB images and depth maps.

## Scoring recipe

```python
def compute_metrics(pred_img, gt_img):
    mse = np.mean((pred_img - gt_img) ** 2)
    psnr = 10 * np.log10(255**2 / mse) if mse > 0 else 0
    ssim = structural_similarity(pred_img, gt_img, multichannel=True)
    lpips = perceptual_loss(pred_img, gt_img, model='alexnet')
    return psnr, ssim, lpips
```

## Common pitfalls

- Pose drift heavily penalizes PSNR; hierarchical pose optimization is required to avoid misleadingly low scores.
- LPIPS directionality: lower values indicate better perceptual quality, opposite to PSNR and SSIM.
- Data splits are non-standard; Argoverse 2 uses every 10th frame held out, while KITTI/VKITTI2 follow the specific protocol from SUDS [61].

## Evidence (verbatim from paper)

> Following [61], we measure image synthesis quality with PSNR, SSIM [62], and LPIPS (AlexNet) [72]. To evaluate against competing methods on our proposed benchmark on Argoverse 2 [64], we hold out every 10th sample in uniform time intervals where a sample corresponds to seven ring-camera images. To compare our methods against prior art on KITTI [15] and VKITTI2 [4], we follow the experimental protocol and data splits in [61].

## Citation

```bibtex
@misc{fischer2024multilevelneuralscenegraphs,
  title={Multi-Level Neural Scene Graphs for Dynamic Urban Environments},
  author={Tobias Fischer et al.},
  year={2024},
  note={arXiv:2404.00168}
}
```

- arXiv: 2404.00168