Scannerf Eval

---
name: scannerf-eval
description: Evaluates the rendering quality and novel-view synthesis capability of Neural Radiance Field (NeRF) methods on real-world inward-facing object scans. It probes how well models generalize to unseen camera poses when trained with varying image densities and localized acquisition patterns. Use when the user wants to benchmark on ScanNeRF, or asks about evaluating this task. Reports PSNR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2211.13762
  bibtex_key: deluigi2022scannerf
  confidence: high
---

# scannerf-eval

> ScanNeRF: a Scalable Benchmark for Neural Radiance Fields — De Luigi et al. (2022) (arXiv:2211.13762, 2022)

## What this evaluates

Evaluates the rendering quality and novel-view synthesis capability of Neural Radiance Field (NeRF) methods on real-world inward-facing object scans. It probes how well models generalize to unseen camera poses when trained with varying image densities and localized acquisition patterns.

## Datasets

- **ScanNeRF** — total 35; splits: train (-1), test (-1)

## Metrics

- `PSNR` **(primary)** — range: dB
  - Peak Signal-to-Noise Ratio between rendered images x_hat and ground truth test images x. Computed as -10 log10((x - x_hat)^2) per the paper (standard MSE-based PSNR). Results are averaged over 35 scanned objects.

## Input / output format

**Input**: RGB images with known camera poses for training and testing inward-facing object scans.

**Output**: Rendered RGB images at the test camera viewpoints.

## Scoring recipe

```python
def compute_psnr(rendered, ground_truth):
    mse = np.mean((rendered - ground_truth) ** 2)
    if mse == 0:
        return float('inf')
    return 10 * np.log10(255.0**2 / mse)
# Average PSNR across all test images and 35 objects
```

## Common pitfalls

- DVGO experiments were run at half resolution due to memory constraints, making direct quantitative comparison with full-resolution methods unfair.
- Instant-NGP training steps were reduced from 100K to 10K to save time, which may affect convergence compared to official defaults.
- Densely localized splits stress generalization to unseen hemispherical regions, but performance heavily depends on the specific train/test split combination.

## Evidence (verbatim from paper)

> Evaluation metrics. To assess the quality of the rendered images, we compute the Peak Signal Noise Ratio (PSNR) between the rendered $(\hat{x})$ and real test $(x)$ images: $$ \operatorname {P S N R} (\hat {x}, x) = - 1 0 \log_ {1 0} (x - \hat {x}) ^ {2}. \tag {7} $$

## Citation

```bibtex
@misc{deluigi2022scannerf,
  title={ScanNeRF: a Scalable Benchmark for Neural Radiance Fields},
  author={De Luigi et al. (2022)},
  year={2022},
  note={arXiv:2211.13762}
}
```

- arXiv: 2211.13762