Nerf View Synthesis Eval

---
name: nerf-view-synthesis-eval
description: Evaluates the ability of a neural radiance field to synthesize photorealistic novel views of 3D scenes from a sparse set of input images. It probes geometric reconstruction fidelity, appearance modeling (including non-Lambertian materials), and multi-view consistency across synthetic and real-world captures. Use when the user wants to benchmark on Diffuse Synthetic 360° (DeepVoxels), Realistic Synthetic 360°, Real Forward-Facing, or asks about evaluating this task. Reports PSNR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2003.08934
  bibtex_key: [REDACTED]
  confidence: high
---

# nerf-view-synthesis-eval

> NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis — Ben Mildenhall et al. (arXiv:2003.08934, 2020)

## What this evaluates

Evaluates the ability of a neural radiance field to synthesize photorealistic novel views of 3D scenes from a sparse set of input images. It probes geometric reconstruction fidelity, appearance modeling (including non-Lambertian materials), and multi-view consistency across synthetic and real-world captures.

## Datasets

- **Diffuse Synthetic 360° (DeepVoxels)** — total 1479; splits: train (479), test (1000)
- **Realistic Synthetic 360°** — total 300; splits: train (100), test (200)
- **Real Forward-Facing** — total ?; splits: train (-1), test (-1)

## Metrics

- `PSNR` **(primary)** — range: dB
  - Peak Signal-to-Noise Ratio measured in decibels (dB). Computed as 10 * log10(255^2 / MSE) between the rendered and ground truth images.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index measuring perceived structural changes between two images. Ranges from 0 to 1, where 1 indicates identical structure.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity. Uses a pre-trained network to compare feature activations of image patches. Lower values indicate higher perceptual similarity.

## Input / output format

**Input**: A set of input images with known camera poses (intrinsics and extrinsics) capturing a scene from multiple viewpoints.

**Output**: A rendered RGB image at a novel camera viewpoint, generated via differentiable volume rendering.

## Scoring recipe

```python
def evaluate(rendered_img, gt_img):
    mse = np.mean((rendered_img - gt_img) ** 2)
    psnr = 10 * np.log10(255**2 / mse)
    ssim = compute_ssim(rendered_img, gt_img)
    lpips = compute_lpips(rendered_img, gt_img)
    return {'PSNR': psnr, 'SSIM': ssim, 'LPIPS': lpips}
```

## Common pitfalls

- LPIPS is lower-is-better, while PSNR and SSIM are higher-is-better; confusing the direction leads to incorrect performance claims.
- The evaluation requires training a separate network for each individual scene, so metrics are averaged over independently trained models rather than a single generalizable network.
- Camera pose accuracy is critical; errors in input poses directly degrade all metrics on test views, making pose estimation quality a hidden confounder.

## Evidence (verbatim from paper)

> Table 1: Our method quantitatively outperforms prior work on datasets of both synthetic and real images. We report PSNR/SSIM (higher is better) and LPIPS [50] (lower is better).

## Citation

```bibtex
@misc{mildenhall2020nerf,
  title={NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis},
  author={Ben Mildenhall et al.},
  year={2020},
  note={arXiv:2003.08934}
}
```

- arXiv: 2003.08934