Gen Nerf Eval

---
name: gen-nerf-eval
description: Evaluates the rendering quality and computational efficiency of a generalizable Neural Radiance Field (NeRF) model for novel view synthesis. It measures how accurately the model reconstructs unseen scenes from a few source views, balancing image fidelity against computational cost and hardware throughput. Use when the user wants to benchmark on NeRF Synthetic, LLFF, DeepVoxels, or asks about evaluating this task. Reports PSNR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2304.11842
  bibtex_key: fu2023gennerf
  confidence: high
---

# gen-nerf-eval

> Gen-NeRF: Efficient and Generalizable Neural Radiance Fields via Algorithm-Hardware Co-Design — Yonggan Fu et al. (2023) (arXiv:2304.11842, 2023)

## What this evaluates

Evaluates the rendering quality and computational efficiency of a generalizable Neural Radiance Field (NeRF) model for novel view synthesis. It measures how accurately the model reconstructs unseen scenes from a few source views, balancing image fidelity against computational cost and hardware throughput.

## Datasets

- **NeRF Synthetic** — total ?; splits: test (-1)
- **LLFF** — total ?; splits: test (-1)
- **DeepVoxels** — total ?; splits: test (-1)

## Metrics

- `PSNR` **(primary)** — range: other
  - Peak Signal-to-Noise Ratio, calculated as 10 * log10(255^2 / MSE) between rendered and ground-truth images. Higher values indicate better reconstruction quality.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity, a deep learning-based metric that measures perceptual difference between images. Lower values indicate higher perceptual similarity.
- `MFLOPs/pixel` — range: other
  - Million Floating Point Operations per pixel, measuring the computational cost of rendering a single pixel. Lower values indicate higher efficiency.
- `FPS` — range: other
  - Frames Per Second, measuring the rendering throughput of the hardware accelerator. Higher values indicate faster real-time performance.

## Input / output format

**Input**: A set of source RGB images (typically 4-10 views) and their corresponding camera poses for a target scene.

**Output**: A single rendered RGB image of a novel viewpoint for the target scene.

## Scoring recipe

```python
def compute_metrics(rendered_img, gt_img):
    mse = np.mean((rendered_img - gt_img) ** 2)
    psnr = 10 * np.log10(255.0**2 / mse)
    lpips_val = lpips_model(rendered_img, gt_img)
    return psnr, lpips_val

# Efficiency metrics are computed separately:
# MFLOPs/pixel = total_model_flops / (image_width * image_height)
# FPS = 1 / (latency_per_frame_in_seconds)
```

## Common pitfalls

- Hardware performance metrics (FPS, area, power) are derived from cycle-accurate simulation and gate-level synthesis, not physical silicon measurements.
- MFLOPs/pixel values are highly dependent on the coarse-then-focus sampling strategy and the number of source views used; comparing across different source view counts without normalization is misleading.
- The evaluation focuses on zero-shot generalizable NeRF; per-scene finetuning results are reported separately and should not be conflated with the base generalizable model.

## Evidence (verbatim from paper)

> As shown in rows 2-4 of Tab.[2], we can observe that (1) removing the ray transformer leads to a significant PSNR drop due to erroneous density estimation, and (2) integrating Ray-Mixer results in considerably better reconstruction accuracy over that w/o Ray-Mixer, e.g., a 3.37 PSNR improvement on average across four scenes from LLFF.

## Citation

```bibtex
@misc{fu2023gennerf,
  title={Gen-NeRF: Efficient and Generalizable Neural Radiance Fields via Algorithm-Hardware Co-Design},
  author={Yonggan Fu et al. (2023)},
  year={2023},
  note={arXiv:2304.11842}
}
```

- arXiv: 2304.11842