Muvienefr Eval

---
name: muvienefr-eval
description: Evaluates a model's ability to perform multi-task view synthesis by predicting multiple scene properties (RGB, surface normals, shading, edges, keypoints, semantic segmentation) from novel viewpoints, given a set of source-view annotations and camera poses. Use when the user wants to benchmark on Replica, SceneNet RGB-D, or asks about evaluating this task. Reports RGB.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2309.17450
  bibtex_key: zheng2023muvienefr
  confidence: high
---

# muvienefr-eval

> Multi-task View Synthesis with Neural Radiance Fields — Zheng et al. (2023) (arXiv:2309.17450, 2023)

## What this evaluates

Evaluates a model's ability to perform multi-task view synthesis by predicting multiple scene properties (RGB, surface normals, shading, edges, keypoints, semantic segmentation) from novel viewpoints, given a set of source-view annotations and camera poses.

## Datasets

- **Replica** — total ?; splits: training (-1), testing (-1)
- **SceneNet RGB-D** — total ?; splits: training (-1), testing (-1)

## Metrics

- `RGB` **(primary)** — range: dB
  - Peak Signal-to-Noise Ratio (PSNR) in dB. Higher is better (↑).
- `SN` — range: other
  - Surface Normal L1 error. Lower is better (↓).
- `SH` — range: other
  - Shading L1 error. Lower is better (↓).
- `ED` — range: other
  - Edge Detection L1 error. Lower is better (↓).
- `KP` — range: other
  - Keypoint L1 error. Lower is better (↓).
- `SL` — range: [0, 1]
  - Semantic Labeling mIoU. Higher is better (↑).

## Input / output format

**Input**: A set of V source-view task annotations (RGB and K other task annotations) with corresponding camera poses, plus a target camera pose.

**Output**: Predicted task annotations (RGB, surface normals, shading, edge, keypoint, semantic segmentation) for the target view.

## Scoring recipe

```python
Compute per-pixel or per-image metrics between predicted and ground-truth annotations:
- RGB: PSNR = 10 * log10(255^2 / MSE)
- SN, SH, ED, KP: L1 loss = mean(|pred - gt|)
- SL: mIoU = mean(class IoU)
Average across all test scenes and views.
```

## Common pitfalls

- Confusing Inference Setting I (source-view annotations available at test time) with Setting II (RGB-only input at test time).
- Evaluating on seen training scenes instead of strictly unseen novel scenes, which inflates performance.
- Not normalizing or aligning predictions and ground truth for geometric tasks (SN, SH, ED, KP) before computing L1 error.

## Evidence (verbatim from paper)

> During the evaluation, $\Phi$ is supposed to be generalized to novel scenes that are not seen during training. Table 1: Averaged performance of MuvieNeRF on Replica and SceneNet RGB-D datasets on both training scenes and testing scenes. Metrics reported: RGB (↑), SN (↓), SH (↓), ED (↓), KP (↓), SL (↑).

## Citation

```bibtex
@misc{zheng2023muvienefr,
  title={Multi-task View Synthesis with Neural Radiance Fields},
  author={Zheng et al. (2023)},
  year={2023},
  note={arXiv:2309.17450}
}
```

- arXiv: 2309.17450