Wildrayzer Nvs Eval

---
name: wildrayzer-nvs-eval
description: Evaluates novel view synthesis and motion mask estimation in dynamic environments where both camera and objects move. It probes a model's ability to remove transient objects, complete occluded backgrounds, and preserve scene geometry from sparse input views without 3D supervision or ground-truth poses. Use when the user wants to benchmark on D-RE10K-Mask, D-RE10K-iPhone, or asks about evaluating this task. Reports PSNR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.10716
  bibtex_key: chen2026wildrayzer
  confidence: high
---

# wildrayzer-nvs-eval

> WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments — Chen et al. (2026) (arXiv:2601.10716, 2026)

## What this evaluates

Evaluates novel view synthesis and motion mask estimation in dynamic environments where both camera and objects move. It probes a model's ability to remove transient objects, complete occluded backgrounds, and preserve scene geometry from sparse input views without 3D supervision or ground-truth poses.

## Datasets

- **D-RE10K-Mask** — total 74; splits: test (-1)
- **D-RE10K-iPhone** — total 50; splits: test (-1)

## Metrics

- `PSNR` **(primary)** — range: dB
  - Peak Signal-to-Noise Ratio, computed as 10 * log10(MAX^2 / MSE) between predicted and ground truth images.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index, measuring luminance, contrast, and structure similarity between images.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity, using deep features to measure perceptual distance.
- `mIoU` — range: [0, 1]
  - Mean Intersection over Union for motion mask segmentation against human-verified annotations.
- `Recall` — range: [0, 1]
  - True positive rate for motion mask pixels against ground truth annotations.

## Input / output format

**Input**: 2–4 input images (sparse views) for scene and pose estimation; 6 target views for evaluation.

**Output**: Synthesized novel view images and/or binary motion masks for transient regions.

## Scoring recipe

```python
def compute_nvs_metrics(pred, gt):
    mse = np.mean((pred - gt) ** 2)
    psnr = 10 * np.log10(255**2 / mse)
    ssim = compute_ssim(pred, gt)
    lpips = compute_lpips(pred, gt)
    return psnr, ssim, lpips

def compute_mask_metrics(pred_mask, gt_mask):
    intersection = np.logical_and(pred_mask, gt_mask).sum()
    union = np.logical_or(pred_mask, gt_mask).sum()
    miou = intersection / union if union > 0 else 0.0
    recall = intersection / gt_mask.sum() if gt_mask.sum() > 0 else 0.0
    return miou, recall
```

## Common pitfalls

- Full-image PSNR can be misleading because transient objects typically cover only small image areas, masking background reconstruction quality.
- Off-the-shelf motion estimators are tuned on the D-RE10K validation set but must be evaluated without modification on the iPhone benchmark to test cross-dataset generalization.
- Sparse-view settings (2–4 inputs) cause over-masking or diffuse boundaries in baseline methods, requiring learned cross-view completion rather than simple token dropping.

## Evidence (verbatim from paper)

> We adopt a sparse-view NVS protocol: two, three or four input images are used for pose and scene estimation, and six target views are used for evaluation. We report static-region metrics for D-RE10K-Mask and report full-image metrics for D-RE10K-iPhone. Image quality is evaluated using PSNR, SSIM, and LPIPS. Furthermore, when evaluating motion mask, motion-mask accuracy is measured using mIoU and Recall against human verified annotations.

## Citation

```bibtex
@misc{chen2026wildrayzer,
  title={WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments},
  author={Chen et al. (2026)},
  year={2026},
  note={arXiv:2601.10716}
}
```

- arXiv: 2601.10716