Df3dv 1k Eval

---
name: df3dv-1k-eval
description: Evaluates the robustness of distractor-free novel view synthesis methods against large-scale, diverse distractor scenarios. It measures how well radiance field and 3D Gaussian Splatting models can reconstruct clean 3D scenes from cluttered or dynamically changing inputs without degrading static background quality. Use when the user wants to benchmark on DF3DV-1K, DF3DV-41, or asks about evaluating this task. Reports PSNR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.13416
  bibtex_key: lu2026df3dv1k
  confidence: high
---

# df3dv-1k-eval

> DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis — Lu et al. (2026) (arXiv:2604.13416, 2026)

## What this evaluates

Evaluates the robustness of distractor-free novel view synthesis methods against large-scale, diverse distractor scenarios. It measures how well radiance field and 3D Gaussian Splatting models can reconstruct clean 3D scenes from cluttered or dynamically changing inputs without degrading static background quality.

## Datasets

- **DF3DV-1K** — total 1048; splits: test (-1)
- **DF3DV-41** — total 41; splits: test (-1)

## Metrics

- `PSNR` **(primary)** — range: dB
  - Peak Signal-to-Noise Ratio in decibels, calculated as 10 * log10(MAX^2 / MSE), where MAX is the maximum possible pixel value (typically 1.0 for normalized images) and MSE is the mean squared error between predicted and ground truth images.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index, measuring perceived changes in structural information, luminance, and contrast between two images. Values range from -1 to 1, with 1 indicating identical images.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity, a perceptual metric based on deep network features that correlates better with human judgment than pixel-wise metrics. Lower values indicate higher perceptual similarity.

## Input / output format

**Input**: Cluttered/distorted image sequences containing distractors (e.g., dynamic objects, fluid phenomena, nighttime conditions) used to train or evaluate radiance field reconstruction.

**Output**: Rendered novel-view RGB images from the reconstructed 3D scene or enhanced by the 2D diffusion model.

## Scoring recipe

```python
def compute_metrics(pred, gold):
    mse = np.mean((pred - gold) ** 2)
    psnr = 10 * np.log10(1.0 / mse)
    ssim = compute_ssim(pred, gold)
    lpips = compute_lpips(pred, gold)
    return {'PSNR': psnr, 'SSIM': ssim, 'LPIPS': lpips}
```

## Common pitfalls

- Methods relying on semantic features or fixed thresholds fail on semantically similar distractors or low-light scenes.
- Qualitative 'zoom-in' comparisons can mask performance gaps; the benchmark requires direct, unzoomed visual comparison.
- Fine-tuning enhancers on smaller prior benchmarks (e.g., RobustNeRF) severely degrades generalization on DF3DV-1K/41.

## Evidence (verbatim from paper)

> DI2FIX improves the average rendering quality of 3DGS*[kerbl20233d]* and distractor-free radiance-field methods, particularly in PSNR and LPIPS, achieving a 0.96 dB PSNR gain and a 0.057 reduction in LPIPS.

## Citation

```bibtex
@misc{lu2026df3dv1k,
  title={DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis},
  author={Lu et al. (2026)},
  year={2026},
  note={arXiv:2604.13416}
}
```

- arXiv: 2604.13416