Synthetic4relight Eval

---
name: synthetic4relight-eval
description: Evaluates a 3D scene representation's capability for novel view synthesis, relighting, and inverse rendering (estimating diffuse albedo and roughness) from posed RGB images. Use when the user wants to benchmark on Synthetic4Relight, or asks about evaluating this task. Reports PSNR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2412.15867
  bibtex_key: gu2024irgs
  confidence: high
---

# synthetic4relight-eval

> IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing — Gu et al. (2024) (arXiv:2412.15867, 2024)

## What this evaluates

Evaluates a 3D scene representation's capability for novel view synthesis, relighting, and inverse rendering (estimating diffuse albedo and roughness) from posed RGB images.

## Datasets

- **Synthetic4Relight** — total ?; splits: test (-1)

## Metrics

- `PSNR` **(primary)** — range: dB
  - Peak Signal-to-Noise Ratio, computed in decibels (dB) between rendered and ground truth images. Higher values indicate better reconstruction quality.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index, measures perceptual structural similarity between images. Ranges from 0 to 1, where higher is better.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity, measures perceptual distance using deep network features. Lower values indicate better perceptual quality.
- `MSE` — range: [0, inf]
  - Mean Squared Error between estimated and ground truth material maps (e.g., roughness). Lower values indicate more accurate material estimation.

## Input / output format

**Input**: A set of posed RGB images of a scene, along with object masks.

**Output**: Rendered RGB images for novel views and relighting scenarios, plus estimated material maps (albedo, roughness) and lighting parameters.

## Scoring recipe

```python
def compute_metrics(pred, gt):
    mse = np.mean((pred - gt) ** 2)
    psnr = 10 * np.log10(1.0 / mse)
    ssim = compute_ssim(pred, gt)
    lpips = compute_lpips(pred, gt)
    return psnr, ssim, lpips

# For material maps (e.g., roughness):
# mse_material = np.mean((pred_material - gt_material) ** 2)
```

## Common pitfalls

- Relighting and novel view synthesis are evaluated under different lighting conditions; metrics must be computed separately for each task and not averaged together.
- Material estimation (albedo/roughness) requires disentangling appearance from lighting, so ground truth maps are compared directly rather than rendered appearance.
- LPIPS measures perceptual distance, meaning lower values indicate better quality, which is the opposite convention of PSNR and SSIM.

## Evidence (verbatim from paper)

> Table 1: Quantitative comparison on the Synthetic4Relight dataset[[43]]. A higher intensity of the red color signifies a better result.

|  | Novel view synthesis | | | Relighting | | | Albedo | | | Roughness | Time |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | PSNR↑ | SSIM↑ | LPIPS↓ | MSE↓ | Time (hours) |

## Citation

```bibtex
@misc{gu2024irgs,
  title={IRGS: Inter-Reflective Gaussian Splatting with 2D Gaussian Ray Tracing},
  author={Gu et al. (2024)},
  year={2024},
  note={arXiv:2412.15867}
}
```

- arXiv: 2412.15867