Image Denoising Eval

---
name: image-denoising-eval
description: Evaluates the ability of generative models with discrete latents to restore clean images from noisy inputs using a zero-shot, patch-based variational optimization framework. Use when the user wants to benchmark on Standard denoising benchmarks (e.g., House image), or asks about evaluating this task. Reports PSNR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2012.12294
  bibtex_key: drefs2020evolutionary
  confidence: high
---

# image-denoising-eval

> Evolutionary Variational Optimization of Generative Models — Drefs et al. (2020) (arXiv:2012.12294, 2020)

## What this evaluates

Evaluates the ability of generative models with discrete latents to restore clean images from noisy inputs using a zero-shot, patch-based variational optimization framework.

## Datasets

- **Standard denoising benchmarks (e.g., House image)** — total ?; splits: test (-1)

## Metrics

- `PSNR` **(primary)** — range: dB
  - Peak Signal-to-Noise Ratio, calculated as 10 * log10(MAX_I^2 / MSE), where MAX_I is the maximum possible pixel value (typically 255 for 8-bit images) and MSE is the mean squared error between the ground-truth clean image and the reconstructed image.

## Input / output format

**Input**: Noisy image patches extracted via a sliding window from a single corrupted image. Patches are processed individually without external training data or known noise levels.

**Output**: Reconstructed clean image pixels, obtained by taking expectations over the posterior predictive distribution for each patch and aggregating overlapping pixel estimates via weighted averaging.

## Scoring recipe

```python
def compute_psnr(clean_img, reconstructed_img):
    mse = np.mean((clean_img - reconstructed_img) ** 2)
    if mse == 0:
        return float('inf')
    max_pixel = 255.0  # Assuming 8-bit images
    psnr = 10 * np.log10((max_pixel ** 2) / mse)
    return psnr
```

## Common pitfalls

- The variational lower bound (learning objective) is not perfectly correlated with PSNR at convergence; the run with the highest bound may not yield the highest PSNR.
- Algorithms require varying amounts of prior knowledge (e.g., noise level, clean training data, specific missing patterns), making direct comparisons difficult without strict categorization.
- Patch-based reconstruction requires careful handling of overlapping regions; incorrect weighting or aggregation of pixel estimates will distort the final PSNR.

## Evidence (verbatim from paper)

> Finally, we were interested in how well EBSC and ES3C could denoise a given image in terms of the standard peak-signal-to-noise ratio (PSNR) evaluation measure. ... While the PSNR measure requires access to the clean target image, the learning objective can be evaluated without such ground-truth knowledge.

## Citation

```bibtex
@misc{drefs2020evolutionary,
  title={Evolutionary Variational Optimization of Generative Models},
  author={Drefs et al. (2020)},
  year={2020},
  note={arXiv:2012.12294}
}
```

- arXiv: 2012.12294