Mv Adapter Multi View Eval

---
name: mv-adapter-multi-view-eval
description: Evaluates a diffusion adapter's ability to generate geometrically consistent multi-view images conditioned on text prompts or reference images with camera parameters. It measures visual fidelity, image-text alignment, and multi-view structural similarity against ground-truth 3D scans. Use when the user wants to benchmark on Objaverse, Google Scanned Objects (GSO), or asks about evaluating this task. Reports FID.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2412.03632
  bibtex_key: huang2024mvadapter
  confidence: high
---

# mv-adapter-multi-view-eval

> MV-Adapter: Multi-view Consistent Image Generation Made Easy — Huang et al. (2024) (arXiv:2412.03632, 2024)

## What this evaluates

Evaluates a diffusion adapter's ability to generate geometrically consistent multi-view images conditioned on text prompts or reference images with camera parameters. It measures visual fidelity, image-text alignment, and multi-view structural similarity against ground-truth 3D scans.

## Datasets

- **Objaverse** — total ?; splits: (unstated)
- **Google Scanned Objects (GSO)** — total ?; splits: (unstated)

## Metrics

- `FID` **(primary)** — range: [0, ∞)
  - Fréchet Inception Distance. Computes the Fréchet distance between multivariate Gaussians fitted to feature representations of real and generated images.
- `PSNR` — range: [0, ∞) dB
  - Peak Signal-to-Noise Ratio. Measures pixel-level reconstruction fidelity in decibels based on the mean squared error between reference and generated images.
- `SSIM` — range: [0, 1]
  - Structural Similarity Index. Quantifies perceived change in structural information, luminance, and contrast between two images.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity. Computes perceptual distance using deep features from a pre-trained network (e.g., VGG or AlexNet).
- `CLIP Score` — range: [-1, 1]
  - Cosine similarity between the text prompt embedding and the image embedding in the CLIP latent space.
- `IS` — range: [0, ∞)
  - Inception Score. Measures image quality and diversity by training a classifier on generated images and computing the KL divergence between conditional and marginal label distributions.
- `KID` — range: [0, ∞)
  - Kernel Inception Distance. Estimates the distance between real and generated image distributions using polynomial kernels on Inception features.

## Input / output format

**Input**: Text prompt + camera pose parameters (text-to-multiview); Reference image + camera pose/depth map (image-to-multiview)

**Output**: A set of multi-view images (typically 4 or 8 views) corresponding to specified camera poses

## Scoring recipe

```python
def evaluate(pred_imgs, gt_imgs, text_prompts=None):
    fid = frechet_inception_distance(pred_imgs, gt_imgs)
    psnr = peak_signal_noise_ratio(gt_imgs, pred_imgs)
    ssim = structural_similarity(gt_imgs, pred_imgs)
    lpips = learned_perceptual_image_patch_similarity(pred_imgs, gt_imgs)
    clip = cosine_similarity(text_embeds, image_embeds) if text_prompts else None
    return {'FID': fid, 'PSNR': psnr, 'SSIM': ssim, 'LPIPS': lpips, 'CLIP': clip}
```

## Common pitfalls

- Standard 2D metrics (FID, PSNR, LPIPS) are computed per-view and do not directly quantify 3D geometric consistency or cross-view alignment.
- The exact subset of Objaverse used for training and the 1,000-prompt test split are not specified, hindering exact reproduction.
- Inference time is reported on a single RTX 4090 but varies with batch size and resolution, limiting fair cross-method timing comparisons.

## Evidence (verbatim from paper)

> Table 3: Quantitative comparison on 3D texture generation. FID and KID (×10−4) are evaluated on multi-view renderings. Our models achieves best texture quality with faster inference.

## Citation

```bibtex
@misc{huang2024mvadapter,
  title={MV-Adapter: Multi-view Consistent Image Generation Made Easy},
  author={Huang et al. (2024)},
  year={2024},
  note={arXiv:2412.03632}
}
```

- arXiv: 2412.03632