Mvpaint T2t Eval

---
name: mvpaint-t2t-eval
description: This evaluation probes a model's ability to generate high-quality, multi-view consistent 3D textures on arbitrary meshes conditioned on text instructions. It measures visual fidelity, distributional similarity to ground truth, and cross-view consistency through both automated generative metrics and human preference studies. Use when the user wants to benchmark on Objaverse T2T benchmark, GSO T2T benchmark, or asks about evaluating this task. Reports FID.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2411.02336
  bibtex_key: cheng2024mvpaint
  confidence: high
---

# mvpaint-t2t-eval

> MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D — Wei Cheng et al. (2024) (arXiv:2411.02336, 2024)

## What this evaluates

This evaluation probes a model's ability to generate high-quality, multi-view consistent 3D textures on arbitrary meshes conditioned on text instructions. It measures visual fidelity, distributional similarity to ground truth, and cross-view consistency through both automated generative metrics and human preference studies.

## Datasets

- **Objaverse T2T benchmark** — total 1000; splits: train (102000), test (1000)
- **GSO T2T benchmark** — total 1032; splits: test (1032)

## Metrics

- `FID` **(primary)** — range: other
  - Fréchet Inception Distance between the feature distributions of rendered images from generated textures versus ground truth textures. Lower is better.
- `KID` — range: other
  - Kernel Inception Distance, scaled by 10^3 in reported tables. Lower is better.
- `CLIP score` — range: other
  - Cosine similarity between image and text embeddings in CLIP space. Higher is better.
- `User Study (Overall/Seamless/Consistency)` — range: other
  - Average rating on a 1-5 Likert scale provided by 10 human participants evaluating rendered 3D meshes.

## Input / output format

**Input**: Text instruction (prompt) and a 3D mesh geometry (with or without UVs).

**Output**: A UV texture map or synchronized multi-view RGB images that are baked onto the input mesh.

## Scoring recipe

```python
def score(predictions, gold, text_prompt):
    gen_imgs = render_mesh(predictions.mesh, predictions.texture, viewpoints=16, elevation=15)
    gt_imgs = render_mesh(gold.mesh, gold.texture, viewpoints=16, elevation=15)
    fid = compute_fid(gen_imgs, gt_imgs)
    kid = compute_kid(gen_imgs, gt_imgs) * 1000
    clip = compute_clip_similarity(gen_imgs, text_prompt)
    return {'FID': fid, 'KID': kid, 'CLIP': clip}
```

## Common pitfalls

- Janus multi-view inconsistency artifacts can artificially inflate CLIP scores, so high CLIP alone does not guarantee visual quality or multi-view consistency.
- Reported KID values in tables are scaled by 10^3; raw KID values are orders of magnitude smaller and should not be compared directly to the table numbers.
- User study ratings use a 1-5 scale, not a percentage or 0-100 scale, which can cause misinterpretation of subjective performance.

## Evidence (verbatim from paper)

> After generating 3D textures, 512-resolution images of the mesh are rendered with the generated textures from 16 fixed viewpoints at the same elevation, i.e., 15° for fairness. Then, we compare the rendered images with the true image distribution generated using the ground truth textures. For a thorough evaluation, we use common generative metrics: Fréchet Inception Distance (FID), Kernel Inception Distance (KID), and CLIP score to assess image distribution, quality, and richness.

## Citation

```bibtex
@misc{cheng2024mvpaint,
  title={MVPaint: Synchronized Multi-View Diffusion for Painting Anything 3D},
  author={Wei Cheng et al. (2024)},
  year={2024},
  note={arXiv:2411.02336}
}
```

- arXiv: 2411.02336