Gt23d Bench Eval

---
name: gt23d-bench-eval
description: Evaluates the quality and alignment of generated 3D assets against text prompts across multiple dimensions, including textual alignment, texture fidelity, geometry correctness, and multi-view consistency. It measures how well automated metrics correlate with human preferences to provide a reliable assessment of general text-to-3D generation methods. Use when the user wants to benchmark on GT23D-Bench, or asks about evaluating this task. Reports Texture Fidelity.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2412.09997
  bibtex_key: su2024gt23dbench
  confidence: high
---

# gt23d-bench-eval

> GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmark — Sitong Su et al. (2024) (arXiv:2412.09997, 2024)

## What this evaluates

Evaluates the quality and alignment of generated 3D assets against text prompts across multiple dimensions, including textual alignment, texture fidelity, geometry correctness, and multi-view consistency. It measures how well automated metrics correlate with human preferences to provide a reliable assessment of general text-to-3D generation methods.

## Datasets

- **GT23D-Bench** — total 400000; splits: test (380)

## Metrics

- `Textual-PointCloud` — range: [0, 1] or arbitrary score scale
  - Measures alignment between the text prompt and the generated point cloud representation using cross-modal matching.
- `Textual-MultiView` — range: [0, 1] or arbitrary score scale
  - Measures alignment between the text prompt and the generated multi-view images.
- `Textual-Attribute` — range: [0, 1] or arbitrary score scale
  - Measures alignment between the text prompt and specific semantic attributes of the generated 3D object.
- `Texture Fidelity` **(primary)** — range: [0, 1] or arbitrary score scale
  - Evaluates the richness, clarity, and visual quality of textures on the generated 3D surface. Comprises sub-metrics CC, TR, and Mv-IQ.
- `Geometry Correctness` — range: [0, 1] or arbitrary score scale
  - Assesses the structural accuracy, shape completeness, and geometric validity of the generated 3D asset. Comprises sub-metrics 3D-Ali, Shape-C, and Geo-V.
- `Multi-View Consistency` — range: [0, 1] or arbitrary score scale
  - Measures how consistent the generated 3D asset appears across different camera viewpoints, penalizing view-inconsistent artifacts.

## Input / output format

**Input**: Text prompt (caption) describing a 3D object or scene.

**Output**: 3D representation (e.g., point cloud, multi-view images, or implicit representation).

## Scoring recipe

```python
def evaluate_generation(prompts, generated_assets, human_scores=None):
    scores = {}
    for prompt, asset in zip(prompts, generated_assets):
        scores[prompt] = {
            'Textual-PointCloud': compute_alignment(asset, prompt, modality='pointcloud'),
            'Textual-MultiView': compute_alignment(asset, prompt, modality='multiview'),
            'Textual-Attribute': compute_alignment(asset, prompt, modality='attribute'),
            'Texture Fidelity': compute_texture_quality(asset),
            'Geometry Correctness': compute_geometry_quality(asset),
            'Multi-View Consistency': compute_consistency(asset)
        }
    if human_scores is not None:
        return {dim: pearson_spearman_kendall(metric_scores, human_scores) for dim, metric_scores in scores.items()}
    return scores
```

## Common pitfalls

- Methods that only generate multi-view images (e.g., MVDream) cannot be evaluated on point-cloud-based metrics like Textual-PointCloud or 3D-Ali, leading to missing scores.
- Aesthetic scores may favor visually appealing but geometrically inconsistent outputs, leading to misleading quality assessments.
- Correlation metrics (Pearson/Spearman/Kendall) measure human alignment reliability, not absolute generation quality.

## Evidence (verbatim from paper)

> Multi-view image-based methods like MVDream achieve high visual quality in single-view images (high Texture Fidelity score) but lack 3D supervision, resulting in poor 3D quality (low Multi-View Consistency score).

## Citation

```bibtex
@misc{su2024gt23dbench,
  title={GT23D-Bench: A Comprehensive General Text-to-3D Generation Benchmark},
  author={Sitong Su et al. (2024)},
  year={2024},
  note={arXiv:2412.09997}
}
```

- arXiv: 2412.09997