Video To 4d Mesh Eval

---
name: video-to-4d-mesh-eval
description: Evaluates a model's ability to generate temporally consistent, animated 3D meshes from input videos. It probes per-frame geometric reconstruction accuracy, overall 4D sequence fidelity, and motion transfer quality while maintaining topology consistency across frames. Use when the user wants to benchmark on Objaverse, Consistent4D, DAVIS, or asks about evaluating this task. Reports CD-3D.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.16148
  bibtex_key: sabathier2026actionmesh
  confidence: high
---

# video-to-4d-mesh-eval

> ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusion — Sabathier et al. (2026) (arXiv:2601.16148, 2026)

## What this evaluates

Evaluates a model's ability to generate temporally consistent, animated 3D meshes from input videos. It probes per-frame geometric reconstruction accuracy, overall 4D sequence fidelity, and motion transfer quality while maintaining topology consistency across frames.

## Datasets

- **Objaverse** — total 32; splits: test (32)
- **Consistent4D** — total ?; splits: test (-1)
- **DAVIS** — total ?; splits: test (-1)

## Metrics

- `CD-3D` **(primary)** — range: other (lower is better)
  - Per-frame Chamfer Distance. Aligns the predicted mesh to the ground-truth mesh for each frame using Iterative Closest Point (ICP), then computes the bidirectional chamfer distance between corresponding points.
- `CD-4D` — range: other (lower is better)
  - 4D sequence Chamfer Distance. Aligns the entire predicted mesh sequence to the ground-truth sequence using a global ICP on the first frame, then averages the chamfer distance across all frames.
- `CD-M` — range: other (lower is better)
  - Motion Chamfer Distance. After global ICP alignment, establishes nearest-neighbor correspondences using the first frame's mesh. Computes the bidirectional chamfer distance between corresponding points across all remaining frames to quantify motion fidelity.

## Input / output format

**Input**: A video sequence (temporal frames of an object or scene)

**Output**: A sequence of 3D meshes (one per frame) with consistent topology representing the animated object

## Scoring recipe

```python
def compute_metrics(pred_meshes, gt_meshes):
    # CD-3D: per-frame ICP alignment & chamfer
    cd3d = sum(chamfer_distance(icp_align(p, g), g) for p, g in zip(pred_meshes, gt_meshes)) / len(pred_meshes)
    # CD-4D: global ICP on first frame, then average chamfer
    p_aligned = global_icp_align(pred_meshes, gt_meshes[0])
    cd4d = sum(chamfer_distance(p, g) for p, g in zip(p_aligned, gt_meshes)) / len(pred_meshes)
    # CD-M: motion fidelity via first-frame correspondences
    cd_m = sum(chamfer_distance(p_aligned[i], gt_meshes[i]) for i in range(1, len(pred_meshes))) / (len(pred_meshes) - 1)
    return cd3d, cd4d, cd_m
```

## Common pitfalls

- ICP alignment is sensitive to initialization and topology mismatches; improper alignment can artificially inflate CD scores.
- The primary quantitative benchmark (Objaverse subset) contains only 32 scenes and is in-house, limiting generalizability and cross-paper comparability.
- CD-M assumes stable point correspondences from the first frame; severe self-occlusion or non-rigid deformation can break this assumption.

## Evidence (verbatim from paper)

> First, we evaluate the per-frame 3D reconstruction quality by aligning, for each frame, the predicted mesh with ICP and computing the chamfer distance between ground-truth and prediction (CD-3D). Second, the 4D reconstruction quality is evaluated by aligning the predicted mesh sequence with a global ICP applied on the first mesh, and averaging the chamfer distance (CD-4D). Third, we evaluate motion fidelity with a chamfer-like distance tailored to quantify motion (CD-M). Specifically, after aligning the mesh sequence with a global ICP, we establish nearest neighbor correspondences using the first mesh. Then, for each remaining frame, we evaluate the bidirectional distance between corresponding points.

## Citation

```bibtex
@misc{sabathier2026actionmesh,
  title={ActionMesh: Animated 3D Mesh Generation with Temporal 3D Diffusion},
  author={Sabathier et al. (2026)},
  year={2026},
  note={arXiv:2601.16148}
}
```

- arXiv: 2601.16148