Holistic Motion2d Eval

---
name: holistic-motion2d-eval
description: Evaluates the quality, text-motion alignment, and diversity of generated 2D whole-body human motion sequences conditioned on text prompts. It probes the model's ability to capture fine-grained spatial-temporal dynamics and handle occlusions or noisy 2D pose data. Use when the user wants to benchmark on Holistic-Motion2D, or asks about evaluating this task. Reports FID.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2406.11253
  bibtex_key: wang2024holisticmotion2d
  confidence: high
---

# holistic-motion2d-eval

> Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space — Wang et al. (2024) (arXiv:2406.11253, 2024)

## What this evaluates

Evaluates the quality, text-motion alignment, and diversity of generated 2D whole-body human motion sequences conditioned on text prompts. It probes the model's ability to capture fine-grained spatial-temporal dynamics and handle occlusions or noisy 2D pose data.

## Datasets

- **Holistic-Motion2D** — total ?; splits: D1 (-1), D2 (-1), D3 (-1)

## Metrics

- `FID` **(primary)** — range: other
  - Fréchet Inception Distance measuring the distance between feature distributions of real and generated motion sequences. Lower values indicate higher fidelity and realism.
- `R-Precision` — range: [0, 1]
  - Retrieval precision: given a text prompt, the model retrieves top-k motion sequences from a candidate pool. R-Precision indicates whether the ground-truth motion is ranked in the top-k results (typically k=3). Higher is better.
- `Top1` — range: [0, 1]
  - Retrieval precision at k=1. Indicates whether the ground-truth motion is ranked first among candidates. Higher is better.
- `Top3` — range: [0, 1]
  - Retrieval precision at k=3. Indicates whether the ground-truth motion is ranked in the top-3 among candidates. Higher is better.
- `Diversity` — range: other
  - Average pairwise distance between generated motion sequences, measuring the variety of outputs. Higher is better.

## Input / output format

**Input**: Text prompt describing a human action or motion.

**Output**: 2D whole-body human motion sequence (keypoint coordinates over time).

## Scoring recipe

```python
def compute_metrics(real_motions, generated_motions, prompts, candidates, ground_truth_indices):
    # FID
    real_feats = feature_extractor(real_motions)
    gen_feats = feature_extractor(generated_motions)
    fid = frechet_distance(real_feats.mean(), real_feats.cov(), gen_feats.mean(), gen_feats.cov())
    
    # Retrieval (Top1/Top3/R-Precision)
    text_feats = text_encoder(prompts)
    motion_feats = motion_encoder(candidates)
    scores = cosine_similarity(text_feats, motion_feats)
    top_k = argsort(scores, k=3)
    r_prec = mean([1.0 if gt in top_k[i] else 0.0 for i, gt in enumerate(ground_truth_indices)])
    
    # Diversity
    diversity = mean_pairwise_distance(generated_motions)
    return {'FID': fid, 'Top1': r_prec_at_1, 'Top3': r_prec_at_3, 'Diversity': diversity}
```

## Common pitfalls

- FID requires a fixed feature extractor and consistent motion representation; mismatched preprocessing or dimensionality skews results.
- R-Precision/Top1/Top3 scores are highly sensitive to the retrieval pool size, quality, and whether ground-truth motions are properly excluded from candidates during retrieval.
- Diversity is not explicitly defined in the paper; implementations vary (e.g., mean pairwise distance vs. variance), making cross-paper comparisons difficult.
- Low-confidence keypoints and occlusions in 2D data can artificially degrade FID or retrieval accuracy if evaluation does not account for pose confidence or filtering.

## Evidence (verbatim from paper)

> We achieve the best performance in FID and R-Precision, outperforming other three baseline methods on generation quality and text-motion alignment. Table 3 reports Top1, FID, and Diversity across different data scales.

## Citation

```bibtex
@misc{wang2024holisticmotion2d,
  title={Holistic-Motion2D: Scalable Whole-body Human Motion Generation in 2D Space},
  author={Wang et al. (2024)},
  year={2024},
  note={arXiv:2406.11253}
}
```

- arXiv: 2406.11253