Susuinteracts Eval

---
name: susuinteracts-eval
description: Evaluates the quality of generated 3D human motions for interactive dialogue, measuring semantic alignment with text/audio, motion quality, audio-motion synchronization, and diversity. Use when the user wants to benchmark on SuSuInterActs, or asks about evaluating this task. Reports R@K.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.02908
  bibtex_key: jin2026sentiavatar
  confidence: high
---

# susuinteracts-eval

> SentiAvatar: Towards Expressive and Interactive Digital Humans — Chuhao Jin et al. (2026) (arXiv:2604.02908, 2026)

## What this evaluates

Evaluates the quality of generated 3D human motions for interactive dialogue, measuring semantic alignment with text/audio, motion quality, audio-motion synchronization, and diversity.

## Datasets

- **SuSuInterActs** — total 22234; splits: train (20982), val (710), test (542)

## Metrics

- `R@K` **(primary)** — range: [0, 1]
  - Text-to-motion retrieval recall at top-K matches. Higher is better.
- `FID` — range: other
  - Fréchet Inception Distance measuring overall motion quality distribution match. Lower is better.
- `ESD` — range: other
  - Event Sync Distance: average nearest temporal distance between detected audio onset events and motion velocity peak events. Lower is better.
- `Diversity` — range: other
  - Mean pairwise L2 distance among motion latent features. Higher is better.

## Input / output format

**Input**: Chinese text labels and/or audio tokens conditioning a motion generation model.

**Output**: Generated 3D human motion sequences (sparse keyframes followed by frame-level interpolated motion).

## Scoring recipe

```python
R@K: For each query, retrieve top-K generated motions; recall = matches / total queries.
FID: Compute mean and covariance of real vs generated motion latent features; FID = ||μ_r - μ_g||^2 + Tr(Σ_r + Σ_g - 2(Σ_r Σ_g)^0.5).
ESD: Detect audio onsets and motion velocity peaks; compute average nearest temporal distance between them.
Diversity: Compute mean pairwise L2 distance among generated motion latent features.
```

## Common pitfalls

- ESD measures audio onset vs motion velocity peaks, not frame-level alignment; lower is better.
- FID and ESD are lower-is-better, while R@K and Diversity are higher-is-better.
- Diversity is computed on motion latent features, not raw coordinates.

## Evidence (verbatim from paper)

> For objective evaluation on the SuSuInterActs dataset, following previous works, we report R@K ($\uparrow$, $K\in{1,2,3}$), the text-to-motion retrieval recall measuring semantic alignment; FID ($\downarrow$), the Fréchet Inception Distance measuring overall motion quality; ESD ($\downarrow$), Event Sync Distance, a bidirectional event-level audio–motion synchronization metric that computes the average nearest temporal distance between detected audio onset events and motion velocity peak events—lower values indicate tighter synchronization ... and Diversity ($\uparrow$), the mean pairwise L2 distance among motion latent features.

## Citation

```bibtex
@misc{jin2026sentiavatar,
  title={SentiAvatar: Towards Expressive and Interactive Digital Humans},
  author={Chuhao Jin et al. (2026)},
  year={2026},
  note={arXiv:2604.02908}
}
```

- arXiv: 2604.02908