Interact Eval

---
name: interact-eval
description: Evaluates the ability of generative models to synthesize physically plausible, contact-consistent 3D human-object interaction sequences conditioned on text, actions, or object shapes. It probes motion realism, contact accuracy, and alignment between linguistic/action prompts and generated kinematics. Use when the user wants to benchmark on InterAct, or asks about evaluating this task. Reports FID.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.09555
  bibtex_key: xu2025interact
  confidence: high
---

# interact-eval

> InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation — Xu et al. (2025) (arXiv:2509.09555, 2025)

## What this evaluates

Evaluates the ability of generative models to synthesize physically plausible, contact-consistent 3D human-object interaction sequences conditioned on text, actions, or object shapes. It probes motion realism, contact accuracy, and alignment between linguistic/action prompts and generated kinematics.

## Datasets

- **InterAct** — total ?; splits: train (-1), test (-1); repo https://github.com/wzyabcas/InterAct

## Metrics

- `FID` **(primary)** — range: [0, ∞)
  - Fréchet Inception Distance computed between feature distributions of generated and ground-truth HOI sequences to measure distributional similarity.
- `R-Precision` — range: [0, 1]
  - Accuracy of retrieving the correct ground-truth interaction sequence when given a set of candidate sequences and a text prompt.
- `MPMPE` — range: meters
  - Mean Per-Marker Position Error, calculated as the average L2 distance between predicted and ground-truth marker positions over time.
- `Contact Ratio` — range: [0, 1]
  - Average ratio of human vertices whose distance to the object surface falls below a predefined threshold.
- `Penetration` — range: meters
  - Average across the sequence of the maximum negative signed distance from human vertices to the object surface, measuring intersection depth.

## Input / output format

**Input**: Text descriptions, action labels, or object shapes (encoded via BPS), optionally with contact conditions or human pose priors.

**Output**: 3D human-object interaction sequences represented as time-series of marker-based or joint-based poses.

## Scoring recipe

```python
def score(predictions, gold):
    # predictions, gold: (T, V, 3) marker positions
    mpmpe = np.mean(np.linalg.norm(predictions - gold, axis=-1))
    dists = np.linalg.norm(predictions - gold, axis=-1)
    contact_ratio = np.mean(dists < CONTACT_THRESHOLD)
    penetration = np.mean(np.max(np.where(dists < 0, -dists, 0), axis=-1))
    return {'MPMPE': mpmpe, 'Contact_Ratio': contact_ratio, 'Penetration': penetration}
```

## Common pitfalls

- Feature extractors for FID and R-Precision must be retrained on the full InterAct dataset; using extractors trained on limited data degrades evaluation quality.
- Contact metrics are highly sensitive to the distance threshold used to define contact, making cross-dataset comparisons difficult.
- Marker-based and joint-based representations yield different penetration/contact values, so evaluations must specify the representation used.

## Evidence (verbatim from paper)

> Metrics. Following the literature on text-to-motion generation, we develop five metrics for evaluation. The Fréchet Inception Distance (FID) quantifies the similarity between generated HOI features and the ground truth. The Multimodality and Diversity metrics assess the variety within the generated HOI. R-Precision measures the alignment between the textual descriptions and the generated HOI. The Multimodal Distance (MM Dist) evaluates the disparity between HOI features and corresponding text features.

## Citation

```bibtex
@misc{xu2025interact,
  title={InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation},
  author={Xu et al. (2025)},
  year={2025},
  note={arXiv:2509.09555}
}
```

- arXiv: 2509.09555