Hoivg Bench Eval

---
name: hoivg-bench-eval
description: Evaluates a video generation model's ability to synthesize high-fidelity videos conditioned on multimodal inputs (reference images, audio, pose, and text) while maintaining reference consistency, audio-visual synchronization, and temporal coherence. Use when the user wants to benchmark on HOIVG-Bench, EMTD, or asks about evaluating this task. Reports NexusScore.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.11804
  bibtex_key: zhou2026omnishow
  confidence: high
---

# hoivg-bench-eval

> OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation — Zhou et al. (2026) (arXiv:2604.11804, 2026)

## What this evaluates

Evaluates a video generation model's ability to synthesize high-fidelity videos conditioned on multimodal inputs (reference images, audio, pose, and text) while maintaining reference consistency, audio-visual synchronization, and temporal coherence.

## Datasets

- **HOIVG-Bench** — total ?; splits: test (-1)
- **EMTD** — total ?; splits: test (-1)

## Metrics

- `NexusScore` **(primary)** — range: [0, 1]
  - Composite metric evaluating overall video quality, reference preservation, and condition adherence across multimodal settings.
- `FaceSim` — range: [0, 1]
  - Measures facial similarity and identity preservation between the generated video frames and the input reference image.
- `TA` — range: [0, 1]
  - Text Alignment score measuring the degree to which the generated video adheres to the provided text prompt.
- `Sync-C` — range: [0, 1]
  - Audio-visual synchronization consistency score evaluating temporal alignment between audio cues and visual actions.
- `Sync-D` — range: [0, 1]
  - Audio-visual synchronization detail score measuring fine-grained frame-wise interaction and temporal coherence.

## Input / output format

**Input**: Text prompt, reference image(s), audio clip, and pose sequence (optional depending on setting: R2V, RA2V, RP2V, RAP2V)

**Output**: Video sequence (480p or 720p)

## Scoring recipe

```python
def evaluate(predictions, gold):
    metrics = {}
    metrics['FaceSim'] = compute_face_similarity(predictions, gold['ref_img'])
    metrics['NexusScore'] = compute_nexus_score(predictions, gold['text'], gold['ref_img'])
    metrics['TA'] = compute_text_alignment(predictions, gold['text'])
    metrics['Sync-C'] = compute_sync_consistency(predictions, gold['audio'])
    metrics['Sync-D'] = compute_sync_detail(predictions, gold['audio'])
    return metrics
```

## Common pitfalls

- Baselines often lack support for the full set of four multimodal conditions, requiring evaluation across different input subsets (R2V, RA2V, RP2V) rather than a single unified setting.
- Objective metrics like FaceSim may drop in pose-conditioned settings (RP2V) due to viewpoint shifts and facial morphology changes, which does not necessarily indicate poor generation quality.
- Human evaluation is conducted on small subsets (20 samples) with limited participants (30-33), which may not fully capture temporal dynamics or perceived realism compared to frame-level metrics.

## Evidence (verbatim from paper)

> In the R2V setting, our method matches the reference preservation capabilities of specialized methods like Phantom-14B, as evidenced by our comparable FaceSim and NexusScore. Notably, OmniShow exhibits distinct advantages as a unified framework in more complex scenarios. In the RA2V setting, while dedicated baselines like HuMo-17B may show slight gains in TA, our approach delivers leading performance in other metrics like NexusScore and Sync-C.

## Citation

```bibtex
@misc{zhou2026omnishow,
  title={OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generation},
  author={Zhou et al. (2026)},
  year={2026},
  note={arXiv:2604.11804}
}
```

- arXiv: 2604.11804