Mobilei2v I2v Eval

---
name: mobilei2v-i2v-eval
description: Evaluates the visual quality and generation speed of image-to-video diffusion models optimized for mobile deployment. It probes the model's ability to generate temporally coherent 17-frame videos from a single reference image while maintaining high resolution and low latency on mobile hardware. Use when the user wants to benchmark on Unspecified (FVD benchmarks), or asks about evaluating this task. Reports FVDhum.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.21475
  bibtex_key: zhang2025mobilei2v
  confidence: medium
---

# mobilei2v-i2v-eval

> MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices — Zhang et al. (2025) (arXiv:2511.21475, 2025)

## What this evaluates

Evaluates the visual quality and generation speed of image-to-video diffusion models optimized for mobile deployment. It probes the model's ability to generate temporally coherent 17-frame videos from a single reference image while maintaining high resolution and low latency on mobile hardware.

## Datasets

- **Unspecified (FVD benchmarks)** — total ?; splits: (unstated)

## Metrics

- `FVDhum` **(primary)** — range: other
  - Frechet Video Distance computed on human video clips. Lower is better. Calculated as the Fréchet distance between the multivariate Gaussian distributions of features extracted from a pretrained video classifier (typically I3D) for real and generated videos.
- `FVDscen` — range: other
  - Frechet Video Distance computed on scene video clips. Lower is better. Uses the same feature extraction and distance calculation as FVDhum but on a different subset of videos.

## Input / output format

**Input**: A reference image and an optical flow/motion score condition.

**Output**: A 17-frame video at 1280×720 resolution.

## Scoring recipe

```python
def compute_fvd(real_videos, generated_videos, feature_extractor):
    real_features = feature_extractor(real_videos)
    gen_features = feature_extractor(generated_videos)
    mu_real, sigma_real = np.mean(real_features, axis=0), np.cov(real_features, rowvar=False)
    mu_gen, sigma_gen = np.mean(gen_features, axis=0), np.cov(gen_features, rowvar=False)
    diff = mu_real - mu_gen
    covmean, _ = scipy.linalg.sqrtm(sigma_real.dot(sigma_gen), disp=False)
    if np.iscomplexobj(covmean):
        covmean = covmean.real
    fvd = np.sum(diff**2) + np.trace(sigma_real + sigma_gen - 2.0 * covmean)
    return fvd
```

## Common pitfalls

- FVD scores are highly sensitive to the batch size used for feature extraction and the specific version of the I3D classifier weights.
- Mobile latency measurements can vary significantly due to thermal throttling and background OS processes on the iPhone 16 Pro.
- The table reports FVD for 17-frame videos, but standard FVD benchmarks often use different frame counts; ensure consistent temporal sampling.

## Evidence (verbatim from paper)

> Table 1: Speed and quality comparison of different models performing 17-frame I2V tasks. | Model | Type | Steps | Params | Resolution | LatencyA100 | LatencyMobile | FVDhum↓ | FVDscen↓ |

## Citation

```bibtex
@misc{zhang2025mobilei2v,
  title={MobileI2V: Fast and High-Resolution Image-to-Video on Mobile Devices},
  author={Zhang et al. (2025)},
  year={2025},
  note={arXiv:2511.21475}
}
```

- arXiv: 2511.21475