Mobiface Eval

---
name: mobiface-eval
description: This benchmark evaluates the capability of visual tracking algorithms to maintain robust face localization in unconstrained, mobile-captured video sequences. It specifically probes resilience to challenging real-world conditions such as rapid camera motion, out-of-plane rotations, scale changes, and partial occlusions. Use when the user wants to benchmark on iBUG MobiFace, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1805.09749
  bibtex_key: lin2018mobiface
  confidence: high
---

# mobiface-eval

> MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild — Lin et al. (2018) (arXiv:1805.09749, 2018)

## What this evaluates

This benchmark evaluates the capability of visual tracking algorithms to maintain robust face localization in unconstrained, mobile-captured video sequences. It specifically probes resilience to challenging real-world conditions such as rapid camera motion, out-of-plane rotations, scale changes, and partial occlusions.

## Datasets

- **iBUG MobiFace** — total 50736; splits: test (50736)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area Under the Curve of the success plot, which measures the percentage of frames where the overlap ratio between predicted and ground-truth bounding boxes exceeds a given threshold (typically integrated over thresholds from 0 to 1).
- `Precision@20px` — range: [0, 1]
  - The percentage of frames where the Euclidean distance between the center of the predicted bounding box and the ground-truth center is within 20 pixels.
- `FPS` — range: other
  - Frames per second processed by the tracker, computed as total frames divided by total inference time.

## Input / output format

**Input**: Sequential video frames captured from smartphones, along with ground-truth bounding box annotations for the target face in each frame.

**Output**: Predicted bounding box coordinates (e.g., top-left x, y and width, height) for the target face in each frame.

## Scoring recipe

```python
def compute_metrics(predictions, ground_truths, frame_times):
    overlaps = []
    center_errors = []
    for pred, gt in zip(predictions, ground_truths):
        overlaps.append(intersection_over_union(pred, gt))
        center_errors.append(center_distance(pred, gt))
    # AUC: integrate success curve over thresholds [0, 1]
    auc = np.trapz([sum(o >= t) / len(o) for t in np.linspace(0, 1, 100)], np.linspace(0, 1, 100))
    # Precision@20px
    precision = sum(e <= 20 for e in center_errors) / len(center_errors)
    # FPS
    fps = len(predictions) / sum(frame_times)
    return {'AUC': auc, 'Precision@20px': precision, 'FPS': fps}
```

## Common pitfalls

- Forcing trackers to output a bounding box even when the target is completely missing, which unfairly penalizes re-identification capabilities compared to methods that correctly output null/zero.
- Ignoring online model adaptation; trackers with fixed weights (e.g., SiamFC) degrade significantly compared to those that update parameters during tracking.
- Evaluating speed on desktop GPUs (GTX 1060) rather than actual mobile hardware, making FPS comparisons less representative of real-world deployment constraints.

## Evidence (verbatim from paper)

> For success plot, the trackers' name is shown with their corresponding AUC. For precision plot, the score at 20 pixel threshold is shown.

## Citation

```bibtex
@misc{lin2018mobiface,
  title={MobiFace: A Novel Dataset for Mobile Face Tracking in the Wild},
  author={Lin et al. (2018)},
  year={2018},
  note={arXiv:1805.09749}
}
```

- arXiv: 1805.09749