Mobiface Lfw Megaface Eval

---
name: mobiface-lfw-megaface-eval
description: Evaluates the accuracy and robustness of lightweight face recognition models on unconstrained face verification tasks, specifically measuring pair-wise verification accuracy and identification rates under extreme distractor conditions. Use when the user wants to benchmark on Labeled Faces in the Wild (LFW), MegaFace, or asks about evaluating this task. Reports Accuracy, TAR@FAR=10^-6.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1811.11080
  bibtex_key: duong2018mobiface
  confidence: high
---

# mobiface-lfw-megaface-eval

> MobiFace: A Lightweight Deep Learning Face Recognition on Mobile Devices — Chi Nhan Duong et al. (2018) (arXiv:1811.11080, 2018)

## What this evaluates

Evaluates the accuracy and robustness of lightweight face recognition models on unconstrained face verification tasks, specifically measuring pair-wise verification accuracy and identification rates under extreme distractor conditions.

## Datasets

- **Labeled Faces in the Wild (LFW)** — total 13233; splits: test (6000)
- **MegaFace** — total ?; splits: test (100200)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Percentage of correctly verified face pairs out of the 6000 fixed test pairs.
- `TAR@FAR=10^-6` **(primary)** — range: percent
  - True Accepted Rate at a False Accepted Rate of 10^-6, measuring identification accuracy when allowing one false match per million probe-gallery comparisons.

## Input / output format

**Input**: Aligned and cropped face images of size 112x112x3, normalized to [-1, 1] by subtracting 127.5 and dividing by 128. Input consists of face pairs for verification or probe-gallery pairs for identification.

**Output**: Verification score or binary decision per pair; identification rank or match decision per probe image against a gallery.

## Scoring recipe

```python
def score_lfw(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return correct / 6000 * 100

def score_megaface(scores, probes, gallery):
    far_threshold = compute_threshold_for_far(scores, 1e-6)
    correct = sum(1 for p in probes if p.id in gallery and model(p, gallery) > far_threshold)
    return correct / len(probes) * 100
```

## Common pitfalls

- LFW protocol uses exactly 6000 fixed pairs; accuracy must be computed over this specific set, not the full 13k images.
- MegaFace evaluation is highly sensitive to the 1M distractor gallery; TAR@FAR=1e-6 requires precise threshold calibration to avoid inflated rates due to false matches.

## Evidence (verbatim from paper)

> According to the testing protocol of LFW, there are 6000 face pairs where half of them are positive pairs. ... The accuracy is reported on the True Accepted Rate (TAR) at the False Accepted Rate (FAR) of 10^-6.

## Citation

```bibtex
@misc{duong2018mobiface,
  title={MobiFace: A Lightweight Deep Learning Face Recognition on Mobile Devices},
  author={Chi Nhan Duong et al. (2018)},
  year={2018},
  note={arXiv:1811.11080}
}
```

- arXiv: 1811.11080