Crl Biometric Eval

---
name: crl-biometric-eval
description: Evaluates a model's ability to learn generalizable biometric feature representations in a continual learning setting, specifically measuring generalization to unseen identities across sequential learning steps rather than retaining knowledge of previously seen classes. Use when the user wants to benchmark on CRL-face, CRL-person, LFW, Megaface, or asks about evaluating this task. Reports Top 1 accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2006.04455
  bibtex_key: zhao2020continual
  confidence: high
---

# crl-biometric-eval

> Continual Representation Learning for Biometric Identification — Zhao et al. (2020) (arXiv:2006.04455, 2020)

## What this evaluates

Evaluates a model's ability to learn generalizable biometric feature representations in a continual learning setting, specifically measuring generalization to unseen identities across sequential learning steps rather than retaining knowledge of previously seen classes.

## Datasets

- **CRL-face** — total ?; splits: train (-1), test (-1); repo https://github.com/PatrickZH/Continual-Representation-Learning-for-Biometric-Identification
- **CRL-person** — total ?; splits: train (-1), test (-1); repo https://github.com/PatrickZH/Continual-Representation-Learning-for-Biometric-Identification
- **LFW** — total ?; splits: test (-1)
- **Megaface** — total ?; splits: test (-1)

## Metrics

- `Top 1 accuracy` **(primary)** — range: percent
  - Percentage of correctly identified identities or matched query-gallery pairs out of the total test set, computed at the final learning step.
- `mAP` — range: percent
  - Mean Average Precision, calculated by averaging the Average Precision scores across all query-gallery matching pairs in the person re-identification test set.

## Input / output format

**Input**: RGB images aligned and resized to 112x112 for face recognition, or 256x128 for person re-identification.

**Output**: Feature embeddings (256-dimensional for face, 2048-dimensional for person re-id). Classification/retrieval is performed via Euclidean distance similarity.

## Scoring recipe

```python
def compute_metrics(embeddings, labels, query_ids, gallery_ids):
    correct = 0
    for emb, label in zip(embeddings, labels):
        pred = argmin([euclidean_dist(emb, class_center) for class_center in class_centers])
        if pred == label: correct += 1
    top1_acc = (correct / len(labels)) * 100
    
    dists = pairwise_euclidean(query_embs, gallery_embs)
    mAP = mean([ap(dists[i], query_labels[i], gallery_labels) for i in range(len(query_embs))])
    return top1_acc, mAP
```

## Common pitfalls

- Evaluating performance on old (seen) classes instead of unseen classes, as the paper shows old-class performance does not decrease significantly and is unsuitable for measuring CRL.
- Ignoring the continual learning setting (5-step or 10-step sequential training) and treating it as standard single-step training.
- Not reporting standard deviation over 5 random seeds/runs, as the protocol requires repeating experiments five times.

## Evidence (verbatim from paper)

> We evaluate model’s generalization ability on unseen classes in CRL setting, which is more suitable. ... Top 1 accuracy (%) is reported. ... Top1 and mAP accuracy (%) in the final learning step.

## Citation

```bibtex
@misc{zhao2020continual,
  title={Continual Representation Learning for Biometric Identification},
  author={Zhao et al. (2020)},
  year={2020},
  note={arXiv:2006.04455}
}
```

- arXiv: 2006.04455