Svc Ongoing Eval

---
name: svc-ongoing-eval
description: Evaluates on-line signature verification systems across office (stylus), mobile (finger), and hybrid scenarios. It measures robustness against both skilled and random forgeries, testing generalization across different acquisition devices and intra-user variability. Use when the user wants to benchmark on DeepSignDB, SVC2021_EvalDB, or asks about evaluating this task. Reports EER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2108.06090
  bibtex_key: tolosana2021svcongoing
  confidence: high
---

# svc-ongoing-eval

> SVC-onGoing: Signature Verification Competition — Tolosana et al. (2021) (arXiv:2108.06090, 2021)

## What this evaluates

Evaluates on-line signature verification systems across office (stylus), mobile (finger), and hybrid scenarios. It measures robustness against both skilled and random forgeries, testing generalization across different acquisition devices and intra-user variability.

## Datasets

- **DeepSignDB** — total 442; splits: eval (442)
- **SVC2021_EvalDB** — total ?; splits: test (-1); repo https://github.com/BiDAlab/SVC2021_EvalDB

## Metrics

- `EER` **(primary)** — range: percent
  - Equal Error Rate, defined as the operating point where the False Acceptance Rate (FAR) equals the False Rejection Rate (FRR). Reported as a percentage.

## Input / output format

**Input**: On-line signature traces comprising X and Y spatial coordinates, time, and pressure signals.

**Output**: Binary verification decision (genuine vs. forgery) or a continuous similarity score used to compute FAR/FRR curves.

## Scoring recipe

```python
def compute_eer(scores, labels):
    # scores: similarity scores, labels: 1 (genuine), 0 (forgery)
    thresholds = np.unique(scores)
    best_eer = 1.0
    for t in thresholds:
        far = np.mean(scores[labels == 0] >= t)
        frr = np.mean(scores[labels == 1] < t)
        eer = max(far, frr)
        if eer < best_eer:
            best_eer = eer
    return best_eer * 100
```

## Common pitfalls

- Models trained exclusively on stylus data often fail to generalize to finger-written mobile signatures due to high intra-user variability and device-specific acquisition noise.
- Evaluating systems on only one forgery type (skilled or random) is insufficient; the protocol requires simultaneous robustness against both to simulate real-world conditions.
- Performance on the development set (DeepSignDB) frequently overestimates final evaluation results (SVC2021_EvalDB) due to distribution shifts in subjects and acquisition devices.

## Evidence (verbatim from paper)

> Concretely, for each of the tasks, the DLVC-Lab team achieves relative improvements of 74.54%, 50.34%, and 58.3% EER compared to the Baseline DTW. These results prove the high potential of deep learning approaches such as DSDTW and TA-RNN for the on-line signature verification field, as commented in previous studies.

## Citation

```bibtex
@misc{tolosana2021svcongoing,
  title={SVC-onGoing: Signature Verification Competition},
  author={Tolosana et al. (2021)},
  year={2021},
  note={arXiv:2108.06090}
}
```

- arXiv: 2108.06090