Tampar Eval

---
name: tampar-eval
description: This benchmark probes a model's ability to detect visual tampering on parcel logistics items by comparing a single RGB image to a reference database. It evaluates the pipeline's robustness in detecting corner keypoints, performing perspective transformation to generate viewpoint-invariant views, and accurately identifying appearance changes across varying angles, lighting, and lens distortions. Use when the user wants to benchmark on TAMPAR, or asks about evaluating this task. Reports F1-Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2311.03124
  bibtex_key: [REDACTED]
  confidence: high
---

# tampar-eval

> TAMPAR: Visual Tampering Detection for Parcel Logistics in Postal Supply Chains — Alexander Naumann et al. (arXiv:2311.03124, 2023)

## What this evaluates

This benchmark probes a model's ability to detect visual tampering on parcel logistics items by comparing a single RGB image to a reference database. It evaluates the pipeline's robustness in detecting corner keypoints, performing perspective transformation to generate viewpoint-invariant views, and accurately identifying appearance changes across varying angles, lighting, and lens distortions.

## Datasets

- **TAMPAR** — total 3306; splits: test (3306)

## Metrics

- `F1-Score` **(primary)** — range: [0, 1]
  - Harmonic mean of precision and recall: 2 * (precision * recall) / (precision + recall). Thresholds are determined via a depth-one decision tree trained on similarity metrics.
- `Keypoint AP` — range: [0, 1]
  - Average Precision for keypoint detection, computed with annotation precision thresholds κ_i (0.05 for visible corners, 0.1 for self-occluded corners).

## Input / output format

**Input**: Single RGB image of a parcel to be inspected, compared against a reference image of the same parcel.

**Output**: Binary tampering label (tampered/not tampered) and/or a binary change map, along with 8 corner keypoint coordinates.

## Scoring recipe

```python
def compute_f1(predictions, gold):
    tp = sum(1 for p, g in zip(predictions, gold) if p == 1 and g == 1)
    fp = sum(1 for p, g in zip(predictions, gold) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(predictions, gold) if p == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    return 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
```

## Common pitfalls

- Models are trained exclusively on synthetic data (Parcel3D) but evaluated on real-world datasets, causing a domain gap that lowers performance.
- Ground truth keypoints are derived from 3D bounding boxes, causing frequent misalignment with actual parcel corners, especially for damaged parcels.
- Strong lens distortions can push keypoints outside the image boundaries or corrupt annotations, degrading detection accuracy.

## Evidence (verbatim from paper)

> Results in Tab. 2 using predicted keypoints show that (SimSaC, LPIPS) yields the best performance and reaches 0.81 accuracy and an F1-Score of 0.83.

## Citation

```bibtex
@misc{naumann2023tampar,
  title={TAMPAR: Visual Tampering Detection for Parcel Logistics in Postal Supply Chains},
  author={Alexander Naumann et al.},
  year={2023},
  note={arXiv:2311.03124}
}
```

- arXiv: 2311.03124