Webuav 3m Eval

---
name: webuav-3m-eval
description: Evaluates the robustness and accuracy of deep visual tracking algorithms on large-scale, real-world UAV video sequences. It probes how well trackers handle diverse environmental conditions, motion dynamics, and target appearance changes without parameter tuning. Use when the user wants to benchmark on WebUAV-3M, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2201.07425
  bibtex_key: [REDACTED]
  confidence: high
---

# webuav-3m-eval

> WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking — Zhang et al. (2022) (arXiv:2201.07425, 2022)

## What this evaluates

Evaluates the robustness and accuracy of deep visual tracking algorithms on large-scale, real-world UAV video sequences. It probes how well trackers handle diverse environmental conditions, motion dynamics, and target appearance changes without parameter tuning.

## Datasets

- **WebUAV-3M** — total 3300000; splits: test (-1); repo https://github.com/983632847/WebUAV-3M

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area under the success plot curve. It calculates the percentage of frames where the overlap score (IoU) between predicted and ground-truth bounding boxes exceeds a given threshold, averaged over thresholds from 0 to 1.
- `Pre` — range: [0, 1]
  - Precision plot score. Measures the percentage of frames whose center location error is within a predefined threshold (e.g., 20 pixels).
- `nPre` — range: [0, 1]
  - Normalized precision plot. Normalizes the precision score over the size of the ground-truth bounding box to reduce sensitivity to target size and image resolution.
- `mAcc` — range: [0, 1]
  - Mean accuracy. Encourages trackers to output invisible predictions when the target disappears, measuring robustness to target loss.
- `cAUC` — range: [0, 1]
  - Complete success plot. Area under the curve of a complete overlap score $S_c = \frac{|B_G \cap B_P|}{|B_G \cup B_P|} - \frac{d^2(b^G, b^P)}{c^2} - \alpha v$, which combines IoU, normalized center distance, and aspect ratio consistency.

## Input / output format

**Input**: Video frames and an initial ground-truth bounding box for the target.

**Output**: Predicted bounding box coordinates (x, y, w, h) for each frame in the sequence.

## Scoring recipe

```python
def compute_auc(pred_boxes, gt_boxes):
    ious = [intersection_over_union(p, g) for p, g in zip(pred_boxes, gt_boxes)]
    thresholds = np.linspace(0, 1, 101)
    success_rates = [sum(iou >= t for iou in ious) / len(ious) for t in thresholds]
    return np.trapz(success_rates, thresholds)

def compute_pre(pred_boxes, gt_boxes, threshold=20):
    errors = [center_distance(p, g) for p, g in zip(pred_boxes, gt_boxes)]
    return sum(e <= threshold for e in errors) / len(errors)
```

## Common pitfalls

- Evaluation is strictly one-pass (OPE) with no re-initialization, providing a lower-bound performance estimate compared to multi-pass protocols.
- Standard metrics (Pre, AUC) ignore aspect ratio changes; use cAUC for complete geometric evaluation.
- No hyperparameter tuning is performed on the benchmark; all trackers use default weights and fixed parameters.

## Evidence (verbatim from paper)

> In this work, we perform a one-pass evaluation (OPE) and adopt four popular metrics, i.e., the precision plot (Pre), normalized precision plot (nPre), success plot (AUC) and mean accuracy (mAcc) measures, and a newly proposed metric, the complete success plot (cAUC), to assess the performance of different tracking algorithms.

## Citation

```bibtex
@misc{zhang2022webuav3m,
  title={WebUAV-3M: A Benchmark for Unveiling the Power of Million-Scale Deep UAV Tracking},
  author={Zhang et al. (2022)},
  year={2022},
  note={arXiv:2201.07425}
}
```

- arXiv: 2201.07425