Webuot 1m Eval

---
name: webuot-1m-eval
description: Evaluates the robustness and accuracy of deep object trackers in challenging underwater environments. It probes cross-domain adaptation from open-air to underwater domains, as well as within-domain fine-tuning capabilities, while also assessing performance under varying frame rates and complex visual conditions like occlusion and low visibility. Use when the user wants to benchmark on WebUOT-1M, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2405.19818
  bibtex_key: zhang2024webuot1m
  confidence: high
---

# webuot-1m-eval

> WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark — Chunhui Zhang et al. (2024) (arXiv:2405.19818, 2024)

## What this evaluates

Evaluates the robustness and accuracy of deep object trackers in challenging underwater environments. It probes cross-domain adaptation from open-air to underwater domains, as well as within-domain fine-tuning capabilities, while also assessing performance under varying frame rates and complex visual conditions like occlusion and low visibility.

## Datasets

- **WebUOT-1M** — total 1500; splits: train (-1), test (-1)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area Under the Curve of the success rate (IoU overlap) across thresholds from 0 to 1. Standard in tracking benchmarks.
- `mACC` — range: other
  - Mean Accuracy, calculated as the average center location error (in pixels) between predicted and ground truth bounding boxes across all frames.
- `Pre` — range: percent
  - Precision, the percentage of frames where the center distance between predicted and ground truth boxes is within a 20-pixel threshold.
- `nPre` — range: percent
  - Normalized Precision, similar to Pre but normalized by the ground truth box size to account for varying target scales.
- `cAUC` — range: [0, 1]
  - Complete Success Rate AUC, computed using a stricter IoU threshold range or complete overlap metric as defined in standard tracking protocols.

## Input / output format

**Input**: Sequential video frames (underwater imagery), an initial ground-truth bounding box for the first frame, and optionally a language prompt describing the target.

**Output**: A sequence of predicted bounding box coordinates (x, y, width, height) for each frame in the video sequence.

## Scoring recipe

```python
def compute_metrics(predictions, ground_truths):
    ious = [iou(pred, gt) for pred, gt in zip(predictions, ground_truths)]
    auc = np.mean([np.mean([iou > t for iou in ious]) for t in np.linspace(0, 1, 101)])
    precision = np.mean([center_dist(pred, gt) <= 20 for pred, gt in zip(predictions, ground_truths)]) * 100
    macc = np.mean([center_dist(pred, gt) for pred, gt in zip(predictions, ground_truths)])
    return {'AUC': auc, 'Pre': precision, 'mACC': macc}
```

## Common pitfalls

- Confusing Protocol I (cross-domain evaluation of pre-trained open-air trackers) with Protocol II (within-domain retraining on WebUOT-1M).
- Simulating low frame rates by randomly discarding frames can break temporal continuity, requiring careful interpolation or state reset in trackers.
- Vision-language trackers using only language prompts without bounding box cues perform significantly worse, contrary to open-air multimodal tracking trends.

## Evidence (verbatim from paper)

> Following*[zhang2022webuav] ; [fan2021lasot]*, we perform the one-pass evaluation (OPE) and measure trackers using five evaluation metrics (i.e., percision (Pre), normalized precision (nPre), success rate (AUC), complete success rate (cAUC), and mean accuracy (mACC)) under two protocols.

## Citation

```bibtex
@misc{zhang2024webuot1m,
  title={WebUOT-1M: Advancing Deep Underwater Object Tracking with A Million-Scale Benchmark},
  author={Chunhui Zhang et al. (2024)},
  year={2024},
  note={arXiv:2405.19818}
}
```

- arXiv: 2405.19818