Got10k Eval

---
name: got10k-eval
description: Evaluates tracking performance on a large-scale dataset of 10,000 videos with diverse object categories, testing generalization to real-world scenarios with language guidance. Use when the user wants to benchmark on GOT-10k, or asks about evaluating this task. Reports AO.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.05221
  bibtex_key: wang2025reasoningtrack
  confidence: high
---

# got10k-eval

> ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking — Xiao Wang et al. (2025) (arXiv:2508.05221, 2025)

## What this evaluates

Evaluates tracking performance on a large-scale dataset of 10,000 videos with diverse object categories, testing generalization to real-world scenarios with language guidance.

## Datasets

- **GOT-10k** — total 10000; splits: test (-1)

## Metrics

- `AO` **(primary)** — range: percent
  - Average Overlap: mean IoU between predicted and ground-truth boxes across all frames.
- `SR0.5` — range: percent
  - Success Rate at IoU threshold 0.5: percentage of frames with IoU > 0.5.
- `SR0.75` — range: percent
  - Success Rate at IoU threshold 0.75: percentage of frames with IoU > 0.75.

## Input / output format

**Input**: Video frames, initial language description, and ground-truth bounding boxes.

**Output**: Predicted bounding box coordinates per frame.

## Scoring recipe

```python
For each frame, compute IoU(pred, GT).
AO = mean(IoU)
SR0.5 = (count(IoU > 0.5) / N) * 100
SR0.75 = (count(IoU > 0.75) / N) * 100
```

## Common pitfalls

- GOT-10k test set lacks language annotations; authors use external annotations from Guo's work.
- Large scale requires efficient inference; language update overhead must be managed.

## Evidence (verbatim from paper)

> The experimental results presented in Table[IV] indicate that after introducing the strategy of COT text update mechanism, our method achieves competitive results (AO:77.8%, SR0.5:88.5, SR0.75:77.0) on this dataset, significantly enhancing the performance of the baseline tracker.

## Citation

```bibtex
@misc{wang2025reasoningtrack,
  title={ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking},
  author={Xiao Wang et al. (2025)},
  year={2025},
  note={arXiv:2508.05221}
}
```

- arXiv: 2508.05221