Trek 150 Eval

---
name: trek-150-eval
description: Evaluates single-object visual tracking performance in first-person vision videos, specifically testing robustness to object manipulation, occlusions, and dynamic interactions under real-time execution constraints. Use when the user wants to benchmark on TREK-150, or asks about evaluating this task. Reports SS, NPS, GSR.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2209.13502
  bibtex_key: dunnhofer2022visual
  confidence: high
---

# trek-150-eval

> Visual Object Tracking in First Person Vision — Dunnhofer et al. (2022) (arXiv:2209.13502, 2022)

## What this evaluates

Evaluates single-object visual tracking performance in first-person vision videos, specifically testing robustness to object manipulation, occlusions, and dynamic interactions under real-time execution constraints.

## Datasets

- **TREK-150** — total 150; splits: test (-1)

## Metrics

- `SS, NPS, GSR` **(primary)** — range: [0, 1]
  - Standard VOT metrics: Success Score (SS) based on IoU threshold, Normalized Precision Score (NPS) measuring center location error, and Generalized Success Rate (GSR). Computed per the VOT2017 protocol.
- `FPS` — range: other
  - Frames per second, measuring processing speed/efficiency.

## Input / output format

**Input**: Video sequence of first-person vision frames with a ground-truth bounding box provided only for the first frame.

**Output**: A sequence of bounding boxes (one per executed frame) tracking the target object throughout the video.

## Scoring recipe

```python
def evaluate_rte(tracker, video_frames, fps):
    predictions = []
    t = 0
    while t < len(video_frames):
        box = tracker.run(video_frames[t])
        predictions.append(box)
        exec_time = tracker.get_execution_time()
        t += int(exec_time * fps)
    ss = compute_success_score(predictions, ground_truth)
    nps = compute_normalized_precision(predictions, ground_truth)
    gsr = compute_generalized_success_rate(predictions, ground_truth)
    return ss, nps, gsr
```

## Common pitfalls

- Assuming standard One-Pot Evaluation (OPE) where the tracker processes every frame sequentially; RTE skips frames during tracker execution.
- Ignoring the real-time constraint: trackers must report processing time per frame to correctly align predictions with the actual video timeline.
- Using generic tracking metrics without accounting for FPV-specific challenges like rapid appearance changes due to object manipulation.

## Evidence (verbatim from paper)

> The overall performance scores (SS, NPS, GSR) are ultimately obtained as for the OPE protocol. Together with those values, we evaluate the trackers’ processing speed in frames per second (FPS) to quantify their efficiency.

## Citation

```bibtex
@misc{dunnhofer2022visual,
  title={Visual Object Tracking in First Person Vision},
  author={Dunnhofer et al. (2022)},
  year={2022},
  note={arXiv:2209.13502}
}
```

- arXiv: 2209.13502