Mot15 Tracking Eval

---
name: mot15-tracking-eval
description: Evaluates multi-object tracking accuracy in a tracking-by-detection pipeline on edge hardware, measuring how well backbones support SORT-based tracking under strict computational and power constraints. Use when the user wants to benchmark on MOT15, or asks about evaluating this task. Reports MOTA.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2110.00337
  bibtex_key: paissan2021phinet
  confidence: high
---

# mot15-tracking-eval

> PhiNets: a scalable backbone for low-power AI at the edge — Paissan et al. (2021) (arXiv:2110.00337, 2021)

## What this evaluates

Evaluates multi-object tracking accuracy in a tracking-by-detection pipeline on edge hardware, measuring how well backbones support SORT-based tracking under strict computational and power constraints.

## Datasets

- **MOT15** — total ?; splits: test (-1)

## Metrics

- `MOTA` **(primary)** — range: percent
  - Multi Object Tracking Accuracy, combining false positives, false negatives, and ID switches relative to the number of ground truth tracks.

## Input / output format

**Input**: Video frames processed through a detection backbone followed by the SORT tracker, with detector training augmented by 360 epochs on the benchmark data.

**Output**: Tracked object IDs and trajectories over time for each video sequence.

## Scoring recipe

```python
def compute_MOTA(tracks, gts):
    fp = count_false_positives(tracks, gts)
    fn = count_false_negatives(tracks, gts)
    ids = count_id_switches(tracks, gts)
    total_gt = len(gts)
    return 1 - (fp + fn + ids) / total_gt
```

## Common pitfalls

- Tracking performance is heavily dependent on detection quality; poor detector IoU directly lowers tracking scores, making it hard to isolate backbone efficiency from detection head performance.
- Detectors are trained with 360 epochs of augmentation directly on the benchmark data, which risks data leakage or overfitting to the test set.

## Evidence (verbatim from paper)

> This choice of hardware and software allowed for a state of the art power consumption of under 1.3mJ for the 1.2MMACC PhiNet (53.7 / 60.3 mAP on a subset of the COCO/VOC2012 datasets) and 11.8mJ for the 9.8MMACC PhiNet (64.1 / 73.9 mAP on COCO/VOC2012, 60.8 MOTA on MOT15).

## Citation

```bibtex
@misc{paissan2021phinet,
  title={PhiNets: a scalable backbone for low-power AI at the edge},
  author={Paissan et al. (2021)},
  year={2021},
  note={arXiv:2110.00337}
}
```

- arXiv: 2110.00337