Uvh 26 Eval

---
name: uvh-26-eval
description: Evaluates object detection models on a domain-specific Indian traffic dataset, probing their ability to localize and classify 14 heterogeneous vehicle types under surveillance viewpoints with varying occlusion and scale. Use when the user wants to benchmark on UVH-26, or asks about evaluating this task. Reports mAP(50:95).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.02563
  bibtex_key: sharma2025uvh
  confidence: high
---

# uvh-26-eval

> The Urban Vision Hackathon Dataset and Models: Towards Image Annotations and Accurate Vision Models for Indian Traffic — Akash Sharma et al. (2025) (arXiv:2511.02563, 2025)

## What this evaluates

Evaluates object detection models on a domain-specific Indian traffic dataset, probing their ability to localize and classify 14 heterogeneous vehicle types under surveillance viewpoints with varying occlusion and scale.

## Datasets

- **UVH-26** — total 26646; splits: test (400)

## Metrics

- `mAP(50:95)` **(primary)** — range: [0, 1]
  - Mean of Average Precision (AP) values computed at Intersection over Union (IoU) thresholds from 0.50 to 0.95 in steps of 0.05.
- `mAP(75)` — range: [0, 1]
  - AP computed at a strict IoU threshold of 0.75, emphasizing precise localization quality.
- `mAP(50)` — range: [0, 1]
  - AP computed at a lenient IoU threshold of 0.50, reflecting coarse detection capability.

## Input / output format

**Input**: High-resolution traffic images captured from surveillance/CCTV viewpoints.

**Output**: Bounding boxes with class labels for 14 India-specific vehicle categories.

## Scoring recipe

```python
def compute_map(predictions, ground_truth, iou_thresholds):
    aps = []
    for iou in iou_thresholds:
        ap = 0.0
        for class_id in classes:
            # Standard COCO-style AP: match predictions to GT using IoU >= iou,
            # compute precision-recall curve, integrate area under curve
            ap += calculate_ap(predictions[class_id], ground_truth[class_id], iou)
        aps.append(ap / len(classes))
    return sum(aps) / len(aps)

# mAP(50:95) = compute_map(preds, gts, np.arange(0.50, 0.96, 0.05))
```

## Common pitfalls

- COCO and UVH-26 annotate 2-Wheelers/Cycles differently (COCO excludes riders, UVH-26 includes them), so these classes were explicitly excluded from cross-dataset comparisons.
- The 'Others' umbrella class was excluded from evaluation due to low instance counts and lack of semantic specificity.
- Domain shift between ego-view (COCO) and top-down surveillance (UVH-26) significantly impacts baseline generalization, requiring careful fine-tuning rather than direct transfer.

## Evidence (verbatim from paper)

> We use a held-out test set curated from our gold dataset comprising of 400 images, sampled to ensure diverse coverage of all fourteen UVH-26 vehicle classes. ... Performance assessment follows standard practices widely adopted in the object detection literature. The primary metric is the mean Average Precision (mAP), evaluated across a range of Intersection over Union (IoU) thresholds. In particular, we report: 1. mAP(50:95): The main benchmark metric, defined as the mean of AP values at IoU thresholds from 0.50 to 0.95 in steps of 0.05. 2. mAP(75): AP computed at a stricter IoU threshold of 0.75, which emphasizes precise localization quality. 3. mAP(50): AP computed at a lenient IoU threshold of 0.50, reflecting the model’s capacity for coarse but correct detections.

## Citation

```bibtex
@misc{sharma2025uvh,
  title={The Urban Vision Hackathon Dataset and Models: Towards Image Annotations and Accurate Vision Models for Indian Traffic},
  author={Akash Sharma et al. (2025)},
  year={2025},
  note={arXiv:2511.02563}
}
```

- arXiv: 2511.02563