Mavrec Eval

---
name: mavrec-eval
description: Evaluates object detection performance on aerial and ground-view imagery, probing how geographic context and multi-view data fusion affect detection accuracy across different object scales. Use when the user wants to benchmark on MAVREC, or asks about evaluating this task. Reports mAP.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2312.04548
  bibtex_key: dutta2023mavrec
  confidence: high
---

# mavrec-eval

> Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception? — Dutta et al. (2023) (arXiv:2312.04548, 2023)

## What this evaluates

Evaluates object detection performance on aerial and ground-view imagery, probing how geographic context and multi-view data fusion affect detection accuracy across different object scales.

## Datasets

- **MAVREC** — total 11024; splits: train (8605), val (805), test (1614)

## Metrics

- `mAP` **(primary)** — range: [0, 100] percent
  - Mean Average Precision computed over object categories, averaging the area under the precision-recall curve across IoU thresholds. The paper references COCO-style evaluation [44], which typically averages AP over IoU thresholds from 0.50 to 0.95. Sub-metrics include AP at IoU=0.50 (AP_50), and AP for small (AP_S) and medium (AP_M) objects.

## Input / output format

**Input**: Image frames from either ground or aerial perspectives, provided as labeled pairs for supervised training, labeled/unlabeled pairs for semi-supervised training, or single annotated frames for validation/testing.

**Output**: Predicted bounding boxes with class labels and confidence scores for each detected object in the input image.

## Scoring recipe

```python
def compute_mAP(predictions, ground_truths, iou_thresh=0.5):
    ap_scores = []
    for cls in classes:
        gt = ground_truths[cls]
        pred = predictions[cls]
        pred = sorted(pred, key=lambda x: x.confidence, reverse=True)
        tp, fp = [], []
        for p in pred:
            best_iou = max(iou(p.box, g.box) for g in gt)
            if best_iou >= iou_thresh and not g.used:
                tp.append(1); g.used = True
            else:
                fp.append(1)
        ap_scores.append(calculate_ap(tp, fp))
    return sum(ap_scores) / len(ap_scores) * 100
```

## Common pitfalls

- Assuming pre-training on large ground-view datasets (like COCO) generalizes well to aerial views without geographic alignment; the paper shows strong domain shift across geographies.
- Overlooking scale-specific performance: aerial views contain significantly smaller objects, leading to poor AP_S scores that mask overall mAP performance.
- Treating semi-supervised results as purely supervised: the curriculum learning phase uses labeled ground/aerial images in a specific burn-in order, which critically affects pseudo-label quality.

## Evidence (verbatim from paper)

> We evaluate the models with the widely used metric for object detection, mean average precision (mAP) [[44]]; see a detailed discussion in §C.2.

## Citation

```bibtex
@misc{dutta2023mavrec,
  title={Multiview Aerial Visual Recognition (MAVREC): Can Multi-view Improve Aerial Visual Perception?},
  author={Dutta et al. (2023)},
  year={2023},
  note={arXiv:2312.04548}
}
```

- arXiv: 2312.04548