Ov Vg Eval

---
name: ov-vg-eval
description: Evaluates a model's ability to localize objects in images based on natural language descriptions without prior exposure to those specific categories. It probes visual-linguistic alignment, handling of novel vocabulary, and robustness to varying object scales and complex scenes. Use when the user wants to benchmark on OV-VG, or asks about evaluating this task. Reports Acc50.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2310.14374
  bibtex_key: wang2023ovvg
  confidence: high
---

# ov-vg-eval

> OV-VG: A Benchmark for Open-Vocabulary Visual Grounding — Wang et al. (2023) (arXiv:2310.14374, 2023)

## What this evaluates

Evaluates a model's ability to localize objects in images based on natural language descriptions without prior exposure to those specific categories. It probes visual-linguistic alignment, handling of novel vocabulary, and robustness to varying object scales and complex scenes.

## Datasets

- **OV-VG** — total 7272; splits: test (7272); repo https://github.com/cv516Buaa/OV-VG

## Metrics

- `Acc50` **(primary)** — range: percent
  - Accuracy at an Intersection over Union (IoU) threshold of 0.5. For each text-image query, a prediction is correct if the IoU between the predicted bounding box and the ground truth bounding box is ≥ 0.5. The final score is the percentage of correctly localized queries across the dataset.

## Input / output format

**Input**: An image and a natural language query describing the target object to be localized.

**Output**: A single bounding box (typically [x_min, y_min, x_max, y_max] or [x, y, w, h]) representing the predicted location of the target object.

## Scoring recipe

```python
correct = 0
for query, pred_box, gt_box in zip(queries, predictions, ground_truths):
    iou = calculate_iou(pred_box, gt_box)
    if iou >= 0.5:
        correct += 1
acc50 = (correct / len(queries)) * 100
```

## Common pitfalls

- Data leakage: Many open-vocabulary baselines are pre-trained on datasets (e.g., COCO, RefCOCO, LVIS) that contain images or categories identical to the test set, artificially inflating performance.
- Small target detection: Objects smaller than 32x32 pixels are frequently missed or mislocalized due to resolution limits and poor visual-linguistic alignment.
- Vocabulary confusion: Complex or ambiguous descriptions lead to predictions of completely unrelated objects.

## Evidence (verbatim from paper)

> Since the DETR in VLTVG has been pre-trained with BERT, the Acc50 performance revealed in our OV-VG dataset is 2.78%.

## Citation

```bibtex
@misc{wang2023ovvg,
  title={OV-VG: A Benchmark for Open-Vocabulary Visual Grounding},
  author={Wang et al. (2023)},
  year={2023},
  note={arXiv:2310.14374}
}
```

- arXiv: 2310.14374