Venusbench Gd Eval

---
name: venusbench-gd-eval
description: This benchmark evaluates GUI grounding capabilities across a hierarchical taxonomy of basic (element, visual, spatial) and advanced (functional, reasoning, refusal) tasks. It probes a model's ability to accurately locate UI elements in screenshots and handle complex, domain-specific, or unanswerable instructions across web, mobile, and desktop platforms. Use when the user wants to benchmark on VenusBench-GD, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.16501
  bibtex_key: [REDACTED]
  confidence: high
---

# venusbench-gd-eval

> VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks — Zhou et al. (2025) (arXiv:2512.16501, 2025)

## What this evaluates

This benchmark evaluates GUI grounding capabilities across a hierarchical taxonomy of basic (element, visual, spatial) and advanced (functional, reasoning, refusal) tasks. It probes a model's ability to accurately locate UI elements in screenshots and handle complex, domain-specific, or unanswerable instructions across web, mobile, and desktop platforms.

## Datasets

- **VenusBench-GD** — total 6166; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - A prediction is correct if the predicted point lies within the ground-truth bounding box. For refusal tasks, correctness requires strict adherence to a prescribed refusal format (e.g., coordinates [-1,-1] or a specific rejection phrase). The final metric is the accuracy averaged over all UI elements in the benchmark.

## Input / output format

**Input**: A GUI screenshot image paired with a natural language instruction or query.

**Output**: Predicted coordinates (e.g., [x, y]) or a designated refusal phrase/format.

## Scoring recipe

```python
correct = 0
total = 0
for inst in dataset:
    pred = model.predict(inst.image, inst.instruction)
    if inst.task_type == 'refusal':
        if pred == inst.refusal_format:
            correct += 1
    else:
        if point_in_box(pred, inst.gt_box):
            correct += 1
    total += 1
return (correct / total) * 100
```

## Common pitfalls

- Refusal tasks require strict format adherence rather than coordinate proximity, which many models fail to follow.
- Basic grounding tasks are near-saturation for modern VLMs, so evaluating only element/visual/spatial accuracy will not differentiate model capabilities.
- Advanced tasks (functional/reasoning) require domain-specific knowledge and robustness to out-of-distribution queries, which specialized models often lack due to overfitting.

## Evidence (verbatim from paper)

> Consistent with previous works, we define a prediction as correct when the predicted point lies within the ground-truth bounding box and report the accuracy averaged over all UI elements. For refusal grounding tasks, we consider a model’s output correct only when it adheres strictly to the prescribed refusal format, such as returning coordinates $[-1,-1]$ or a designated rejection phrase as specified in the prompt.

## Citation

```bibtex
@misc{zhou2025venusbenchgd,
  title={VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks},
  author={Zhou et al. (2025)},
  year={2025},
  note={arXiv:2512.16501}
}
```

- arXiv: 2512.16501