Showdown Clicks Eval

---
name: showdown-clicks-eval
description: Evaluates a vision-language model's ability to accurately predict the correct UI element to click based on a screenshot and a task instruction. It isolates low-level visual grounding and interaction skills in ambiguous or icon-heavy interfaces. Use when the user wants to benchmark on Showdown-Clicks, or asks about evaluating this task. Reports Top-1 Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2508.16987
  bibtex_key: bhathal2025websight
  confidence: high
---

# showdown-clicks-eval

> WebSight: A Vision-First Architecture for Robust Web Agents — Bhathal et al. (2025) (arXiv:2508.16987, 2025)

## What this evaluates

Evaluates a vision-language model's ability to accurately predict the correct UI element to click based on a screenshot and a task instruction. It isolates low-level visual grounding and interaction skills in ambiguous or icon-heavy interfaces.

## Datasets

- **Showdown-Clicks** — total 5679; splits: test (5679), dev (557)

## Metrics

- `Top-1 Accuracy` **(primary)** — range: percent
  - Percentage of test cases where the model's predicted click location exactly matches the ground-truth human click. Calculated as (correct predictions / total examples) * 100.

## Input / output format

**Input**: A screenshot of a macOS desktop/UI element and a natural language task instruction.

**Output**: A single click coordinate (x, y) or UI element identifier corresponding to the target.

## Scoring recipe

```python
correct = 0
for pred, gold in zip(predictions, golds):
    if pred == gold:
        correct += 1
accuracy = (correct / len(golds)) * 100
```

## Common pitfalls

- Ambiguous UI contexts where multiple elements could satisfy the instruction.
- Difficulty distinguishing interactive icons from static visual elements or text labels.
- Model may output actions like scrolling or text input when a direct click is required.

## Evidence (verbatim from paper)

> The showdown-clicks track contains 5679 human-collected left-click events on macOS, with a public dev subset of 557 examples. ... WEBSIGHT-7B attains 58.84% accuracy, achieving higher accuracy than VLMs with almost 10x more parameters. Table 1. Top-1 Accuracy on the Showdown/Clicks Benchmark [59]

## Citation

```bibtex
@misc{bhathal2025websight,
  title={WebSight: A Vision-First Architecture for Robust Web Agents},
  author={Bhathal et al. (2025)},
  year={2025},
  note={arXiv:2508.16987}
}
```

- arXiv: 2508.16987