Iplotbench Eval

---
name: iplotbench-eval
description: Evaluates a visualization agent's ability to reconstruct interactive charts from static images and answer binary questions about them. It probes the agent's capacity for spec-grounded introspection and view-grounded interaction to resolve visual ambiguities like overlapping geometries. Use when the user wants to benchmark on iPlotBench, or asks about evaluating this task. Reports Question-level accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.21134
  bibtex_key: lu2026beyondpixels
  confidence: high
---

# iplotbench-eval

> Beyond Pixels: Introspective and Interactive Grounding for Visualization Agents — Lu et al. (2026) (arXiv:2604.21134, 2026)

## What this evaluates

Evaluates a visualization agent's ability to reconstruct interactive charts from static images and answer binary questions about them. It probes the agent's capacity for spec-grounded introspection and view-grounded interaction to resolve visual ambiguities like overlapping geometries.

## Datasets

- **iPlotBench** — total 500; splits: test (500); repo https://github.com/HexSys-lab/iPlotBench

## Metrics

- `Semantic Structural Similarity (S_Type, S_Data, S_Text, S_Style)` — range: [0, 1]
  - Measures fidelity of the recreated chart to the reference across four dimensions: trace typing, data fidelity, text extraction, and style. S_Data uses an exponential penalty parameter λ to score deviations from the ground-truth specification.
- `Question-level accuracy` **(primary)** — range: [0, 1]
  - Fraction of correctly answered binary questions. Per-figure accuracy averages accuracy within each figure before averaging across figures to avoid overweighting chart types with more questions.

## Input / output format

**Input**: Static reference image of a Plotly chart (Task 1) or the agent's recreated interactive figure plus a binary question (Task 2).

**Output**: Task 1: Recreated interactive chart code/spec. Task 2: Binary answer (yes/no) to the question.

## Scoring recipe

```python
def score_task1(recreated, reference, lambda=5):
    s_data = compute_fidelity(recreated, reference, penalty=lambda)
    s_type, s_text, s_style = compute_component_scores(recreated, reference)
    return s_type, s_data, s_text, s_style

def score_task2(predictions, golds):
    correct = sum(1 for p, g in zip(predictions, golds) if p == g)
    return correct / len(golds)

def per_figure_accuracy(fig_preds, fig_golds):
    fig_accs = [sum(p==g for p,g in zip(ps, gs))/len(gs) for ps, gs in zip(fig_preds, fig_golds)]
    return sum(fig_accs) / len(fig_accs)
```

## Common pitfalls

- Per-figure accuracy is computed by averaging within-figure accuracies first, then across figures, rather than a simple global accuracy, to prevent chart types with more questions from dominating the score.
- Conditional QA accuracy filters for reconstruction fidelity (S_Data >= 0.9), so low scores may reflect poor chart recreation rather than flawed reasoning.
- Tool usage (e.g., zoom, toggle) is optional; agents may rely solely on introspection or interaction, and adding interaction can sometimes distract from spec-level fixes during reconstruction.

## Evidence (verbatim from paper)

> As shown in Table 4, equipping agents with introspection (+Intro) drives the most significant gains in semantic reconstruction, achieving the highest scores in trace typing (S_Type), data fidelity (S_Data), and style (S_Style). While interaction (+Inter) slightly outperforms in text extraction (S_Text), likely by exposing occluded labels, it provides limited structural benefit.

## Citation

```bibtex
@misc{lu2026beyondpixels,
  title={Beyond Pixels: Introspective and Interactive Grounding for Visualization Agents},
  author={Lu et al. (2026)},
  year={2026},
  note={arXiv:2604.21134}
}
```

- arXiv: 2604.21134