Chartverse Eval

---
name: chartverse-eval
description: Evaluates visual language models on complex chart understanding and reasoning tasks, including data extraction, comparison, and trend analysis across diverse chart types and domains. Use when the user wants to benchmark on ChartQA-Pro, CharXiv, ChartMuseum, ChartX, ChartBench, EvoChart, or asks about evaluating this task. Reports average score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.13606
  bibtex_key: [REDACTED]
  confidence: medium
---

# chartverse-eval

> ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch — Zheng Liu et al. (arXiv:2601.13606, 2026)

## What this evaluates

Evaluates visual language models on complex chart understanding and reasoning tasks, including data extraction, comparison, and trend analysis across diverse chart types and domains.

## Datasets

- **ChartQA-Pro** — total ?; splits: test (-1)
- **CharXiv** — total ?; splits: test (-1)
- **ChartMuseum** — total ?; splits: test (-1)
- **ChartX** — total ?; splits: test (-1)
- **ChartBench** — total ?; splits: test (-1)
- **EvoChart** — total ?; splits: test (-1)

## Metrics

- `average score` **(primary)** — range: percent
  - Arithmetic mean of the per-benchmark accuracy or score values across all evaluated chart reasoning datasets.

## Input / output format

**Input**: Chart image (PNG/JPG) paired with a natural language question requiring reasoning or data extraction.

**Output**: Natural language text containing the final answer, often preceded by chain-of-thought reasoning.

## Scoring recipe

```python
scores = []
for dataset in [ChartQA-Pro, CharXiv, ChartMuseum, ChartX, ChartBench, EvoChart]:
    preds = model.generate(image, question)
    acc = compute_accuracy(preds, gold_answers)
    scores.append(acc)
average_score = sum(scores) / len(scores)
return average_score
```

## Common pitfalls

- The reported 'average score' aggregates heterogeneous metrics from different benchmarks without specifying normalization or weighting schemes.
- The paper does not disclose exact evaluation prompts, temperature settings, or decoding parameters used during inference.

## Evidence (verbatim from paper)

> We evaluate performance across 6 benchmarks necessitating complex chart understanding and reasoning: ChartQA-Pro [[26]], CharXiv [[35]], ChartMuseum [[33]], ChartX [[36]], ChartBench [[40]], and EvoChart [[9]]. ... ChartVerse-2B achieves an average score of 54.3, exceeding all chart-specific baselines, including ECD-7B, START-7B, and Chart-R1-7B.

## Citation

```bibtex
@misc{liu2026chartverse,
  title={ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch},
  author={Zheng Liu et al.},
  year={2026},
  note={arXiv:2601.13606}
}
```

- arXiv: 2601.13606