Booststep Eval

---
name: booststep-eval
description: This evaluation probes the mathematical reasoning capability of large language models, specifically focusing on single-step reasoning and the effectiveness of step-aligned in-context learning. It measures how well models can solve challenging math problems across text and multi-modal domains when provided with fine-grained, step-level examples. Use when the user wants to benchmark on MATH500, AQuA, OlympiadBench-TO, MATHBench, AMC-10, AMC-12, MathVision, MathVerse, AIME, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2501.03226
  bibtex_key: zhang2025booststep
  confidence: high
---

# booststep-eval

> BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning — Zhang et al. (2025) (arXiv:2501.03226, 2025)

## What this evaluates

This evaluation probes the mathematical reasoning capability of large language models, specifically focusing on single-step reasoning and the effectiveness of step-aligned in-context learning. It measures how well models can solve challenging math problems across text and multi-modal domains when provided with fine-grained, step-level examples.

## Datasets

- **MATH500** — total 500; splits: test (-1)
- **AQuA** — total ?; splits: test (-1)
- **OlympiadBench-TO** — total ?; splits: test (-1)
- **MATHBench** — total ?; splits: test (-1)
- **AMC-10** — total ?; splits: test (-1)
- **AMC-12** — total ?; splits: test (-1)
- **MathVision** — total ?; splits: test (-1)
- **MathVerse** — total ?; splits: test (-1)
- **AIME** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of problems where the model's final predicted answer exactly matches the ground truth answer. Calculated as (number of correct predictions / total number of problems) * 100.

## Input / output format

**Input**: A mathematical problem (text or image-based for multi-modal benchmarks) accompanied by a prompt containing step-aligned in-context examples retrieved via TF-IDF cosine similarity. For tree search variants, multiple reasoning candidates are generated.

**Output**: A step-by-step reasoning chain followed by a final answer. The model may generate multiple candidates during tree search, from which a verifier selects the best path.

## Scoring recipe

```python
correct = 0
for pred, gold in zip(predictions, gold_answers):
    if extract_final_answer(pred) == gold:
        correct += 1
accuracy = (correct / len(predictions)) * 100
```

## Common pitfalls

- Using grammatical segmentation (e.g., splitting by periods) instead of reasoning-content-based step splitting for the example problem bank, which significantly degrades performance.
- Assuming problem-level few-shot learning is universally superior; step-level ICL can fail or underperform if the retrieval strategy lacks the 'first-try' mechanism or if example similarity is too low.
- Forgetting to adjust the temperature parameter: it must be set to 0 for standard generation but 0.3 when using step-level tree search to allow random sampling.

## Evidence (verbatim from paper)

> Considering that the AIME consists of only 30 questions each year, making the results prone to fluctuations, we evaluated the questions three times annually and reported the average accuracy.

## Citation

```bibtex
@misc{zhang2025booststep,
  title={BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning},
  author={Zhang et al. (2025)},
  year={2025},
  note={arXiv:2501.03226}
}
```

- arXiv: 2501.03226