Sparse Rl Eval

---
name: sparse-rl-eval
description: This evaluation protocol assesses the mathematical reasoning capabilities of LLMs trained with sparse reinforcement learning under strict memory constraints. It measures how well models maintain accuracy on standard math benchmarks when policy rollouts are generated using compressed KV caches instead of full context. Use when the user wants to benchmark on GSM8K, MATH500, Gaokao, Minerva Math, OlympiadBench, AIME24, AMC23, or asks about evaluating this task. Reports Pass@1 / Avg@32 accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.10079
  bibtex_key: luo2026sparserl
  confidence: high
---

# sparse-rl-eval

> Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts — Luo et al. (2026) (arXiv:2601.10079, 2026)

## What this evaluates

This evaluation protocol assesses the mathematical reasoning capabilities of LLMs trained with sparse reinforcement learning under strict memory constraints. It measures how well models maintain accuracy on standard math benchmarks when policy rollouts are generated using compressed KV caches instead of full context.

## Datasets

- **GSM8K** — total ?; splits: test (-1)
- **MATH500** — total ?; splits: test (-1)
- **Gaokao** — total ?; splits: test (-1)
- **Minerva Math** — total ?; splits: test (-1)
- **OlympiadBench** — total ?; splits: test (-1)
- **AIME24** — total ?; splits: test (-1); HF `AI-MO/aimo-validation-aime`
- **AMC23** — total ?; splits: test (-1); HF `AI-MO/aimo-validation-amc`

## Metrics

- `Pass@1 / Avg@32 accuracy` **(primary)** — range: [0, 1]
  - Pass@1: binary accuracy (1 if the single generated response matches the gold answer, 0 otherwise), averaged over the dataset. Avg@32: for each problem, sample 32 responses, compute the mean accuracy across the 32 samples, then average these means across all problems in the dataset.

## Input / output format

**Input**: Mathematical reasoning problem prompts.

**Output**: Text response containing reasoning steps and a final answer.

## Scoring recipe

```python
def compute_metric(predictions, golds, metric_type):
    scores = []
    for pred, gold in zip(predictions, golds):
        if metric_type == 'pass1':
            scores.append(1.0 if pred.strip() == gold.strip() else 0.0)
        elif metric_type == 'avg32':
            scores.append(sum(1.0 if p.strip() == gold.strip() else 0.0 for p in pred) / 32.0)
    return sum(scores) / len(scores)
```

## Common pitfalls

- For AIME24 and AMC23, failing to sample 32 responses per problem and average them will drastically underestimate performance compared to the reported Avg@32 metric.
- Confusing the training KV cache budget (512 tokens) with inference constraints; the bonus evaluation explicitly enforces the same R-KV compression during inference, which shifts the distribution if not replicated.
- Using dense rollout evaluation for the main Sparse-RL results instead of sparse rollout evaluation, which would invalidate the memory-constrained performance claims.

## Evidence (verbatim from paper)

> We evaluate the performance of our method on 7 standard mathematical reasoning benchmarks: GSM8K*Cobbe et al. ([2021])*, MATH500*Hendrycks et al. ([2021])*, Gaokao*Zhang et al. ([2023a])*, Minerva Math*Lewkowycz et al. ([2022])*, OlympiadBench*He et al. ([2024])*, AIME24111huggingface.co/datasets/AI-MO/aimo-validation-aime and AMC23222huggingface.co/datasets/AI-MO/aimo-validation-amc. For AIME24 and AMC23, we report Avg@32: for each problem, we sample 32 responses and calculate the mean accuracy among these 32 responses; the final score is the average of the mean accuracy for each item over the dataset. For the other six benchmarks, we report Pass@1: generating one response per problem.

## Citation

```bibtex
@misc{luo2026sparserl,
  title={Sparse-RL: Breaking the Memory Wall in LLM Reinforcement Learning via Stable Sparse Rollouts},
  author={Luo et al. (2026)},
  year={2026},
  note={arXiv:2601.10079}
}
```

- arXiv: 2601.10079