Entropy Minimization Reasoning Eval

---
name: entropy-minimization-reasoning-eval
description: Evaluates the reasoning capabilities of LLMs on mathematical and coding benchmarks by measuring accuracy under various inference-time scaling and unsupervised entropy minimization techniques. It probes whether reducing output uncertainty improves correctness without labeled data or parameter updates. Use when the user wants to benchmark on AMC, AIME, Minerva, LeetCode Live Contest, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2505.15134
  bibtex_key: agarwal2025entropyminimization
  confidence: medium
---

# entropy-minimization-reasoning-eval

> The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning — Agarwal et al. (2025) (arXiv:2505.15134, 2025)

## What this evaluates

Evaluates the reasoning capabilities of LLMs on mathematical and coding benchmarks by measuring accuracy under various inference-time scaling and unsupervised entropy minimization techniques. It probes whether reducing output uncertainty improves correctness without labeled data or parameter updates.

## Datasets

- **AMC** — total ?; splits: test (-1)
- **AIME** — total ?; splits: test (-1)
- **Minerva** — total ?; splits: test (-1)
- **LeetCode Live Contest** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly solved problems out of the total number of evaluated instances. Calculated as (correct predictions / total predictions) * 100.

## Input / output format

**Input**: Natural language prompts containing mathematical problems or coding tasks requiring code generation.

**Output**: Model-generated text containing either a mathematical answer or executable code.

## Scoring recipe

```python
correct = 0
for pred, gold in zip(predictions, golds):
    if check_correctness(pred, gold):
        correct += 1
return (correct / len(golds)) * 100
```

## Common pitfalls

- EM-INF reduces output diversity, which can cause performance gains to diminish when combined with iterative self-refinement methods that rely on diverse generations.
- Accuracy metrics are reported across heterogeneous benchmarks (math vs. coding) with different difficulty levels, making direct averaging potentially misleading without stratification.

## Evidence (verbatim from paper)

> Accuracy vs. FLOPs for combining EM-INF and self-consistency at inference time on AMC. As expected, EM-INF reduces the diversity of the generation, which causes it’s benefits to diminish over iterations.

## Citation

```bibtex
@misc{agarwal2025entropyminimization,
  title={The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning},
  author={Agarwal et al. (2025)},
  year={2025},
  note={arXiv:2505.15134}
}
```

- arXiv: 2505.15134