Ttsr Eval

---
name: ttsr-eval
description: Evaluates the ability of LLMs to improve reasoning performance at test time through self-reflection and targeted variant question synthesis, without external supervision. It probes how well a model can adapt its policy to difficult mathematical and general reasoning problems by diagnosing its own failures and generating corrective training signals. Use when the user wants to benchmark on AMC23, MATH-500, Minerva, OlympiadBench, AIME 2024, AIME 2025, GPQA-Diamond, MMLU-Pro, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.03297
  bibtex_key: he2026ttsr
  confidence: high
---

# ttsr-eval

> TTSR: Test-Time Self-Reflection for Continual Reasoning Improvement — He et al. (2026) (arXiv:2603.03297, 2026)

## What this evaluates

Evaluates the ability of LLMs to improve reasoning performance at test time through self-reflection and targeted variant question synthesis, without external supervision. It probes how well a model can adapt its policy to difficult mathematical and general reasoning problems by diagnosing its own failures and generating corrective training signals.

## Datasets

- **AMC23** — total ?; splits: test (-1)
- **MATH-500** — total ?; splits: test (-1)
- **Minerva** — total ?; splits: test (-1)
- **OlympiadBench** — total ?; splits: test (-1)
- **AIME 2024** — total ?; splits: test (-1)
- **AIME 2025** — total ?; splits: test (-1)
- **GPQA-Diamond** — total ?; splits: test (-1)
- **MMLU-Pro** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of test instances where the model's final extracted answer exactly matches the ground truth label.

## Input / output format

**Input**: A mathematical or general reasoning problem statement.

**Output**: A step-by-step reasoning trajectory followed by a final answer.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_answers):
    correct = sum(1 for p, g in zip(predictions, gold_answers) if p.strip() == g.strip())
    return (correct / len(predictions)) * 100
```

## Common pitfalls

- Test-time updates may overfit to the specific adaptation instances rather than learning reusable reasoning patterns, leading to poor cross-dataset generalization.
- Self-consistency based pseudo-labels can be highly noisy when the majority of sampled trajectories are incorrect, potentially destabilizing the policy update if not properly regularized.

## Evidence (verbatim from paper)

> We report accuracy (%) or the official score for each benchmark.

## Citation

```bibtex
@misc{he2026ttsr,
  title={TTSR: Test-Time Self-Reflection for Continual Reasoning Improvement},
  author={He et al. (2026)},
  year={2026},
  note={arXiv:2603.03297}
}
```

- arXiv: 2603.03297