Ragen Agent Eval

---
name: ragen-agent-eval
description: Evaluates LLM agents' multi-turn decision-making and reasoning capabilities across symbolic planning, risk-sensitive reasoning, and realistic web interaction environments. It probes the agent's ability to complete interactive tasks under noisy or probabilistic feedback while maintaining exploration and training stability. Use when the user wants to benchmark on Bandit, Sokoban, Frozen Lake, WebShop, or asks about evaluating this task. Reports success rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2504.20073
  bibtex_key: wang2025ragen
  confidence: high
---

# ragen-agent-eval

> RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning — Wang et al. (2025) (arXiv:2504.20073, 2025)

## What this evaluates

Evaluates LLM agents' multi-turn decision-making and reasoning capabilities across symbolic planning, risk-sensitive reasoning, and realistic web interaction environments. It probes the agent's ability to complete interactive tasks under noisy or probabilistic feedback while maintaining exploration and training stability.

## Datasets

- **Bandit** — total 256; splits: val (256)
- **Sokoban** — total 256; splits: val (256)
- **Frozen Lake** — total 256; splits: val (256)
- **WebShop** — total 256; splits: val (256)

## Metrics

- `success rate` **(primary)** — range: [0, 1]
  - Fraction of the 256 fixed prompts where the agent successfully completes the task within the turn limit.
- `rollout entropy` — range: other
  - Entropy of the action distribution across N=16 rollouts per prompt, measuring exploration diversity.
- `in-group reward variability` — range: other
  - Variance or standard deviation of rewards within a batch of rollouts, indicating behavioral diversity.
- `response length` — range: other
  - Average number of actions or tokens generated per episode, capped at 5 turns and 10 actions per turn.
- `gradient norm` — range: other
  - L2 norm of the policy gradients computed during updates, used to monitor training stability.

## Input / output format

**Input**: 256 fixed prompts per environment, processed by the LLM agent in an interactive environment with a maximum of 5 turns and 10 actions per turn.

**Output**: Agent's action sequence (up to 10 actions per turn, max 5 turns) and final task completion status.

## Scoring recipe

```python
def compute_metrics(prompts, agent_actions, rewards, gradients):
    success_rate = sum(1 for r in rewards if r == 1) / len(rewards)
    rollout_entropy = compute_entropy(agent_actions)
    reward_var = np.var(rewards)
    response_len = np.mean([len(actions) for actions in agent_actions])
    grad_norm = np.linalg.norm(gradients)
    return success_rate, rollout_entropy, reward_var, response_len, grad_norm
```

## Common pitfalls

- Confusing training diagnostics (gradient norm, in-group reward variability) with final evaluation metrics; these are computed over validation instances but primarily track training dynamics.
- Ignoring the hard truncation limit of 5 turns and 10 actions per turn, which artificially caps response length and distorts entropy/variability calculations if not enforced.
- Using evaluation temperature T=0.5 inconsistently; the paper explicitly fixes T=0.5 during evaluation, which significantly impacts exploration and success rates compared to greedy decoding.

## Evidence (verbatim from paper)

> We evaluate on 256 fixed prompts per environment with temperature T=0.5, truncating episodes after 5 turns. Metrics include: (i) success rate (task completion), (ii) rollout entropy (exploration), (iii) in-group reward variability (behavioral diversity), (iv) response length (reasoning verbosity), and (v) gradient norm (training stability). All are computed over validation instances.

## Citation

```bibtex
@misc{wang2025ragen,
  title={RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning},
  author={Wang et al. (2025)},
  year={2025},
  note={arXiv:2504.20073}
}
```

- arXiv: 2504.20073