Super Ddqn Eval

---
name: super-ddqn-eval
description: Evaluates a decentralized multi-agent reinforcement learning algorithm that selectively shares high-temporal-difference-error experiences. It probes cooperative and competitive multi-agent coordination, credit assignment, and communication efficiency in anonymous environments with separate per-agent reward signals. Use when the user wants to benchmark on PettingZoo (Pursuit, Battle, Adversarial-Pursuit), or asks about evaluating this task. Reports total mean episode reward.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2311.00865
  bibtex_key: gerstgrasser2023super
  confidence: high
---

# super-ddqn-eval

> Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning — Gerstgrasser et al. (2023) (arXiv:2311.00865, 2023)

## What this evaluates

Evaluates a decentralized multi-agent reinforcement learning algorithm that selectively shares high-temporal-difference-error experiences. It probes cooperative and competitive multi-agent coordination, credit assignment, and communication efficiency in anonymous environments with separate per-agent reward signals.

## Datasets

- **PettingZoo (Pursuit, Battle, Adversarial-Pursuit)** — total ?; splits: train (-1); repo https://github.com/Farama-Foundation/PettingZoo

## Metrics

- `total mean episode reward` **(primary)** — range: other
  - Average of the sum of rewards obtained by all agents (or the designated sharing team) across all evaluation episodes.

## Input / output format

**Input**: Per timestep: individual observations for each agent, joint action space, and per-agent reward signals. Environment state evolves according to multi-agent dynamics.

**Output**: Action selected by each agent's independent policy network at each timestep.

## Scoring recipe

```python
def compute_metric(rollouts):
    episode_rewards = []
    for rollout in rollouts:
        episode_rewards.append(sum(rollout.rewards))
    return sum(episode_rewards) / len(episode_rewards)
```

## Common pitfalls

- Pre-training opposing teams in adversarial domains (Battle, Adversarial-Pursuit) before main training is strictly required; failing to do so breaks the evaluation setup.
- Sharing all experiences indiscriminately ('all' bandwidth) actually degrades performance compared to no sharing, contrary to intuition.
- Parameter sharing is a centralized baseline that shares policy weights, not joint control; confusing the two misrepresents the comparison.

## Evidence (verbatim from paper)

> For Pursuit, performance is the total mean episode reward from all agents. For Battle and Adversarial-Pursuit, performance is the total mean episode reward from all agents in the sharing team (blue team in Battle, prey team in Adversarial-Pursuit).

## Citation

```bibtex
@misc{gerstgrasser2023super,
  title={Selectively Sharing Experiences Improves Multi-Agent Reinforcement Learning},
  author={Gerstgrasser et al. (2023)},
  year={2023},
  note={arXiv:2311.00865}
}
```

- arXiv: 2311.00865