Shpi Recommendation Eval

---
name: shpi-recommendation-eval
description: Evaluates offline reinforcement learning methods for session-based recommendation systems in optimizing long-term user retention versus short-term clicks. It tests the ability of algorithms to learn from fixed logging policies and generalize to online rollouts across synthetic, simulated, and real-world recommendation environments. Use when the user wants to benchmark on Synthetic recommendation problem, RecoGym, HIV treatment simulator, Private dataset X, or asks about evaluating this task. Reports undiscounted test performance on true environment rewards.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2106.00589
  bibtex_key: mazoure2021improving
  confidence: high
---

# shpi-recommendation-eval

> Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning — Mazoure et al. (2021) (arXiv:2106.00589, 2021)

## What this evaluates

Evaluates offline reinforcement learning methods for session-based recommendation systems in optimizing long-term user retention versus short-term clicks. It tests the ability of algorithms to learn from fixed logging policies and generalize to online rollouts across synthetic, simulated, and real-world recommendation environments.

## Datasets

- **Synthetic recommendation problem** — total ?; splits: (unstated)
- **RecoGym** — total ?; splits: (unstated)
- **HIV treatment simulator** — total ?; splits: (unstated)
- **Private dataset X** — total 120000; splits: (unstated)

## Metrics

- `undiscounted test performance on true environment rewards` **(primary)** — range: other
  - Mean of true environment rewards collected over 200 online rollouts, averaged across 5 random seeds. Higher values indicate better long-term performance.

## Input / output format

**Input**: State/context features (e.g., moving average of previous contexts, user features, action features) and available action set.

**Output**: Discrete action selection (recommendation) from the available action space.

## Scoring recipe

```python
rewards = []
for seed in range(5):
  for rollout in range(200):
    state = env.reset()
    total_r = 0
    while not done:
      action = policy(state)
      state, reward, done, _ = env.step(action)
      total_r += reward
    rewards.append(total_r)
return sum(rewards) / len(rewards)
```

## Common pitfalls

- The evaluation is strictly offline-to-online: models are trained on a fixed dataset D from a logging policy and then evaluated via online rollouts, not standard offline test sets.
- Long-term reward (LTR) is distinct from short-term click rewards; optimizing for clicks does not guarantee high LTR due to policy-induced drift.
- The private dataset lacks explicit train/val/test splits; it is used as a single fixed offline dataset for training, with evaluation done via simulation.

## Evidence (verbatim from paper)

> All experiments report undiscounted test performance on true environment rewards over 200 rollouts and 5 random seeds.

## Citation

```bibtex
@misc{mazoure2021improving,
  title={Improving Long-Term Metrics in Recommendation Systems using Short-Horizon Reinforcement Learning},
  author={Mazoure et al. (2021)},
  year={2021},
  note={arXiv:2106.00589}
}
```

- arXiv: 2106.00589