Repeatnet Session Rec Eval

---
name: repeatnet-session-rec-eval
description: Evaluates session-based recommendation models on predicting the next item in a user session. It probes the model's ability to capture sequential patterns and handle repeat consumption behaviors across e-commerce and music domains. Use when the user wants to benchmark on YOOCHOOSE, DIGINETICA, LASTFM, or asks about evaluating this task. Reports Recall@k.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1812.02646
  bibtex_key: ren2018repeatnet
  confidence: high
---

# repeatnet-session-rec-eval

> RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation — Pengjie Ren et al. (2018) (arXiv:1812.02646, 2018)

## What this evaluates

Evaluates session-based recommendation models on predicting the next item in a user session. It probes the model's ability to capture sequential patterns and handle repeat consumption behaviors across e-commerce and music domains.

## Datasets

- **YOOCHOOSE** — total ?; splits: train (5325971), val (591775), test (55898)
- **DIGINETICA** — total ?; splits: train (647532), val (71947), test (60858)
- **LASTFM** — total ?; splits: train (2690424), val (333537), test (338115)

## Metrics

- `Recall@k` **(primary)** — range: percent
  - The proportion of test cases where the ground-truth next item appears in the top-k recommended items.
- `MRR@k` — range: percent
  - The average of reciprocal ranks of the ground-truth items, truncated to 0 if the rank exceeds k.

## Input / output format

**Input**: A sequence of items visited by a user in a session, ordered by time.

**Output**: A ranked list of candidate items for the next interaction.

## Scoring recipe

```python
def compute_metrics(predictions, targets, k=20):
    recalls = []
    mrrs = []
    for pred_list, target in zip(predictions, targets):
        recalls.append(1.0 if target in pred_list[:k] else 0.0)
        rank = pred_list.index(target) + 1 if target in pred_list else k + 1
        mrrs.append(1.0 / rank if rank <= k else 0.0)
    return {
        f'Recall@{k}': sum(recalls) / len(recalls) * 100,
        f'MRR@{k}': sum(mrrs) / len(mrrs) * 100
    }
```

## Common pitfalls

- Filtering criteria differ across datasets (e.g., session length bounds, minimum item frequency).
- MRR@k truncates reciprocal rank to 0 if the target item is ranked beyond k.
- Datasets are split identically to prior work (Li et al. 2017b), not randomly.

## Evidence (verbatim from paper)

> Recall@k: The primary evaluation metric is Recall@k, which is the proportion of cases when the desired item is amongst the top-k items in all test cases.  MRR@k: Another used metric is MRR@k (Mean Reciprocal Rank), which is the average of reciprocal ranks of the desire items. The reciprocal rank is set to zero if the rank is larger than k.

## Citation

```bibtex
@misc{ren2018repeatnet,
  title={RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation},
  author={Pengjie Ren et al. (2018)},
  year={2018},
  note={arXiv:1812.02646}
}
```

- arXiv: 1812.02646