Multi Session Rec Eval

---
name: multi-session-rec-eval
description: Evaluates a model's ability to perform next-item recommendation by leveraging both current session context and historical multi-session information. It probes how well the model handles varying session lengths (short vs. long) and filters out noise from irrelevant historical sessions. Use when the user wants to benchmark on Delicious, Reddit, or asks about evaluating this task. Reports Recall@20.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2205.04366
  bibtex_key: wang2022effectively
  confidence: high
---

# multi-session-rec-eval

> Effectively Using Long and Short Sessions for Multi-Session-based Recommendations — Wang et al. (2022) (arXiv:2205.04366, 2022)

## What this evaluates

Evaluates a model's ability to perform next-item recommendation by leveraging both current session context and historical multi-session information. It probes how well the model handles varying session lengths (short vs. long) and filters out noise from irrelevant historical sessions.

## Datasets

- **Delicious** — total ?; splits: train (-1), val (-1), test (-1)
- **Reddit** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `Recall@5` — range: [0, 1]
  - Proportion of ground-truth next items that appear in the top-5 recommended items, averaged over all test queries.
- `Recall@20` **(primary)** — range: [0, 1]
  - Proportion of ground-truth next items that appear in the top-20 recommended items, averaged over all test queries.
- `MRR@5` — range: [0, 1]
  - Mean Reciprocal Rank at 5: average of 1/rank for the first relevant item in the top-5 list, or 0 if none appear.
- `MRR@20` — range: [0, 1]
  - Mean Reciprocal Rank at 20: average of 1/rank for the first relevant item in the top-20 list, or 0 if none appear.

## Input / output format

**Input**: An ordered sequence of item IDs representing the user's current session, optionally augmented with historical session sequences or long-term interest features.

**Output**: A ranked list of candidate item IDs (top-K) predicted for the next interaction.

## Scoring recipe

```python
def compute_metrics(preds, golds, k=20):
    recalls = []
    mrrs = []
    for pred, gold in zip(preds, golds):
        top_k = pred[:k]
        if gold in top_k:
            recalls.append(1.0)
            rank = top_k.index(gold) + 1
            mrrs.append(1.0 / rank)
        else:
            recalls.append(0.0)
            mrrs.append(0.0)
    return sum(recalls) / len(recalls), sum(mrrs) / len(mrrs)
```

## Common pitfalls

- The train/val/test splits overlap (train: 0-80%, val: 70-80%, test: 80-100%), meaning validation and test sets share user history. This can leak information if historical sessions are not strictly partitioned.
- Sessions are split using a strict 3600-second time gap, which differs from standard SBR benchmarks that often use fixed session IDs or different gap thresholds.
- Item frequency (<10) and session length (<2, >20) filtering are applied iteratively, which can drastically reduce dataset size compared to raw downloads.

## Evidence (verbatim from paper)

> So we choose the widely used ranking metrics, i.e., Recall@20 and Recall@5, and Mean Reciprocal Rank MRR@20 and MRR@5 to evaluate the recommendation performance in the experiments. ... Then we divide the sessions of each user into training set, validation set and test set according to the proportion of 0 - 80%, 70 - 80% and 80 - 100%...

## Citation

```bibtex
@misc{wang2022effectively,
  title={Effectively Using Long and Short Sessions for Multi-Session-based Recommendations},
  author={Wang et al. (2022)},
  year={2022},
  note={arXiv:2205.04366}
}
```

- arXiv: 2205.04366