Session Rec Eval

---
name: session-rec-eval
description: This evaluation probes a model's ability to predict the next item in a user session based on historical interactions. It measures ranking quality across multiple benchmark datasets, testing how well the model captures temporal patterns and prospective user preferences without relying on fixed recency heuristics. Use when the user wants to benchmark on Six session-based recommendation benchmarks (DG, GA, YC, TM, LF, NP), or asks about evaluating this task. Reports recall@k, MRR@k, NDCG@k.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2206.01875
  bibtex_key: peng2022prospective
  confidence: high
---

# session-rec-eval

> Prospective Preference Enhanced Mixed Attentive Model for Session-based Recommendation — Bo Peng et al. (arXiv:2206.01875, 2022)

## What this evaluates

This evaluation probes a model's ability to predict the next item in a user session based on historical interactions. It measures ranking quality across multiple benchmark datasets, testing how well the model captures temporal patterns and prospective user preferences without relying on fixed recency heuristics.

## Datasets

- **Six session-based recommendation benchmarks (DG, GA, YC, TM, LF, NP)** — total ?; splits: test (-1)

## Metrics

- `recall@k, MRR@k, NDCG@k` **(primary)** — range: [0, 1]
  - Recall@k is 1 if the gold item appears in the top-k predictions, else 0. MRR@k is 1/rank if the gold item is in top-k, else 0. NDCG@k is 1/log2(rank+1) normalized by ideal DCG. Evaluated at k=10 and k=20.

## Input / output format

**Input**: A sequence of item IDs representing a user's session history.

**Output**: A ranked list of candidate item IDs (top-k) for the next item prediction.

## Scoring recipe

```python
def score(predictions, gold):
    recall = 1.0 if gold in predictions[:10] else 0.0
    mrr = 0.0
    ndcg = 0.0
    for i, item in enumerate(predictions[:10]):
        if item == gold:
            mrr = 1.0 / (i + 1)
            ndcg = 1.0 / math.log2(i + 2)
            break
    return recall, mrr, ndcg
```

## Common pitfalls

- Failing to exclude items already present in the session history from the candidate pool during evaluation.
- Using fixed recency-based heuristics (e.g., last item embedding) instead of the model's learned prospective preference estimates.
- Ignoring position embeddings, which causes significant performance drops on datasets with strong temporal patterns.

## Evidence (verbatim from paper)

> Table 3 presents the overall performance of different methods at recall@k, MRR@k and NDCG@k in recommending the next item.

## Citation

```bibtex
@misc{peng2022prospective,
  title={Prospective Preference Enhanced Mixed Attentive Model for Session-based Recommendation},
  author={Bo Peng et al.},
  year={2022},
  note={arXiv:2206.01875}
}
```

- arXiv: 2206.01875