Cotrec Eval

---
name: cotrec-eval
description: Evaluates a model's ability to predict the next item in a user session based on historical click sequences. It probes the model's capacity to capture sequential dependencies and session-level patterns in sparse e-commerce or media interaction data. Use when the user wants to benchmark on Tmall, RetailRocket, Diginetica, or asks about evaluating this task. Reports P@10.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2108.10560
  bibtex_key: xia2021selfsupervised
  confidence: high
---

# cotrec-eval

> Self-Supervised Graph Co-Training for Session-based Recommendation — Xin Xia et al. (arXiv:2108.10560, 2021)

## What this evaluates

Evaluates a model's ability to predict the next item in a user session based on historical click sequences. It probes the model's capacity to capture sequential dependencies and session-level patterns in sparse e-commerce or media interaction data.

## Datasets

- **Tmall** — total ?; splits: train (-1), test (-1)
- **RetailRocket** — total ?; splits: train (-1), test (-1)
- **Diginetica** — total ?; splits: train (-1), test (-1)

## Metrics

- `P@10` **(primary)** — range: percent
  - Precision at K: the proportion of test instances where the ground-truth next item appears in the model's top-K recommended list.
- `MRR@10` — range: percent
  - Mean Reciprocal Rank at K: the average of 1/rank for each test instance, where rank is the position of the ground-truth item in the top-K list (0 if not in top-K).

## Input / output format

**Input**: A sequence of item IDs representing a user's session history (e.g., [item_1, item_2, ..., item_m]).

**Output**: A ranked list of top-K predicted item IDs for the next click.

## Scoring recipe

```python
def compute_metrics(predictions, golds, K=10):
    prec_scores = []
    mrr_scores = []
    for pred_list, gold in zip(predictions, golds):
        top_k = pred_list[:K]
        prec_scores.append(1.0 if gold in top_k else 0.0)
        if gold in top_k:
            rank = top_k.index(gold) + 1
            mrr_scores.append(1.0 / rank)
        else:
            mrr_scores.append(0.0)
    return {
        f'P@{K}': sum(prec_scores) / len(prec_scores) * 100,
        f'MRR@{K}': sum(mrr_scores) / len(mrr_scores) * 100
    }
```

## Common pitfalls

- The dataset preprocessing splits each original session into multiple training/test sequences via a sliding window, which must be replicated exactly to avoid data leakage or mismatched evaluation counts.
- The paper reports metrics as percentages (e.g., 13.10), but standard implementations often output [0,1] decimals; ensure consistent scaling.
- Items appearing fewer than 5 times and sessions of length 1 are filtered out before splitting; omitting this step will inflate performance.

## Evidence (verbatim from paper)

> Following [38, 41], we use P@K (Precision) and MRR@K (Mean Reciprocal Rank) to evaluate the recommendation results where K is 10 or 20.

## Citation

```bibtex
@misc{xia2021selfsupervised,
  title={Self-Supervised Graph Co-Training for Session-based Recommendation},
  author={Xin Xia et al.},
  year={2021},
  note={arXiv:2108.10560}
}
```

- arXiv: 2108.10560