Tsrec Eval

---
name: tsrec-eval
description: Evaluates a model's ability to perform sequential recommendation with a focus on capturing repeat-aware temporal patterns. It measures how well the model balances predicting new items versus recurring items based on user interaction history and time intervals. Use when the user wants to benchmark on RetailRocket, LastFM, Diginetica, or asks about evaluating this task. Reports HR@K.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.08531
  bibtex_key: quan2025tsrec
  confidence: high
---

# tsrec-eval

> TSRec: Enhancing Repeat-Aware Recommendation from a Temporal-Sequential Perspective — Quan et al. (2025) (arXiv:2506.08531, 2025)

## What this evaluates

Evaluates a model's ability to perform sequential recommendation with a focus on capturing repeat-aware temporal patterns. It measures how well the model balances predicting new items versus recurring items based on user interaction history and time intervals.

## Datasets

- **RetailRocket** — total ?; splits: train (-1), val (-1), test (-1); repo https://www.kaggle.com/retailrocket/ecommerce-dataset
- **LastFM** — total ?; splits: train (-1), val (-1), test (-1)
- **Diginetica** — total ?; splits: train (-1), val (-1), test (-1); repo http://cikm2016.cs.iupui.edu/cikm-cup

## Metrics

- `HR@K` **(primary)** — range: [0, 1]
  - HR@K = (1/|U|) * Σ_{u∈U} I(rank_u ≤ K), where I is an indicator function that is 1 if the ground-truth item appears in the top-K recommendations for user u, and 0 otherwise. It measures the proportion of users for whom the correct item is ranked within the top K.
- `NDCG@K` — range: [0, 1]
  - NDCG@K = (1/|U|) * Σ_{u∈U} (1/log2(rank_u+1)) / (1/log2(2)), averaged over users. It measures the quality of the ranking by discounting the relevance of the ground-truth item based on its position, with higher ranks receiving less weight.

## Input / output format

**Input**: User interaction history sequence (item IDs with timestamps) up to a maximum length L, used to predict the next item in the sequence.

**Output**: A ranked list of candidate items (top-K) for the next interaction, or a probability score for each item in the catalog.

## Scoring recipe

```python
def compute_metrics(predictions, ground_truth, K=20):
    hr_scores = []
    ndcg_scores = []
    for u in users:
        rank = predictions[u].index(ground_truth[u]) + 1
        hr_scores.append(1.0 if rank <= K else 0.0)
        ndcg_scores.append(1.0 / math.log2(rank + 1))
    return {
        f"HR@{K}": sum(hr_scores) / len(users),
        f"NDCG@{K}": sum(ndcg_scores) / len(users)
    }
```

## Common pitfalls

- Negative sampling strategy differs between validation (3 negatives) and test (100 negatives) sets, which can skew ranking metrics if not handled consistently.
- Data is split chronologically by timestamp rather than randomly, requiring strict temporal ordering to prevent data leakage.
- Explicit ratings are converted to implicit feedback, meaning the evaluation focuses on ranking accuracy rather than rating prediction.

## Evidence (verbatim from paper)

> Following (Kang and McAuley, [2016]; Sun et al., [2019]), we use the Top-K Hit Rate (HR@K) and Top-K Normalized Discounted Cumulative Gain (NDCG@K), which are commonly used in related research (Kang and McAuley, [2016]; Sun et al., [2019]). Our reported results are based on HR@{5, 10, 20} and NDCG@{5, 10, 20}. We calculate all metrics based on item ranking and report the average scores.

## Citation

```bibtex
@misc{quan2025tsrec,
  title={TSRec: Enhancing Repeat-Aware Recommendation from a Temporal-Sequential Perspective},
  author={Quan et al. (2025)},
  year={2025},
  note={arXiv:2506.08531}
}
```

- arXiv: 2506.08531