H2seqrec Eval

---
name: h2seqrec-eval
description: Evaluates sequential recommendation models on predicting the next item a user will interact with, capturing temporal dynamics and handling sparse user-item interactions. Use when the user wants to benchmark on AMT, Goodreads, or asks about evaluating this task. Reports HR@K, NDCG@K.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2108.08134
  bibtex_key: li2021hyperbolic
  confidence: high
---

# h2seqrec-eval

> Hyperbolic Hypergraphs for Sequential Recommendation — Li et al. (2021) (arXiv:2108.08134, 2021)

## What this evaluates

Evaluates sequential recommendation models on predicting the next item a user will interact with, capturing temporal dynamics and handling sparse user-item interactions.

## Datasets

- **AMT** — total 59495; splits: train (-1), val (-1), test (-1)
- **Goodreads** — total 1084781; splits: train (-1), val (-1), test (-1)

## Metrics

- `HR@K` **(primary)** — range: [0, 1]
  - Hit Ratio at rank K: binary indicator (1 if ground truth item appears in top-K predicted list, 0 otherwise). Evaluated at K={1,5,10,20}.
- `NDCG@K` **(primary)** — range: [0, 1]
  - Normalized Discounted Cumulative Gain at rank K: sum of graded relevance (1 for hit) discounted by log2(rank+1), normalized by ideal DCG. Evaluated at K={1,5,10,20}.

## Input / output format

**Input**: User's historical interaction sequence (ordered list of item IDs).

**Output**: Ranked list of candidate items (top-K predictions or full ranking over 101/501 items including sampled negatives).

## Scoring recipe

```python
def compute_metrics(predictions, ground_truth, k=10):
    top_k = predictions[:k]
    hr = 1.0 if ground_truth in top_k else 0.0
    ndcg = 0.0
    if ground_truth in top_k:
        rank = top_k.index(ground_truth) + 1
        ndcg = 1.0 / math.log2(rank + 1)
    return hr, ndcg
```

## Common pitfalls

- Negative sampling size (100 vs 500) drastically changes absolute metric values; results are not directly comparable across different sampling settings.
- Datasets use time-based chronological splits (last interactions in a specific year are validation/test), not random user/item splits.
- Evaluation is performed over a sampled candidate set (101 or 501 items) rather than the full item catalog due to computational constraints.

## Evidence (verbatim from paper)

> Our proposed method focuses on recommending next item, and therefore we use Top K Hit Ratio (HR@K) and Top K Normalized Discounted Cumulative Gain (NDCG@K) as our evaluation metrics. We choose K={1,5,10,20} in the baseline comparison experiment. ... in our experiment, we randomly choose {100,500} negative samples and rank {101,501} items to calculate the HR@K and NDCG@K scores.

## Citation

```bibtex
@misc{li2021hyperbolic,
  title={Hyperbolic Hypergraphs for Sequential Recommendation},
  author={Li et al. (2021)},
  year={2021},
  note={arXiv:2108.08134}
}
```

- arXiv: 2108.08134