Lightgcn Rec Eval

---
name: lightgcn-rec-eval
description: Evaluates the ability of graph-based collaborative filtering models to rank relevant items for users based on sparse user-item interaction graphs. It probes how well neighborhood aggregation and embedding smoothing capture latent preferences without relying on node semantic features. Use when the user wants to benchmark on Gowalla, Yelp2018, Amazon-Book, or asks about evaluating this task. Reports recall@20.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2002.02126
  bibtex_key: he2020lightgcn
  confidence: high
---

# lightgcn-rec-eval

> LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation — He et al. (2020) (arXiv:2002.02126, 2020)

## What this evaluates

Evaluates the ability of graph-based collaborative filtering models to rank relevant items for users based on sparse user-item interaction graphs. It probes how well neighborhood aggregation and embedding smoothing capture latent preferences without relying on node semantic features.

## Datasets

- **Gowalla** — total 1027370; splits: train (-1), test (-1)
- **Yelp2018** — total 1561406; splits: train (-1), test (-1)
- **Amazon-Book** — total 2984108; splits: train (-1), test (-1)

## Metrics

- `recall@20` **(primary)** — range: [0, 1]
  - Recall@20 measures the fraction of a user's ground-truth interacted items that appear in the top-20 recommended items. It is averaged across all users in the test set.
- `ndcg@20` — range: [0, 1]
  - NDCG@20 measures the quality of the top-20 ranked list by weighting hits by their position using logarithmic discounting, normalized by the ideal DCG. It is averaged across all users in the test set.

## Input / output format

**Input**: User and item identifiers with a bipartite interaction graph representing observed user-item engagements.

**Output**: A ranked list of candidate items for each user, evaluated at the top-20 cutoff.

## Scoring recipe

```python
def compute_metrics(predictions, ground_truth, k=20):
    recalls, ndcgs = [], []
    for u in ground_truth:
        relevant = ground_truth[u]
        ranked = predictions[u][:k]
        hits = sum(1 for item in ranked if item in relevant)
        recalls.append(hits / len(relevant))
        dcg = sum(hits_at_i / math.log2(i + 2) for i, item in enumerate(ranked) if item in relevant)
        idcg = sum(1 / math.log2(i + 2) for i in range(min(len(relevant), k)))
        ndcgs.append(dcg / idcg if idcg > 0 else 0)
    return {'recall@20': sum(recalls)/len(recalls), 'ndcg@20': sum(ndcgs)/len(ndcgs)}
```

## Common pitfalls

- Using a fixed negative sampling set instead of the all-ranking protocol (the paper explicitly states all non-interacted items are candidates).
- Including cold-start items in the test set without filtering, which was a known issue in the original Yelp2018 split requiring a revised version.
- Comparing models trained or evaluated on different train-test splits, as the authors explicitly requested the exact same splits from the NGCF authors for fairness.

## Evidence (verbatim from paper)

> The evaluation metrics are recall@20 and ndcg@20 computed by the all-ranking protocol — all items that are not interacted by a user are the candidates.

## Citation

```bibtex
@misc{he2020lightgcn,
  title={LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation},
  author={He et al. (2020)},
  year={2020},
  note={arXiv:2002.02126}
}
```

- arXiv: 2002.02126