Mlp4rec Eval

---
name: mlp4rec-eval
description: Evaluates a model's ability to predict the next item in a user's sequential interaction history. It probes the model's capacity to capture temporal dependencies, cross-channel correlations in item embeddings, and cross-feature interactions using only MLP-based architectures. Use when the user wants to benchmark on MovieLens-100k, Amazon Beauty, or asks about evaluating this task. Reports NDCG@10.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2204.11510
  bibtex_key: li2022mlp4rec
  confidence: high
---

# mlp4rec-eval

> MLP4Rec: A Pure MLP Architecture for Sequential Recommendations — Li et al. (2022) (arXiv:2204.11510, 2022)

## What this evaluates

Evaluates a model's ability to predict the next item in a user's sequential interaction history. It probes the model's capacity to capture temporal dependencies, cross-channel correlations in item embeddings, and cross-feature interactions using only MLP-based architectures.

## Datasets

- **MovieLens-100k** — total 100000; splits: train (-1), val (-1), test (-1)
- **Amazon Beauty** — total 2023070; splits: train (-1), val (-1), test (-1)

## Metrics

- `HR@10` — range: [0, 1]
  - Binary indicator of whether the ground-truth item appears in the model's top-10 recommendations.
- `NDCG@10` **(primary)** — range: [0, 1]
  - Measures the quality of the top-10 ranked list by weighting the ground-truth item's position with a logarithmic discount factor.
- `MRR@10` — range: [0, 1]
  - Averages the reciprocal of the rank of the ground-truth item within the top-10 recommendations across all test instances.

## Input / output format

**Input**: A zero-padded sequence of item IDs and associated features (e.g., brand, category) up to a maximum length of 50, representing a user's historical interactions.

**Output**: A ranked list of top-10 candidate items predicted for the next interaction.

## Scoring recipe

```python
def compute_metrics(preds, golds, k=10):
    hr, ndcg, mrr = 0.0, 0.0, 0.0
    for pred, gold in zip(preds, golds):
        top_k = pred[:k]
        if gold in top_k:
            hr += 1.0
            rank = top_k.index(gold) + 1
            mrr += 1.0 / rank
            ndcg += 1.0 / math.log2(rank + 1)
    n = len(preds)
    return hr/n, ndcg/n, mrr/n
```

## Common pitfalls

- Negative sampling must pair exactly 100 negative items with each ground-truth item during prediction, as specified in the evaluation settings.
- Users and items with fewer than 5 interactions must be filtered out before dataset splitting to match the reported statistics.
- The maximum sequence length is fixed at 50 with zero-padding; ignoring this padding can cause misalignment during evaluation.

## Evidence (verbatim from paper)

> Metrics. We apply three commonly used evaluation metrics in recommender system, namely hit ratio (HR), normalized discounted cumulative gain (NDCG) and Mean Reciprocal Rank (MRR). HR measures the probability of the ground-truth item that appears in model's top-K recommendation, NDCG measures the order of the top-K recommended items generated by the recommender, and finally MRR takes the reciprocal of the ground-truth item's ranking in top-K recommendation and averages across all the evaluated items.

## Citation

```bibtex
@misc{li2022mlp4rec,
  title={MLP4Rec: A Pure MLP Architecture for Sequential Recommendations},
  author={Li et al. (2022)},
  year={2022},
  note={arXiv:2204.11510}
}
```

- arXiv: 2204.11510