Contrastvae Eval

---
name: contrastvae-eval
description: Evaluates sequential recommendation models on Amazon review datasets, measuring ranking quality for next-item prediction. It specifically probes performance on long-tail items, sequence sparsity, and robustness to noisy inputs. Use when the user wants to benchmark on Amazon Beauty, Amazon Toys, Amazon Tools, Amazon Office, or asks about evaluating this task. Reports Recall@20.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2209.00456
  bibtex_key: wang2022contrastvae
  confidence: high
---

# contrastvae-eval

> ContrastVAE: Contrastive Variational AutoEncoder for Sequential Recommendation — Wang et al. (2022) (arXiv:2209.00456, 2022)

## What this evaluates

Evaluates sequential recommendation models on Amazon review datasets, measuring ranking quality for next-item prediction. It specifically probes performance on long-tail items, sequence sparsity, and robustness to noisy inputs.

## Datasets

- **Amazon Beauty** — total 198502; splits: train (-1), val (-1), test (-1)
- **Amazon Toys** — total 167597; splits: train (-1), val (-1), test (-1)
- **Amazon Tools** — total 134476; splits: train (-1), val (-1), test (-1)
- **Amazon Office** — total 53258; splits: train (-1), val (-1), test (-1)

## Metrics

- `Recall@20` **(primary)** — range: [0, 1]
  - Fraction of relevant items found in the top-20 recommended items. Calculated as |predicted_top20 ∩ relevant| / |relevant|.
- `Recall@40` — range: [0, 1]
  - Fraction of relevant items found in the top-40 recommended items. Calculated as |predicted_top40 ∩ relevant| / |relevant|.
- `NDCG@20` — range: [0, 1]
  - Normalized Discounted Cumulative Gain at rank 20. Measures ranking quality by discounting gains logarithmically based on position.
- `NDCG@40` — range: [0, 1]
  - Normalized Discounted Cumulative Gain at rank 40. Measures ranking quality by discounting gains logarithmically based on position.

## Input / output format

**Input**: A chronological sequence of user-item interactions (implicit feedback) sorted by timestamp.

**Output**: A ranked list of top-N candidate items for recommendation.

## Scoring recipe

```python
def recall_at_n(pred_list, relevant_set, n):
    top_n = set(pred_list[:n])
    return len(top_n & relevant_set) / len(relevant_set)

def ndcg_at_n(pred_list, relevant_set, n):
    dcg = sum(1.0 / math.log2(i + 2) for i, item in enumerate(pred_list[:n]) if item in relevant_set)
    idcg = sum(1.0 / math.log2(i + 2) for i in range(min(len(relevant_set), n)))
    return dcg / idcg if idcg > 0 else 0.0
```

## Common pitfalls

- Failing to filter users and items with fewer than 5 interactions before splitting the data.
- Using random splits instead of chronological order (last item for test, penultimate for validation, rest for training).
- Treating rating scores as explicit feedback instead of implicit binary interactions.

## Evidence (verbatim from paper)

> We compute each user's relevance scores for all items and choose items with top-N scores for the recommendation. Then we adopt two widely used top-N metrics, Recall and NDCG, as our top-N ranking evaluation metrics. We report the experimental results when N = 20 and N = 40.

## Citation

```bibtex
@misc{wang2022contrastvae,
  title={ContrastVAE: Contrastive Variational AutoEncoder for Sequential Recommendation},
  author={Wang et al. (2022)},
  year={2022},
  note={arXiv:2209.00456}
}
```

- arXiv: 2209.00456