Cross Domain Sequential Rec Eval

---
name: cross-domain-sequential-rec-eval
description: This evaluation probes a model's ability to perform cross-domain sequential recommendation by leveraging user interaction histories across two domains, even when user overlap is minimal or absent. It measures how well the model captures domain-specific and shared sequential patterns to rank candidate items accurately. Use when the user wants to benchmark on Micro Video, Amazon, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2311.08272
  bibtex_key: lin2023mixed
  confidence: high
---

# cross-domain-sequential-rec-eval

> Mixed Attention Network for Cross-domain Sequential Recommendation — Guanyu Lin et al. (arXiv:2311.08272, 2023)

## What this evaluates

This evaluation probes a model's ability to perform cross-domain sequential recommendation by leveraging user interaction histories across two domains, even when user overlap is minimal or absent. It measures how well the model captures domain-specific and shared sequential patterns to rank candidate items accurately.

## Datasets

- **Micro Video** — total ?; splits: (unstated)
- **Amazon** — total ?; splits: (unstated)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve. Measures the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance.
- `MRR` — range: [0, 1]
  - Mean Reciprocal Rank. The average of the reciprocal of the rank of the first relevant item in the predicted list.
- `NDCG` — range: [0, 1]
  - Normalized Discounted Cumulative Gain. Measures ranking quality by discounting the relevance of items based on their position in the list, normalized by the ideal DCG.
- `WAUC` — range: [0, 1]
  - Weighted AUC. AUC metric weighted by user activity or sequence length to account for data sparsity and user engagement differences.

## Input / output format

**Input**: User item interaction sequences (history) from one or two domains, optionally with the second domain's sequence empty for single-domain baselines.

**Output**: Predicted relevance scores or ranked list of candidate items for the next interaction.

## Scoring recipe

```python
def compute_metrics(predictions, gold):
    auc_scores = [roc_auc_score(g, p) for g, p in zip(gold, predictions)]
    mrr_scores = [reciprocal_rank(g, p) for g, p in zip(gold, predictions)]
    ndcg_scores = [ndcg_at_k(g, p, k=10) for g, p in zip(gold, predictions)]
    return {
        'AUC': np.mean(auc_scores),
        'MRR': np.mean(mrr_scores),
        'NDCG@10': np.mean(ndcg_scores)
    }
```

## Common pitfalls

- Assuming cross-domain recommenders require fully overlapped users; the paper shows baselines like PiNet and DASL degrade significantly without user overlap.
- Training both domains simultaneously on a shared backbone causes optimization conflict and negative transfer, which the authors explicitly avoid by using separate local and global modules.

## Evidence (verbatim from paper)

> All models are evaluated on two popular accuracy metrics AUC and GAUC [8], as well as two ranking metrics, MRR and NDCG [2].

## Citation

```bibtex
@misc{lin2023mixed,
  title={Mixed Attention Network for Cross-domain Sequential Recommendation},
  author={Guanyu Lin et al.},
  year={2023},
  note={arXiv:2311.08272}
}
```

- arXiv: 2311.08272