Dsin Ctr Eval

---
name: dsin-ctr-eval
description: This benchmark evaluates a model's ability to predict click-through rates (CTR) by leveraging session-aware user behavior sequences. It probes how well a system can decompose historical interactions into time-separated sessions, model cross-session interest evolution, and adaptively weight session interests relative to a target item. Use when the user wants to benchmark on Advertising Dataset, Recommender Dataset, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1905.06482
  bibtex_key: feng2019dsin
  confidence: high
---

# dsin-ctr-eval

> Deep Session Interest Network for Click-Through Rate Prediction — Feng et al. (2019) (arXiv:1905.06482, 2019)

## What this evaluates

This benchmark evaluates a model's ability to predict click-through rates (CTR) by leveraging session-aware user behavior sequences. It probes how well a system can decompose historical interactions into time-separated sessions, model cross-session interest evolution, and adaptively weight session interests relative to a target item.

## Datasets

- **Advertising Dataset** — total 26000000; splits: train (-1), test (-1)
- **Recommender Dataset** — total 6000000000; splits: train (-1), test (-1)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - AUC = (1 / (m^+ * m^-)) * Σ_{x^+ ∈ D^+} Σ_{x^- ∈ D^-} I(f(x^+) > f(x^-)), where D^+ and D^- are positive and negative examples, f(·) is the model's prediction score, and I(·) is the indicator function. It measures the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative instance.

## Input / output format

**Input**: User ID, target item ID, and a sequence of up to 200 historical user behaviors (display/click logs) partitioned into time-separated sessions.

**Output**: A continuous prediction score (probability) representing the likelihood of the user clicking the target item.

## Scoring recipe

```python
def compute_auc(predictions, labels):
    pos_scores = [p for p, l in zip(predictions, labels) if l == 1]
    neg_scores = [p for p, l in zip(predictions, labels) if l == 0]
    if not pos_scores or not neg_scores:
        return 0.0
    concordant = sum(1 for ps in pos_scores for ns in neg_scores if ps > ns)
    return concordant / (len(pos_scores) * len(neg_scores))
```

## Common pitfalls

- The dataset splits are strictly time-based (specific dates in 2017 and 2018), not random, so temporal leakage must be avoided.
- User behavior sequences are capped at the recent 200 interactions; truncating or padding incorrectly will alter session boundaries and model performance.
- AUC is computed over all positive/negative pairs in the test set; using approximation methods or incorrect negative sampling can skew results.

## Evidence (verbatim from paper)

> AUC (Area Under ROC Curve) reflects the ranking ability of the model. It is defined as follows: $$ \mathrm {A U C} = \frac {1}{m ^ {+} m ^ {-}} \sum_ {x ^ {+} \in D ^ {+}} \sum_ {x ^ {-} \in D ^ {-}} (I (f (x ^ {+}) > f (x ^ {-}))) \tag {11} $$ where $D^{+}$ is the collection of all positive examples, $D^{-}$ is the collection of all negative examples, $f(\cdot)$ is the result of the model's prediction of the sample $\mathbf{x}$ and $I(\cdot)$ is the indicator function.

## Citation

```bibtex
@misc{feng2019dsin,
  title={Deep Session Interest Network for Click-Through Rate Prediction},
  author={Feng et al. (2019)},
  year={2019},
  note={arXiv:1905.06482}
}
```

- arXiv: 1905.06482