Clickbait Mitigation Eval

---
name: clickbait-mitigation-eval
description: This evaluation probes a recommender system's ability to mitigate clickbait by measuring performance exclusively on user interactions that result in positive post-click feedback (likes), rather than raw click-through rates. Use when the user wants to benchmark on Unspecified in provided section, or asks about evaluating this task. Reports post-click satisfaction (likes).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2009.09945
  bibtex_key: wang2020clickbait
  confidence: medium
---

# clickbait-mitigation-eval

> Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue — Wenjie Wang et al. (2020) (arXiv:2009.09945, 2020)

## What this evaluates

This evaluation probes a recommender system's ability to mitigate clickbait by measuring performance exclusively on user interactions that result in positive post-click feedback (likes), rather than raw click-through rates.

## Datasets

- **Unspecified in provided section** — total ?; splits: test (-1)

## Metrics

- `post-click satisfaction (likes)` **(primary)** — range: [0, 1]
  - Recommendation performance is computed exclusively on the subset of test interactions where the user provided positive post-click feedback (likes). Interactions lacking post-click feedback are excluded from evaluation.

## Input / output format

**Input**: User features $u$ and item features $i=(e,t)$, where $e$ denotes exposure features (e.g., title, cover) and $t$ denotes content features.

**Output**: A predicted preference score $Y_{u,i} = s_{\bar{\theta}}(u,i)$ used to rank items.

## Scoring recipe

```python
# 1. Filter test interactions to keep only those with positive post-click feedback (likes)
liked_interactions = [(u, i) for (u, i, y) in test_data if y == 'like']
# 2. Exclude interactions lacking post-click feedback
# 3. Compute standard recommendation performance metric (e.g., AUC/NDCG) on the filtered subset
metric_value = compute_recommender_metric(model, liked_interactions)
```

## Common pitfalls

- Conventional recommender evaluations treat all clicks as positive signals; this protocol explicitly excludes clicks without post-click feedback to isolate clickbait effects.
- Evaluating on raw click-through rate (CTR) will overestimate performance on clickbait items; this protocol requires post-click satisfaction signals.

## Evidence (verbatim from paper)

> Distinct from the conventional recommender evaluation that treats all clicks in the testing period as positive samples*(Wei et al., [2019](#bib.bib51 "")); He and McAuley, [2016](#bib.bib18 ""))*, we evaluate recommendation performance only over clicks that end with positive post-click feedback (i.e., likes)*(Wen et al., [2020](#bib.bib53 ""))*. We do not use the clicks that lack post-click feedback due to the unawareness of user satisfaction.

## Citation

```bibtex
@misc{wang2020clickbait,
  title={Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue},
  author={Wenjie Wang et al. (2020)},
  year={2020},
  note={arXiv:2009.09945}
}
```

- arXiv: 2009.09945