Fcn Ctr Eval

---
name: fcn-ctr-eval
description: Evaluates the effectiveness, efficiency, and interpretability of explicit feature interaction models for click-through rate (CTR) prediction on large-scale, highly sparse advertising and recommendation datasets. Use when the user wants to benchmark on Avazu, Criteo, ML-1M, KDD12, iPinYou, KKBox, or asks about evaluating this task. Reports AUC.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2407.13349
  bibtex_key: li2024fcn
  confidence: high
---

# fcn-ctr-eval

> FCN: Fusing Exponential and Linear Cross Network for Click-Through Rate Prediction — Li et al. (2024) (arXiv:2407.13349, 2024)

## What this evaluates

Evaluates the effectiveness, efficiency, and interpretability of explicit feature interaction models for click-through rate (CTR) prediction on large-scale, highly sparse advertising and recommendation datasets.

## Datasets

- **Avazu** — total 40428967; splits: train/val/test (-1)
- **Criteo** — total 45840617; splits: train/val/test (-1)
- **ML-1M** — total 739012; splits: train/val/test (-1)
- **KDD12** — total 141371038; splits: train/val/test (-1)
- **iPinYou** — total 19495974; splits: train/val/test (-1)
- **KKBox** — total 7377418; splits: train/val/test (-1)

## Metrics

- `AUC` **(primary)** — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve. Measures the probability that a randomly chosen positive instance is ranked higher than a randomly chosen negative one.
- `Logloss` — range: other
  - Negative log-likelihood loss. Calculated as -1/N * sum(y_true * log(y_pred) + (1 - y_true) * log(1 - y_pred)). Lower values indicate better model fit.

## Input / output format

**Input**: Sparse categorical and numerical feature vectors representing user-item interactions. Numerical features are discretized via floor(log^2(x)) for x>2, and infrequent categories are replaced with an 'OOV' token.

**Output**: A single scalar probability score representing the likelihood of a click.

## Scoring recipe

```python
import numpy as np
from sklearn.metrics import roc_auc_score, log_loss

def compute_metrics(y_true, y_pred):
    auc = roc_auc_score(y_true, y_pred)
    ll = log_loss(y_true, y_pred)
    return {'AUC': auc, 'Logloss': ll}
```

## Common pitfalls

- Small absolute improvements (e.g., 0.1% in AUC or 0.001 in Logloss) are considered statistically significant and meaningful in CTR tasks.
- Dataset class imbalance can cause Logloss to plateau at similar low values (e.g., ~0.0055) across models, making AUC a more reliable differentiator.
- Parameter count does not directly correlate with training runtime or time complexity; some lightweight models have high time complexity.

## Evidence (verbatim from paper)

> To compare the performance, we utilize two commonly used metrics in CTR models: Logloss, AUC (Song et al., [2019]; Wang et al., [2023a]; Zhu et al., [2022b]). AUC stands for Area Under the ROC Curve, which measures the probability that a positive instance will be ranked higher than a randomly chosen negative one. Logloss is the result of the calculation of L in Equation [7]. A lower Logloss suggests a better capacity for fitting the data.

## Citation

```bibtex
@misc{li2024fcn,
  title={FCN: Fusing Exponential and Linear Cross Network for Click-Through Rate Prediction},
  author={Li et al. (2024)},
  year={2024},
  note={arXiv:2407.13349}
}
```

- arXiv: 2407.13349