Credit Card Fraud Detection Eval

---
name: credit-card-fraud-detection-eval
description: Evaluates a model's ability to detect fraudulent credit card transactions in a streaming context by learning topological and sequential patterns from transaction graphs without manual feature engineering. Use when the user wants to benchmark on Credit Card Transaction Dataset (Feb-Sep), or asks about evaluating this task. Reports AP.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2501.19267
  bibtex_key: deng2025transformerfinancialfraud
  confidence: high
---

# credit-card-fraud-detection-eval

> Transformer-Based Financial Fraud Detection with Cloud-Optimized Real-Time Streaming — Deng et al. (2025) (arXiv:2501.19267, 2025)

## What this evaluates

Evaluates a model's ability to detect fraudulent credit card transactions in a streaming context by learning topological and sequential patterns from transaction graphs without manual feature engineering.

## Datasets

- **Credit Card Transaction Dataset (Feb-Sep)** — total 141861; splits: train (-1), test (-1)

## Metrics

- `AP` **(primary)** — range: [0, 1]
  - Average Precision: AP = (1/N) * Σ P(n) for n=1 to N, measuring precision at each threshold across the dataset.
- `AUC` — range: [0, 1]
  - Area Under the ROC Curve: AUC = ∫ TPR(FPR) d(FPR) from 0 to 1, measuring the overall ability to distinguish between fraudulent and normal transactions.

## Input / output format

**Input**: Original credit card transaction records (merchant number, card number, amount, etc.) converted into a dynamic transaction graph where nodes represent transactions/entities and edges represent relationships.

**Output**: Fraud probability score for each transaction.

## Scoring recipe

```python
def compute_metrics(predictions, labels):
    # predictions: list of fraud probabilities
    # labels: list of binary fraud labels (1=fraud, 0=normal)
    # Calculate AP (Average Precision)
    ap = sum(precision_at_threshold(t) for t in thresholds) / len(thresholds)
    # Calculate AUC (Area Under ROC Curve)
    auc = trapezoidal_integration(tpr, fpr)
    return ap, auc
```

## Common pitfalls

- Temporal split: training on Feb-Jun and testing on Jul-Sep, so results are not independent and identically distributed across months.
- Imbalanced data: requires negative sampling on normal transactions, which can skew precision/recall if not handled consistently.
- Evaluation reported monthly (July, August, September) rather than as a single aggregate metric, making cross-month comparison sensitive to seasonal fraud patterns.

## Evidence (verbatim from paper)

> The experimental data set in this paper is based on partial credit card transaction records from February 1 to September 30 of a certain year. ... During the training process, data from February 1 to June 30 were selected as the training set and data from July 1 to September 30 as the test set. ... In this experiment, AUC (Area Under the ROC Curve) and AP (Average Precision) were used as the measurement indexes of credit card fraudulent transaction detection tasks.

## Citation

```bibtex
@misc{deng2025transformerfinancialfraud,
  title={Transformer-Based Financial Fraud Detection with Cloud-Optimized Real-Time Streaming},
  author={Deng et al. (2025)},
  year={2025},
  note={arXiv:2501.19267}
}
```

- arXiv: 2501.19267