Transfer Fraud Detection Eval

---
name: transfer-fraud-detection-eval
description: Evaluates machine learning models for detecting fraudulent bank transfers by optimizing instance-dependent cost-sensitive objectives. It probes the model's ability to minimize financial losses and maximize expected savings under highly imbalanced transaction data. Use when the user wants to benchmark on Credit Card Transaction Data, Bank data set, or asks about evaluating this task. Reports Expected Savings.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2005.02488
  bibtex_key: hoppner2020instance
  confidence: high
---

# transfer-fraud-detection-eval

> Instance-Dependent Cost-Sensitive Learning for Detecting Transfer Fraud — Höppner et al. (2020) (arXiv:2005.02488, 2020)

## What this evaluates

Evaluates machine learning models for detecting fraudulent bank transfers by optimizing instance-dependent cost-sensitive objectives. It probes the model's ability to minimize financial losses and maximize expected savings under highly imbalanced transaction data.

## Datasets

- **Credit Card Transaction Data** — total 284807; splits: full (284807)
- **Bank data set** — total 31763; splits: full (31763)

## Metrics

- `Expected Savings` **(primary)** — range: other
  - Net financial gain maximized by minimizing Average Expected Cost (AEC). Calculated as the difference between gains from correctly identified frauds and costs incurred from false positives.
- `Savings` — range: other
  - Total financial savings achieved by the model's binary decisions compared to a baseline, computed after applying instance-dependent cost-related thresholds.
- `Precision` — range: [0, 1]
  - Ratio of true positive fraud predictions to all positive predictions.
- `Recall` — range: [0, 1]
  - Ratio of true positive fraud predictions to all actual fraud cases.
- `F1` — range: [0, 1]
  - Harmonic mean of Precision and Recall.

## Input / output format

**Input**: Numerical and categorical features representing bank transactions (e.g., PCA-transformed features V1-V28, Time, Amount, or proprietary bank features).

**Output**: Binary classification decision (fraud or not) derived from predicted probabilities using an instance-dependent cost-related threshold.

## Scoring recipe

```python
tp = sum(y_true == 1 & y_pred == 1)
fp = sum(y_true == 0 & y_pred == 1)
fn = sum(y_true == 1 & y_pred == 0)
precision = tp / (tp + fp) if (tp + fp) > 0 else 0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
# Expected Savings & Savings are computed by applying instance-dependent cost thresholds to predicted probabilities, converting them to binary decisions, and calculating net financial gain/loss per transaction.
```

## Common pitfalls

- Folds must be stratified by both fraud label and transaction amount category (low/middle/high) to maintain distribution balance across the 5x2-fold cross-validation replications.
- Using standard fixed thresholds instead of instance-dependent cost-related thresholds leads to suboptimal financial performance despite potentially higher F1 scores.
- Optimizing purely for accuracy-related metrics (Precision, Recall, F1) can result in higher overall financial costs compared to cost-sensitive objectives.

## Evidence (verbatim from paper)

> All methods are evaluated using Savings, Expected Savings, Precision, Recall and $F_{1}$ measure where we use the instance-dependent thresholds. For each data set, we perform 5 replications of two-fold cross validation. To keep the analysis of the data sets manageable, we only consider main effects and we do not include interactions of any degree.

## Citation

```bibtex
@misc{hoppner2020instance,
  title={Instance-Dependent Cost-Sensitive Learning for Detecting Transfer Fraud},
  author={Höppner et al. (2020)},
  year={2020},
  note={arXiv:2005.02488}
}
```

- arXiv: 2005.02488