Bwgnn Anomaly Detection Eval

---
name: bwgnn-anomaly-detection-eval
description: Evaluates graph neural networks and baselines for node anomaly detection in heterogeneous and homogeneous graphs. It probes the model's ability to identify fraudulent or anomalous accounts/users based on node features and graph structure under both supervised and semi-supervised settings. Use when the user wants to benchmark on YelpChi, Amazon, T-Finance, T-Social, or asks about evaluating this task. Reports F1-macro.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2205.15508
  bibtex_key: tang2022rethinking
  confidence: high
---

# bwgnn-anomaly-detection-eval

> Rethinking Graph Neural Networks for Anomaly Detection — Tang et al. (2022) (arXiv:2205.15508, 2022)

## What this evaluates

Evaluates graph neural networks and baselines for node anomaly detection in heterogeneous and homogeneous graphs. It probes the model's ability to identify fraudulent or anomalous accounts/users based on node features and graph structure under both supervised and semi-supervised settings.

## Datasets

- **YelpChi** — total ?; splits: train (-1), val (-1), test (-1)
- **Amazon** — total ?; splits: train (-1), val (-1), test (-1)
- **T-Finance** — total ?; splits: train (-1), val (-1), test (-1)
- **T-Social** — total ?; splits: train (-1), val (-1), test (-1)

## Metrics

- `F1-macro` **(primary)** — range: percent
  - The unweighted mean of the F1-score of two classes (normal and anomaly), which neglects the imbalance ratio between normal and anomaly labels.
- `AUC` — range: [0, 1]
  - The area under the Receiver Operating Characteristic (ROC) Curve, measuring the model's ability to discriminate between normal and anomalous nodes across all classification thresholds.

## Input / output format

**Input**: Graph data containing node features and edge connections. Multi-relational graphs are processed either by treating all edges as a single homogeneous type or by performing separate propagation per relation.

**Output**: Binary anomaly label (normal vs. anomaly) or continuous anomaly score for each node.

## Scoring recipe

```python
def compute_metrics(y_true, y_scores, threshold=0.5):
    y_pred = (y_scores >= threshold).astype(int)
    f1_normal = f1_score(y_true, y_pred, pos_label=0)
    f1_anomaly = f1_score(y_true, y_pred, pos_label=1)
    f1_macro = (f1_normal + f1_anomaly) / 2
    auc = roc_auc_score(y_true, y_scores)
    return f1_macro, auc
```

## Common pitfalls

- F1-macro is used specifically to ignore class imbalance between normal and anomaly nodes, unlike standard accuracy or weighted F1.
- Multi-relational graphs (YelpChi, Amazon) require explicit handling strategies (homogeneous vs. heterogeneous propagation) that drastically affect results.
- Semi-supervised settings use extremely low training ratios (1% or 0.01%), making performance highly sensitive to the exact number of labeled anomalies.

## Evidence (verbatim from paper)

> Metrics. We choose two widely used metrics to measure the performance of all the methods, namely F1-macro and AUC. F1-macro is the unweighted mean of the F1-score of two classes, which neglects the imbalance ratio between normal and anomaly labels. AUC (Davis & Goadrich, 2006) is the area under the ROC Curve.

## Citation

```bibtex
@misc{tang2022rethinking,
  title={Rethinking Graph Neural Networks for Anomaly Detection},
  author={Tang et al. (2022)},
  year={2022},
  note={arXiv:2205.15508}
}
```

- arXiv: 2205.15508