Temporal Graph Anomaly Eval

---
name: temporal-graph-anomaly-eval
description: Evaluates the ability of various data-driven models to detect emerging anomalies in temporal graphs derived from social media interactions. It probes how well different architectures generalize across different social platforms and remain robust to parameter variations and temporal/spatial shifts. Use when the user wants to benchmark on Twitter, Facebook, or asks about evaluating this task. Reports weighted F1 score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2307.05268
  bibtex_key: lazebnik2023benchmarking
  confidence: high
---

# temporal-graph-anomaly-eval

> Temporal Graphs Anomaly Emergence Detection: Benchmarking For Social Media Interactions — Lazebnik et al. (2023) (arXiv:2307.05268, 2023)

## What this evaluates

Evaluates the ability of various data-driven models to detect emerging anomalies in temporal graphs derived from social media interactions. It probes how well different architectures generalize across different social platforms and remain robust to parameter variations and temporal/spatial shifts.

## Datasets

- **Twitter** — total ?; splits: (unstated)
- **Facebook** — total ?; splits: (unstated)

## Metrics

- `weighted F1 score` **(primary)** — range: [0, 1]
  - Harmonic mean of precision and recall, weighted by the support (number of true instances) for each class. Reported as mean ± standard deviation over 100 instances per dataset.

## Input / output format

**Input**: Temporal graph data representing social media interactions.

**Output**: Anomaly scores or binary labels indicating the presence of emerging anomalies.

## Scoring recipe

```python
def weighted_f1(y_true, y_pred, labels):
    precisions, recalls, supports = precision_recall_fscore_support(y_true, y_pred, labels=labels, average=None)
    weights = supports / supports.sum()
    f1s = 2 * (precisions * recalls) / (precisions + recalls + 1e-8)
    return np.average(f1s, weights=weights)
```

## Common pitfalls

- Performance is highly dataset-dependent; Facebook consistently yields lower scores than Twitter across all models.
- No single model consistently outperforms others across both datasets or sensitivity tests, indicating poor generalizability.
- Sensitivity to prediction lag and temporal concept drift varies significantly by model architecture, complicating hyperparameter tuning.

## Evidence (verbatim from paper)

> Fig. [2](#S4.F2 "Figure 2 ‣ 4 Results ‣ Temporal Graphs Anomaly Emergence Detection: Benchmarking For Social Media Interactions") summarizes the main results obtained where Figs. [2(a)](#S4.F2.sf1 "In Figure 2 ‣ 4 Results ‣ Temporal Graphs Anomaly Emergence Detection: Benchmarking For Social Media Interactions") and 2(b) show the weighted $F_{1}$ score of each model for the Twitter and Facebook datasets, respectively. The results are shown as the mean $\pm$ standard deviation of $n\=100$ instances for each dataset.

## Citation

```bibtex
@misc{lazebnik2023benchmarking,
  title={Temporal Graphs Anomaly Emergence Detection: Benchmarking For Social Media Interactions},
  author={Lazebnik et al. (2023)},
  year={2023},
  note={arXiv:2307.05268}
}
```

- arXiv: 2307.05268