Fettadbench Eval

---
name: fettadbench-eval
description: This benchmark evaluates federated time-series anomaly detection systems by measuring how well models maintain detection accuracy when trained across decentralized clients compared to centralized baselines. It probes the robustness of anomaly detection architectures under federated learning protocols and varying degrees of non-IID data partitioning. Use when the user wants to benchmark on Time-series anomaly detection datasets (specific names not provided in excerpt), or asks about evaluating this task. Reports detection accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2212.09518
  bibtex_key: [REDACTED]
  confidence: medium
---

# fettadbench-eval

> FedTADBench: Federated Time-Series Anomaly Detection Benchmark — Liu et al. (2022) (arXiv:2212.09518, 2022)

## What this evaluates

This benchmark evaluates federated time-series anomaly detection systems by measuring how well models maintain detection accuracy when trained across decentralized clients compared to centralized baselines. It probes the robustness of anomaly detection architectures under federated learning protocols and varying degrees of non-IID data partitioning.

## Datasets

- **Time-series anomaly detection datasets (specific names not provided in excerpt)** — total ?; splits: train (-1), test (-1); repo https://github.com/fanxingliu2020/FedTADBench

## Metrics

- `detection accuracy` **(primary)** — range: [0, 1]
  - Standard anomaly detection metric computed on test splits, typically AUC-ROC or F1-score, measuring the model's ability to distinguish normal from anomalous time-series segments.
- `training efficiency` — range: other
  - Measures the computational and communication cost of federated training, typically tracked via convergence time, communication rounds, or epoch count.

## Input / output format

**Input**: Multivariate or univariate time-series sequences partitioned across multiple federated clients, each containing timestamped sensor readings and corresponding ground-truth anomaly labels.

**Output**: Per-timestep anomaly scores or binary classification labels indicating normal vs. anomalous behavior.

## Scoring recipe

```python
def compute_detection_accuracy(predictions, labels):
    tp = sum(p == 1 and l == 1 for p, l in zip(predictions, labels))
    fp = sum(p == 1 and l == 0 for p, l in zip(predictions, labels))
    fn = sum(p == 0 and l == 1 for p, l in zip(predictions, labels))
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    return f1
```

## Common pitfalls

- Ignoring the impact of non-IID data partitioning across clients, which significantly skews federated performance compared to centralized baselines.
- Evaluating only final model accuracy without tracking federated training efficiency (communication rounds, convergence time) and stability across heterogeneous clients.
- Applying centralized evaluation protocols directly to federated settings without accounting for client dropout or partial data availability.

## Evidence (verbatim from paper)

> evaluating five representative anomaly detection models across three dimensions: (1) performance degradation under federated learning vs. centralized training, (2) compatibility of four federated learning methods (FedAvg, FedProx, SCAFFOLD, MOON) with anomaly detection models, including training efficiency, and (3) impact of client data heterogeneity (partitioning) on detection accuracy.

## Citation

```bibtex
@misc{liu2022fettadbench,
  title={FedTADBench: Federated Time-Series Anomaly Detection Benchmark},
  author={Liu et al. (2022)},
  year={2022},
  note={arXiv:2212.09518}
}
```

- arXiv: 2212.09518