Nab Yahoo Anomaly Detection Eval

---
name: nab-yahoo-anomaly-detection-eval
description: Evaluates unsupervised anomaly detection algorithms on streaming time-series data. It probes the model's ability to identify point, contextual, and collective anomalies in highly imbalanced real-world and synthetic datasets without labeled training data. Use when the user wants to benchmark on Numenta Anomaly Benchmark, Yahoo Anomaly Dataset, or asks about evaluating this task. Reports F-measure.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1912.08785
  bibtex_key: maciag2019unsupervised
  confidence: high
---

# nab-yahoo-anomaly-detection-eval

> Unsupervised Anomaly Detection in Stream Data with Online Evolving Spiking Neural Networks — Maciąg et al. (2019) (arXiv:1912.08785, 2019)

## What this evaluates

Evaluates unsupervised anomaly detection algorithms on streaming time-series data. It probes the model's ability to identify point, contextual, and collective anomalies in highly imbalanced real-world and synthetic datasets without labeled training data.

## Datasets

- **Numenta Anomaly Benchmark** — total 58; splits: test (-1)
- **Yahoo Anomaly Dataset** — total 367; splits: test (-1)

## Metrics

- `Precision` — range: [0, 1]
  - |TP| / (|TP| + |FP|)
- `Recall` — range: [0, 1]
  - |TP| / (|TP| + |FN|)
- `F-measure` **(primary)** — range: [0, 1]
  - 2 * (Precision * Recall) / (Precision + Recall)
- `Balanced Accuracy (BA)` — range: [0, 1]
  - 0.5 * (|TP|/(|TP|+|FN|) + |TN|/(|TN|+|FP|))
- `Matthews Correlation Coefficient (MCC)` — range: [-1, 1]
  - (|TP|*|TN| - |FP|*|FN|) / sqrt((|TP|+|FP|)(|TP|+|FN|)(|TN|+|FP|)(|TN|+|FN|))

## Input / output format

**Input**: CSV files containing timestamp and input value time series. Data is processed sequentially as a streaming sequence of scalar values.

**Output**: Binary classification label per input value indicating whether it is classified as anomalous or non-anomalous.

## Scoring recipe

```python
tp = sum(pred == 1 and gold == 1)
fp = sum(pred == 1 and gold == 0)
fn = sum(pred == 0 and gold == 1)
tn = sum(pred == 0 and gold == 0)
precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
ba = 0.5 * (recall + (tn / (tn + fp) if (tn + fp) > 0 else 0.0))
mcc = (tp*tn - fp*fn) / math.sqrt((tp+fp)*(tp+fn)*(tn+fp)*(tn+fn)) if denominator > 0 else 0.0
```

## Common pitfalls

- Labeling inconsistencies in NAB and Yahoo where some true anomalies are unlabeled or normal points are incorrectly labeled, which can artificially lower recall and F-measure.
- Strong class imbalance (<10% anomalies, often <1%) makes standard accuracy misleading; balanced accuracy or MCC should be prioritized for evaluation.
- Grid search for window size (W_size) and anomaly factor (epsilon) is performed per dataset/file, which may lead to overfitting to the evaluation set if not properly separated from tuning.

## Evidence (verbatim from paper)

> In the experimental phase, we compare anomaly detection quality of our approach to the other state-of-the-art methods and algorithms provided in the literature. To this end, we use five measures of detection quality: precision, recall, F-measure, balanced accuracy (BA) and Matthews correlation coefficient (MCC).

## Citation

```bibtex
@misc{maciag2019unsupervised,
  title={Unsupervised Anomaly Detection in Stream Data with Online Evolving Spiking Neural Networks},
  author={Maciąg et al. (2019)},
  year={2019},
  note={arXiv:1912.08785}
}
```

- arXiv: 1912.08785