Network Intrusion Detection Eval

---
name: network-intrusion-detection-eval
description: Evaluates the ability of machine learning classifiers to detect network intrusions across different traffic datasets, with a focus on handling severe class imbalance and identifying rare attack types. Use when the user wants to benchmark on KDD-99, NSL-KDD, UNSW-NB15, or asks about evaluating this task. Reports Weighted F1-Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1811.05372
  bibtex_key: divekar2018benchmarking
  confidence: high
---

# network-intrusion-detection-eval

> Benchmarking datasets for Anomaly-based Network Intrusion Detection: KDD CUP 99 alternatives — Divekar et al. (2018) (arXiv:1811.05372, 2018)

## What this evaluates

Evaluates the ability of machine learning classifiers to detect network intrusions across different traffic datasets, with a focus on handling severe class imbalance and identifying rare attack types.

## Datasets

- **KDD-99** — total ?; splits: test (-1)
- **NSL-KDD** — total ?; splits: test (-1)
- **UNSW-NB15** — total ?; splits: test (-1)

## Metrics

- `Weighted F1-Score` **(primary)** — range: percent
  - The average of per-class F1-scores weighted by the number of true instances (support) for each class. Calculated as sum(w_i * F1_i) / sum(w_i), where w_i is the support of class i.

## Input / output format

**Input**: Network traffic records represented as feature vectors containing numerical and categorical attributes describing connection properties (e.g., duration, protocol, service, flag, byte counts).

**Output**: Predicted class label from a predefined set of intrusion categories (e.g., DoS, Normal, Probe, R2L, U2R) or a binary attack/normal label.

## Scoring recipe

```python
def compute_weighted_f1(y_true, y_pred, classes):
    f1s = []
    weights = []
    for c in classes:
        tp = sum(1 for t, p in zip(y_true, y_pred) if t == c and p == c)
        fp = sum(1 for t, p in zip(y_true, y_pred) if t != c and p == c)
        fn = sum(1 for t, p in zip(y_true, y_pred) if t == c and p != c)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        f1s.append(f1)
        weights.append(sum(1 for t in y_true if t == c))
    return sum(w * f for w, f in zip(weights, f1s)) / sum(weights)
```

## Common pitfalls

- Using raw accuracy as the primary metric, which is misleading due to extreme class imbalance in network traffic data.
- Comparing multi-class F1-scores directly across datasets with different class taxonomies without binarization or proper weighting.
- Overlooking minority attack classes (e.g., R2L, U2R, Shell Code) which are critical for security but often have near-zero scores on legacy datasets.

## Evidence (verbatim from paper)

> Unlike other works [[17]], we consider primarily the Weighted F1-Score, as accuracy is misleading for such data distributions.

## Citation

```bibtex
@misc{divekar2018benchmarking,
  title={Benchmarking datasets for Anomaly-based Network Intrusion Detection: KDD CUP 99 alternatives},
  author={Divekar et al. (2018)},
  year={2018},
  note={arXiv:1811.05372}
}
```

- arXiv: 1811.05372