Cic Ids2017 Nids Eval

---
name: cic-ids2017-nids-eval
description: Evaluates the classification accuracy and adversarial robustness of a Graph Neural Network-based Network Intrusion Detection System (NIDS) on distinguishing benign traffic from various attack types in network flow data. Use when the user wants to benchmark on CIC-IDS2017, or asks about evaluating this task. Reports weighted F1-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2107.14756
  bibtex_key: pujolperich2021gnnids
  confidence: high
---

# cic-ids2017-nids-eval

> Unveiling the potential of Graph Neural Networks for robust Intrusion Detection — Pujol-Perich et al. (2021) (arXiv:2107.14756, 2021)

## What this evaluates

Evaluates the classification accuracy and adversarial robustness of a Graph Neural Network-based Network Intrusion Detection System (NIDS) on distinguishing benign traffic from various attack types in network flow data.

## Datasets

- **CIC-IDS2017** — total 1119250; splits: train (895400), val (223850)

## Metrics

- `weighted F1-score` **(primary)** — range: [0, 1]
  - Harmonic mean of precision and recall, weighted by the number of true instances (support) in each class. Computed as the sum of per-class F1-scores multiplied by their relative class frequencies.

## Input / output format

**Input**: Host-connection graphs constructed from network flow records, where each flow aggregates 80 features representing packet sizes, inter-arrival times, and other traffic statistics.

**Output**: Discrete class label indicating either 'Benign' or one of 11 attack sub-classes (e.g., SSH-Patator, DoS GoldenEye, DDoS, etc.).

## Scoring recipe

```python
def compute_weighted_f1(y_true, y_pred, classes):
    precisions, recalls, supports = [], [], []
    for c in classes:
        tp = sum(1 for t, p in zip(y_true, y_pred) if t == c and p == c)
        fp = sum(1 for t, p in zip(y_true, y_pred) if t != c and p == c)
        fn = sum(1 for t, p in zip(y_true, y_pred) if t == c and p != c)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        sup = sum(1 for t in y_true if t == c)
        precisions.append(prec); recalls.append(rec); supports.append(sup)
    total_sup = sum(supports)
    return sum(f1 * (s / total_sup) for f1, s in zip(precisions, supports))
```

## Common pitfalls

- Dataset is highly imbalanced (~88% benign, ~12% attacks); the authors drop 90% of benign training graphs to over-represent attacks, altering the evaluation distribution compared to raw data.
- Evaluation only includes classes with >100 flow samples, excluding rare attack types from the final metric calculation.
- Adversarial robustness is tested only by perturbing packet size and inter-arrival time, not by modifying the underlying graph structure or other flow features.

## Evidence (verbatim from paper)

> We use a standard weighted F1-score to measure the per-class accuracy, which unifies in a single metric the precision and recall of solutions. From these results, we can observe that the proposed model achieves a level of accuracy comparable to state-of-the-art ML methods, obtaining a weighted F1-score of 0.99 over all traffic flows.

## Citation

```bibtex
@misc{pujolperich2021gnnids,
  title={Unveiling the potential of Graph Neural Networks for robust Intrusion Detection},
  author={Pujol-Perich et al. (2021)},
  year={2021},
  note={arXiv:2107.14756}
}
```

- arXiv: 2107.14756