Iot Nids Eval

---
name: iot-nids-eval
description: Evaluates the ability of a graph neural network to detect network intrusions in IoT environments by classifying traffic flows as benign or malicious, and identifying specific attack types. It probes the model's capacity to leverage topological graph structures and edge features for robust intrusion detection across imbalanced, real-world network traffic datasets. Use when the user wants to benchmark on BoT-IoT, NF-BoT-IoT, ToN-IoT, NF-ToN-IoT, or asks about evaluating this task. Reports F1-Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2103.16329
  bibtex_key: lo2021egraphsage
  confidence: high
---

# iot-nids-eval

> E-GraphSAGE: A Graph Neural Network based Intrusion Detection System for IoT — Lo et al. (2021) (arXiv:2103.16329, 2021)

## What this evaluates

Evaluates the ability of a graph neural network to detect network intrusions in IoT environments by classifying traffic flows as benign or malicious, and identifying specific attack types. It probes the model's capacity to leverage topological graph structures and edge features for robust intrusion detection across imbalanced, real-world network traffic datasets.

## Datasets

- **BoT-IoT** — total ?; splits: train (-1), test (-1)
- **NF-BoT-IoT** — total ?; splits: train (-1), test (-1)
- **ToN-IoT** — total ?; splits: train (-1), test (-1)
- **NF-ToN-IoT** — total ?; splits: train (-1), test (-1)

## Metrics

- `F1-Score` **(primary)** — range: [0, 1]
  - Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall). For multiclass tasks, the paper reports the Weighted Average F1-Score, which computes the F1 per class and averages them weighted by class support.
- `Detection Rate (Recall)` — range: [0, 1]
  - True Positives divided by the sum of True Positives and False Negatives: TP / (TP + FN).
- `Precision` — range: [0, 1]
  - True Positives divided by the sum of True Positives and False Positives: TP / (TP + FP).
- `Accuracy` — range: [0, 1]
  - Sum of True Positives and True Negatives divided by total samples: (TP + TN) / (TP + FP + TN + FN).
- `False Alarm Rate (FAR)` — range: [0, 1]
  - False Positives divided by the sum of False Positives and True Negatives: FP / (FP + TN).

## Input / output format

**Input**: Network flow records represented as graphs with edge features (e.g., packet counts, duration) and topological structure.

**Output**: Classification label: binary (attack vs. benign) or multiclass (specific attack type + benign).

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    tp = sum(1 for t, p in zip(y_true, y_pred) if t == p == 1)
    tn = sum(1 for t, p in zip(y_true, y_pred) if t != 1 and p != 1)
    fp = sum(1 for t, p in zip(y_true, y_pred) if t != 1 and p == 1)
    fn = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p != 1)
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    f1 = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0
    accuracy = (tp + tn) / (tp + fp + tn + fn)
    far = fp / (fp + tn) if (fp + tn) > 0 else 0
    return {"F1-Score": f1, "Recall": recall, "Precision": precision, "Accuracy": accuracy, "FAR": far}
```

## Common pitfalls

- Class imbalance makes Accuracy misleading; the paper explicitly states F1-Score is more relevant and should be used for SOTA comparison.
- NetFlow dataset variants use generic features instead of engineered ones, causing significant performance drops compared to original datasets.
- ToN-IoT evaluation uses only a 10% random subset due to dataset size, which may affect generalization claims.

## Evidence (verbatim from paper)

> Since the considered datasets are generally highly imbalanced, the F1-Score is a more relevant performance metric. We use the F1-Score to compare our classifier with the state-of-the-art, i.e., the best classification results reported in the literature for each of the four NIDS datasets.

## Citation

```bibtex
@misc{lo2021egraphsage,
  title={E-GraphSAGE: A Graph Neural Network based Intrusion Detection System for IoT},
  author={Lo et al. (2021)},
  year={2021},
  note={arXiv:2103.16329}
}
```

- arXiv: 2103.16329