Bs Gat Nf Iot Eval

---
name: bs-gat-nf-iot-eval
description: Evaluates network intrusion detection capabilities in IoT environments by classifying network flows into benign or multiple attack categories using graph-based and traditional machine learning models. Use when the user wants to benchmark on NF-BoT-IoT-v2, NF-ToN-IoT-v2, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2304.07226
  bibtex_key: wang2023bsgat
  confidence: high
---

# bs-gat-nf-iot-eval

> BS-GAT Behavior Similarity Based Graph Attention Network for Network Intrusion Detection — Wang et al. (2023) (arXiv:2304.07226, 2023)

## What this evaluates

Evaluates network intrusion detection capabilities in IoT environments by classifying network flows into benign or multiple attack categories using graph-based and traditional machine learning models.

## Datasets

- **NF-BoT-IoT-v2** — total 2018770; splits: train (-1), val (-1), test (-1)
- **NF-ToN-IoT-v2** — total 934576; splits: train (-1), val (-1), test (-1)

## Metrics

- `Accuracy` **(primary)** — range: [0, 1]
  - Calculated as (TP + TN) / (TP + FP + FN + TN), where TP, TN, FP, FN are True/False Positives/Negatives from the confusion matrix.
- `Precision` — range: [0, 1]
  - Calculated as TP / (TP + FP), measuring the proportion of positive identifications that were actually correct.
- `Recall` — range: [0, 1]
  - Calculated as TP / (TP + FN), measuring the proportion of actual positives that were correctly identified.
- `F1-Score` — range: [0, 1]
  - Calculated as 2 × (Recall × Precision) / (Recall + Precision), the harmonic mean of Precision and Recall.

## Input / output format

**Input**: 43-dimensional feature vector representing a network flow/node.

**Output**: Multi-class label from {Benign, Reconnaissance, DDos, Dos, Theft, Backdoor, Injection, MITM, Password, Scanning, XSS}.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    tp = sum(1 for t, p in zip(y_true, y_pred) if t == p and t == 1)
    tn = sum(1 for t, p in zip(y_true, y_pred) if t == p and t == 0)
    fp = sum(1 for t, p in zip(y_true, y_pred) if t != p and p == 1)
    fn = sum(1 for t, p in zip(y_true, y_pred) if t != p and t == 1)
    accuracy = (tp + tn) / (tp + fp + fn + tn)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0
    return {'Accuracy': accuracy, 'Precision': precision, 'Recall': recall, 'F1-Score': f1}
```

## Common pitfalls

- The datasets are heavily subsampled (only 5% of majority classes used) to manage memory, which alters the original class distribution and may inflate performance metrics.
- The Gini coefficient evaluates graph construction quality (neighbor uniformity), not model prediction accuracy.
- Baseline hyperparameters are fixed to original paper values rather than tuned on the shared validation set, potentially creating an unfair comparison.

## Evidence (verbatim from paper)

> To evaluate the performance of the proposed algorithm, the evaluation metrics shown in Table 3 were used. Where TP, FP, FN, and TN represent True Positive, False Positive, True Negative, and False Negative, respectively, in the Confusion Matrix. Table 3: Evaluation metrics adopted in this paper | Metric | Definition | Recall | TP/TP + FN | Precision | TP/TP + FP | F1-Score | 2 × Recall × Precision/Recall + Precision | Accuracy | TP + TN/TP + FP + FN + TN |

## Citation

```bibtex
@misc{wang2023bsgat,
  title={BS-GAT Behavior Similarity Based Graph Attention Network for Network Intrusion Detection},
  author={Wang et al. (2023)},
  year={2023},
  note={arXiv:2304.07226}
}
```

- arXiv: 2304.07226