Dinids Cross Domain Eval

---
name: dinids-cross-domain-eval
description: This benchmark evaluates the robustness of network intrusion detection models against distribution shifts between different network environments. It specifically probes cross-domain generalization by training on one NetFlow dataset and testing on another, measuring how well domain-invariant feature extraction mitigates performance degradation when facing unseen attack distributions. Use when the user wants to benchmark on NFv2-UNSW-NB15, NFv2-CIC-2018, or asks about evaluating this task. Reports F1-Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2210.08252
  bibtex_key: layeghy2022dinids
  confidence: high
---

# dinids-cross-domain-eval

> DI-NIDS: Domain Invariant Network Intrusion Detection System — Layeghy et al. (2022) (arXiv:2210.08252, 2022)

## What this evaluates

This benchmark evaluates the robustness of network intrusion detection models against distribution shifts between different network environments. It specifically probes cross-domain generalization by training on one NetFlow dataset and testing on another, measuring how well domain-invariant feature extraction mitigates performance degradation when facing unseen attack distributions.

## Datasets

- **NFv2-UNSW-NB15** — total 2390275; splits: train (-1), test (-1)
- **NFv2-CIC-2018** — total 18893708; splits: train (-1), test (-1)

## Metrics

- `F1-Score` **(primary)** — range: percent
  - Harmonic mean of precision and recall for binary classification (Benign vs. Attack). Calculated as 2 * (precision * recall) / (precision + recall). Reported as a percentage in the paper.

## Input / output format

**Input**: 43 NetFlow version 9 fields representing bi-directional network flows, with binary labels (Benign or Attack).

**Output**: Binary prediction indicating whether a network flow is Benign or Attack.

## Scoring recipe

```python
def compute_f1_score(preds, gold):
    tp = sum(1 for p, g in zip(preds, gold) if p == 1 and g == 1)
    fp = sum(1 for p, g in zip(preds, gold) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(preds, gold) if p == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return f1 * 100
```

## Common pitfalls

- Cross-domain evaluation direction drastically affects baseline performance; reporting only one direction (e.g., CIC→UNSW) hides severe degradation in the reverse direction.
- Aggregating diverse attack types into a single 'Attack' class changes the anomaly detection task from multi-class classification to binary anomaly detection, which may not reflect real-world multi-attack scenarios.

## Evidence (verbatim from paper)

> Table 4 shows the results (F1-Score) of domain-specific performance evaluation for DI-NIDS, and the six baseline ML models used for comparison.

## Citation

```bibtex
@misc{layeghy2022dinids,
  title={DI-NIDS: Domain Invariant Network Intrusion Detection System},
  author={Layeghy et al. (2022)},
  year={2022},
  note={arXiv:2210.08252}
}
```

- arXiv: 2210.08252