Flowxpert Mawi Eval

---
name: flowxpert-mawi-eval
description: Evaluates a network intrusion detection model's ability to classify benign versus malicious traffic flows in real-world IoT environments. It specifically probes robustness to severe class imbalance, feature sparsity mitigation via context-aware embeddings, and temporal generalization across different time periods. Use when the user wants to benchmark on MAWI, or asks about evaluating this task. Reports F1-Score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2509.20861
  bibtex_key: zha2025flowxpert
  confidence: high
---

# flowxpert-mawi-eval

> FlowXpert: Context-Aware Flow Embedding for Enhanced Traffic Detection in IoT Network — Zha et al. (2025) (arXiv:2509.20861, 2025)

## What this evaluates

Evaluates a network intrusion detection model's ability to classify benign versus malicious traffic flows in real-world IoT environments. It specifically probes robustness to severe class imbalance, feature sparsity mitigation via context-aware embeddings, and temporal generalization across different time periods.

## Datasets

- **MAWI** — total ?; splits: train (-1), test (-1), generalization (-1)

## Metrics

- `F1-Score` **(primary)** — range: percent
  - Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall). Evaluated separately for benign and malicious classes to account for class imbalance.
- `Precision` — range: percent
  - Ratio of true positive predictions to all positive predictions: TP / (TP + FP).
- `Recall` — range: percent
  - Ratio of true positive predictions to all actual positives: TP / (TP + FN).
- `Latency` — range: other
  - Time taken per inference or processing cycle, measured to assess real-time deployability.
- `Throughput` — range: other
  - Number of flows processed per unit time, measured to assess real-time deployability.

## Input / output format

**Input**: Context-aware flow embeddings derived from source-host-to-destination-node associations. Raw IP addresses and port numbers are explicitly excluded to prevent label leakage. Includes both encrypted (e.g., HTTPS) and unencrypted traffic flows.

**Output**: Binary classification label: 'Benign' or 'Malicious'.

## Scoring recipe

```python
def compute_metrics(preds, gold):
    tp = sum(1 for p, g in zip(preds, gold) if p == g == 1)
    fp = sum(1 for p, g in zip(preds, gold) if p == 1 and g == 0)
    fn = sum(1 for p, g in zip(preds, gold) if p == 0 and g == 1)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return precision, recall, f1
```

## Common pitfalls

- Using overall accuracy as the primary metric, which the authors explicitly avoid due to severe class imbalance between benign and malicious traffic.
- Including raw IP addresses or port numbers in the feature set, which can reveal label information and lead to cheating behaviors or overfitting.
- Evaluating on simulated datasets like NSL-KDD or CICIDS-2017 instead of real-world MAWI data, which fails to capture practical applicability and encrypted traffic challenges.

## Evidence (verbatim from paper)

> We use five metrics to evaluate the performance of FlowXpert, including three commonly used metrics in machine learning algorithms: Precision, Recall, and F1-Score [[35]]. Furthermore, we include two metrics that are crucial for practical deployment: Latency and Throughput. Due to the inherent class imbalance between benign and malicious traffic, a metric such as accuracy was not used. Instead, detection performance metrics were separately evaluated for benign and malicious traffic to provide a more comprehensive and fair assessment.

## Citation

```bibtex
@misc{zha2025flowxpert,
  title={FlowXpert: Context-Aware Flow Embedding for Enhanced Traffic Detection in IoT Network},
  author={Zha et al. (2025)},
  year={2025},
  note={arXiv:2509.20861}
}
```

- arXiv: 2509.20861