Ciciomt2024 Ifl Eval

---
name: ciciomt2024-ifl-eval
description: Evaluates incremental federated learning models for intrusion detection in IoT networks under evolving threat distributions. Probes the model's ability to adapt to concept drift over time while mitigating catastrophic forgetting in a federated setting. Use when the user wants to benchmark on CICIoMT2024, or asks about evaluating this task. Reports Accuracy (Acc).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.10776
  bibtex_key: rehman2026incremental
  confidence: high
---

# ciciomt2024-ifl-eval

> Incremental Federated Learning for Intrusion Detection in IoT Networks under Evolving Threat Landscape — Rehman et al. (2026) (arXiv:2603.10776, 2026)

## What this evaluates

Evaluates incremental federated learning models for intrusion detection in IoT networks under evolving threat distributions. Probes the model's ability to adapt to concept drift over time while mitigating catastrophic forgetting in a federated setting.

## Datasets

- **CICIoMT2024** — total ?; splits: train (-1), test (-1)

## Metrics

- `Accuracy (Acc)` **(primary)** — range: [0, 1]
  - Micro-averaged accuracy computed as the ratio of correctly predicted samples to the total number of test samples: sum(TP_i) / sum(TP_i + FP_i + FN_i) across all classes C.

## Input / output format

**Input**: 45 normalized network traffic features including header/flow metadata, TCP/IP flag indicators, protocol identifiers, and statistical descriptors.

**Output**: 6-class categorical prediction: Benign, MQTT, DoS, DDoS, Reconnaissance, or ARP_Spoofing.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    total = len(gold_labels)
    return correct / total if total > 0 else 0.0
```

## Common pitfalls

- The dataset uses a strict class-capping mechanism (max 10k train, 2k test per class per time period) to control imbalance, which differs from the raw dataset distribution.
- Evaluation is conducted across a temporal timeline (t0 to t6) simulating concept drift, not as a single static train/test split.
- Only micro-averaged accuracy is reported, despite precision, recall, F1, and FAR being computed internally.

## Evidence (verbatim from paper)

> Model performance was assessed using standard metrics derived from the confusion matrix, including Accuracy, Precision, Recall, F1-score, False Alarm Rate (FAR). Trends across Precision, Recall and F1-score were consistent with accuracy results, therefore, we report the overall accuracy (Acc) at each time period in the paper. The overall accuracy (Acc) is computed using micro-averaging as the ratio of correctly predicted samples to the total number of test samples. $$ \mathrm {A c c} = \frac {\sum_ {i = 1} ^ {C} \mathrm {T P} _ {i}}{\sum_ {i = 1} ^ {C} \left(\mathrm {T P} _ {i} + \mathrm {F P} _ {i} + \mathrm {F N} _ {i}\right)} \tag {1} $$

## Citation

```bibtex
@misc{rehman2026incremental,
  title={Incremental Federated Learning for Intrusion Detection in IoT Networks under Evolving Threat Landscape},
  author={Rehman et al. (2026)},
  year={2026},
  note={arXiv:2603.10776}
}
```

- arXiv: 2603.10776