Ids Detection Eval

---
name: ids-detection-eval
description: Evaluates machine learning classifiers for network intrusion detection on imbalanced, high-dimensional traffic data. Probes the model's ability to distinguish benign from malicious traffic across binary and multilabel settings using standard classification metrics. Use when the user wants to benchmark on UNSW-NB15, CIC-IDS2017, CIC-IDS2018, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2401.12262
  bibtex_key: [REDACTED]
  confidence: high
---

# ids-detection-eval

> Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction — Talukder et al. (2024) (arXiv:2401.12262, 2024)

## What this evaluates

Evaluates machine learning classifiers for network intrusion detection on imbalanced, high-dimensional traffic data. Probes the model's ability to distinguish benign from malicious traffic across binary and multilabel settings using standard classification metrics.

## Datasets

- **UNSW-NB15** — total ?; splits: train (-1), test (-1)
- **CIC-IDS2017** — total ?; splits: train (-1), test (-1)
- **CIC-IDS2018** — total ?; splits: train (-1), test (-1)

## Metrics

- `Accuracy` **(primary)** — range: [0, 1]
  - Proportion of correctly predicted observations to total observations. Formula: (TP + TN) / (TP + FP + FN + TN).
- `Precision` — range: [0, 1]
  - Ratio of correctly predicted positive values to total predicted positive values. Formula: TP / (TP + FP).
- `Recall` — range: [0, 1]
  - Ratio of correctly predicted positive values to all actual positive values. Formula: TP / (TP + FN).
- `F1-Score` — range: [0, 1]
  - Harmonic mean of precision and recall. Formula: 2 * (Precision * Recall) / (Precision + Recall).
- `AUC` — range: [0, 1]
  - Area Under the Receiver Operating Characteristic Curve. Measures the model's ability to distinguish between class labels by plotting True Positive Rate against False Positive Rate at various thresholds.

## Input / output format

**Input**: Feature vectors extracted from network traffic records, preprocessed via Random Oversampling, Stacking Feature Embedding, and PCA.

**Output**: Predicted class label (binary or multilabel classification).

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    tp = sum((t==1) & (p==1) for t, p in zip(y_true, y_pred))
    tn = sum((t==0) & (p==0) for t, p in zip(y_true, y_pred))
    fp = sum((t==0) & (p==1) for t, p in zip(y_true, y_pred))
    fn = sum((t==1) & (p==0) for t, p in zip(y_true, y_pred))
    acc = (tp + tn) / (tp + fp + fn + tn)
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * (prec * rec) / (prec + rec) if (prec + rec) > 0 else 0
    return acc, prec, rec, f1
```

## Common pitfalls

- Data imbalance is handled via Random Oversampling on the training folds, so test metrics must be computed on the original, unbalanced test split to reflect real-world performance.
- The protocol evaluates both binary and multilabel classification; metric aggregation (e.g., macro vs micro averaging) is not explicitly specified and may vary across implementations.
- 10-fold cross-validation is used with a 90/10 train/test split per fold, meaning reported scores are averages across 10 runs rather than a single holdout evaluation.

## Evidence (verbatim from paper)

> Several measures, such as accuracy, precision, recall, F1-score, ROC curve, and confusion matrix, are used to evaluate the performance of our proposed model. ... Accuracy is a fundamental performance metric, representing the proportion of correctly predicted observations to the total observations. It is calculated as follows: $$ Accuracy = rac {T P + T N}{T P + F P + F N + T N} $$

## Citation

```bibtex
@misc{talukder2024network,
  title={Machine learning-based network intrusion detection for big and imbalanced data using oversampling, stacking feature embedding and feature extraction},
  author={Talukder et al. (2024)},
  year={2024},
  note={arXiv:2401.12262}
}
```

- arXiv: 2401.12262