Kdd Cup Eval

---
name: kdd-cup-eval
description: Evaluates the capability of an unsupervised cellular automata-based framework to detect network intrusions and anomalous traffic patterns. It probes the model's ability to learn spatial-temporal dependencies from raw network connection records and distinguish between normal and malicious activities. Use when the user wants to benchmark on KDD Cup, or asks about evaluating this task. Reports Intrusion Detection Vs Positive Rate.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1401.3046
  bibtex_key: kiran2014nidwca
  confidence: medium
---

# kdd-cup-eval

> Investigating Cellular Automata Based Network Intrusion Detection System For Fixed Networks (NIDWCA) — P.Kiran Sree and I. Ramesh Babu (2014) (arXiv:1401.3046, 2014)

## What this evaluates

Evaluates the capability of an unsupervised cellular automata-based framework to detect network intrusions and anomalous traffic patterns. It probes the model's ability to learn spatial-temporal dependencies from raw network connection records and distinguish between normal and malicious activities.

## Datasets

- **KDD Cup** — total ?; splits: train (-1), test (-1)

## Metrics

- `Intrusion Detection Vs Positive Rate` **(primary)** — range: [0, 1]
  - Plots the True Positive Rate (Intrusion Detection) against the False Positive Rate (Positive Rate) across classification thresholds. Detection Rate = TP / (TP + FN); Positive Rate = FP / (FP + TN).

## Input / output format

**Input**: Network traffic records represented as 41-dimensional feature vectors extracted from system call audit files.

**Output**: Binary classification labels (intrusion vs. normal) per record, aggregated into detection and positive rates for plotting.

## Scoring recipe

```python
def compute_ids_metrics(predictions, labels):
    tp = sum(1 for p, l in zip(predictions, labels) if p == 1 and l == 1)
    fp = sum(1 for p, l in zip(predictions, labels) if p == 1 and l == 0)
    fn = sum(1 for p, l in zip(predictions, labels) if p == 0 and l == 1)
    tn = sum(1 for p, l in zip(predictions, labels) if p == 0 and l == 0)
    detection_rate = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    positive_rate = fp / (fp + tn) if (fp + tn) > 0 else 0.0
    return detection_rate, positive_rate
```

## Common pitfalls

- The paper tests the trained algorithm on the same dataset used for training, risking data leakage and overoptimistic performance estimates.
- KDD Cup lacks standardized train/test splits, making cross-study comparisons difficult without explicit partitioning details.
- Reporting only aggregate detection/positive rates obscures performance on specific attack categories (e.g., DoS vs. R2L).

## Evidence (verbatim from paper)

> We have applied our algorithm to a set of standard benchmark data, namely the KDD Cup data. Network traffic can be captured using packet-capturing utilities or operating system utilities at the system call level. It is stored as a collection of records in the audit file. Each data point is described by 41 features. The algorithm was trained with the KDD Cup data set. Figure 3, 4, 5 depicts the findings. Intrusion Detection Vs Positive Rate in Sample Network for Classifier I

## Citation

```bibtex
@misc{kiran2014nidwca,
  title={Investigating Cellular Automata Based Network Intrusion Detection System For Fixed Networks (NIDWCA)},
  author={P.Kiran Sree and I. Ramesh Babu (2014)},
  year={2014},
  note={arXiv:1401.3046}
}
```

- arXiv: 1401.3046