Iot Malware Image Classification Eval

---
name: iot-malware-image-classification-eval
description: Evaluates the ability of a lightweight CNN to classify IoT binary files as benign or belonging to specific DDoS malware families (Mirai, Linux.Gafgyt) by converting raw binaries into 64x64 grayscale images. Use when the user wants to benchmark on IoTPOT IoT DDoS Malware Dataset, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1802.03714
  bibtex_key: su2018lightweight
  confidence: high
---

# iot-malware-image-classification-eval

> Lightweight Classification of IoT Malware based on Image Recognition — Su et al. (2018) (arXiv:1802.03714, 2018)

## What this evaluates

Evaluates the ability of a lightweight CNN to classify IoT binary files as benign or belonging to specific DDoS malware families (Mirai, Linux.Gafgyt) by converting raw binaries into 64x64 grayscale images.

## Datasets

- **IoTPOT IoT DDoS Malware Dataset** — total 365; splits: train (320), test (45); repo https://github.com/Carina02/IotMalwareImage

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly classified samples out of the total test samples. Reported for both 2-class (benign vs. malicious) and 3-class (benign, Mirai, Gafgyt) settings.

## Input / output format

**Input**: Raw binary file converted to a 64x64 one-channel grayscale image, where each pixel value corresponds to the decimal encoding of an 8-bit byte from the binary.

**Output**: Class label: 'Benign', 'Malicious' (2-class), or 'Benign', 'Gafgyt', 'Mirai' (3-class).

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return (correct / len(gold)) * 100
# Average the accuracy over 5 independent random train/test splits
```

## Common pitfalls

- The dataset was manually balanced to exactly 365 samples (15 test per class); using the original unbalanced IoTPOT collection will change results.
- Evaluation is averaged over 5 independent random splits, not a standard fixed k-fold or single hold-out set.
- Input must be strictly 64x64 grayscale; resizing or using RGB channels breaks the protocol.

## Evidence (verbatim from paper)

> The experiments were conducted five times which each time with a completely different training/test data combination (i.e., there are no shared test samples between any two of five test data sets). According to the results of two-class classification, we find the proposed system can predict the existence of maliciousness with about 94.0% accuracy on the average.

## Citation

```bibtex
@misc{su2018lightweight,
  title={Lightweight Classification of IoT Malware based on Image Recognition},
  author={Su et al. (2018)},
  year={2018},
  note={arXiv:1802.03714}
}
```

- arXiv: 1802.03714