Malware Image Classification Eval

---
name: malware-image-classification-eval
description: Evaluates a deep learning model's ability to classify malware images across multiple tasks, including binary classification, malware family classification, and detection of obfuscation techniques across Windows, Android, macOS, and Linux platforms. Use when the user wants to benchmark on Maling benchmark dataset, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2405.05906
  bibtex_key: [REDACTED]
  confidence: medium
---

# malware-image-classification-eval

> Deep Multi-Task Learning for Malware Image Classification — Bensaoud et al. (2024) (arXiv:2405.05906, 2024)

## What this evaluates

Evaluates a deep learning model's ability to classify malware images across multiple tasks, including binary classification, malware family classification, and detection of obfuscation techniques across Windows, Android, macOS, and Linux platforms.

## Datasets

- **Maling benchmark dataset** — total ?; splits: test (-1); repo https://github.com/abensaou-uccs/Colorado-MalColorImg

## Metrics

- `accuracy` **(primary)** — range: percent
  - Calculated as the percentage of correctly classified samples out of the total test samples. Accuracy = (Correct Predictions / Total Samples) * 100.

## Input / output format

**Input**: Bitmap (BMP/PNG) images representing structural features of PE, APK, Mach-O, and ELF binaries.

**Output**: Discrete class labels corresponding to one of seven classification tasks (binary classification, malware family classification, or obfuscation detection).

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return (correct / len(gold)) * 100
```

## Common pitfalls

- The paper does not specify the exact train/validation/test split ratios or data augmentation pipeline details beyond CycleGAN for macOS, making exact reproduction of error rates difficult.
- Tasks 3 (iOS) and 6 (macOS) explicitly lack state-of-the-art comparisons due to a scarcity of prior image-based research, which limits the completeness of the benchmark evaluation.

## Evidence (verbatim from paper)

> For the malware image binary classification task1, we obtain an accuracy of 99.88%, 99.94% for task2, 99.91% for task3, 99.89% for task5, 99.92% for task6, and 99.95% for task7, while task4 for the malware family classification yields a classification accuracy of 99.97% as shown in Table 5.

## Citation

```bibtex
@misc{bensaoud2024deep,
  title={Deep Multi-Task Learning for Malware Image Classification},
  author={Bensaoud et al. (2024)},
  year={2024},
  note={arXiv:2405.05906}
}
```

- arXiv: 2405.05906