Big2015 Malware Detection Eval

---
name: big2015-malware-detection-eval
description: Evaluates the effectiveness of multimodal visual feature fusion (grayscale, entropy graph, SimHash) using VGG16 for binary malware classification and family detection. It probes the model's ability to handle imbalanced malware datasets and detect obfuscated binaries. Use when the user wants to benchmark on BIG2015, or asks about evaluating this task. Reports F1-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2405.14311
  bibtex_key: [REDACTED]
  confidence: high
---

# big2015-malware-detection-eval

> Deep Learning Fusion For Effective Malware Detection: Leveraging Visual Features — Johny et al. (2024) (arXiv:2405.14311, 2024)

## What this evaluates

Evaluates the effectiveness of multimodal visual feature fusion (grayscale, entropy graph, SimHash) using VGG16 for binary malware classification and family detection. It probes the model's ability to handle imbalanced malware datasets and detect obfuscated binaries.

## Datasets

- **BIG2015** — total ?; splits: test (-1)

## Metrics

- `F1-score` **(primary)** — range: [0, 1]
  - F1 = 2 * (Precision * Recall) / (Precision + Recall), where Precision = TP / (TP + FP) and Recall = TP / (TP + FN).
- `Accuracy` — range: [0, 1]
  - Accuracy = (TP + TN) / (TP + FP + TN + FN).
- `Precision` — range: [0, 1]
  - Precision = TP / (TP + FP).
- `Recall` — range: [0, 1]
  - Recall = TP / (TP + FN).
- `Prediction Time` — range: seconds
  - Wall-clock time in seconds required to predict a single file.

## Input / output format

**Input**: Hexadecimal binary content of malware files (header stripped), converted into grayscale images, entropy graphs, and SimHash images.

**Output**: Predicted malware family/class label and inference time in seconds per file.

## Scoring recipe

```python
def compute_metrics(tp, tn, fp, fn):
    accuracy = (tp + tn) / (tp + fp + tn + fn)
    precision = tp / (tp + fp)
    recall = tp / (tp + fn)
    f1 = 2 * (precision * recall) / (precision + recall)
    return accuracy, precision, recall, f1
```

## Common pitfalls

- Dataset is imbalanced, so Accuracy can be misleading; F1-score is explicitly prioritized.
- Visual features are generated via a custom hex-to-image conversion pipeline not fully detailed, which may hinder exact reproducibility.
- Experiments are repeated 5 times and averaged to minimize random error, but variance is not reported.

## Evidence (verbatim from paper)

> Given the imbalanced nature of the dataset, we prioritize the F1-score over other metrics to assess the model's performance. Besides the above-mentioned metrics, we also estimate the time required to predict a file using our proposed models.

## Citation

```bibtex
@misc{johny2024deeplearningfusion,
  title={Deep Learning Fusion For Effective Malware Detection: Leveraging Visual Features},
  author={Johny et al. (2024)},
  year={2024},
  note={arXiv:2405.14311}
}
```

- arXiv: 2405.14311