Malware Detection Eval

---
name: malware-detection-eval
description: Binary classification of software binaries as benign or malicious based on their control flow graphs. It probes the model's ability to learn graph-structured representations and route them through specialized experts for accurate detection. Use when the user wants to benchmark on BODMAS, DikeDataset, PMML, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.19025
  bibtex_key: shokouhinejad2026routing
  confidence: high
---

# malware-detection-eval

> Routing-Aware Explanations for Mixture of Experts Graph Models in Malware Detection — Shokouhinejad et al. (2026) (arXiv:2602.19025, 2026)

## What this evaluates

Binary classification of software binaries as benign or malicious based on their control flow graphs. It probes the model's ability to learn graph-structured representations and route them through specialized experts for accurate detection.

## Datasets

- **BODMAS** — total 122; splits: train (-1), test (-1)
- **DikeDataset** — total 319; splits: train (-1), test (-1)
- **PMML** — total 390; splits: train (-1), test (-1)

## Metrics

- `Accuracy` **(primary)** — range: [0, 1]
  - Proportion of correctly classified samples out of the total number of samples. Calculated as (TP + TN) / Total.
- `F1` — range: [0, 1]
  - Harmonic mean of precision and recall. Calculated as 2 * (Precision * Recall) / (Precision + Recall). Reported separately for benign and malicious classes.

## Input / output format

**Input**: Control flow graphs (CFGs) extracted from binaries using angr, with node features compressed to 64 dimensions via a symmetric autoencoder.

**Output**: Binary classification label (Benign or Malware) per graph, plus routing gate weights and expert selections for MoE variants.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    correct = sum(1 for t, p in zip(y_true, y_pred) if t == p)
    accuracy = correct / len(y_true)
    tp = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 1)
    fp = sum(1 for t, p in zip(y_true, y_pred) if t == 0 and p == 1)
    fn = sum(1 for t, p in zip(y_true, y_pred) if t == 1 and p == 0)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return {'accuracy': accuracy, 'f1': f1}
```

## Common pitfalls

- Datasets have highly imbalanced class distributions (only DikeDataset is benign, BODMAS/PMML are malware), requiring careful stratified splitting to preserve class proportions.
- Node features are heavily preprocessed (439D to 64D via autoencoder); evaluating raw features without this compression will yield different results.
- Explainability metrics (Fidelity+/Fidelity-) are evaluated across sparsity levels (5%-95%), not just a single threshold.

## Evidence (verbatim from paper)

> The dataset was stratified and split into 80% for training and 20% for testing, preserving equal class proportions for benign and malicious samples in both subsets. Table[2] reports classwise precision, recall, F1, and overall accuracy for all models.

## Citation

```bibtex
@misc{shokouhinejad2026routing,
  title={Routing-Aware Explanations for Mixture of Experts Graph Models in Malware Detection},
  author={Shokouhinejad et al. (2026)},
  year={2026},
  note={arXiv:2602.19025}
}
```

- arXiv: 2602.19025