Malware Hmm Eval

---
name: malware-hmm-eval
description: Evaluates static, dynamic, and hybrid analysis pipelines for malware detection. Models are trained on opcode and API call sequences to distinguish malware families from benign Windows executables. Use when the user wants to benchmark on Malware Detection Dataset, or asks about evaluating this task. Reports Area under the ROC curve.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2203.09938
  bibtex_key: damodaran2022malware
  confidence: high
---

# malware-hmm-eval

> A Comparison of Static, Dynamic, and Hybrid Analysis for Malware Detection — Damodaran et al. (2022) (arXiv:2203.09938, 2022)

## What this evaluates

Evaluates static, dynamic, and hybrid analysis pipelines for malware detection. Models are trained on opcode and API call sequences to distinguish malware families from benign Windows executables.

## Datasets

- **Malware Detection Dataset** — total 785; splits: train (-1), test (-1)

## Metrics

- `Area under the ROC curve` **(primary)** — range: [0, 1]
  - Computed from the HMM-generated likelihood scores for malware versus benign samples. Plots true positive rate against false positive rate across thresholds to yield a value between 0 and 1.

## Input / output format

**Input**: Opcode mnemonics or API call names extracted as sequences from each executable, either statically via disassembly or dynamically via sandbox execution.

**Output**: A continuous probability score from the Hidden Markov Model indicating sequence likelihood.

## Scoring recipe

```python
# 5-fold cross-validation loop
for fold in range(5):
    train_seqs, test_seqs = split_dataset(fold)
    hmm = train_hmm(train_seqs)
    scores_malware = hmm.score(test_seqs)
    scores_benign = hmm.score(benign_seqs)
    labels = [1]*len(scores_malware) + [0]*len(scores_benign)
    preds = scores_malware + scores_benign
    auc = compute_auc(labels, preds)
```

## Common pitfalls

- Dynamic analysis execution is time-boxed, potentially missing late-stage malware behavior (e.g., Zbot vanishes after 5-10 minutes).
- Benign samples are restricted to Windows System 32 binaries, which may not represent real-world third-party software distributions.
- The paper evaluates four static/dynamic training-scoring combinations but does not report per-family detection rates, only aggregate AUC.

## Evidence (verbatim from paper)

> The scores from a given experiment are used to form a scatterplot, from which an ROC curve is generated. The area under the ROC curve serving as our measure of success, as discussed in Section[2.4]...

## Citation

```bibtex
@misc{damodaran2022malware,
  title={A Comparison of Static, Dynamic, and Hybrid Analysis for Malware Detection},
  author={Damodaran et al. (2022)},
  year={2022},
  note={arXiv:2203.09938}
}
```

- arXiv: 2203.09938