Cic Malmem 2022 Eval

---
name: cic-malmem-2022-eval
description: Evaluates the capability of machine learning models (traditional classifiers and CNNs on barcode-encoded features) to classify malware samples into benign or specific malware families. It probes how well structural patterns in 2D barcodes (QR and Aztec codes) capture executable features for downstream classification tasks. Use when the user wants to benchmark on CIC-MalMem-2022, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2412.08514
  bibtex_key: khadilkar2024imagebasedmalware
  confidence: high
---

# cic-malmem-2022-eval

> Image-Based Malware Classification Using QR and Aztec Codes — Khadilkar et al. (2024) (arXiv:2412.08514, 2024)

## What this evaluates

Evaluates the capability of machine learning models (traditional classifiers and CNNs on barcode-encoded features) to classify malware samples into benign or specific malware families. It probes how well structural patterns in 2D barcodes (QR and Aztec codes) capture executable features for downstream classification tasks.

## Datasets

- **CIC-MalMem-2022** — total 24000; splits: train (16800), validation (3600), test (3600)

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Ratio of correctly classified samples to the total number of samples.
- `F1-score` — range: [0, 1]
  - Harmonic mean of precision and recall, averaged across classes.

## Input / output format

**Input**: 1D numerical feature vectors (55 or top 10 features) or 2D QR/Aztec code images (128x128 pixels) derived from those features.

**Output**: Multiclass label: benign, ransomware, spyware, or Trojan.

## Scoring recipe

```python
accuracy = sum(pred == gold) / len(gold)
f1 = f1_score(gold, pred, average='macro')
return accuracy, f1
```

## Common pitfalls

- Resizing QR/Aztec images to 128x128 pixels may lose fine-grained structural patterns critical for CNN learning.
- Feature selection using SelectKBest with chi2/ANOVA drastically reduces dimensionality, potentially discarding weak but predictive malware indicators.
- Classic ML models (Random Forest) significantly outperform CNNs on the BODMAS dataset, highlighting dataset-dependent encoding efficacy.

## Evidence (verbatim from paper)

> We split the 24,000 samples 70:15:15 for train:validation:test. ... The test accuracy achieved for the CNN on the QR image representation was 0.9998 for this multiclass classification problem.

## Citation

```bibtex
@misc{khadilkar2024imagebasedmalware,
  title={Image-Based Malware Classification Using QR and Aztec Codes},
  author={Khadilkar et al. (2024)},
  year={2024},
  note={arXiv:2412.08514}
}
```

- arXiv: 2412.08514