Paella Malware Detection Eval

---
name: paella-malware-detection-eval
description: Evaluates machine learning models' ability to detect malware and anomalies in data center compute nodes using high-resolution power consumption data and hardware performance counters. It probes real-time anomaly detection capabilities under severe class imbalance and varying computational workloads. Use when the user wants to benchmark on pAElla Malware & Benchmark Dataset, or asks about evaluating this task. Reports weighted F1-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2004.03670
  bibtex_key: libri2020paella
  confidence: high
---

# paella-malware-detection-eval

> pAElla: Edge-AI based Real-Time Malware Detection in Data Centers — Libri et al. (2020) (arXiv:2004.03670, 2020)

## What this evaluates

Evaluates machine learning models' ability to detect malware and anomalies in data center compute nodes using high-resolution power consumption data and hardware performance counters. It probes real-time anomaly detection capabilities under severe class imbalance and varying computational workloads.

## Datasets

- **pAElla Malware & Benchmark Dataset** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/alibriee/paella

## Metrics

- `weighted F1-score` **(primary)** — range: [0, 1]
  - Accounts for class imbalance: F1 = (2·TP·W_M) / (2·TP·W_M + FP·W_H + FN·W_M), where W_M and W_H are the counts of malware and healthy instances. Also reports False Alarm (FA) rate (FP rate) and Malware Miss (MM) rate (FN rate).

## Input / output format

**Input**: Time-series feature vectors per 20 ms window containing hardware performance counters (per-core and per-cpu), Power Spectral Density (PSD) estimates from high-resolution power measurements, and coarse-grain power statistics (mean, std, max, min).

**Output**: Binary label: 'malware/anomaly' or 'healthy/normal'.

## Scoring recipe

```python
def compute_metrics(tp, fp, fn, tn, W_M, W_H):
    fa_rate = fp / (fp + tn)  # False Alarm rate (FP rate)
    mm_rate = fn / (tp + fn)  # Malware Miss rate (FN rate)
    f1_weighted = (2 * tp * W_M) / (2 * tp * W_M + fp * W_H + fn * W_M)
    return fa_rate, mm_rate, f1_weighted
```

## Common pitfalls

- Dataset is highly imbalanced (~95% malware), so unweighted F1 or accuracy is misleading.
- Threshold selection for anomaly scoring (e.g., reconstruction error or outlier percentage) significantly impacts FA and MM rates.
- Performance counter feature space is often not linearly separable for oc-SVM, requiring careful PCA/kernel tuning.

## Evidence (verbatim from paper)

> Table III reports the results of our analysis in the test set. Namely, the (i) False Alarm (FA) rate (= False Positive - FP - rate) - i.e., healthy benchmarks erroneously labeled as malware; (ii) the Malware Miss (MM) rate (= False Negative - FN - rate) - i.e., malware not detected; and (iii) the weighted F1-score[28] (best value at 1, and worst at 0), which measure the test accuracy based on the following formula: F1score = (2TP·WM)/(2TP·WM + FP·WH + FN·WM) where we weighted the True Positives (TP), FP, and FN by the number of instances of each class (Malware and Healthy), via the two weights WM and WH, to take into account the imbalance of the dataset between number of malware and number of healthy acquisitions.

## Citation

```bibtex
@misc{libri2020paella,
  title={pAElla: Edge-AI based Real-Time Malware Detection in Data Centers},
  author={Libri et al. (2020)},
  year={2020},
  note={arXiv:2004.03670}
}
```

- arXiv: 2004.03670