Clam Adversarial Eval

---
name: clam-adversarial-eval
description: Evaluates the robustness of classical neural networks and quantum neural networks against adversarial attacks by measuring performance degradation on a malware classification task after injecting random noise into input features. Use when the user wants to benchmark on ClaMP_Integrated, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2305.19593
  bibtex_key: akter2023adversarial
  confidence: high
---

# clam-adversarial-eval

> Exploring the Vulnerabilities of Machine Learning and Quantum Machine Learning to Adversarial Attacks using a Malware Dataset: A Comparative Analysis — Akter et al. (2023) (arXiv:2305.19593, 2023)

## What this evaluates

Evaluates the robustness of classical neural networks and quantum neural networks against adversarial attacks by measuring performance degradation on a malware classification task after injecting random noise into input features.

## Datasets

- **ClaMP_Integrated** — total 5210; splits: train (-1), val (-1), test (-1)

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Ratio of correctly classified instances to the total number of instances.
- `precision` — range: [0, 1]
  - Ratio of true positive predictions to the total number of positive predictions.
- `recall` — range: [0, 1]
  - Ratio of true positive predictions to the total number of actual positives.
- `F-score` — range: [0, 1]
  - Harmonic mean of precision and recall.

## Input / output format

**Input**: 68-dimensional feature vector extracted from PE headers (raw, expanded, derived features), or 16-dimensional PCA-reduced vector for QNN models. Binary labels mapped to -1 and 1.

**Output**: Binary classification prediction (malware or benign), typically output as logits or probabilities.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    tp = np.sum((y_true == 1) & (y_pred == 1))
    tn = np.sum((y_true == -1) & (y_pred == -1))
    fp = np.sum((y_true == -1) & (y_pred == 1))
    fn = np.sum((y_true == 1) & (y_pred == -1))
    accuracy = (tp + tn) / (tp + tn + fp + fn)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    return accuracy, precision, recall, f1
```

## Common pitfalls

- The adversarial perturbation uses np.random.randint without explicit bounds, which may generate large integers depending on the NumPy version, drastically altering feature scales.
- PCA dimensionality reduction to 16 components is strictly required for the QNN due to simulator qubit limits, making direct comparison with the full 68D feature space impossible without this step.
- Labels are mapped to -1 and 1 for hinge loss training, which requires careful thresholding (e.g., >0 for positive) during metric computation.

## Evidence (verbatim from paper)

> In order to design effective adversarial attacks, we define the experimental settings, where we use accuracy [24], precision [25], recall [26], and F-score metrics to evaluate the robustness of models to attacks.

## Citation

```bibtex
@misc{akter2023adversarial,
  title={Exploring the Vulnerabilities of Machine Learning and Quantum Machine Learning to Adversarial Attacks using a Malware Dataset: A Comparative Analysis},
  author={Akter et al. (2023)},
  year={2023},
  note={arXiv:2305.19593}
}
```

- arXiv: 2305.19593