Foca Malware Classification Eval

---
name: foca-malware-classification-eval
description: This evaluation probes a model's ability to classify Android malware by fusing audio and visual representations derived from raw APK binaries. It measures supervised classification performance across multiple malware families and benign samples using standard accuracy and macro-F1 metrics. Use when the user wants to benchmark on CICMalDroid-2020, Mal-Net, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.17638
  bibtex_key: choudhury2026foca
  confidence: high
---

# foca-malware-classification-eval

> FOCA: Multimodal Malware Classification via Hyperbolic Cross-Attention — Choudhury et al. (2026) (arXiv:2601.17638, 2026)

## What this evaluates

This evaluation probes a model's ability to classify Android malware by fusing audio and visual representations derived from raw APK binaries. It measures supervised classification performance across multiple malware families and benign samples using standard accuracy and macro-F1 metrics.

## Datasets

- **CICMalDroid-2020** — total 17341; splits: 5-fold cross-validation (-1)
- **Mal-Net** — total 8000; splits: 5-fold cross-validation (-1)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Number of correctly predicted class labels divided by the total number of samples, reported as a percentage.
- `macro-F1` — range: percent
  - Unweighted mean of the F1-score computed independently for each class, reported as a percentage.

## Input / output format

**Input**: Multimodal feature representations of APK files: audio features (generated via binary-to-audio transformation) and image features (generated via binary-to-image transformation).

**Output**: A single categorical class label indicating the malware family (e.g., Adware, Banking malware, SMS malware, Riskware) or benign category.

## Scoring recipe

```python
def compute_metrics(predictions, labels, num_classes):
    accuracy = sum(p == l for p, l in zip(predictions, labels)) / len(labels)
    f1_scores = []
    for c in range(num_classes):
        tp = sum(1 for p, l in zip(predictions, labels) if p == c and l == c)
        fp = sum(1 for p, l in zip(predictions, labels) if p == c and l != c)
        fn = sum(1 for p, l in zip(predictions, labels) if p != c and l == c)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        f1_scores.append(f1)
    macro_f1 = sum(f1_scores) / num_classes
    return accuracy * 100, macro_f1 * 100
```

## Common pitfalls

- Using the full Mal-Net dataset (>1.2M images) instead of the paper's explicitly filtered 8,000-sample subset (800 per class).
- Reporting results from a single train/test split instead of averaging over the specified 5-fold cross-validation strategy.
- Confusing the fusion operators: $+$ (simple concatenation), $\otimes$ (Euclidean cross-attention), and $\boxplus$ (hyperbolic cross-attention/FOCA).

## Evidence (verbatim from paper)

> Accuracy and macro-F1 are reported in %; Here, $+$ and $\otimes$ represent concatenation and cross-modal attention in Euclidean space, and $\boxplus$ defines hyperbolic cross-attention fusion via FOCA, respectively; The scores are average of five folds;

## Citation

```bibtex
@misc{choudhury2026foca,
  title={FOCA: Multimodal Malware Classification via Hyperbolic Cross-Attention},
  author={Choudhury et al. (2026)},
  year={2026},
  note={arXiv:2601.17638}
}
```

- arXiv: 2601.17638