Lfree Da Eval

---
name: lfree-da-eval
description: Evaluates a malware detection model's ability to adapt to natural concept drift over time using a rolling monthly update setup on real-world Windows malware binaries. Use when the user wants to benchmark on MB-24+, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2511.14963
  bibtex_key: li2025lfreeeda
  confidence: high
---

# lfree-da-eval

> LFreeDA: Label-Free Drift Adaptation for Windows Malware Detection — Li et al. (2025) (arXiv:2511.14963, 2025)

## What this evaluates

Evaluates a malware detection model's ability to adapt to natural concept drift over time using a rolling monthly update setup on real-world Windows malware binaries.

## Datasets

- **MB-24+** — total ?; splits: source_train (-1), source_test (-1), target_train (-1), target_test (-1)

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Standard classification accuracy: fraction of correctly predicted labels out of total samples.
- `macro-F1` — range: [0, 1]
  - Macro-averaged F1 score: harmonic mean of precision and recall computed per class (malware, benign) and then averaged.

## Input / output format

**Input**: Windows PE binaries represented as Control-Flow Graphs (CFGs) or image-based representations (DIReps).

**Output**: Binary classification label (malware vs. benign).

## Scoring recipe

```python
def score(predictions, gold):
    acc = sum(p == g for p, g in zip(predictions, gold)) / len(gold)
    classes = [0, 1]
    f1s = []
    for c in classes:
        tp = sum(1 for p, g in zip(predictions, gold) if p == c and g == c)
        fp = sum(1 for p, g in zip(predictions, gold) if p == c and g != c)
        fn = sum(1 for p, g in zip(predictions, gold) if p != c and g == c)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        f1s.append(f1)
    macro_f1 = sum(f1s) / len(f1s)
    return acc, macro_f1
```

## Common pitfalls

- Temporal leakage must be strictly avoided; source and target data are partitioned by month, with June skipped to ensure clean separation.
- Benign samples lack temporal labels and are assumed distributionally stable, but CFG extraction fails for some files due to disassembly errors.
- Pseudo-labels are generated without ground truth during adaptation, requiring confidence thresholding and outlier detection to filter noise.

## Evidence (verbatim from paper)

> Performance is measured by accuracy and macro-F1, and all results are averaged over five independent runs.

## Citation

```bibtex
@misc{li2025lfreeeda,
  title={LFreeDA: Label-Free Drift Adaptation for Windows Malware Detection},
  author={Li et al. (2025)},
  year={2025},
  note={arXiv:2511.14963}
}
```

- arXiv: 2511.14963