Hstf Trojan Detection Eval

---
name: hstf-trojan-detection-eval
description: Evaluates a deep learning model's ability to detect HTTP-based Trojan malware in network traffic by analyzing hierarchical spatio-temporal features. It probes the model's binary classification accuracy, robustness to training set class imbalance, and cross-dataset generalization performance. Use when the user wants to benchmark on BTHT-2018, ISCX-2012, or asks about evaluating this task. Reports F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2309.03724
  bibtex_key: xie2023hstfmodel
  confidence: high
---

# hstf-trojan-detection-eval

> HSTF-Model: an HTTP-based Trojan Detection Model via the Hierarchical Spatio-Temporal Features of Traffics — Jiang Xie et al. (2023) (arXiv:2309.03724, 2023)

## What this evaluates

Evaluates a deep learning model's ability to detect HTTP-based Trojan malware in network traffic by analyzing hierarchical spatio-temporal features. It probes the model's binary classification accuracy, robustness to training set class imbalance, and cross-dataset generalization performance.

## Datasets

- **BTHT-2018** — total 4082588; splits: train (-1), test (-1)
- **ISCX-2012** — total 245348; splits: train (-1), test (-1)

## Metrics

- `F1` **(primary)** — range: [0, 1]
  - F1 = 2 * (Precision * Recall) / (Precision + Recall), where Precision = TP/(TP+FP) and Recall = TP/(TP+FN).
- `Precision` — range: [0, 1]
  - Precision = TP / (TP + FP).
- `Recall` — range: [0, 1]
  - Recall = TP / (TP + FN).
- `FPR` — range: [0, 1]
  - FPR = FP / (TN + FP).
- `TPR` — range: [0, 1]
  - TPR = TP / (TP + FN) (equivalent to Recall).

## Input / output format

**Input**: Preprocessed HTTP flow data represented as sequences of request/response packets, payload sequences, and statistical features (packet size, flow size).

**Output**: Binary classification label: 'Malicious' or 'Benign', derived from a probability score compared against a threshold λ.

## Scoring recipe

```python
def compute_metrics(tp, fp, tn, fn):
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0.0
    fpr = fp / (tn + fp) if (tn + fp) > 0 else 0.0
    tpr = recall
    return {'precision': precision, 'recall': recall, 'f1': f1, 'fpr': fpr, 'tpr': tpr}
```

## Common pitfalls

- Test set is artificially balanced 1:1 (malicious:benign), which masks real-world extreme class imbalance and may inflate perceived practical utility.
- Generalization is evaluated by training on BTHT-2018 and testing on ISCX-2012, but the datasets differ in traffic types and collection environments, making direct performance comparison difficult.
- The decision threshold λ is mentioned but never explicitly defined or tuned, making exact reproduction of binary predictions challenging.

## Evidence (verbatim from paper)

> Precision and recall are used as primary evaluation indicators to verify the detection performance of the model, as shown in Eq(14). Fβ is also calculated as a comprehensive evaluation index. A represents the weight of P and R in this index. The larger β(>1) represents R is more important, and the smaller β(<1) represents P is more important. We set β=1 to show that both are equally important.

## Citation

```bibtex
@misc{xie2023hstfmodel,
  title={HSTF-Model: an HTTP-based Trojan Detection Model via the Hierarchical Spatio-Temporal Features of Traffics},
  author={Jiang Xie et al. (2023)},
  year={2023},
  note={arXiv:2309.03724}
}
```

- arXiv: 2309.03724