Traffic Classification Eval

---
name: traffic-classification-eval
description: Evaluates the ability of deep learning models to classify mobile network traffic into specific application categories (e.g., video, music, games) and background services using extracted packet flow features. It specifically probes the effectiveness of semi-supervised VAE-CNN architectures and model pruning techniques for resource-constrained edge devices. Use when the user wants to benchmark on Private Campus Network Dataset, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2308.06924
  bibtex_key: wang2023fededge
  confidence: high
---

# traffic-classification-eval

> FedEdge AI-TC: A Semi-supervised Traffic Classification Method based on Trusted Federated Deep Learning for Mobile Edge Computing — Wang et al. (2023) (arXiv:2308.06924, 2023)

## What this evaluates

Evaluates the ability of deep learning models to classify mobile network traffic into specific application categories (e.g., video, music, games) and background services using extracted packet flow features. It specifically probes the effectiveness of semi-supervised VAE-CNN architectures and model pruning techniques for resource-constrained edge devices.

## Datasets

- **Private Campus Network Dataset** — total 78277; splits: train (-1), test (41451)

## Metrics

- `Accuracy` **(primary)** — range: [0, 1]
  - Overall classification accuracy calculated as the number of correctly predicted flows divided by the total number of flows in the test set. Reported alongside per-class Precision, Recall, and F1-score.

## Input / output format

**Input**: Numerical features extracted from network packet flows (PCAP files) using CICFlowMeter.

**Output**: Categorical application label (e.g., Bilibili, QQ music, Honor of Kings, Teamfight Tactics, Game for peace, Background, IQiyi).

## Scoring recipe

```python
def compute_accuracy(predictions, labels):
    correct = sum(1 for p, l in zip(predictions, labels) if p == l)
    return correct / len(labels)

def compute_f1_macro(predictions, labels, classes):
    tp = sum(1 for p, l in zip(predictions, labels) if p == l)
    fp = sum(1 for p, l in zip(predictions, labels) if p != l)
    fn = sum(1 for p, l in zip(predictions, labels) if p != l)
    return 2 * tp / (2 * tp + fp + fn) if (2 * tp + fp + fn) > 0 else 0.0
```

## Common pitfalls

- Training time and hardware-dependent metrics are explicitly excluded from the evaluation protocol.
- The dataset uses a semi-supervised setup where a VAE encoder is pre-trained on unlabeled flows before the CNN classifier is fine-tuned, which differs from standard supervised baselines.
- The test set size (41,451 flows) corresponds to a fixed 45% split ratio of the total collected flows, not a standard public benchmark split.

## Evidence (verbatim from paper)

> The following is a description of the evaluation metrics: Precision, Recall, F1, Accuracy, and AUC. ... We have set the partition ratio to 0.45. After training and testing, we obtained the respective Confusion Matrix and Classification Report for E-CNN and single CNN using this ratio. ... resulting in a total classification accuracy of 0.7422.

## Citation

```bibtex
@misc{wang2023fededge,
  title={FedEdge AI-TC: A Semi-supervised Traffic Classification Method based on Trusted Federated Deep Learning for Mobile Edge Computing},
  author={Wang et al. (2023)},
  year={2023},
  note={arXiv:2308.06924}
}
```

- arXiv: 2308.06924