Ubb Ntp Eval

---
name: ubb-ntp-eval
description: Evaluates the predictive accuracy and computational efficiency of a user-behavior-based network traffic forecasting method against statistical and neural network baselines on real-world SMS traffic data. Use when the user wants to benchmark on Guangzhou and Milan SMS datasets, or asks about evaluating this task. Reports R2.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2304.09811
  bibtex_key: wang2023analytic
  confidence: high
---

# ubb-ntp-eval

> Analytic Network Traffic Prediction Based on User Behavior Modeling — Wang et al. (2023) (arXiv:2304.09811, 2023)

## What this evaluates

Evaluates the predictive accuracy and computational efficiency of a user-behavior-based network traffic forecasting method against statistical and neural network baselines on real-world SMS traffic data.

## Datasets

- **Guangzhou and Milan SMS datasets** — total ?; splits: train (first 2 weeks) (-1), test (rest) (-1)

## Metrics

- `MSE` — range: other
  - Mean Square Error: the average of the squares of the errors between predicted and actual values.
- `RMSE` — range: other
  - Root Mean Square Error: the square root of the MSE, providing error magnitude in the original units.
- `MAE` — range: other
  - Mean Absolute Error: the average of the absolute differences between predicted and actual values.
- `R2` **(primary)** — range: [0, 1]
  - Coefficient of determination: 1 minus the ratio of the residual sum of squares to the total sum of squares, indicating the proportion of variance explained by the model.

## Input / output format

**Input**: Time-series network traffic data (SMS message counts) over a multi-week period, used to fit nine normal-distribution parameters representing user behavior cycles.

**Output**: Predicted traffic values for the test period, fitted behavioral parameters (peak, variance, time), and total elapsed time for training and prediction.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred):
    n = len(y_true)
    y_mean = sum(y_true) / n
    mse = sum((y - yp)**2 for y, yp in zip(y_true, y_pred)) / n
    rmse = mse**0.5
    mae = sum(abs(y - yp) for y, yp in zip(y_true, y_pred)) / n
    ss_res = sum((y - yp)**2 for y, yp in zip(y_true, y_pred))
    ss_tot = sum((y - y_mean)**2 for y in y_true)
    r2 = 1 - (ss_res / ss_tot)
    return {'MSE': mse, 'RMSE': rmse, 'MAE': mae, 'R2': r2}
```

## Common pitfalls

- The train/test split is strictly time-based (first two weeks vs. remaining weeks), not random, which may limit generalizability to different time horizons.
- Computational efficiency metrics combine training and prediction time, but the proposed method requires no iterative offline training, making the comparison with LSTM/ARIMA slightly asymmetric.
- Metrics are reported on two geographically distinct datasets with different traffic magnitudes, so absolute error values (MSE/MAE) are not directly comparable across cities without normalization.

## Evidence (verbatim from paper)

> Predictive accuracy is measured by Mean Square Error (MSE), Root MSE (RMSE), Mean Absolute Error (MAE), and coefficient of determination (R2), which can be clearly defined by the following formulas: ... We use the SMS data of the first two weeks to build the proposed mathematical model and the rest to evaluate the performance.

## Citation

```bibtex
@misc{wang2023analytic,
  title={Analytic Network Traffic Prediction Based on User Behavior Modeling},
  author={Wang et al. (2023)},
  year={2023},
  note={arXiv:2304.09811}
}
```

- arXiv: 2304.09811