Csi Bert2 Eval

---
name: csi-bert2-eval
description: Evaluates a transformer-based framework for Channel State Information (CSI) time-series prediction and wireless sensing classification. It probes the model's ability to recover missing data, predict future CSI sequences, and classify human actions or environmental states from Wi-Fi signals. Use when the user wants to benchmark on WiGesture, WiFall, WiCount, CommPre, or asks about evaluating this task. Reports Accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2412.06861
  bibtex_key: zhao2024csibert2
  confidence: high
---

# csi-bert2-eval

> CSI-BERT2: A BERT-inspired Framework for Efficient CSI Prediction and Classification in Wireless Communication and Sensing — Zijian Zhao et al. (2024) (arXiv:2412.06861, 2024)

## What this evaluates

Evaluates a transformer-based framework for Channel State Information (CSI) time-series prediction and wireless sensing classification. It probes the model's ability to recover missing data, predict future CSI sequences, and classify human actions or environmental states from Wi-Fi signals.

## Datasets

- **WiGesture** — total ?; splits: train (-1), test (-1); HF `RS2002/WiGesture`
- **WiFall** — total ?; splits: train (-1), test (-1); HF `RS2002/WiFall`
- **WiCount** — total ?; splits: train (-1), test (-1)
- **CommPre** — total 892; splits: train (-1), test (-1)

## Metrics

- `MSE` — range: other
  - Mean Squared Error: average of squared differences between ground truth c and prediction c_hat across N packets and M subcarriers.
- `SMAPE` — range: percent
  - Symmetric Mean Absolute Percentage Error: average of 2*|c - c_hat| / (|c| + |c_hat|) across all elements.
- `MAPE` — range: percent
  - Mean Absolute Percentage Error: average of |c - c_hat| / (c + epsilon) across all elements.
- `Accuracy` **(primary)** — range: percent
  - Percentage of correctly classified samples out of the total test set for gesture recognition, fall detection, and people counting tasks.

## Input / output format

**Input**: 1-second CSI time-series samples (100 time steps × 52 subcarriers). For prediction tasks, the input is the first 80 steps (0.8s) and the target is the subsequent 20 steps (0.2s). For classification, the full 100-step sequence is used.

**Output**: Predicted CSI values (complex magnitude/phase or real/imaginary components) for reconstruction/prediction, or discrete class labels for sensing tasks.

## Scoring recipe

```python
def compute_metrics(y_true, y_pred, task='prediction'):
    if task == 'prediction':
        mse = np.mean((y_true - y_pred) ** 2)
        smape = np.mean(2 * np.abs(y_true - y_pred) / (np.abs(y_true) + np.abs(y_pred) + 1e-8))
        mape = np.mean(np.abs(y_true - y_pred) / (np.abs(y_true) + 1e-8))
        return mse, smape, mape
    else:
        return np.mean(y_true == y_pred) * 100
```

## Common pitfalls

- Recovery error must be calculated only on the 15% of deleted/masked packets, not the entire sequence.
- Distinguish between 'recover' (filling only masked positions) and 'replace' (overwriting the whole sequence) strategies when training downstream classifiers.
- The model is explicitly tested on discontinuous sequences and varying sampling rates, which breaks conventional fixed-rate baselines.

## Evidence (verbatim from paper)

> We employ mean squared error (MSE), symmetric mean absolute percentage error (SMAPE), and mean absolute percentage error (MAPE) to quantify the error: ... The metrics are calculated only on the 15% of the deleted CSI.

## Citation

```bibtex
@misc{zhao2024csibert2,
  title={CSI-BERT2: A BERT-inspired Framework for Efficient CSI Prediction and Classification in Wireless Communication and Sensing},
  author={Zijian Zhao et al. (2024)},
  year={2024},
  note={arXiv:2412.06861}
}
```

- arXiv: 2412.06861