Gwrfrf Spatial Spectrum Eval

---
name: gwrfrf-spatial-spectrum-eval
description: This benchmark evaluates a model's ability to synthesize accurate spatial radio-frequency spectra at target transmitter locations using neighboring spectra and scene geometry. It probes both single-scene prediction accuracy and cross-scene generalization capabilities in wireless propagation environments. Use when the user wants to benchmark on RFID Dataset, MATLAB Dataset, or asks about evaluating this task. Reports MSE.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2502.05708
  bibtex_key: yang2025gwrfrf
  confidence: high
---

# gwrfrf-spatial-spectrum-eval

> Generalizable Radio-Frequency Radiance Fields for Spatial Spectrum Synthesis — Kang Yang, Yuning Chen, Wan Du (2025) (arXiv:2502.05708, 2025)

## What this evaluates

This benchmark evaluates a model's ability to synthesize accurate spatial radio-frequency spectra at target transmitter locations using neighboring spectra and scene geometry. It probes both single-scene prediction accuracy and cross-scene generalization capabilities in wireless propagation environments.

## Datasets

- **RFID Dataset** — total 6123; splits: train (-1), test (-1)
- **MATLAB Dataset** — total 32625; splits: train (-1), test (-1)

## Metrics

- `MSE` **(primary)** — range: other
  - Mean Squared Error: average of the squared differences between predicted and ground truth spectrum values. Standard regression loss.
- `LPIPS` — range: [0, 1]
  - Learned Perceptual Image Patch Similarity: measures perceptual difference using features from a pre-trained network. Lower is better.
- `PSNR` — range: other
  - Peak Signal-to-Noise Ratio: 10 * log10(max_val^2 / MSE). Higher is better, typically measured in dB.
- `SSIM` — range: other
  - Structural Similarity Index Measure: evaluates luminance, contrast, and structure similarity between two images. Higher is better.

## Input / output format

**Input**: 2D spatial spectrum images (360×90) from neighboring transmitter locations, along with scene geometry/CAD models.

**Output**: Predicted 2D spatial spectrum image (360×90) at the target transmitter location.

## Scoring recipe

```python
import numpy as np
from skimage.metrics import peak_signal_noise_ratio, structural_similarity

def compute_metrics(y_true, y_pred):
    mse = np.mean((y_true - y_pred) ** 2)
    psnr = peak_signal_noise_ratio(y_true, y_pred)
    ssim = structural_similarity(y_true, y_pred)
    lpips = compute_lpips(y_true, y_pred)  # Requires LPIPS library
    return {'MSE': mse, 'PSNR': psnr, 'SSIM': ssim, 'LPIPS': lpips}
```

## Common pitfalls

- The dataset is synthetically generated via MATLAB ray tracing and not publicly hosted; users must replicate the simulation setup or use the provided NeRF² dataset.
- Evaluation settings differ significantly between single-scene (train/test on same layout) and cross-scene/generalization (train on V1, test on V2/V3 or different layouts); results are not directly comparable across these settings.
- Spectra are treated as 2D images (360×90) for metric computation, not raw 1D signal vectors.

## Evidence (verbatim from paper)

> Signal power prediction is evaluated as a regression task with Mean Squared Error (MSE). Since the spatial spectrum, visualized in Figure 2(b), resembles an image, image quality metrics assess pixel-level differences and structural information (Wang et al., 2004), capturing directional patterns. Therefore, the evaluation involves four widely adopted metrics: MSE↓, Learned Perceptual Image Patch Similarity (LPIPS↓), Peak Signal-to-Noise Ratio (PSNR↑), and Structural Similarity Index Measure (SSIM↑).

## Citation

```bibtex
@misc{yang2025gwrfrf,
  title={Generalizable Radio-Frequency Radiance Fields for Spatial Spectrum Synthesis},
  author={Kang Yang, Yuning Chen, Wan Du (2025)},
  year={2025},
  note={arXiv:2502.05708}
}
```

- arXiv: 2502.05708