Isac Lawn Multimodal Eval

---
name: isac-lawn-multimodal-eval
description: Probes the capability of multimodal fusion and adaptive expert routing for integrated sensing and communication tasks in low-altitude wireless networks. Specifically, it evaluates how well models leverage synchronized visual, lidar, radar, GPS, and RF channel data to predict beam indices, estimate path loss, and track UAV trajectories under dynamic environmental conditions. Use when the user wants to benchmark on Public Multimodal ISAC Dataset for Low-Altitude Scenarios, or asks about evaluating this task. Reports top-1 beam accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2512.01750
  bibtex_key: zhang2025multimodal
  confidence: high
---

# isac-lawn-multimodal-eval

> Multimodal Mixture-of-Experts for ISAC in Low-Altitude Wireless Networks — Kai Zhang et al. (2025) (arXiv:2512.01750, 2025)

## What this evaluates

Probes the capability of multimodal fusion and adaptive expert routing for integrated sensing and communication tasks in low-altitude wireless networks. Specifically, it evaluates how well models leverage synchronized visual, lidar, radar, GPS, and RF channel data to predict beam indices, estimate path loss, and track UAV trajectories under dynamic environmental conditions.

## Datasets

- **Public Multimodal ISAC Dataset for Low-Altitude Scenarios** — total ?; splits: train (-1), test (-1)

## Metrics

- `top-1 beam accuracy` **(primary)** — range: percent
  - Percentage of correctly predicted optimal downlink beam indices out of the total number of test samples.
- `MSE` — range: other
  - Mean Squared Error between predicted and ground-truth values for path loss or position.
- `NMSE` — range: dB
  - Normalized Mean Squared Error, calculated as the ratio of MSE to the variance of the ground-truth signal, then converted to decibels.
- `average Euclidean distance` — range: other
  - Average 3D Euclidean distance between predicted and ground-truth UAV positions across all time steps.

## Input / output format

**Input**: Synchronized multimodal inputs per time slot: RGB images, lidar point clouds, mmWave radar point clouds, GPS coordinates, and complex-valued RF channel impulse responses.

**Output**: Task-dependent: (1) discrete beam index, (2) scalar path loss value, or (3) 3D UAV position coordinates.

## Scoring recipe

```python
# Beam Prediction
accuracy = (preds == golds).mean() * 100

# Path Loss / Trajectory (MSE & NMSE)
mse = ((preds - golds) ** 2).mean()
nmse_db = 10 * np.log10(mse / golds.var())

# Trajectory (Euclidean Distance)
dist = np.sqrt(((preds - golds) ** 2).sum(axis=1))
avg_dist = dist.mean()
```

## Common pitfalls

- NMSE is explicitly reported in dB scale, which can lead to misinterpretation if treated as a linear ratio without the 10*log10 conversion.
- The dataset contains mismatched time slot counts across modalities (3,000 for sensing vs. 30,000 for RF), requiring careful synchronization or subsampling during evaluation.
- Sparse MoE baselines only activate 5 of 15 experts, drastically reducing compute but sometimes yielding slightly lower accuracy that must be weighed against efficiency.

## Evidence (verbatim from paper)

> i) sensing-aided beam prediction, where the model predicts the optimal beam index from sensing inputs and is evaluated by top-1 beam accuracy; ii) sensing-aided path loss prediction, where the model predicts downlink path loss and is evaluated by the MSE between estimated and ground-truth values; and iii) communication-aided UAV trajectory tracking, where the UAV position is regressed from sensing and communication features, and evaluated by MSE and average Euclidean distance between predicted and ground-truth positions.

## Citation

```bibtex
@misc{zhang2025multimodal,
  title={Multimodal Mixture-of-Experts for ISAC in Low-Altitude Wireless Networks},
  author={Kai Zhang et al. (2025)},
  year={2025},
  note={arXiv:2512.01750}
}
```

- arXiv: 2512.01750