Librispeech Sr Semantic Comm Eval

---
name: librispeech-sr-semantic-comm-eval
description: Evaluates the ability of semantic communication systems to transmit speech spectra over noisy wireless channels (AWGN and Rayleigh) and accurately recover text transcriptions, comparing performance against traditional speech and text transceivers. Use when the user wants to benchmark on LibriSpeech, or asks about evaluating this task. Reports Character Error Rate (CER).
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2107.11190
  bibtex_key: weng2021semantic
  confidence: high
---

# librispeech-sr-semantic-comm-eval

> Semantic Communications for Speech Recognition — Weng et al. (2021) (arXiv:2107.11190, 2021)

## What this evaluates

Evaluates the ability of semantic communication systems to transmit speech spectra over noisy wireless channels (AWGN and Rayleigh) and accurately recover text transcriptions, comparing performance against traditional speech and text transceivers.

## Datasets

- **LibriSpeech** — total ?; splits: train (-1), test (-1)

## Metrics

- `Character Error Rate (CER)` **(primary)** — range: [0, 1]
  - Standard character-level edit distance normalized by the length of the reference transcription.
- `Word Error Rate (WER)` — range: [0, 1]
  - Standard word-level edit distance normalized by the length of the reference transcription.

## Input / output format

**Input**: Speech sample sequences M converted to spectra S, channel conditions H, and SNR values.

**Output**: Recovered text transcriptions ̂T (or ̂S).

## Scoring recipe

```python
def compute_cer_wer(predictions, references):
    cer = sum(edit_distance(p, r) for p, r in zip(predictions, references)) / sum(len(r) for r in references)
    wer = sum(edit_distance(p.split(), r.split()) for p, r in zip(predictions, references)) / sum(len(r.split()) for r in references)
    return cer, wer
```

## Common pitfalls

- Training is performed under fixed channel conditions, but testing evaluates robustness across varying SNR regimes and channel types (AWGN vs Rayleigh).
- Benchmarks use a two-stage pipeline (source coding + ASR) rather than end-to-end semantic extraction, making direct architectural comparison non-trivial.
- The output format requires decoding CTC probabilities into text transcriptions before computing CER/WER.

## Evidence (verbatim from paper)

> The model is trained under fixed channel conditions and demonstrates robustness across varying channel environments, especially in low SNR regimes, outperforming traditional systems in character- and word-error rates.

## Citation

```bibtex
@misc{weng2021semantic,
  title={Semantic Communications for Speech Recognition},
  author={Weng et al. (2021)},
  year={2021},
  note={arXiv:2107.11190}
}
```

- arXiv: 2107.11190