Chime4 Aishell4 Asr Eval

---
name: chime4-aishell4-asr-eval
description: Evaluates multi-channel end-to-end speech recognition systems in noisy, real-world environments using neural beamforming front-ends combined with CTC-CRF acoustic models. It probes the model's ability to leverage single-channel data via pre-training, data scheduling, or simulation to improve robustness and accuracy. Use when the user wants to benchmark on CHiME4, AISHELL-4, or asks about evaluating this task. Reports WER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2107.02670
  bibtex_key: an2021exploiting
  confidence: high
---

# chime4-aishell4-asr-eval

> Exploiting Single-Channel Speech For Multi-channel End-to-end Speech Recognition — An et al. (2021) (arXiv:2107.02670, 2021)

## What this evaluates

Evaluates multi-channel end-to-end speech recognition systems in noisy, real-world environments using neural beamforming front-ends combined with CTC-CRF acoustic models. It probes the model's ability to leverage single-channel data via pre-training, data scheduling, or simulation to improve robustness and accuracy.

## Datasets

- **CHiME4** — total ?; splits: dev_real (-1), dev_simu (-1), eval_real (-1), eval_simu (-1)
- **AISHELL-4** — total ?; splits: train (-1), test (-1)

## Metrics

- `WER` **(primary)** — range: percent
  - Word Error Rate, calculated as the minimum number of word edits (insertions, deletions, substitutions) required to transform the predicted transcript into the reference transcript, divided by the total number of words in the reference.

## Input / output format

**Input**: Multi-channel audio recordings processed into STFT features, passed through a neural beamformer to produce enhanced single-channel STFT features, converted to 40-dim log fbank features, mean-variance normalized, and appended with delta/delta-delta features, then subsampled by a factor of 3.

**Output**: Word-level or character-level transcript predictions generated by the CTC-CRF decoder.

## Scoring recipe

```python
def compute_wer(predictions, references):
    total_errors = 0
    total_words = 0
    for pred, ref in zip(predictions, references):
        dist = levenshtein_distance(pred.split(), ref.split())
        total_errors += dist
        total_words += len(ref.split())
    return (total_errors / total_words) * 100 if total_words > 0 else 0.0
```

## Common pitfalls

- The authors explicitly state they do not use model ensemble, unlike some top CHiME4 challenge systems, so comparisons must account for this.
- AISHELL-4 lacks an official lexicon; the evaluation uses the AISHELL-1 lexicon combined with Jieba segmentation, which may affect Chinese word boundary alignment.
- Language model rescoring is applied post-decoding (3-gram initial, then 5-gram/RNN/LSTM), so the reported WER reflects the full decoding pipeline, not just the acoustic model.

## Evidence (verbatim from paper)

> It can be seen that the WER drops dramatically as stronger language models are used. We also compare our model with models from literature in Table[2]. It can be seen that our model outperforms the CHiME4 baseline system significantly.

## Citation

```bibtex
@misc{an2021exploiting,
  title={Exploiting Single-Channel Speech For Multi-channel End-to-end Speech Recognition},
  author={An et al. (2021)},
  year={2021},
  note={arXiv:2107.02670}
}
```

- arXiv: 2107.02670