Bengali Asr Diarization Eval

---
name: bengali-asr-diarization-eval
description: Evaluates automatic speech recognition (ASR) accuracy and speaker diarization performance on long-form Bengali speech. It measures phonetic robustness and computational efficiency using public and private test splits under strict hardware constraints. Use when the user wants to benchmark on Lipi-Ghor-882, or asks about evaluating this task. Reports WER.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.23070
  bibtex_key: hasan2026bengali
  confidence: high
---

# bengali-asr-diarization-eval

> Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment — Hasan et al. (2026) (arXiv:2602.23070, 2026)

## What this evaluates

Evaluates automatic speech recognition (ASR) accuracy and speaker diarization performance on long-form Bengali speech. It measures phonetic robustness and computational efficiency using public and private test splits under strict hardware constraints.

## Datasets

- **Lipi-Ghor-882** — total 882; splits: test (22)

## Metrics

- `WER` **(primary)** — range: percent
  - Word Error Rate: 100 * (Substitutions + Deletions + Insertions) / Total Reference Words.
- `DER` — range: percent
  - Diarization Error Rate: 100 * (False Alarms + Missed Detections + Speaker Confusions) / Total Speech Duration.
- `RTF` — range: ratio
  - Real-Time Factor: Inference Processing Time / Audio Duration.

## Input / output format

**Input**: Long-form Bengali audio recordings from public and private test sets.

**Output**: ASR: Transcribed text. Diarization: Speaker turn labels and timestamps.

## Scoring recipe

```python
def compute_wer(hypothesis, reference):
    return 100 * (substitutions + deletions + insertions) / len(reference)

def compute_der(pred_spk_assignments, gt_spk_assignments):
    return 100 * (false_alarms + missed_detections + confusions) / total_duration

def compute_rtf(process_time, audio_duration):
    return process_time / audio_duration
```

## Common pitfalls

- Public and private test splits are evaluated separately; results must be reported for both.
- RTF is measured under strict hardware constraints (2x T4 GPUs for inference), making cross-hardware comparisons invalid.
- Diarization accuracy relies on heuristic post-processing of baseline outputs rather than end-to-end model retraining.

## Evidence (verbatim from paper)

> Trainings were done using L40S GPU, 48GB VRAM. All inference processes were evaluated under strict hardware constraints (2x T4 GPUs). The separation of the ASR and Diarization pipelines allowed for maximum computational efficiency without relying on complex, end-to-end joint modeling.

*TABLE II: ASR Performance Benchmarks on 22-Hours Test Set*

| Model | RTF | Pub WER | Priv WER |
| --- | --- | --- | --- |

*TABLE III: Speaker Diarization Benchmarks*

| Model | Retrained | Post-Proc | Pub DER | Priv DER | RTF |

## Citation

```bibtex
@misc{hasan2026bengali,
  title={Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment},
  author={Hasan et al. (2026)},
  year={2026},
  note={arXiv:2602.23070}
}
```

- arXiv: 2602.23070