Bangla Key2text Eval

---
name: bangla-key2text-eval
description: Evaluates a model's ability to generate coherent, faithful Bangla text conditioned on a set of extracted keywords, testing sequence-to-sequence generation capabilities in a low-resource language setting. Use when the user wants to benchmark on Bangla Key2Text, or asks about evaluating this task. Reports generation_quality.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.19508
  bibtex_key: talukder2026banglakey2text
  confidence: medium
---

# bangla-key2text-eval

> Bangla Key2Text: Text Generation from Keywords for a Low Resource Language — Talukder et al. (2026) (arXiv:2604.19508, 2026)

## What this evaluates

Evaluates a model's ability to generate coherent, faithful Bangla text conditioned on a set of extracted keywords, testing sequence-to-sequence generation capabilities in a low-resource language setting.

## Datasets

- **Bangla Key2Text** — total 2600000; splits: train (-1), val (-1), test (-1); repo https://github.com/TonmoyTalukder/Bangla-Key2Text

## Metrics

- `generation_quality` **(primary)** — range: qualitative
  - Not explicitly defined in the provided section; assessed via keyword-text alignment, fluency, and faithfulness during inference.

## Input / output format

**Input**: A list of keywords extracted from a source text using a pre-trained BERT model.

**Output**: A generated Bangla text sequence, truncated to a maximum of 64 tokens.

## Scoring recipe

```python
def evaluate(keywords, gold_text, model):
    preds = model.generate(
        input_ids=keywords,
        max_length=64,
        num_beams=2,
        top_k=50,
        top_p=0.95,
        repetition_penalty=2.5,
        length_penalty=1.0
    )
    # Coherence and faithfulness scoring (human or automated)
    score = compute_alignment_and_fluency(preds, gold_text)
    return score
```

## Common pitfalls

- Input sequences are truncated to 512 tokens, which may discard relevant context for keyword extraction.
- Decoding hyperparameters (beam size, top-k, top-p, repetition penalty) vary across experiments and must be fixed for reproducible comparison.
- Keyword extraction uses default BERT parameters without fine-tuning, potentially introducing noise into the conditioning signal.

## Evidence (verbatim from paper)

> During inference, we experimented with different decoding techniques i.e. greedy decoding, beam search with beam size 2, top-k sampling with k=50 and top-p sampling with p=0.95. The repetition penalty and length penalty were set to 2.5 and α=1.0 respectively. For 4-bit quantized LLMs, we utilized model-specific default text generation parameters during inference.

## Citation

```bibtex
@misc{talukder2026banglakey2text,
  title={Bangla Key2Text: Text Generation from Keywords for a Low Resource Language},
  author={Talukder et al. (2026)},
  year={2026},
  note={arXiv:2604.19508}
}
```

- arXiv: 2604.19508