Bangla Sentiment Eval

---
name: bangla-sentiment-eval
description: Evaluates the capability of NLP models to classify sentiment in Bangla text. It compares classical machine learning, CNN, FastText, and transformer-based architectures to determine which model family performs best on low-resource Bangla sentiment tasks. Use when the user wants to benchmark on Multiple publicly available Bangla sentiment datasets, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2011.10106
  bibtex_key: hasan2020sentiment
  confidence: medium
---

# bangla-sentiment-eval

> Sentiment Classification in Bangla Textual Content: A Comparative Study — Md. Arid Hasan et al. (2020) (arXiv:2011.10106, 2020)

## What this evaluates

Evaluates the capability of NLP models to classify sentiment in Bangla text. It compares classical machine learning, CNN, FastText, and transformer-based architectures to determine which model family performs best on low-resource Bangla sentiment tasks.

## Datasets

- **Multiple publicly available Bangla sentiment datasets** — total ?; splits: train (-1), dev (-1), test (-1); repo https://github.com/banglanlp/bangla-sentiment-classification

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - The proportion of correctly predicted sentiment labels out of the total number of instances. Calculated as (number of correct predictions) / (total number of predictions).

## Input / output format

**Input**: Preprocessed Bangla text, converted to bag-of-n-gram vectors (unigram, bigram, tri-gram weighted with TF-IDF) for classical models, or tokenized sequences for CNN/FastText/Transformer models.

**Output**: A discrete sentiment class label (e.g., positive, negative, neutral).

## Scoring recipe

```python
def calculate_accuracy(predictions, gold_labels):
    correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
    return correct / len(gold_labels)
```

## Common pitfalls

- Transformers are evaluated by running each experiment 10 times with different random seeds and selecting the single best model based on development set accuracy, rather than averaging results across runs.
- Early stopping is applied during training based on development set accuracy with a patience of 200 epochs, which may cause models to stop training before convergence if the dev set is small or noisy.
- Different models use different pre-trained embeddings (Word2Vec, GloVe, FastText official, or Transformer Toolkit tokenizers), making direct architectural comparisons confounded by embedding quality.

## Evidence (verbatim from paper)

> To reduce the computation without impacting the performance we used early stopping based on the accuracy on the development set. The patience size in early stopping was set to 200. ... we run each experiment 10 times using different random seeds and select the model that performs the best on the development set.

## Citation

```bibtex
@misc{hasan2020sentiment,
  title={Sentiment Classification in Bangla Textual Content: A Comparative Study},
  author={Md. Arid Hasan et al. (2020)},
  year={2020},
  note={arXiv:2011.10106}
}
```

- arXiv: 2011.10106