Low Resource Ner Transfer Eval

---
name: low-resource-ner-transfer-eval
description: Evaluates cross-lingual transfer learning for Named Entity Recognition in low-resource Indian languages (Hindi and Marathi) by measuring how well models trained on combined or assisting-language datasets generalize to target language test sets compared to monolingual baselines. Use when the user wants to benchmark on IIT Bombay (Marathi), IJCNLP (Hindi), Wiki ANN (Hindi and Marathi), or asks about evaluating this task. Reports scores.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2306.06477
  bibtex_key: sabane2023enhancing
  confidence: medium
---

# low-resource-ner-transfer-eval

> Enhancing Low Resource NER Using Assisting Language And Transfer Learning — Sabane et al. (2023) (arXiv:2306.06477, 2023)

## What this evaluates

Evaluates cross-lingual transfer learning for Named Entity Recognition in low-resource Indian languages (Hindi and Marathi) by measuring how well models trained on combined or assisting-language datasets generalize to target language test sets compared to monolingual baselines.

## Datasets

- **IIT Bombay (Marathi)** — total ?; splits: train (-1), test (-1)
- **IJCNLP (Hindi)** — total ?; splits: train (-1), test (-1)
- **Wiki ANN (Hindi and Marathi)** — total ?; splits: train (-1), test (-1)

## Metrics

- `scores` **(primary)** — range: percent
  - Unspecified in the text; refers to standard NER performance metrics (typically F1-score or accuracy) used to compare monolingual vs. cross-lingual/mixed training setups.

## Input / output format

**Input**: Tokenized text sequences with corresponding entity labels (e.g., BIO/IOB tagging) for NER.

**Output**: Predicted entity labels for each token in the input sequence.

## Scoring recipe

```python
def compute_ner_scores(preds, golds):
    pred_spans = extract_spans(preds)
    gold_spans = extract_spans(golds)
    tp = len(pred_spans & gold_spans)
    fp = len(pred_spans - gold_spans)
    fn = len(gold_spans - pred_spans)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
    return f1
```

## Common pitfalls

- Blindly merging datasets from different domains (e.g., Wiki ANN vs. news/social media) without domain or tag-distribution alignment degrades performance.
- Assuming cross-lingual transfer always improves results; performance gains depend heavily on script similarity, lexical overlap, and data curation.

## Evidence (verbatim from paper)

> The observations portray that XLM Roberta, RoBERTa Hindi, MahaBERT [9], and Maha Roberta perform better on the mixed dataset than on the monolingual IIT Bombay dataset (Marathi). Similarly, when the mixed data model is tested on IJCNLP (Hindi), the scores observed are better for all the models except with mBERT where monolingual IJCNLP performs better.

## Citation

```bibtex
@misc{sabane2023enhancing,
  title={Enhancing Low Resource NER Using Assisting Language And Transfer Learning},
  author={Sabane et al. (2023)},
  year={2023},
  note={arXiv:2306.06477}
}
```

- arXiv: 2306.06477