Fgner Eval

---
name: fgner-eval
description: Evaluates fine-grained named entity recognition (FgNER) capabilities across multiple languages by measuring how accurately models identify and classify specific entity types within text sequences. The protocol assesses sequence labeling performance using standard span-based metrics. Use when the user wants to benchmark on Various NER datasets (cited in text), or asks about evaluating this task. Reports F1-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2601.10161
  bibtex_key: kaushik2026awedfiner
  confidence: medium
---

# fgner-eval

> AWED-FiNER: Agents, Web applications, and Expert Detectors for Fine-grained Named Entity Recognition across 36 Languages for 6.6 Billion Speakers — Kaushik et al. (2026) (arXiv:2601.10161, 2026)

## What this evaluates

Evaluates fine-grained named entity recognition (FgNER) capabilities across multiple languages by measuring how accurately models identify and classify specific entity types within text sequences. The protocol assesses sequence labeling performance using standard span-based metrics.

## Datasets

- **Various NER datasets (cited in text)** — total ?; splits: (unstated)

## Metrics

- `F1-score` **(primary)** — range: [0, 1]
  - Standard sequence labeling F1-score computed over exact match of predicted and gold entity spans and types, following the evaluation protocol in Golde et al. (2025).

## Input / output format

**Input**: Tokenized text sequences with corresponding fine-grained entity labels for sequence labeling.

**Output**: Predicted fine-grained entity spans and type labels per token/sequence.

## Scoring recipe

```python
def compute_f1(predictions, gold):
    pred_spans = extract_spans(predictions)
    gold_spans = extract_spans(gold)
    tp = len(pred_spans & gold_spans)
    fp = len(pred_spans - gold_spans)
    fn = len(gold_spans - pred_spans)
    precision = tp / (tp + fp) if (tp + fp) > 0 else 0
    recall = tp / (tp + fn) if (tp + fn) > 0 else 0
    return 2 * precision * recall / (precision + recall) if (precision + recall) > 0 else 0
```

## Common pitfalls

- Exact metric computation details are deferred to an external citation (Golde et al., 2025) rather than being defined inline.
- Specific NER datasets used for training/evaluation are only referenced via citations and not explicitly named or sized in the text.

## Evidence (verbatim from paper)

> Training was performed on an NVIDIA A100 GPU, with evaluation based on SeqEval metrics, and the best performance determined by the F1-score following Golde et al. ([2025]).

## Citation

```bibtex
@misc{kaushik2026awedfiner,
  title={AWED-FiNER: Agents, Web applications, and Expert Detectors for Fine-grained Named Entity Recognition across 36 Languages for 6.6 Billion Speakers},
  author={Kaushik et al. (2026)},
  year={2026},
  note={arXiv:2601.10161}
}
```

- arXiv: 2601.10161