Klue Re Eval

---
name: klue-re-eval
description: Tests a model's ability to extract relational triples between entities in Korean text, probing structured information extraction capabilities. Use when the user wants to benchmark on KLUE-RE, or asks about evaluating this task. Reports F1.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2105.09680
  bibtex_key: park2021klue
  confidence: medium
---

# klue-re-eval

> KLUE: Korean Language Understanding Evaluation — Sungjoon Park et al. (arXiv:2105.09680, 2021)

## What this evaluates

Tests a model's ability to extract relational triples between entities in Korean text, probing structured information extraction capabilities.

## Datasets

- **KLUE-RE** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/KLUE-benchmark/KLUE

## Metrics

- `F1` **(primary)** — range: [0, 1]
  - Harmonic mean of precision and recall for exact triple (head, relation, tail) matching.

## Input / output format

**Input**: Korean sentence with annotated entity spans.

**Output**: List of predicted relation triples.

## Scoring recipe

```python
def compute_f1(pred_triples, gold_triples):
    tp = len(set(pred_triples) & set(gold_triples))
    fp = len(set(pred_triples) - set(gold_triples))
    fn = len(set(gold_triples) - set(pred_triples))
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    return 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
```

## Common pitfalls

- Overlapping entities require careful disambiguation.
- Relation label granularity varies across Korean corpora.

## Evidence (verbatim from paper)

> KLUE introduces a comprehensive, ethically designed benchmark for Korean NLU with 8 tasks (Topic Classification, STS, NLI, NER, RE, DP, MRC, DST) built from scratch using diverse, copyright-respected corpora.

## Citation

```bibtex
@misc{park2021klue,
  title={KLUE: Korean Language Understanding Evaluation},
  author={Sungjoon Park et al.},
  year={2021},
  note={arXiv:2105.09680}
}
```

- arXiv: 2105.09680