Klue Dp Eval

---
name: klue-dp-eval
description: Evaluates a model's ability to predict syntactic dependency relations between words in Korean sentences, testing grammatical structure understanding. Use when the user wants to benchmark on KLUE-DP, or asks about evaluating this task. Reports LAS.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2105.09680
  bibtex_key: park2021klue
  confidence: medium
---

# klue-dp-eval

> KLUE: Korean Language Understanding Evaluation — Sungjoon Park et al. (arXiv:2105.09680, 2021)

## What this evaluates

Evaluates a model's ability to predict syntactic dependency relations between words in Korean sentences, testing grammatical structure understanding.

## Datasets

- **KLUE-DP** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/KLUE-benchmark/KLUE

## Metrics

- `LAS` **(primary)** — range: [0, 1]
  - Labeled Attachment Score: percentage of words where both the head word and dependency relation label are predicted correctly.

## Input / output format

**Input**: Korean sentence with tokenized input.

**Output**: List of (head_index, relation_label) pairs per token.

## Scoring recipe

```python
def compute_las(pred_deps, gold_deps):
    correct = sum(1 for p, g in zip(pred_deps, gold_deps) if p == g)
    return correct / len(gold_deps)
```

## Common pitfalls

- Korean agglutinative morphology complicates token-to-word alignment.
- Head-finding rules differ between UD guidelines and paper-specific annotations.

## Evidence (verbatim from paper)

> KLUE introduces a comprehensive, ethically designed benchmark for Korean NLU with 8 tasks (Topic Classification, STS, NLI, NER, RE, DP, MRC, DST) built from scratch using diverse, copyright-respected corpora.

## Citation

```bibtex
@misc{park2021klue,
  title={KLUE: Korean Language Understanding Evaluation},
  author={Sungjoon Park et al.},
  year={2021},
  note={arXiv:2105.09680}
}
```

- arXiv: 2105.09680