Ccks2017 Cner Eval

---
name: ccks2017-cner-eval
description: Evaluates a model's ability to identify five types of clinical named entities (diseases, symptoms, exams, treatments, body parts) in Chinese medical texts. It specifically probes character-level sequence labeling performance and the impact of integrating external dictionary features. Use when the user wants to benchmark on CCKS-2017 Task 2, or asks about evaluating this task. Reports F1-score.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 1808.08669
  bibtex_key: qiu2018fast
  confidence: high
---

# ccks2017-cner-eval

> Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions — Jiahui Qiu et al. (arXiv:1808.08669, 2018)

## What this evaluates

Evaluates a model's ability to identify five types of clinical named entities (diseases, symptoms, exams, treatments, body parts) in Chinese medical texts. It specifically probes character-level sequence labeling performance and the impact of integrating external dictionary features.

## Datasets

- **CCKS-2017 Task 2** — total 10024; splits: train (7906), test (2118); repo http://www.ccks2017.com/en/index.php/sharedtask/

## Metrics

- `F1-score` **(primary)** — range: percent
  - Harmonic mean of precision and recall: F1 = 2 * (Precision * Recall) / (Precision + Recall). Precision = TP / (TP + FP), Recall = TP / (TP + FN).

## Input / output format

**Input**: Chinese clinical sentences or clauses (split by commas), processed at the character level, optionally augmented with dictionary feature embeddings.

**Output**: Sequence of BIO-style entity tags corresponding to each character in the input sentence.

## Scoring recipe

```python
def compute_ner_f1(pred_tags, gold_tags):
    def get_spans(tags):
        spans = []
        start = None
        for i, tag in enumerate(tags):
            if tag.startswith('B-'):
                start = (i, tag[2:])
            elif start and (tag == 'O' or tag.startswith('I-') and not tag.startswith('I-' + start[1])):
                spans.append((start[0], i, start[1]))
                start = None
        return set(spans)
    pred_spans = get_spans(pred_tags)
    gold_spans = get_spans(gold_tags)
    tp = len(pred_spans & gold_spans)
    fp = len(pred_spans - gold_spans)
    fn = len(gold_spans - pred_spans)
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0.0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    return 2 * prec * rec / (prec + rec) * 100
```

## Common pitfalls

- Sentences are split by commas, which may create non-standard boundaries for entity recognition compared to standard NER datasets.
- Character-level labeling avoids word segmentation errors but requires careful handling of rare characters and dictionary feature integration.
- Training speed comparisons are made against Bi-LSTM-CRF, but convergence time varies significantly by hardware and implementation details.

## Evidence (verbatim from paper)

> This dataset contains 1,596 annotated instances (10,024 sentences) with five types of clinical named entities, including diseases, symptoms, exams, treatments and body parts. ... In the following experiments, widely-used performance measures such as precision, recall, and F1-score are used to evaluate the methods.

## Citation

```bibtex
@misc{qiu2018fast,
  title={Fast and Accurate Recognition of Chinese Clinical Named Entities with Residual Dilated Convolutions},
  author={Jiahui Qiu et al.},
  year={2018},
  note={arXiv:1808.08669}
}
```

- arXiv: 1808.08669