Evidencenet Extraction Eval

---
name: evidencenet-extraction-eval
description: This evaluation probes the fidelity of an LLM-assisted pipeline in extracting structured, PICO-style evidence nodes from unstructured full-text biomedical literature, and the precision of subsequent entity normalization against a reference resource. Use when the user wants to benchmark on HCC and CRC PubMed corpus, or asks about evaluating this task. Reports field-level extraction accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.28325
  bibtex_key: zong2026evidencenet
  confidence: low
---

# evidencenet-extraction-eval

> Building evidence-based knowledge graphs from full-text literature for disease-specific biomedical reasoning — Chang Zong et al. (2026) (arXiv:2603.28325, 2026)

## What this evaluates

This evaluation probes the fidelity of an LLM-assisted pipeline in extracting structured, PICO-style evidence nodes from unstructured full-text biomedical literature, and the precision of subsequent entity normalization against a reference resource.

## Datasets

- **HCC and CRC PubMed corpus** — total ?; splits: (unstated); repo https://github.com/ZUST-BIT/EvidenceNet-code

## Metrics

- `field-level extraction accuracy` **(primary)** — range: [0, 1]
  - Proportion of extracted evidence fields that correctly match ground-truth annotations.
- `entity-link accuracy` — range: [0, 1]
  - Proportion of normalized biomedical entities that correctly align with the TarKG reference resource.

## Input / output format

**Input**: Full-text biomedical articles segmented into 3,000-character chunks with 300-character overlap, preserving section labels.

**Output**: Structured evidence objects containing PICO fields (study object, intervention, comparison, outcomes, mechanism, phenotype, design, stage) and quantitative attributes (p-values, sample size, fold change).

## Scoring recipe

```python
Not explicitly defined in the provided text. The paper reports 98.3% field-level extraction accuracy and 100% high-confidence entity-link accuracy, implying exact or fuzzy string matching against curated gold standards. A representative implementation would be:

def compute_accuracy(preds, gold):
    correct = sum(1 for p, g in zip(preds, gold) if p == g)
    return correct / len(gold)
```

## Common pitfalls

- Chunks shorter than 500 characters are discarded during preprocessing, which may bias evaluation results toward longer, more methodologically detailed studies.
- Document-level aggregation removes exact duplicates across sections, potentially undercounting repeated findings and affecting the granularity of extraction accuracy calculations.

## Evidence (verbatim from paper)

> Validation shows 98.3% field-level extraction accuracy, 100% high-confidence entity-link accuracy, and significant improvements in retrieval-augmented QA and link prediction, enabling more reliable, context-aware biomedical reasoning for precision medicine applications.

## Citation

```bibtex
@misc{zong2026evidencenet,
  title={Building evidence-based knowledge graphs from full-text literature for disease-specific biomedical reasoning},
  author={Chang Zong et al. (2026)},
  year={2026},
  note={arXiv:2603.28325}
}
```

- arXiv: 2603.28325