Chinese Tibetan Medicine Qa Eval

---
name: chinese-tibetan-medicine-qa-eval
description: This benchmark evaluates a traceable cross-source retrieval-augmented generation framework for Chinese Tibetan medicine QA. It probes the model's ability to route queries across heterogeneous knowledge bases, fuse cross-source evidence, and generate faithful answers with correct citations. Use when the user wants to benchmark on Chinese Tibetan-medicine QA dataset, or asks about evaluating this task. Reports CrossEv@5.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.05195
  bibtex_key: chen2026traceable
  confidence: high
---

# chinese-tibetan-medicine-qa-eval

> Traceable Cross-Source RAG for Chinese Tibetan Medicine Question Answering — Fengxian Chen et al. (2026) (arXiv:2602.05195, 2026)

## What this evaluates

This benchmark evaluates a traceable cross-source retrieval-augmented generation framework for Chinese Tibetan medicine QA. It probes the model's ability to route queries across heterogeneous knowledge bases, fuse cross-source evidence, and generate faithful answers with correct citations.

## Datasets

- **Chinese Tibetan-medicine QA dataset** — total 500; splits: full (500)

## Metrics

- `CrossEv@5` **(primary)** — range: [0, 1]
  - Checks whether the top-5 retrieved evidence chunks cover all required knowledge bases for cross-KB queries. Computed as 1 if coverage is complete, 0 otherwise.
- `Faithfulness` — range: [0, 1]
  - Reference-free RAGAS-style metric measuring whether generated claims are fully supported by the retrieved context.
- `Citation Correctness (CitCorr)` — range: [0, 1]
  - ALCE-inspired check verifying whether each cited evidence chunk actually supports the corresponding generated statement.
- `EvRecall@5` — range: [0, 1]
  - Fraction of gold evidence chunks retrieved within the top-5 ranked results.
- `EvNDCG@5` — range: [0, 1]
  - Normalized Discounted Cumulative Gain at rank 5 for evidence chunk ranking.
- `PrimaryAcc` — range: [0, 1]
  - Accuracy of predicting the primary source knowledge base for a query.
- `Top2Hit` — range: [0, 1]
  - Fraction of queries where the primary source KB appears in the top-2 ranked KBs.
- `EDR` — range: [0, 1]
  - Encyclopedia Dominance Rate; fraction of encyclopedia chunks in the final evidence list.
- `AuthCov` — range: [0, 1]
  - Measures whether evidence from authoritative KBs appears in the top-5 results for relevant queries.
- `Context Precision` — range: [0, 1]
  - RAGAS-style metric measuring the precision of retrieved context relevant to the query.
- `Context Recall` — range: [0, 1]
  - RAGAS-style metric measuring the recall of retrieved context relative to the gold answer.
- `Answer Relevance` — range: [0, 1]
  - RAGAS-style metric measuring how relevant the generated answer is to the query.

## Input / output format

**Input**: Query, gold answer, gold evidence at chunk level (KB labels, document metadata, stable chunk ids). For cross-KB synthesis queries, the required KB set and primary source KB are specified.

**Output**: Generated answer, cited evidence chunks, KB routing scores/rankings, and evidence ranking list.

## Scoring recipe

```python
# CrossEv@5 (cross-KB subset only)
coverage = len(set(e.kb for e in top_5_evidence) & required_kbs) == len(required_kbs)
cross_ev = coverage

# EvRecall@5 & EvNDCG@5
gold_ids = set(gold_chunk_ids)
pred_ids = set(pred_chunk_ids[:5])
ev_recall = len(gold_ids & pred_ids) / len(gold_ids)
ev_ndcg = dcg_at_k(pred_ids, k=5) / idcg_at_k(gold_ids, k=5)

# Faithfulness & CitCorr (LLM-judged)
faithfulness = judge_model.check_claims_supported(answer, context)
cit_corr = judge_model.check_citations_support_statements(answer, citations)
```

## Common pitfalls

- No train/dev/test splits exist; all 500 queries are evaluated together, so metrics reflect a single fixed configuration rather than split-averaged performance.
- Evidence-level metrics (EvRecall@5, EvNDCG@5, CrossEv@5) are computed exclusively on the cross-KB subset, not the full dataset.
- All automatic metrics are computed by a single judge model (GLM-4.7) to avoid evaluator heterogeneity, which may introduce systematic judge bias.
- A fixed cutoff of K=5 is used for all @K metrics, regardless of query type or retrieval depth requirements.

## Evidence (verbatim from paper)

> We evaluate at three levels with a consistent cutoff K=5. We report EvRecall@5 and EvNDCG@5 based on gold chunk ids. For cross-KB queries, we report CrossEv@5, which checks whether the top-5 evidence covers all required KBs. Unless otherwise stated, evidence-level metrics are computed over the cross-KB subset. We report reference-free RAG metrics (faithfulness, context precision/recall, and answer relevance) in the spirit of RAGAS. We also report citation-based checks inspired by ALCE to assess whether cited evidence supports the generated statements.

## Citation

```bibtex
@misc{chen2026traceable,
  title={Traceable Cross-Source RAG for Chinese Tibetan Medicine Question Answering},
  author={Fengxian Chen et al. (2026)},
  year={2026},
  note={arXiv:2602.05195}
}
```

- arXiv: 2602.05195