Hybridrag Bench Eval

---
name: hybridrag-bench-eval
description: Evaluates retrieval-augmented models' ability to perform multi-hop reasoning over hybrid knowledge (unstructured text and knowledge graphs) using time-framed, external scientific literature to prevent parametric memorization. Use when the user wants to benchmark on Arxiv-AI, Arxiv-CY, Arxiv-BIO, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.10210
  bibtex_key: [REDACTED]
  confidence: medium
---

# hybridrag-bench-eval

> How Much Reasoning Do Retrieval-Augmented Models Add beyond LLMs? A Benchmarking Framework for Multi-Hop Inference over Hybrid Knowledge — Lin et al. (2026) (arXiv:2602.10210, 2026)

## What this evaluates

Evaluates retrieval-augmented models' ability to perform multi-hop reasoning over hybrid knowledge (unstructured text and knowledge graphs) using time-framed, external scientific literature to prevent parametric memorization.

## Datasets

- **Arxiv-AI** — total 863; splits: test (-1); repo https://github.com/junhongmit/HybridRAG-Bench
- **Arxiv-CY** — total 966; splits: test (-1); repo https://github.com/junhongmit/HybridRAG-Bench
- **Arxiv-BIO** — total 1040; splits: test (-1); repo https://github.com/junhongmit/HybridRAG-Bench

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Standard exact-match or token-level F1 depending on question type (entity/value vs. open-ended). Not explicitly defined in the provided section.

## Input / output format

**Input**: A question requiring multi-hop reasoning, provided with retrieved unstructured text chunks and/or knowledge graph paths as context.

**Output**: A predicted answer (entity, value, or short natural language response) and optionally supporting evidence.

## Scoring recipe

```python
def score(predictions, gold, question_type):
    if question_type in ['single-hop', 'multi-hop', 'counterfactual']:
        return 1.0 if predictions.strip() == gold.strip() else 0.0
    else:
        return rouge_f1(predictions, gold)
```

## Common pitfalls

- Questions are explicitly filtered to be answerable only from the provided hybrid context, penalizing models that rely on parametric memorization.
- Counterfactual questions require minimal perturbation of original paths; models may fail to recognize the perturbation or hallucinate unsupported conclusions.
- Open-ended questions require synthesizing explanations from multiple evidence pieces, making them sensitive to retrieval precision and context window limits.

## Evidence (verbatim from paper)

> Together, these components enable reproducible construction of retrieval-intensive benchmarks for evaluating RAG and KG-RAG methods under controlled knowledge settings.

## Citation

```bibtex
@misc{lin2026hybridragbench,
  title={How Much Reasoning Do Retrieval-Augmented Models Add beyond LLMs? A Benchmarking Framework for Multi-Hop Inference over Hybrid Knowledge},
  author={Lin et al. (2026)},
  year={2026},
  note={arXiv:2602.10210}
}
```

- arXiv: 2602.10210