Condmedqa Eval

---
name: condmedqa-eval
description: Evaluates a model's ability to perform conditional multi-hop reasoning in biomedical question answering, specifically how well it modulates clinical answers based on patient-specific constraints like comorbidities, contraindications, and special population factors. Use when the user wants to benchmark on CondMedQA, or asks about evaluating this task. Reports performance.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2602.17911
  bibtex_key: parekh2026condmedqa
  confidence: high
---

# condmedqa-eval

> Condition-Gated Reasoning for Context-Dependent Biomedical Question Answering — Parekh et al. (2026) (arXiv:2602.17911, 2026)

## What this evaluates

Evaluates a model's ability to perform conditional multi-hop reasoning in biomedical question answering, specifically how well it modulates clinical answers based on patient-specific constraints like comorbidities, contraindications, and special population factors.

## Datasets

- **CondMedQA** — total 100; splits: test (100)

## Metrics

- `performance` **(primary)** — range: [0, 1]
  - Standard exact-match accuracy for biomedical QA, calculated as the proportion of questions where the model's predicted answer exactly matches the gold answer.

## Input / output format

**Input**: A clinical question containing a specific patient condition/modifier (e.g., pregnancy, comorbidity, drug interaction), often requiring synthesis of information from two provided knowledge sources.

**Output**: The correct clinical answer (e.g., drug name, dosage, diagnostic modality) that applies specifically given the stated patient condition.

## Scoring recipe

```python
correct = 0
for pred, gold in zip(predictions, gold_answers):
    if normalize_text(pred) == normalize_text(gold):
        correct += 1
return correct / len(predictions)
```

## Common pitfalls

- Models may memorize default/general answers and fail to adjust when patient-specific modifiers are present.
- Failing to synthesize information across two separate knowledge sources required to form the conditional reasoning trace.
- Overlooking subtle causal dependencies where removing the modifier reverts the answer to the general case.

## Evidence (verbatim from paper)

> CGR outperforms state-of-the-art methods on condition-sensitive queries while matching or exceeding performance on factual benchmarks, demonstrating that explicit modeling of conditionality is critical for robust clinical reasoning.

## Citation

```bibtex
@misc{parekh2026condmedqa,
  title={Condition-Gated Reasoning for Context-Dependent Biomedical Question Answering},
  author={Parekh et al. (2026)},
  year={2026},
  note={arXiv:2602.17911}
}
```

- arXiv: 2602.17911