Meissa Medical Eval

---
name: meissa-medical-eval
description: Evaluates a 4B multi-modal medical agentic model's ability to perform clinical reasoning, tool use, and multi-step interaction across radiology, pathology, and clinical domains. Probes strategy selection (when to use tools vs direct reasoning) and execution policy under various agent frameworks. Use when the user wants to benchmark on MIMIC-CXR-VQA, ChestAgentBench, PathVQA, SLAKE, VQA-RAD, OmniMed, MedXpertQA, MedQA, PubMedQA, NEJM, NEJM Ext., MIMIC-IV, MedQA Ext., or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2603.09018
  bibtex_key: chen2026meissa
  confidence: high
---

# meissa-medical-eval

> Meissa: Multi-modal Medical Agentic Intelligence — Chen et al. (2026) (arXiv:2603.09018, 2026)

## What this evaluates

Evaluates a 4B multi-modal medical agentic model's ability to perform clinical reasoning, tool use, and multi-step interaction across radiology, pathology, and clinical domains. Probes strategy selection (when to use tools vs direct reasoning) and execution policy under various agent frameworks.

## Datasets

- **MIMIC-CXR-VQA** — total ?; splits: test (-1)
- **ChestAgentBench** — total ?; splits: test (-1)
- **PathVQA** — total ?; splits: test (-1)
- **SLAKE** — total ?; splits: test (-1)
- **VQA-RAD** — total ?; splits: test (-1)
- **OmniMed** — total ?; splits: test (-1)
- **MedXpertQA** — total ?; splits: test (-1)
- **MedQA** — total ?; splits: test (-1)
- **PubMedQA** — total ?; splits: test (-1)
- **NEJM** — total ?; splits: test (-1)
- **NEJM Ext.** — total ?; splits: test (-1)
- **MIMIC-IV** — total ?; splits: test (-1)
- **MedQA Ext.** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: [0, 1]
  - Exact-match accuracy for close-ended questions; soft-match accuracy with medical synonym normalization for open-ended VQA.

## Input / output format

**Input**: Multi-modal medical queries (text, images like CXR/pathology/EHR, or text-only) with varying complexity, often requiring tool use or multi-turn interaction.

**Output**: Final answer or diagnosis, formatted as a direct response or multi-step agentic trajectory ending in a conclusion.

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = 0
    for pred, gold in zip(predictions, gold):
        if is_close_ended(gold):
            if normalize(pred) == normalize(gold):
                correct += 1
        else:
            if medical_synonym_match(pred, gold):
                correct += 1
    return correct / len(gold)
```

## Common pitfalls

- Decontamination only checks question text n-grams (n=8), not image-level leakage, though disjoint DICOM/hashing is used.
- Soft-match accuracy relies on medical synonym normalization, which may vary across benchmarks.
- Results are averaged over 3 seeds; variance is only in supplementary.

## Evidence (verbatim from paper)

> All tasks are formulated as QA or VQA; the evaluation metric is accuracy throughout: exact-match accuracy for close-ended questions and soft-match accuracy with medical synonym normalization*[aasmimic]* for open-ended VQA.

## Citation

```bibtex
@misc{chen2026meissa,
  title={Meissa: Multi-modal Medical Agentic Intelligence},
  author={Chen et al. (2026)},
  year={2026},
  note={arXiv:2603.09018}
}
```

- arXiv: 2603.09018