Clinical Field Recovery Eval

---
name: clinical-field-recovery-eval
description: Evaluates sequential question-selection strategies for recovering target clinical fields from synthetic patient responses under a fixed interaction budget. It probes how well adaptive versus fixed questioning policies handle varying patient communication styles to maximize information coverage within conversational constraints. Use when the user wants to benchmark on Clinical Psychiatric Intake Vignette Benchmark, or asks about evaluating this task. Reports accuracy.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2604.22067
  bibtex_key: panda2026optimalquestionselection
  confidence: high
---

# clinical-field-recovery-eval

> Optimal Question Selection from a Large Question Bank for Clinical Field Recovery in Conversational Psychiatric Intake — Panda et al. (2026) (arXiv:2604.22067, 2026)

## What this evaluates

Evaluates sequential question-selection strategies for recovering target clinical fields from synthetic patient responses under a fixed interaction budget. It probes how well adaptive versus fixed questioning policies handle varying patient communication styles to maximize information coverage within conversational constraints.

## Datasets

- **Clinical Psychiatric Intake Vignette Benchmark** — total ?; splits: test (300)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of target clinical fields correctly recovered by the end of the interview session (max 20 turns or when all fields are recovered).
- `precision` — range: percent
  - Percentage of recovered fields that are correctly labeled by the judge, measuring post-disclosure interpretation accuracy.
- `conversational_cost` — range: other
  - Cumulative word count across the transcript, used to assess efficiency relative to accuracy.

## Input / output format

**Input**: Current conversation transcript, in-loop recovery state, patient behavioral condition profile, and a question bank.

**Output**: A single question selected from the bank to ask the patient next.

## Scoring recipe

```python
def compute_metrics(recovered_fields, gold_fields, transcript):
    total_target = len(gold_fields)
    correct = sum(1 for f, v in recovered_fields.items() if f in gold_fields and v == gold_fields[f])
    accuracy = (correct / total_target) * 100 if total_target > 0 else 0
    precision = (correct / len(recovered_fields)) * 100 if len(recovered_fields) > 0 else 0
    word_count = len(transcript.split())
    return {'accuracy': accuracy, 'precision': precision, 'conversational_cost': word_count}
```

## Common pitfalls

- Assuming higher transcript length or word count directly correlates with better performance; the paper shows the best strategy recovers more information per unit of conversational effort, not just longer transcripts.
- Overestimating the role of post-disclosure interpretation; precision is uniformly high (~98-99%), indicating the bottleneck is topic coverage and question selection, not field labeling.
- Ignoring behavioral condition heterogeneity; performance drops sharply for non-adaptive strategies under 'Guarded + Concise' conditions, masking overall strategy efficacy if averaged without stratification.

## Evidence (verbatim from paper)

> Random reaches 51.7%±19.6% mean accuracy, while form-based reaches 84.8%±15.8% and LLM-guided reaches 95.4%±8.1%. Precision remains uniformly high across all three strategies, ranging from 98.4% for random to 99.4% for LLM-guided.

## Citation

```bibtex
@misc{panda2026optimalquestionselection,
  title={Optimal Question Selection from a Large Question Bank for Clinical Field Recovery in Conversational Psychiatric Intake},
  author={Panda et al. (2026)},
  year={2026},
  note={arXiv:2604.22067}
}
```

- arXiv: 2604.22067