Cad Recovery Evaluation

---
name: cad_recovery_evaluation
description: Evaluate CAD conflict-recovery runs with exact-match metrics and mechanism-check summaries.
---

# CAD Recovery Evaluation

Use this skill when assessing whether a CAD experiment followed supplied context rather than a memorized prior in QA-style conflict items.

## Inputs
- Items containing `id`, `context_answer`, and `prior_answer`.
- Regular decoding predictions.
- CAD predictions.
- Per-item traces showing prompt separation, dual logits, logit adjustment, and token selection.

## Outputs
Exact-match metrics, improvement, per-item records, and mechanism-check booleans suitable for `recovery_result.json`.

## Workflow
1. Normalize answers by lowercasing and stripping whitespace/punctuation.
2. Score regular and CAD predictions against the context answer.
3. Verify every item has prompt separation, dual logits, adjusted logits, and selection trace.
4. Return metric and mechanism summaries.

## Validation
Run `tests/test_recovery_evaluation.py` or Distiller validation.

## Limitations
This evaluator is for reduced conflict QA recovery, not ROUGE or factuality metrics on summarization.