Nce Recovery Evaluation

---
name: nce_recovery_evaluation
description: Evaluate conditional NCE recovery with ratio errors, KL metrics, and explicit mechanism checks for proxy runs.
---

# NCE Recovery Evaluation

Use this skill after ranking and binary NCE recovery outputs are available. It compares the declared module-plan target against the recovery result and emits the mechanism checks needed for a soft-mode proxy recovery.

Do not use this skill to hide target drift. It should fail or report non-acceptance if the recovery target metadata differs from the module plan.

## Inputs

- The `fast_recovery_target` from `module_plan.json`.
- A ranking NCE output JSON.
- A binary NCE diagnostic JSON.
- A training trace or optimizer-update record.
- Source-boundary status from the recovery harness.

## Outputs

- `ranking_ratio_absolute_error`.
- `binary_inconsistency_gap`.
- KL divergence for optional synthetic distributions.
- Mechanism checks:
  - ranking candidate posterior normalized
  - ranking ratio recovered
  - binary self-normalization failed as expected
  - binary limit matched the paper counterexample
  - reduced optimizer step changed parameters

## Workflow

1. Load ranking and binary outputs.
2. Compute absolute error between ranking `x1` ratio and the true `1/3` ratio.
3. Compute the binary gap against the true ratio and check closeness to `3/7`.
4. Verify loss decreased and parameters changed in the reduced optimizer trace.
5. Produce a validator-compatible recovery-result payload.

## Validation

Run the Distiller skill validator with tests. Tests cover ratio metrics, KL computation, and mechanism-check booleans.

## Limitations

This skill evaluates reduced finite-support recovery. It does not compute Penn Treebank perplexity.