Cross Task Split Protocol

---
name: cross_task_split_protocol
description: Create task-level seen and unseen splits with leakage diagnostics for Natural Instructions style generalization experiments.
---

# Cross Task Split Protocol

Use this skill when reconstructing the BART0/Natural Instructions cross-task generalization workflow without relying on the original paper repository. Do not use it for unrelated instruction-following papers unless the task uses a Natural-Instructions-style schema, task-level held-out evaluation, and generation metrics.

## Inputs
- Paper-derived module document and `module_plan.json`.
- Small task records or instruction items supplied by the current recovery attempt.
- Optional output paths for deterministic JSON artifacts.

## Outputs
- Deterministic JSON or Python return values that can be consumed by downstream recovery scripts.
- Validation evidence from the included tests.

## Workflow
1. Read the current attempt inputs and keep source use inside the allowed recovery boundary.
2. Apply the module contract exactly as described by the paper-derived module document.
3. Write machine-readable artifacts when a CLI output path is provided.
4. Cross-check downstream consumers with tests before accepting recovery evidence.

## Validation
Run `python <distiller>/module-to-skill/scripts/validate_skill_tree.py <skill_dir> --run-tests` from any working directory. The included tests are standard-library compatible and do not require the original source repository.

## Limitations
This skill preserves a reusable mechanism from the paper but does not by itself reproduce full BART-base training on all 61 Natural Instructions tasks. Full recovery requires a resolved dataset and model runtime; reduced recovery must remain explicitly marked as proxy evidence.