Kaggle

---
name: kaggle
description: "Generate a Kaggle competition notebook as a Jupytext `# %%` Python script following the user's established ML research style: PTL for DNN training, best-fit tool selection, EDA→Baseline→Train→Inference pipeline with per-stage lens cells, small single-purpose cells each carrying a why. Grounds data schema and submission format through the authenticated `kaggle` CLI (file listing, sample submission, leaderboard) rather than the login-walled competition page. Tuned to win (leakage-safe CV, metric-aligned modeling) as much as to teach. Writes output to .experiments/kaggle/<name>.py. Requires foundry plugin (foundry:sw-engineer, no fallback)."
argument-hint: <competition-name> [<url-or-description>] [--type classification|regression|segmentation|detection|tabular] [--eda-only] [--inference-only] [--offline-setup] [--resume <existing.py>] [--keep "<items>"]
allowed-tools: Read, Write, Edit, Bash, Grep, Glob, Agent, WebFetch, WebSearch, AskUserQuestion, TaskCreate, TaskUpdate, TaskList
disable-model-invocation: true
effort: high
---

<objective>

Generate Kaggle competition notebook script, Jupytext `# %%` format.

Two goals, equal weight — neither traded for other:

- **Win** — leaderboard-competitive: leakage-safe CV, metric-aligned loss/model choice, tuning/ensembling when it moves the score, not style theater
- **Teach** — read top to bottom like a university/seminar lecture on solving this competition: reader new to it follows the full reasoning chain, every decision motivated, nothing left as unexplained code

Follows user's ML research style distilled from past notebooks:

- **PTL always for DNN training** (PyTorch Lightning + torchmetrics) — even simple baselines
- **Tool agnostic** — best-fit library for problem; PTL when training loop needed
- **Stages with lenses** — each major stage: quick sanity check cell (show one batch, print shapes, verify submission format)
- **Small, single-purpose cells** — one action per cell (load, one transform, one plot, one check); never bundle setup + run + verify to save cell count
- **Every cell earns its place** — one-line why (comment or markdown sentence) before/in each cell: the specific reason this step happens now — never a restatement of what the code does
- **Section markdown is extensive and structured** — full explanation of what/why/how-it-advances-the-goal per section, formatted as tables/lists/blockquotes over dense prose paragraphs; markdown before a plot sets up the question, markdown after states the finding and its implication — plot and prose flow as one beat, never an orphaned chart
- **`# !` bash over subprocess** — package installs, `nvidia-smi`, `ls -lh`, `# ! head submission.csv`
- **EDA is visual** — distribution plots, sample grids, dimension scatters before any model
- **Inference included** — model save pattern + separate load-and-infer cells
- **CSVLogger + seaborn** — metrics plotted from `metrics.csv` after every training run

NOT for writing Python packages, modules, production code — notebook scripts only. NOT research literature survey — use `/research:topic` for SOTA literature search.

</objective>

<inputs>

- **$ARGUMENTS**: one of:
  - `<competition-name>` — short slug for output filename; generates blank template
  - `<competition-name> <url>` — fetches competition overview from URL before generating
  - `<competition-name> "<description>"` — inline description of problem and data
  - `--type <type>` — hint: `classification`, `regression`, `segmentation`, `detection`, `tabular` (auto-detected when omitted)
  - `--eda-only` — generate only EDA sections (no model/training/submission); always online (no offline setup)
  - `--inference-only` — generate inference notebook from checkpoint (no EDA, no training); always offline (frozen packages pattern); loads checkpoint from `PATH_CHECKPOINT` constant; output suffix `-inference.py`
  - `--offline-setup` — include offline package setup (frozen_packages pattern) in setup cell; auto-applied when `--inference-only`; ignored when `--eda-only` (EDA always online)
  - `--resume <path>` — read existing `.py` script, extend/improve it

Output: `.experiments/kaggle/<competition-name>.py`

</inputs>

<constants>

```yaml
OUTPUT_DIR:       .experiments/kaggle/
DATA_DIR:         .experiments/kaggle/data/<competition>/  # kaggle CLI downloads land here, gitignored
CELL_MARK:        "# %%"
MD_CELL_MARK:     "# %% [markdown]"
COMPETITORS_DIR:  resources/competitors/  # optional user-project path, not shipped in plugin — Step 1 reads if present
# NOTE: doc-only — not shell vars across Bash() calls (state doesn't persist); keep synced with literal use sites (Steps 1,3,4)
```

</constants>

<compaction>

- Key boundary: end of Step 3 — notebook script generated by `foundry:sw-engineer`, written to OUTFILE.
- Preserve: OUTFILE path (derived from TMPDIR keys), COMPETITION_NAME (TMPDIR key), mode flags (EDA_ONLY, INFERENCE_ONLY, OFFLINE_SETUP).
- Clear at Step 1 start (stale prior run) and after Step 4 package-distillation gate resolves.

</compaction>

<workflow>

**Task hygiene**: call `TaskList` first; close orphaned tasks. Create tasks per phase.

## Step 1: Parse arguments and gather context

```bash
# loads: compaction-contract.md
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
ARGS="$ARGUMENTS"
COMPETITION_NAME=$(echo "$ARGS" | awk '{print $1}')
RESUME_FLAG=""
EDA_ONLY=false
INFERENCE_ONLY=false
OFFLINE_SETUP=false
PROBLEM_TYPE=""

[[ "$ARGS" == *"--eda-only"* ]]      && EDA_ONLY=true
[[ "$ARGS" == *"--inference-only"* ]] && INFERENCE_ONLY=true
[[ "$ARGS" == *"--offline-setup"* ]]  && OFFLINE_SETUP=true
[[ "$ARGS" =~ --type[[:space:]]([a-z]+) ]] && PROBLEM_TYPE="${BASH_REMATCH[1]}"
[[ "$ARGS" =~ --resume[[:space:]]([^[:space:]]+) ]] && RESUME_FLAG="${BASH_REMATCH[1]}"

# inference always offline; EDA always online (overrides --offline-setup)
[ "$INFERENCE_ONLY" = "true" ] && OFFLINE_SETUP=true
[ "$EDA_ONLY" = "true" ]       && OFFLINE_SETUP=false

echo "Competition: $COMPETITION_NAME"
echo "Type: ${PROBLEM_TYPE:-auto-detect}"
echo "EDA only: $EDA_ONLY | Inference only: $INFERENCE_ONLY | Offline setup: $OFFLINE_SETUP"

# Persist for Steps 3+4 (bash state lost across Bash() calls)
echo "$COMPETITION_NAME" > "${TMPDIR:-/tmp}/kaggle-competition-name-${CSID}"
echo "$EDA_ONLY"         > "${TMPDIR:-/tmp}/kaggle-eda-only-${CSID}"
echo "$INFERENCE_ONLY"   > "${TMPDIR:-/tmp}/kaggle-inference-only-${CSID}"
echo "$OFFLINE_SETUP"    > "${TMPDIR:-/tmp}/kaggle-offline-setup-${CSID}"

mkdir -p .experiments/kaggle/  # timeout: 3000
KEEP_ITEMS=""
if [[ "$ARGS" =~ --keep[[:space:]]\"([^\"]+)\" ]]; then
    KEEP_ITEMS="${BASH_REMATCH[1]}"
fi
# Clear stale contract from any prior incomplete run (compaction-contract.md §Lifecycle)
rm -f .temp/state/skill-contract.md  # timeout: 5000
echo "${KEEP_ITEMS:-}" > "${TMPDIR:-/tmp}/kaggle-keep-items-${CSID}"  # persist for Step 3 contract write
```

**Flag mutual-exclusion check** — if `EDA_ONLY` and `INFERENCE_ONLY` are both `true` (both `--eda-only` and `--inference-only` passed): print `` ! Conflicting flags: `--eda-only` and `--inference-only` are mutually exclusive (`--eda-only` is always-online with no training; `--inference-only` is always-offline/frozen-package with no EDA — see `foundation.md`). Pick one. `` then invoke `AskUserQuestion` — (a) **Abort** · (b) **Continue ignoring both** (falls back to full mode: neither eda-only nor inference-only applied). On Abort: stop.

**Unsupported flag check** — scan `$ARGUMENTS` for remaining `--<token>` tokens after supported flags extracted (`--eda-only`, `--inference-only`, `--offline-setup`, `--type`, `--resume`, `--keep`). Found: print `` ! Unknown flag(s): `--<token>`. Supported: `--eda-only`, `--inference-only`, `--offline-setup`, `--type <type>`, `--resume <path>`, `--keep "<items>"`. `` then invoke `AskUserQuestion` — (a) **Abort** · (b) **Continue ignoring**. On Abort: stop.

**Context collection** — run in parallel:

1. URL provided in args: `WebFetch` competition page; extract problem description, target metric, data format, evaluation — read and quote actual text, never paraphrase from training knowledge
2. `--resume`: read existing script (`Read` tool)
3. Scan `.experiments/kaggle/` (`Glob` pattern `*.py`) for prior scripts; read first 30 lines of each — find similar past competitions, use as structural reference
4. Check `resources/competitors/` for `.ipynb`/`.py` files — found: read each, summarise approach (model choice, preprocessing, feature engineering, augmentation). Use findings to inform detection method and domain-specific preprocessing decisions in Step 2.
5. **Kaggle CLI probe** (below) — authoritative source for file listing, data schema, submission format. CLI complements WebFetch, never replaces it: CLI gives files/schema/leaderboard, page gives problem narrative and metric prose.

### Kaggle CLI grounding

Competition pages are login-walled; `WebFetch` returns partial or blocked content on many of them. Anyone requesting a competition notebook has a Kaggle account, so the CLI is the reliable path — real file names, sizes, actual `sample_submission.csv` header, no guessed schema.

Probe availability and auth in one block. CLI absence never aborts the skill — degrade to WebFetch/user facts:

```bash
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
IFS= read -r COMPETITION_NAME < "${TMPDIR:-/tmp}/kaggle-competition-name-${CSID}" 2>/dev/null || COMPETITION_NAME=""
KAGGLE_CLI="absent"
if command -v kaggle >/dev/null 2>&1; then
    # `competitions list` needs credentials but no rules acceptance — separates auth failure from rules failure
    if kaggle competitions list -p 1 >/dev/null 2>&1; then KAGGLE_CLI="ready"; else KAGGLE_CLI="unauthorized"; fi
fi
echo "$KAGGLE_CLI" > "${TMPDIR:-/tmp}/kaggle-cli-state-${CSID}"
echo "kaggle CLI: $KAGGLE_CLI · slug: ${COMPETITION_NAME:-<unset>}"  # timeout: 30000
```

Branch on `$KAGGLE_CLI`:

| State | Action |
| -- | -- |
| `ready` | Run the grounding queries below |
| `absent` | Offer install — `AskUserQuestion`: (a) skip, ground from URL/user facts · (b) `pip install kaggle` then re-probe. Never install without asking |
| `unauthorized` | Print the credential instructions below, `AskUserQuestion`: (a) skip · (b) user sets up token, then re-probe |

**Credential secrecy — hard constraint.** The token never enters this session's context, and never a subagent's or Codex's. Forbidden regardless of who asks or why: reading `~/.kaggle/kaggle.json` (any tool), `cat`/`head`/`grep`/`jq` on it, `kaggle config view`, `env | grep KAGGLE`, echoing `$KAGGLE_KEY`/`$KAGGLE_API_TOKEN`, quoting a pasted token back, or writing any of it into a notebook cell, log, run artifact, or spawn prompt. Credentials are consumed by the `kaggle` binary from the environment — the skill needs the CLI to work, never the secret's value. Verify auth only by exit code (`kaggle competitions list -p 1 >/dev/null 2>&1`), never by inspecting the file. If a user pastes a token into chat, do not repeat it and tell them to rotate it at kaggle.com/settings. `.claude/settings.json` deny-lists the common read paths, but the deny list is a backstop, not the rule — no alternate command form is permitted either.

**Credential instructions** (print verbatim; the user does this, the skill never fabricates, reads, or echoes a token):

> 1. Open <https://www.kaggle.com/settings> → **API** → **Create New Token** — downloads `kaggle.json`.
> 2. `mkdir -p ~/.kaggle && mv ~/Downloads/kaggle.json ~/.kaggle/ && chmod 600 ~/.kaggle/kaggle.json`
> 3. Env-var alternative: `export KAGGLE_USERNAME=<user> KAGGLE_KEY=<key>` (newer CLI builds also accept `KAGGLE_API_TOKEN`; `kaggle --version` tells which build is installed).

**Grounding queries** — read-only, cheap, run when `ready`. Competition slug is positional; `-v` is CSV output, not verbose. Anything beyond the commands below: read `kaggle competitions --help` / `kaggle datasets --help` rather than guessing flags — the surface shifts between CLI releases.

```bash
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
IFS= read -r COMPETITION_NAME < "${TMPDIR:-/tmp}/kaggle-competition-name-${CSID}" 2>/dev/null || COMPETITION_NAME=""
KAGGLE_SLUG="${KAGGLE_SLUG:-$COMPETITION_NAME}"   # override when notebook slug differs from competition slug
echo "=== files ==="; kaggle competitions files "$KAGGLE_SLUG" -v --page-size 200
echo "=== leaderboard head ==="; kaggle competitions leaderboard "$KAGGLE_SLUG" -s -v 2>/dev/null | head -10  # timeout: 60000
```

File listing works without joining the competition (verified against a competition with `userHasEntered=False`); rules acceptance gates **downloads**. A `403` or any "accept the rules" error means the user must open `https://www.kaggle.com/competitions/<slug>/rules` and click **I Understand and Accept** — the CLI cannot accept them. Treat the affected facts as ungrounded until they confirm.

A `404` here almost always means a malformed slug, not a missing competition: `kaggle competitions list -v` returns full URLs in the `ref` column, so take the last path segment (`arc-prize-2026-arc-agi-2`, never `https://www.kaggle.com/competitions/...`). Confirm with `kaggle competitions list -s "<search term>" -v`.

**Grounding download** — sample submission and any small metadata file only. Size threshold: `sample_submission.csv` plus files under ~10 MB from the listing:

```bash
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
IFS= read -r COMPETITION_NAME < "${TMPDIR:-/tmp}/kaggle-competition-name-${CSID}" 2>/dev/null || COMPETITION_NAME=""
KAGGLE_SLUG="${KAGGLE_SLUG:-$COMPETITION_NAME}"
KAGGLE_DATA=".experiments/kaggle/data/${COMPETITION_NAME}"
mkdir -p "$KAGGLE_DATA"
kaggle competitions download "$KAGGLE_SLUG" -f sample_submission.csv -p "$KAGGLE_DATA" -q  # timeout: 120000
head -3 "$KAGGLE_DATA"/sample_submission.csv 2>/dev/null || echo "no sample_submission.csv in this competition"
```

Single-file downloads may arrive zipped — unzip into `$KAGGLE_DATA` before reading the header.

**Full-data gate** — never pull the whole archive unprompted; competition data reaches hundreds of GB and the user may want only the notebook. Show the listing with sizes, then `AskUserQuestion`: (a) skip — notebook targets Kaggle-runtime paths (`/kaggle/input/<slug>/`) · (b) download all (state total size from the listing in the option description). On (b): `kaggle competitions download "$KAGGLE_SLUG" -p "$KAGGLE_DATA"`; `-f <name>` fetches one large file instead.

Data downloaded locally does not change the notebook's path constants: `PATH_DATASET` stays the Kaggle-runtime path unless the user says the notebook runs locally.

**Related-dataset lookup** (optional, when the competition allows external data): `kaggle datasets list -s "<term>" -v`, then `kaggle datasets files <owner>/<name> -v` and `kaggle datasets download <owner>/<name> -p "$KAGGLE_DATA" --unzip`. Same gate applies — list before downloading.

**Grounding protocol — mandatory before Step 2:**

Build fact table. Each fact needs source: `[kaggle-cli:<command>]`, `[fetched]`, `[user]`, `[past-notebook:<file>]`, or `[inferred-from:<fact>]`. Never mark fact `[inferred]` without citing prior fact it derives from.

`[kaggle-cli:*]` outranks `[fetched]` for file names, data schema, and submission format — the CLI reads the real artifact, the page describes it. Keep `[fetched]` for problem narrative and metric definition.

| Fact | Value | Source |
| -- | -- | -- |
| problem_type | ? | ? |
| input_modality | ? | ? |
| output_format | ? | ? |
| eval_metric | ? | ? |
| data schema (CSV columns / image format) | ? | ? |
| submission format | ? | ? |

**Gaps — ask before generating:**

After building fact table, count facts still marked `?` or `[inferred]` without prior grounded fact. Any of these unknown:

- `input_modality` — cannot generate Dataset class
- `eval_metric` — cannot choose torchmetric
- `submission format` — cannot generate Submission section

When the CLI is `ready`, resolve `data schema`, `submission format`, and often `input_modality` from the file listing and the downloaded `sample_submission.csv` header before asking anything — questions are for what the CLI cannot answer.

Invoke `AskUserQuestion` with up to 4 questions covering all unknown required facts. Never guess or hallucinate competition-specific details (column names, file paths, data schema). State "unknown — will use placeholder" if user skips.

Acknowledge past-notebook similarity explicitly: "Found similar past notebook: `<file>` — reusing `<pattern>` from it."

## Step 2: Determine problem profile

From gathered context, determine:

| Property | Value |
| -- | -- |
| `problem_type` | classification / regression / segmentation / detection / tabular |
| `input_modality` | image-2d / image-3d / tabular / time-series / point-cloud / mixed |
| `output_format` | label / scalar / mask / bboxes / rle |
| `eval_metric` | AUC / F1 / RMSE / Dice / IoU / mAP / ... |
| `recommended_model` | see §Model selection below |
| `use_ptl` | true if DNN training; false for pure XGBoost/sklearn pipelines |

**Model selection rules** (best-fit, not default):

- Image classification → `timm.create_model` (EfficientNetV2, ConvNeXt, ViT-B) + PTL
- Image regression → `timm.create_model` backbone (`num_classes=0`) + PTL regression head
- Image segmentation → `segmentation_models_pytorch` (UNet/UNet++) + PTL; MONAI for 3D
- Object detection → `torchvision.models.detection` or `ultralytics YOLO` + PTL wrapper if needed
- Tabular → `xgboost.XGBClassifier/Regressor` with sklearn Pipeline; PTL only if DNN features needed
- Point cloud → MONAI or `pytorch3d`; PTL always
- Time series → `torch.nn.LSTM` or `tsfresh` features + XGBoost; PTL when DNN

**PTL rule**: use PTL whenever training loop needed — even simple single-layer models. Exception: pure sklearn/XGBoost pipelines, no neural network component.

## Step 3: Generate notebook script

**Foundry availability check** — verify before spawning:

```bash
FOUNDRY_AVAILABLE=$({ find ~/.claude/plugins/cache -maxdepth 5 -path "*/foundry/*/agents/sw-engineer.md" 2>/dev/null; ls plugins/cc_foundry/agents/sw-engineer.md 2>/dev/null; } | head -1)  # timeout: 5000
[ -z "$FOUNDRY_AVAILABLE" ] && { printf "⚠ foundry plugin not available — kaggle notebook generation requires foundry:sw-engineer\nInstall: claude plugin install foundry@borda-ai-rig\n"; exit 1; }
```

Spawn prompt assembled from the inline problem profile below plus exactly one resolved composition row:

```bash
# Re-hydrate flags persisted in Step 1 (bash state lost between Bash calls)
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
IFS= read -r COMPETITION_NAME < "${TMPDIR:-/tmp}/kaggle-competition-name-${CSID}" 2>/dev/null || COMPETITION_NAME="$COMPETITION_NAME"
IFS= read -r EDA_ONLY < "${TMPDIR:-/tmp}/kaggle-eda-only-${CSID}" 2>/dev/null || EDA_ONLY="false"
IFS= read -r INFERENCE_ONLY < "${TMPDIR:-/tmp}/kaggle-inference-only-${CSID}" 2>/dev/null || INFERENCE_ONLY="false"
_KAGGLE_MODES="${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/skills/kaggle/modes"
COMPOSITION_FILE="$_KAGGLE_MODES/composition.md"
MODE="full"
[ "$EDA_ONLY" = "true" ] && MODE="eda-only"
[ "$INFERENCE_ONLY" = "true" ] && MODE="inference-only"

# Derive output filename from mode — must match the composition contract before spawning
OUTPUT_SUFFIX=""
[ "$INFERENCE_ONLY" = "true" ] && OUTPUT_SUFFIX="-inference"
OUTFILE=".experiments/kaggle/${COMPETITION_NAME}${OUTPUT_SUFFIX}.py"
echo "$MODE" > "${TMPDIR:-/tmp}/kaggle-mode-${CSID}"
echo "Mode: $MODE · Output: $OUTFILE"
cat "$COMPOSITION_FILE"  # timeout: 5000
```

Select the exact `$MODE` row from `composition.md` (loaded above) and cat each named contract once, from left to right, plus `style-rules.md` once:

```bash
_KAGGLE_MODES="${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/skills/kaggle/modes"
case "$MODE" in
  full) _CONTRACTS="foundation.md eda.md training.md inference.md submission.md" ;;
  eda-only) _CONTRACTS="foundation.md eda.md" ;;
  inference-only) _CONTRACTS="foundation.md inference.md submission.md" ;;
esac
for _c in $_CONTRACTS style-rules.md; do
    echo "=== $_c ==="
    cat "$_KAGGLE_MODES/$_c"
done
```

Load `modality-dispatch.md` only when a selected section requests a modality branch:

```bash
_KAGGLE_MODES="${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/skills/kaggle/modes"
cat "$_KAGGLE_MODES/modality-dispatch.md"  # timeout: 5000
```

Do not load unselected section contracts. Pass the selected row and resolved contract contents to `foundry:sw-engineer` after the problem profile block below.

Spawn **foundry:sw-engineer** with this prompt preamble (inline, then continue with the resolved composition contracts):

```markdown
Write a complete Kaggle competition notebook script to `<OUTFILE>` (substitute expanded path from bash block above).

Format: Jupytext `# %%` Python script — every cell separated by `# %%` (code) or `# %% [markdown]` (markdown).

## Problem profile
- Competition: <competition-name>
- Problem type: <problem_type>
- Input: <input_modality>
- Output: <output_format>
- Metric: <eval_metric>
- Model: <recommended_model>
- Use PTL: <use_ptl>
- Description: <competition description if available>

[Continue with the selected row from composition.md, followed by the resolved section contracts in order, style-rules.md, and the selected modality branch when applicable.]

## Completion

Write `<OUTFILE>`. Return only:

{"status":"done","file":"<OUTFILE>","lines":N,"sections":N,"problem_type":"<type>","mode":"<MODE>","confidence":0.N}
```

**Synchronous spawn note**: `foundry:sw-engineer` spawned synchronously (not `run_in_background=true`), so CLAUDE.md §6 poll-based monitoring unreachable mid-call. After Agent() returns, check agent's output under `.experiments/kaggle/`; missing or empty → treat as timed out, surface with ⏱ marker — never silently omit.

```bash
# boundary: after Step 3 notebook generated (compaction-contract.md)
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
IFS= read -r _COMPETITION < "${TMPDIR:-/tmp}/kaggle-competition-name-${CSID}" 2>/dev/null || _COMPETITION=""
IFS= read -r _INF < "${TMPDIR:-/tmp}/kaggle-inference-only-${CSID}" 2>/dev/null || _INF="false"
IFS= read -r _KEEP < "${TMPDIR:-/tmp}/kaggle-keep-items-${CSID}" 2>/dev/null || _KEEP=""
_SUFFIX=""; [ "$_INF" = "true" ] && _SUFFIX="-inference"
_OUTFILE=".experiments/kaggle/${_COMPETITION}${_SUFFIX}.py"
_KEEP_APPEND=""; [ -n "$_KEEP" ] && _KEEP_APPEND="; user-keep: $_KEEP"
mkdir -p .temp/state  # timeout: 5000
{
    echo "## Active Skill Contract"
    echo "- skill: research:kaggle · phase: verify (after Step 3 notebook generated)"
    echo "- run-dir: .experiments/kaggle"
    echo "- preserve: outfile=${_OUTFILE}, competition=${_COMPETITION}${_KEEP_APPEND}"
    echo "- next: Step 4 verify structure, follow-up gate, package distillation"
} > .temp/state/skill-contract.md  # timeout: 5000
```

## Step 4: Verify and report

After agent completes:

1. Read first 30 lines of generated file to verify `# %%` structure
2. Count cell markers: `grep -c "^# %%" .experiments/kaggle/<name>.py`
3. Resolve the current row from `composition.md`; verify every listed section is present and no unlisted section was generated
4. Mechanically check for bare `#` heading-spacer lines (style-rules.md rule 13) — prose compliance alone proved insufficient in practice; auto-fix rather than trust the generating pass

```bash
# Re-derive OUTFILE from flags persisted in Step 1 (bash state lost between steps)
export CSID="${CLAUDE_CODE_SESSION_ID:-$PPID}"
IFS= read -r COMPETITION_NAME < "${TMPDIR:-/tmp}/kaggle-competition-name-${CSID}" 2>/dev/null || COMPETITION_NAME="$COMPETITION_NAME"
IFS= read -r INFERENCE_ONLY < "${TMPDIR:-/tmp}/kaggle-inference-only-${CSID}" 2>/dev/null || INFERENCE_ONLY="false"
IFS= read -r MODE < "${TMPDIR:-/tmp}/kaggle-mode-${CSID}" 2>/dev/null || MODE="full"
OUTPUT_SUFFIX=""; [ "$INFERENCE_ONLY" = "true" ] && OUTPUT_SUFFIX="-inference"
OUTFILE=".experiments/kaggle/${COMPETITION_NAME}${OUTPUT_SUFFIX}.py"
echo "=== Composition ==="; echo "$MODE"
echo "=== Cell count ==="; grep -c "^# %%" "$OUTFILE"  # timeout: 5000
echo "=== Sections ===";   grep "^# %% \[markdown\]" "$OUTFILE"  # timeout: 5000
echo "=== File size ===";  wc -l "$OUTFILE"  # timeout: 5000

echo "=== Bare '#' heading-spacer check (rule 13) ==="
python3 "${CLAUDE_PLUGIN_ROOT:-plugins/cc_research}/bin/fix_jupytext_blank_md.py" "$OUTFILE"  # timeout: 5000
```

Print to terminal:

- Output path (`$OUTFILE`)
- Mode + resolved composition contracts
- Problem type + recommended model
- Cell count and section list
- Missing required sections flagged with `⚠`
- Bare `#` heading-spacer count found/auto-fixed (`0` when clean)

Invoke `AskUserQuestion` as follow-up gate:

- (a) Open in editor — `code $OUTFILE`
- (b) Extend with additional sections
- (c) Regenerate with different model/approach
- (d) Done

On (a): run `code "$OUTFILE"` via Bash. On (b): re-enter Step 3 with extension directive. On (c): re-enter Step 2 with user-specified changes.

**Package distillation gate** — invoke after follow-up gate resolves to Done:

Benefits to state before asking: shared helpers tested once, used everywhere; wheel attachment on Kaggle faster than re-inlining; subsequent notebooks shorter; package tests catch regressions before submission.

Invoke `AskUserQuestion`:

- (a) Yes — scaffold `src/<package>/` with extracted helpers + tests
- (b) Skip — keep everything inlined for now

If **(a)**:

1. Identify every function in notebook with no hardcoded paths, no `plt.show()`, no `tqdm` calls
2. Write each to `src/<package>/<module>.py` with **full** Google-style docstring + `Example:` block — all standard coding patterns apply (doctests for pure functions, `Args:`/`Returns:` sections, full `if __name__ == "__main__":` guards where appropriate); these are package modules, not notebook cells
3. Create `tests/test_<module>.py` covering each function
4. Create `notebooks/01_<competition-name>_pkg.py` — inline definitions replaced by package imports; **never modify validated baseline `$OUTFILE`**

If **(b)**: skip; repeat this gate offer after next notebook written.

```bash
rm -f .temp/state/skill-contract.md  # clear contract — kaggle notebook complete (compaction-contract.md §Lifecycle)  # timeout: 5000
```

</workflow>