Claude Integration

---
name: claude-integration
description: "Low-level AI CLI spawn mechanics used by ralphctl's provider adapters — headless / interactive spawn, the file-based contract (`signals.json` + `sessionId` files), idle-stdout watchdog, exponential rate-limit backoff, and session resume for in-flight recovery. Use when modifying `src/integration/ai/providers/{claude,copilot,codex,opencode,grok}/` or `src/integration/ai/providers/_engine/`, debugging stdin hangs / slow Claude startups, or wiring a new code path that spawns an AI CLI directly."
when_to_use: 'When touching a provider adapter, the shared `_engine/`, or a readiness probe; when diagnosing a rate-limit / resume / watchdog issue; when a new consumer needs to spawn an AI CLI outside the existing ports. Not needed for higher-level work — the flows already wrap all of this through the `HeadlessAiProvider` / `InteractiveAiProvider` ports.'
---

# AI provider integration (Claude / Copilot / Codex / OpenCode / Grok)

Covers what is **not** in `CLAUDE.md` or `.claude/docs/ARCHITECTURE.md`. For harness signals, exit codes,
sequential task execution, and check-script gating — see `CLAUDE.md`.

**Source of truth:**

- Provider adapters: `src/integration/ai/providers/{claude,copilot,codex,opencode,grok}/` — one folder per tool,
  sibling-isolated. Each owns `headless.ts`, `interactive.ts`, and `model-availability-probe.ts` (Claude,
  Copilot, OpenCode, and Grok also have `parse-stream.ts` for stream-format handling). OpenCode is the aggregator
  backend: its model ids are `provider/model` and its catalog is discovered at runtime via `opencode models`,
  so its probe is the one that cannot check against a static list. Grok's probe is passthrough (not
  `grok models`).
- Shared engine: `src/integration/ai/providers/_engine/` — `spawn.ts` (the `ProviderSpawn` port + default
  `node:child_process.spawn` impl), `run-headless-spawn.ts` (the headless wrapper that wires watchdog +
  signals file + sessionId file + rate-limit backoff), `rate-limit-backoff.ts` (exponential retry policy),
  `idle-watchdog.ts` (kills wedged children), `headless-ai-provider.ts` (the cross-tool port).
- Signal contract:
  `src/integration/ai/contract/_engine/{validate-signals-file,render-sidecars,render-contract-section}.ts`
  with per-kind Zod schemas under `src/integration/ai/contract/_engine/signals/<kind>/schema.ts`.
  Each AI-spawning leaf composes `<leaf>.contract.ts` from these primitives; the AI creates
  `signals.json` with whatever file-creating tool its CLI has (`Write` on four backends,
  `search_replace` on Grok, which has no `write` tool), the harness Zod-validates post-spawn.
- Composition: `src/application/bootstrap/provider-factory.ts` (`createAiProvider`) picks the concrete
  adapter from `settings.ai.provider`.

## Spawn modes

There are two ports under `_engine/`:

```ts
import type { HeadlessAiProvider } from '@src/integration/ai/providers/_engine/headless-ai-provider.ts';
import type { InteractiveAiProvider } from '@src/integration/ai/providers/_engine/interactive-ai-provider.ts';
```

- **Headless** — `-p` (print) mode with prompt + signals output file. The shared
  `runHeadlessSpawn(...)` helper handles arg construction, the file-based contract, the idle watchdog,
  rate-limit backoff retry, and post-spawn parsing. Each per-tool adapter under `providers/<tool>/headless.ts`
  customises the arg builder and JSON-shape parser.
- **Interactive** — TTY handoff, `stdio: 'inherit'`. The AI CLI takes over the terminal (alt-screen swap to
  its own UI); the TUI's alt-screen state is suspended and restored on the way back. Used by `refine` /
  `plan` / `ideate` / `readiness` flows that need a human at the keyboard.

**Prompt delivery — never the body in argv.** Headless claude / codex / opencode pipe the prompt through
stdin. Interactive cannot (stdin piping flips these CLIs out of interactive mode) and neither can copilot
or grok headless (neither CLI has a stdin prompt path), so those write the prompt to a file and pass a
POINTER at it — `_engine/prompt-pointer.ts` for copilot (`-p <pointer>`), `--prompt-file grok-prompt.md`
for grok (Grok does not read piped stdin as the prompt). Interactive grok never passes `--prompt-file`
(that forces headless) and takes a positional pointer instead. Argv caps at 32,767 bytes on Windows and a
rendered plan prompt clears that on its own; inlining it produced `spawn ENAMETOOLONG` before the CLI ever
started. `_engine/argv-budget.ts` names an overflow in the error rather than leaving a bare errno.

## File-based provider contract

ralphctl does NOT parse stdout for signals or session IDs. Instead, every spawn passes paths to two files:

- `signals.json` — structured JSON the adapter (or the AI's wrapper) writes during the run. The harness
  reads it back post-spawn via `validateSignalsFile(...)` and dispatches to the parser registry.
- `session-id.txt` — a single-line text file containing the provider's session id, written the moment
  the stream yields the id (while the child still runs, so a killed harness leaves it behind) and noted
  in the live-run record. Grok reports its id only on the final `end` record, so it lands at exit.

This replaces v0.6.x's brittle stdout-parsing path. Provider JSON-shape drift no longer breaks the harness;
the contract is the file content.

The per-spawn audit / sandbox layout is:

```
<sprintDir>/<flow>/<unit>/rounds/<N>/{generator,evaluator}/
├── prompt.md           ← rendered prompt (input); the CLI is pointed at this file, not handed its text
├── copilot-prompt.md   ← copilot headless only: the exact prompt that spawn was given
├── grok-prompt.md      ← grok headless only: `--prompt-file` body (Grok has no stdin prompt path)
├── signals.json        ← parsed structured signals (output, provider-written)
├── session-id.txt      ← provider's session id (output)
└── evaluation.md       ← rendered evaluator verdict (evaluator role only)
```

## Permission modes (per-tool, NOT portable)

| Provider         | Headless permission flag                 | Why                                                                                                                                                                                                                                                                                                                                                                                                    |
| ---------------- | ---------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `claude-code`    | `--permission-mode bypassPermissions`    | Piped stdin can't answer prompts; `acceptEdits` hangs                                                                                                                                                                                                                                                                                                                                                  |
| `github-copilot` | `--allow-all-tools`                      | Copilot's permission model is all-or-nothing                                                                                                                                                                                                                                                                                                                                                           |
| `openai-codex`   | per-session approval flow                | Codex prompts for approval inline; sandbox handles it                                                                                                                                                                                                                                                                                                                                                  |
| `opencode`       | `--auto` (or nothing)                    | No enforceable read-only mode — see the over-grant below                                                                                                                                                                                                                                                                                                                                               |
| `xai-grok`       | `--trust --always-approve --sandbox off` | One mechanism per gate — `--deny 'Edit(./**)'` (cwd-rooted) for edits, `--disallowed-tools run_terminal_command,run_terminal_cmd` + `--deny 'Bash(*)'` for shell, `--disallowed-tools web_search,web_fetch` for network — plus `--no-subagents`. Grok has no `write` tool: `search_replace` must stay present or `signals.json` cannot be created. No `--add-dir` — extra roots are a named over-grant |

**OpenCode has no enforceable permission gate.** `opencode run` exposes one approval control, `--auto`, and
omitting it does NOT make the run read-only — writes inside `--dir` land either way. The adapter forwards
`--auto` for `permissions.autoApprove`, and also whenever roots outside `--dir` are declared, because
`external_directory` is otherwise auto-rejected and there is no `--add-dir` equivalent. That over-grants
relative to the `--add-dir` adapters; the topology (`--dir` plus `outputDir`) is the real envelope. The
config-level `permission` block would scope it precisely but needs a generated per-session config file and
is not wired.

Interactive flows use lower-privilege modes where available (e.g. Claude's `acceptEdits`) because a human is
there to answer prompts.

The harness enforces actual safety through branch isolation, the per-task `checkScript` gate, and the
dirty-tree preflight — not the CLI permission gate.

## Idle-stdout watchdog

`src/integration/ai/providers/_engine/idle-watchdog.ts` kills a headless child whose stdout has been silent
past a configurable idle threshold. Prevents a stuck child from stranding the harness, whichever tool it
is. The watchdog timer resets on every stdout chunk; killing the child surfaces as a `RateLimitError`-
adjacent failure that the chain's retry policy handles. The kill reaches the child's whole process group
(`killProcessTree`; headless children are spawned `detached` on POSIX), so tool subprocesses die with it. The threshold is operator-configurable via
`settings.harness.idleWatchdogMs` (60_000–3_600_000 ms, default 300_000 = 5 min); `provider-factory.ts`
threads it into each adapter's `deps.idleMs`. Tests lower it via the `idleMs` dep override to exercise the
watchdog path fast.

## Rate-limit retry (exponential backoff)

`src/integration/ai/providers/_engine/rate-limit-backoff.ts` retries on `RateLimitError` with exponential
delay. Per-spawn cap is `settings.harness.rateLimitRetries` (range 0–10). The retry loop captures the
provider's `sessionId` from the previous attempt and resumes that session on the next, so the AI continues
from where it stopped instead of restarting. The resume spelling is per-tool — `--resume <id>` (claude),
`--resume=<id>` (copilot), the `resume` subcommand arg (codex), `-s <id>` (opencode), `-r <id>` (grok) —
each adapter's `headless.ts` owns its own arg builder. Grok's stale-resume wording is "session not found"
/ 404 restore.

## Session resume contract

```bash
# Initial spawn (claude — adjust per provider)
claude -p --output-format stream-json --permission-mode bypassPermissions < prompt.md
# Provider writes session id to the sessionId file

# Resume later
claude -p --resume "<session-id>" --permission-mode bypassPermissions < followup-prompt.md
```

Implementation contract in `runHeadlessSpawn`:

- Post-spawn, `validateSignalsFile(...)` + the sessionId file's contents are returned as part of the spawn
  result.
- `RateLimitError` carries the captured `sessionId` so the retry pass passes `--resume <id>` on the next
  attempt.
- The implement chain stamps the session of an attempt's last generator turn on `Attempt.sessionId` at
  settle. After a harness interruption, `start-attempt` reads the newest generator round's
  `session-id.txt` + `role-meta.json`; when provider, model and cwd still match it seeds the fresh
  attempt with that session and sends the `implement-crash-resume` prompt, with the full brief as
  `coldPrompt` so a vanished session falls back cold.

## Known startup issues

| Symptom                                  | Cause                     | Fix                                                                         |
| ---------------------------------------- | ------------------------- | --------------------------------------------------------------------------- |
| Process stuck at 0 CPU after spawn       | Stdin not closed          | Confirm `child.stdin.end()` is called after writing the prompt              |
| First spawn takes 1–2 min instead of ~5s | Bloated `~/.claude` cache | `rm -rf ~/.claude/plugins ~/.claude/debug`                                  |
| Watchdog kills a healthy spawn           | Idle threshold too tight  | Raise `settings.harness.idleWatchdogMs` (60_000–3_600_000, default 300_000) |

Quick health checks:

```bash
du -sh ~/.claude              # expect < 10 MB
time claude -p "yolo"         # expect ~5s
```

## Relationship to ports

Business code never imports an adapter directly — it goes through `HeadlessAiProvider` / `InteractiveAiProvider`
ports from `_engine/`. When adding a new spawn code path, prefer extending the port + the per-tool adapter;
only drop into `runHeadlessSpawn` / `spawn` if a genuinely new spawn shape is needed.

Sibling-isolation rules apply: `providers/claude/`, `providers/copilot/`, `providers/codex/`,
`providers/opencode/`, and `providers/grok/` cannot import each other. Shared helpers (spawn, watchdog,
backoff, the file-based contract reader/writer) live in `providers/_engine/` and are the only legitimate
cross-tool seam.