Autonomous Orchestration

---
name: autonomous-orchestration
description: Use when user requests autonomous operation across multiple issues. Orchestrates parallel workers, monitors progress, handles SLEEP/WAKE cycles, and works until scope is complete without user intervention.
---

# Autonomous Orchestration

## Overview

Orchestrates long-running autonomous work across multiple issues, spawning parallel workers, monitoring CI, and persisting state across sessions.

**Core principle:** GitHub is the source of truth. Workers are disposable. State survives restarts.

**Announce at start:** "I'm using autonomous-orchestration to work through [SCOPE]. Starting autonomous operation now."

## Prerequisites

- `worker-dispatch` skill for spawning workers
- `worker-protocol` skill for worker behavior
- `ci-monitoring` skill for CI/WAKE handling
- Git worktrees support (workers use isolated worktrees)
- GitHub CLI (`gh`) authenticated
- GitHub Project Board configured

## State Management

**CRITICAL:** All state is stored in GitHub. NO local state files.

| State Store | Purpose | Used For |
|-------------|---------|----------|
| Project Board Status | THE source of truth | Ready, In Progress, In Review, Blocked, Done |
| Issue Comments | Activity log | Worker assignment, progress, deviations |
| Labels | Lineage only | `spawned-from:#N`, `depth:N`, `epic-*` |
| MCP Memory | Fast cache + active marker | Read optimization, **active orchestration detection** |

**See:** `reference/state-management.md` for detailed state queries and updates.

## Context Compaction Survival

**CRITICAL:** Orchestration must survive mid-loop context compaction.

### On Start: Write Active Marker

```bash
# Write to MCP Memory when orchestration starts
mcp__memory__create_entities([{
  "name": "ActiveOrchestration",
  "entityType": "Orchestration",
  "observations": [
    "Status: ACTIVE",
    "Scope: [MILESTONE/EPIC/unbounded]",
    "Tracking Issue: #[NUMBER]",
    "Started: [ISO_TIMESTAMP]",
    "Repository: [owner/repo]",
    "Phase: BOOTSTRAP|MAIN_LOOP",
    "Last Loop: [ISO_TIMESTAMP]"
  ]
}])
```

### On Each Loop Iteration: Update Marker

```bash
mcp__memory__add_observations({
  "observations": [{
    "entityName": "ActiveOrchestration",
    "contents": ["Last Loop: [ISO_TIMESTAMP]", "Phase: MAIN_LOOP"]
  }]
})
```

### On Complete: Remove Marker

```bash
mcp__memory__delete_entities({
  "entityNames": ["ActiveOrchestration"]
})
```

### On Session Resume (After Compaction)

Session-start skill checks for active orchestration:

```bash
# Check MCP Memory for active orchestration
ACTIVE=$(mcp__memory__open_nodes({"names": ["ActiveOrchestration"]}))

if [ -n "$ACTIVE" ]; then
  echo "⚠️ ACTIVE ORCHESTRATION DETECTED"
  echo "Scope: [from ACTIVE]"
  echo "Tracking: [from ACTIVE]"
  echo ""
  echo "Resuming orchestration loop..."
  # Invoke autonomous-orchestration skill to resume
fi
```

**This ensures:** Even if context compacts mid-loop, the next session will detect the active orchestration and resume it.

## Immediate Start (User Consent Implied)

**The user's request for autonomous operation IS their consent.** No additional confirmation required.

When the user requests autonomous work:

1. **Identify scope** - Parse user request for milestone, epic, specific issues, or "all"
2. **Announce intent** - Briefly state what you're about to do
3. **Start immediately** - Begin orchestration without waiting for additional input

```markdown
## Starting Autonomous Operation

**Scope:** [MILESTONE/EPIC/ISSUES or "all open issues"]
**Workers:** Up to 5 parallel
**Mode:** Continuous until complete

Beginning work now...
```

**Do NOT ask for "PROCEED" or any confirmation.** The user asked for autonomous operation - that is the confirmation.

## Automatic Scope Detection

When the user requests autonomous operation without specifying a scope:

### Priority Order

1. **User-specified scope** - If user mentions specific issues, epics, or milestones
2. **Urgent/High Priority standalone issues** - Issues with `priority:urgent` or `priority:high` labels not part of an epic
3. **Epic-based sequential work** - Work through epics in order, completing all issues within each epic
4. **Remaining standalone issues** - Any issues not part of an epic

```bash
detect_work_scope() {
  # 1. Check for urgent/high priority standalone issues first
  PRIORITY_ISSUES=$(gh issue list --state open \
    --label "priority:urgent,priority:high" \
    --json number,labels \
    --jq '[.[] | select(.labels | map(.name) | any(startswith("epic-")) | not)] | .[].number')

  if [ -n "$PRIORITY_ISSUES" ]; then
    echo "priority_standalone"
    echo "$PRIORITY_ISSUES"
    return
  fi

  # 2. Get epics in order (by creation date)
  EPICS=$(gh issue list --state open --label "type:epic" \
    --json number,title,createdAt \
    --jq 'sort_by(.createdAt) | .[].number')

  if [ -n "$EPICS" ]; then
    echo "epics"
    echo "$EPICS"
    return
  fi

  # 3. Fall back to all open issues
  ALL_ISSUES=$(gh issue list --state open --json number --jq '.[].number')
  echo "all_issues"
  echo "$ALL_ISSUES"
}
```

## Continuous Operation Until Complete

Autonomous operation continues until ALL of:
- No open issues remain in scope
- No open PRs awaiting merge
- No issues in "In Progress" or "In Review" status

The operation does NOT pause for:
- Progress updates
- Confirmation between issues
- Switching between epics
- Any user input (unless blocked by a fatal error)

## PR Resolution Bootstrap Phase

**CRITICAL:** Before spawning ANY new workers, resolve all existing open PRs first.

```
┌──────────────────────────────────────────────────────────┐
│                    BOOTSTRAP PHASE                        │
│             (Runs ONCE before main loop)                  │
└─────────────────────────┬────────────────────────────────┘
                          │
                          ▼
               ┌───────────────────┐
               │ GET OPEN PRs      │
               │                   │
               │ Filter out:       │
               │ - release/*       │
               │ - release-        │
               │   placeholder     │
               └─────────┬─────────┘
                         │
              ┌──────────┴──────────┐
              ▼                     ▼
        ┌───────────┐         ┌───────────┐
        │ Has PRs?  │─── No ──│ → MAIN    │
        │           │         │   LOOP    │
        └─────┬─────┘         └───────────┘
              │ Yes
              ▼
        ┌───────────────────────────────┐
        │ FOR EACH PR:                  │
        │                               │
        │ 1. Check CI status            │
        │ 2. Verify review artifact     │
        │ 3. Merge if ready OR          │
        │ 4. Wait/fix if not            │
        └───────────────────────────────┘
                          │
                          ▼
                    MAIN LOOP
```

### Bootstrap Implementation

```bash
resolve_existing_prs() {
  echo "=== PR RESOLUTION BOOTSTRAP ==="

  # Get all open PRs, excluding release placeholders
  OPEN_PRS=$(gh pr list --json number,headRefName,labels \
    --jq '[.[] | select(
      (.headRefName | startswith("release/") | not) and
      (.labels | map(.name) | index("release-placeholder") | not)
    )] | .[].number')

  if [ -z "$OPEN_PRS" ]; then
    echo "No actionable PRs to resolve. Proceeding to main loop."
    return 0
  fi

  echo "Found PRs to resolve: $OPEN_PRS"

  for pr in $OPEN_PRS; do
    echo "Processing PR #$pr..."

    # Get CI status
    ci_status=$(gh pr checks "$pr" --json state --jq '.[].state' 2>/dev/null | sort -u)

    # Get linked issue
    ISSUE=$(gh pr view "$pr" --json body --jq '.body' | grep -oE 'Closes #[0-9]+' | grep -oE '[0-9]+' | head -1)

    if [ -z "$ISSUE" ]; then
      echo "  ⚠ No linked issue found, skipping"
      continue
    fi

    # Check if CI passed
    if echo "$ci_status" | grep -q "FAILURE"; then
      echo "  ❌ CI failing - triggering ci-monitoring for PR #$pr"
      # Invoke ci-monitoring skill to fix
      handle_ci_failure "$pr"
      continue
    fi

    if echo "$ci_status" | grep -q "PENDING"; then
      echo "  ⏳ CI pending for PR #$pr, will check in main loop"
      continue
    fi

    if echo "$ci_status" | grep -q "SUCCESS"; then
      # Verify review artifact
      REVIEW_EXISTS=$(gh api "/repos/$OWNER/$REPO/issues/$ISSUE/comments" \
        --jq '[.[] | select(.body | contains("<!-- REVIEW:START -->"))] | length' 2>/dev/null || echo "0")

      if [ "$REVIEW_EXISTS" = "0" ]; then
        echo "  ⚠ No review artifact - requesting review for #$ISSUE"
        gh issue comment "$ISSUE" --body "## Review Required

PR #$pr has passing CI but no review artifact.

**Action needed:** Complete comprehensive-review and post artifact to this issue.

---
*Bootstrap phase - Orchestrator*"
        continue
      fi

      # All checks pass - merge
      echo "  ✅ Merging PR #$pr"
      gh pr merge "$pr" --squash --delete-branch
      mark_issue_done "$ISSUE"
    fi
  done

  echo "=== BOOTSTRAP COMPLETE ==="
}
```

### Release Placeholder Detection

PRs are excluded from bootstrap resolution if:

| Condition | Example |
|-----------|---------|
| Branch starts with `release/` | `release/v2.0.0`, `release/2025-01` |
| Has `release-placeholder` label | Manual exclusion |
| Has `do-not-merge` label | Explicit hold |

## Orchestration Loop

```
┌──────────────────────────────────────────────────────────┐
│                       MAIN LOOP                          │
└─────────────────────────┬────────────────────────────────┘
                          │
      ┌───────────────────┼───────────────────┐
      ▼                   ▼                   ▼
┌───────────┐      ┌───────────┐      ┌───────────┐
│ CHECK     │      │ CHECK     │      │ SPAWN     │
│ WORKERS   │      │ CI/PRs    │      │ WORKERS   │
└─────┬─────┘      └─────┬─────┘      └─────┬─────┘
      │                  │                  │
      └──────────────────┼──────────────────┘
                         │
                         ▼
               ┌───────────────────┐
               │ EVALUATE STATE    │
               │                   │
               │ All done? → Exit  │
               │ Waiting? → SLEEP  │
               │ Work? → Continue  │
               └───────────────────┘
```

**See:** `reference/loop-implementation.md` for full loop code.

### Loop Steps

1. **Check Deviation Resolution** - Resume issues whose children are all closed
2. **Check CI/PRs** - Monitor for merge readiness, verify review artifacts
3. **MERGE GREEN PRs** - Any PR with passing CI is merged IMMEDIATELY
4. **Spawn Workers** - Up to 5 parallel workers from Ready queue
5. **Evaluate State** - Determine next action (continue, sleep, complete)
6. **Brief Pause** - 30 second interval between iterations

### CRITICAL: Merge Green PRs Immediately

**Every loop iteration must check for and merge passing PRs:**

```bash
# In each loop iteration
for pr in $(gh pr list --json number,statusCheckRollup --jq '.[] | select(.statusCheckRollup | all(.conclusion == "SUCCESS")) | .number'); do
  # Check for do-not-merge label
  if ! gh pr view "$pr" --json labels --jq '.labels[].name' | grep -q "do-not-merge"; then
    echo "Merging PR #$pr (CI passed)"
    gh pr merge "$pr" --squash --delete-branch

    # Get linked issue and mark done
    ISSUE=$(gh pr view "$pr" --json body --jq '.body' | grep -oE 'Closes #[0-9]+' | grep -oE '[0-9]+' | head -1)
    if [ -n "$ISSUE" ]; then
      # Update project board status to Done
      mark_issue_done "$ISSUE"
    fi
  fi
done
```

**Do NOT:**
- Report "PRs are ready for merge" and stop
- Wait for user to request merge
- Summarize completed work and ask for next steps

**The loop continues until scope is complete. Green PR = immediate merge.**

## Scope Types

### Milestone

```bash
gh issue list --milestone "v1.0.0" --state open --json number --jq '.[].number'
```

### Epic

```bash
gh issue list --label "epic-dark-mode" --state open --json number --jq '.[].number'
```

### Unbounded (All Open Issues)

```bash
gh issue list --state open --json number --jq '.[].number'
```

**Do NOT ask for "UNBOUNDED" confirmation.** The user's request is their consent.

## Failure Handling

Workers that fail do NOT immediately become blocked:

```
Attempt 1 → Research → Attempt 2 → Research → Attempt 3 → Research → Attempt 4 → BLOCKED
```

Only after 3+ research cycles is an issue marked as blocked.

**See:** `reference/failure-recovery.md` for research cycle implementation.

### Blocked Determination

An issue is only marked blocked when:
- Multiple research cycles completed (3+)
- Research concludes "impossible without external input"
- Examples: missing credentials, requires human decision, external service down

## SLEEP/WAKE

### Entering SLEEP

Orchestration sleeps when:
- All issues are either blocked or in review
- No work can proceed without external event

State is posted to GitHub tracking issue (survives crashes).

### WAKE Mechanisms

- **Session start** - Checks CI status on new Codex session
- **Manual** - `codex resume [SESSION_ID]`

## Checklist

Before starting orchestration:

- [ ] Scope identified (explicit or auto-detected)
- [ ] Git worktrees available (`git worktree list`)
- [ ] GitHub CLI authenticated (`gh auth status`)
- [ ] No uncommitted changes in main worktree
- [ ] Tracking issue exists with `orchestration-tracking` label
- [ ] Project board configured with Status field

Bootstrap phase:

- [ ] Existing open PRs detected
- [ ] Release placeholders excluded (`release/*`, `release-placeholder`, `do-not-merge` labels)
- [ ] CI status checked for each PR
- [ ] Review artifacts verified before merge
- [ ] PRs merged or flagged for attention
- [ ] Bootstrap complete before spawning workers

During orchestration:

- [ ] Workers spawned with worktree isolation
- [ ] Worker status tracked via Project Board (NOT labels)
- [ ] CI status monitored
- [ ] Review artifacts verified before PR merge
- [ ] Failed workers trigger research cycles
- [ ] Handovers happen at turn limit
- [ ] SLEEP entered when only waiting on CI
- [ ] Deviation resolution checked each loop
- [ ] Status posted to tracking issue

## Review Enforcement

**CRITICAL:** The orchestrator verifies review compliance:

1. **Before PR merge:**
   - Review artifact exists in issue comments
   - Review status is COMPLETE
   - Unaddressed findings = 0

2. **Child issues (from deferred findings):**
   - Follow full `issue-driven-development` process
   - Have their own code reviews
   - Track via `spawned-from:#N` label

3. **Deviation handling:**
   - Parent status set to Blocked on project board
   - Resumes only when all children closed

## Integration

This skill coordinates:

| Skill | Purpose |
|-------|---------|
| `worker-dispatch` | Spawning workers |
| `worker-protocol` | Worker behavior |
| `worker-handover` | Context passing |
| `ci-monitoring` | CI and WAKE handling |
| `research-after-failure` | Research cycles |
| `issue-driven-development` | Worker follows this |
| `comprehensive-review` | Workers must complete before PR |
| `project-board-enforcement` | ALL state queries and updates |

## Reference Files

- `reference/state-management.md` - State queries, updates, deviation handling
- `reference/loop-implementation.md` - Full loop code and helpers
- `reference/failure-recovery.md` - Research cycles, blocked handling, SLEEP/WAKE