Harness Deployment

# Harness Deployment

> CI/CD pipeline analysis, deployment strategy design, and environment management. From commit to production with confidence. Deployment readiness is a hard gate: `harness check-deployment` **blocks** a deploy on unambiguous, incident-causing violations, **advises** on maturity gaps, and **abstains loudly** when a repo does not deploy. This is a gate, not a suggestion.

## When to Use

- Before approving a pull request or a merge that changes deployment configuration
- When setting up or reviewing CI/CD pipelines for a new or existing project
- When evaluating deployment strategies (blue-green, canary, rolling) for a service
- When auditing environment separation and promotion workflows
- When `on_pr` triggers fire and the change touches pipeline, environment, or deploy-script files
- NOT for container image building or registry management (use harness-containerization)
- NOT for infrastructure provisioning (use harness-infrastructure-as-code)
- NOT for application performance under load (use harness-perf)
- NOT for post-ship operational signal ingestion (incidents, live monitoring, error-budget feeds) — that operations half is out of scope here and deferred to a dedicated ops skill

## Process

### Phase 0: ENFORCE -- Run the deployment gate

This is the mechanical gate and it runs first. The phases below (DETECT/ANALYZE/DESIGN/VALIDATE) are the advisory context you use to _fix_ what the gate finds — they never replace it.

1. **Invoke the gate. Never reimplement it.** Run `harness check-deployment` (add `--json` for machine output, `--findings-json` for the trailing findings-contract line). The skill invokes the command; it does not re-derive the detection or the block/advise decision by hand. Hand-rolling the mechanical check instead of calling the command is a Red-Flag pattern.

2. **Read the exit code as the authority.** The gate reports one of four values, and each means exactly one thing:

   | Exit | Meaning                                                                                                                         | What it says about the deploy                                                                                                                        |
   | ---- | ------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
   | `0`  | **Pass** — deployment config detected, no hard violations (or the gate is explicitly disabled via `deployment.enabled: false`). | Cleared. Soft findings may still be listed as advisories.                                                                                            |
   | `1`  | **Blocked** — at least one hard violation.                                                                                      | Do not deploy. Fix the finding; the gate is the authority and does not get hand-waved past.                                                          |
   | `2`  | **Error** — internal failure or misconfiguration (unreadable/malformed `harness.config.json`).                                  | Indeterminate. Fix the tooling/config; never treat a `2` as a pass.                                                                                  |
   | `3`  | **Abstained** — no deployment configuration detected at all.                                                                    | The gate examined **nothing** — abstained, not passed. Never green. Confirm the repo genuinely does not deploy, or that detection missed the config. |

3. **Apply the block-vs-advise contract.** Hard rules block (exit `1`); soft rules advise (surfaced, exit `0`):

   | Code              | Class                   | Fires when                                                                                                                                                                                             |
   | ----------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
   | `DEPLOY-SEC001`   | **HARD — non-waivable** | A hardcoded secret or long-lived cloud credential appears in a pipeline file or a committed environment file. A `${{ secrets.X }}` / `process.env.X` **reference** is not a leak and does not trip it. |
   | `DEPLOY-RB001`    | **HARD**                | A deploy target is detected but no rollback path is wired.                                                                                                                                             |
   | `DEPLOY-ENV001`   | **HARD**                | A production deploy is reachable with no promotion/approval gate (direct-to-prod, no environment protection, no manual approval, no prior staging job).                                                |
   | `DEPLOY-STAGE001` | SOFT                    | Recommended pre-deploy stages are missing (security scan, smoke tests, post-deploy verification), or a pipeline file is unparseable.                                                                   |
   | `DEPLOY-ENV002`   | SOFT                    | Weak environment separation (shared **non-secret** config across environments) that is not an outright leak.                                                                                           |
   | `DEPLOY-HC001`    | SOFT                    | No post-deploy health check wired for a deploy target.                                                                                                                                                 |
   | `DEPLOY-PERF001`  | SOFT                    | Pipeline structure smells (serial stages that could parallelize, missing dependency/build caching).                                                                                                    |

4. **Severity overrides are per-rule and bounded.** `deployment.rules` may downgrade a waivable hard rule (`DEPLOY-RB001`, `DEPLOY-ENV001`) to advisory for a repo where the concept genuinely does not apply — with a config comment saying why. `DEPLOY-SEC001` ignores any downgrade: a leaked credential is never a judgment call. Disabling the whole gate (`deployment.enabled: false`) is an explicit, reviewable opt-out — not a fix for a single finding.

5. **When the gate blocks, drop into the advisory phases.** Use DETECT to locate the offending config, ANALYZE to understand the gap, and DESIGN to write the fix in the project's CI/CD syntax. Re-run the gate until it clears.

---

### Phase 1: DETECT -- Identify Pipeline and Environment Configuration

1. **Scan for CI/CD configuration files.** Search the project root for pipeline definitions:
   - `.github/workflows/*.yml` -- GitHub Actions
   - `.gitlab-ci.yml` -- GitLab CI
   - `Jenkinsfile` -- Jenkins
   - `.circleci/config.yml` -- CircleCI
   - `bitbucket-pipelines.yml` -- Bitbucket Pipelines
   - `azure-pipelines.yml` -- Azure DevOps
   - `deploy/`, `scripts/deploy*` -- custom deployment scripts

2. **Identify deployment targets.** Parse pipeline files for deployment steps and extract:
   - Target environments (dev, staging, production)
   - Deployment mechanisms (kubectl apply, aws ecs update-service, serverless deploy, rsync)
   - Cloud provider and region information
   - Container registry references

3. **Detect environment configuration.** Look for environment-specific config:
   - `.env.production`, `.env.staging` files
   - Environment variable injection in pipeline definitions
   - Secret references (GitHub Secrets, GitLab CI variables, Vault paths)
   - Feature flag provider configuration per environment

4. **Map the deployment topology.** Build a summary of what gets deployed where:
   - Service name, pipeline file, target environment, deployment mechanism
   - Dependencies between services (deploy order constraints)
   - Manual approval gates vs. automatic promotion

5. **Present detection summary.** Output the discovered topology before proceeding:

   ```
   Deployment Topology:
     Platform: GitHub Actions
     Pipelines: 3 workflow files
     Environments: dev, staging, production
     Strategy: Rolling (detected from kubectl rolling-update)
     Approval gates: production (manual)
   ```

---

### Phase 2: ANALYZE -- Evaluate Pipeline Quality and Gaps

1. **Check pipeline stage completeness.** A mature pipeline includes these stages. Flag any that are missing:
   - Build and compile
   - Unit tests
   - Integration tests
   - Security scan (SAST/DAST)
   - Artifact packaging
   - Deploy to staging
   - Smoke tests post-deploy
   - Deploy to production
   - Post-deploy verification

2. **Evaluate environment isolation.** Verify that environments are properly separated:
   - Staging and production use different credentials
   - Environment-specific variables are not shared across environments
   - Database connections point to the correct environment
   - No hardcoded production URLs in non-production configs

3. **Check deployment safety mechanisms.** Verify the pipeline includes:
   - Rollback procedures (automatic or documented manual)
   - Health checks after deployment
   - Timeout configuration on deployment steps
   - Concurrency controls (prevent parallel deploys to the same environment)
   - Branch protection rules that gate production deploys

4. **Analyze pipeline performance.** Identify bottlenecks:
   - Steps that could run in parallel but are sequential
   - Missing caching (dependencies, build artifacts, Docker layers)
   - Redundant steps across workflows
   - Total pipeline duration from commit to production

5. **Check secret hygiene in pipelines.** Verify:
   - No secrets hardcoded in pipeline files
   - Secrets are scoped to the minimum required environment
   - Secret rotation is possible without pipeline changes
   - OIDC or workload identity is used where available instead of long-lived credentials

---

### Phase 3: DESIGN -- Recommend Strategy Improvements

1. **Recommend deployment strategy.** Based on the service characteristics:
   - **Rolling** -- suitable for stateless services with backward-compatible changes
   - **Blue-green** -- suitable when zero-downtime cutover is required and rollback must be instant
   - **Canary** -- suitable for high-traffic services where gradual validation reduces blast radius
   - **Recreate** -- suitable only for development environments or when downtime is acceptable

2. **Design missing pipeline stages.** For each gap identified in Phase 2, provide:
   - The stage definition in the project's CI/CD platform syntax
   - Where it fits in the pipeline order
   - What tools or services it requires
   - Example configuration snippet

3. **Recommend environment promotion workflow.** Design the path from commit to production:
   - Automatic promotion from dev to staging after tests pass
   - Manual approval gate before production (with notification to the team channel)
   - Smoke test suite that runs post-deploy in each environment
   - Rollback trigger conditions (error rate spike, health check failure)

4. **Design rollback procedure.** Every deployment must have a documented rollback:
   - For container deployments: revert to previous image tag
   - For serverless: revert to previous function version
   - For database migrations: backward-compatible migration strategy
   - Maximum rollback time target (e.g., under 5 minutes)

5. **Recommend monitoring integration.** Connect deployment events to observability:
   - Deploy markers in APM tools (Datadog, New Relic, Grafana)
   - Automated alerts on error rate increase after deploy
   - Deployment frequency and lead time tracking

---

### Phase 4: VALIDATE -- Verify Pipeline Correctness

1. **Lint pipeline configuration.** Run syntax validation:
   - GitHub Actions: `actionlint` or YAML schema validation
   - GitLab CI: `gitlab-ci-lint` API endpoint
   - Jenkinsfile: Groovy syntax check
   - General: YAML structure validation for all config files

2. **Verify environment variable completeness.** For each environment:
   - All required variables are defined
   - No placeholder values remain (TODO, CHANGEME, xxx)
   - Variables referenced in code exist in the pipeline configuration

3. **Verify branch protection alignment.** Confirm that:
   - Production deploy pipelines only trigger from protected branches
   - Required status checks match the pipeline stages
   - Force-push is disabled on deployment branches

4. **Generate deployment readiness report.** Summarize findings:

   ```
   Deployment Readiness: [PASS/WARN/FAIL]

   Pipeline stages: 7/9 present (missing: security scan, smoke tests)
   Environment isolation: PASS
   Rollback procedure: WARN (documented but not automated)
   Secret hygiene: PASS
   Pipeline performance: 12m avg (recommend parallelizing test stages)

   Recommendations:
     1. Add SAST scan stage between build and deploy
     2. Add post-deploy smoke test stage
     3. Automate rollback on health check failure
   ```

5. **Present results.** Use `emit_interaction` to deliver the report and ask whether to proceed with implementing recommendations.

---

## Harness Integration

- **`harness check-deployment`** -- The mechanical gate. Verifies deployment readiness and exits per the four-value contract (`0` pass / `1` blocked / `2` error / `3` abstained). `--json` emits the full result; `--findings-json` emits the trailing findings-contract line. The skill invokes this command; it never reimplements the check.
- **`harness skill run harness-deployment`** -- Advisory invocation for the DETECT/ANALYZE/DESIGN/VALIDATE walkthrough used to fix what the gate reports.
- **`harness validate`** -- Run after any pipeline configuration changes to verify project health.
- **`harness check-deps`** -- Verify deployment script dependencies are available.
- **`emit_interaction`** -- Present deployment readiness report and gather decisions on strategy.
- **Rollback seam (pre-ship gate ↔ post-ship circuit breaker).** `harness check-deployment` verifies a rollback _path exists_ — pre-ship readiness answering "can we roll back?" It is satisfied by any of: a `rollback` block in `harness.config.json` (the circuit breaker is wired), a revert/rollback workflow or `deploy/rollback` script, or a documented rollback runbook. It never deploys and never merges a revert. Its complement, **`harness-rollback`**, executes post-ship: when a signal or evaluation fires, it opens a revert PR (propose-only; a human merges). The two are connected by the shared `rollback` config seam. On a `DEPLOY-RB001` block, point the human at `harness-rollback` to establish the missing post-ship path — but the gate's job is only to confirm the path exists before this merges.

## Success Criteria

- All CI/CD configuration files in the project are identified and cataloged
- Pipeline stage completeness is assessed against the standard checklist
- Environment isolation is verified with no cross-environment credential leakage
- A deployment strategy recommendation is provided with rationale
- Rollback procedures are documented or flagged as missing
- Pipeline lint passes without errors

## Examples

### Example: Node.js API with GitHub Actions

```
Phase 1: DETECT
  Found: .github/workflows/ci.yml, .github/workflows/deploy.yml
  Environments: staging (auto), production (manual dispatch)
  Strategy: Rolling (kubectl set image)
  Registry: ghcr.io/org/api-server

Phase 2: ANALYZE
  Missing stages: security scan, post-deploy smoke tests
  Environment isolation: PASS
  Secret hygiene: WARN -- AWS_ACCESS_KEY_ID used instead of OIDC
  Pipeline duration: 18m (test and lint run sequentially)

Phase 3: DESIGN
  Recommendation: Add trivy scan after Docker build
  Recommendation: Switch to AWS OIDC for keyless authentication
  Recommendation: Parallelize lint and test jobs (saves ~4m)
  Recommendation: Add smoke test job after deploy-staging

Phase 4: VALIDATE
  actionlint: PASS
  Environment variables: PASS
  Branch protection: WARN -- main branch allows force-push
  Result: WARN -- 3 recommendations, 1 security improvement needed
```

### Example: Python Service with GitLab CI and Canary Deploy

```
Phase 1: DETECT
  Found: .gitlab-ci.yml with 5 stages
  Environments: dev, staging, production
  Strategy: Canary (Istio VirtualService weight shifting)
  Registry: registry.gitlab.com/org/service

Phase 2: ANALYZE
  All 9 standard stages present
  Environment isolation: PASS
  Canary configuration: 5% -> 25% -> 75% -> 100% over 30 minutes
  Rollback: Automatic on 5xx rate > 1%

Phase 3: DESIGN
  Current strategy is well-configured. Minor recommendations:
  - Add canary duration metrics to Grafana dashboard
  - Add deployment event annotation to Prometheus
  - Consider adding a manual gate between 75% and 100%

Phase 4: VALIDATE
  GitLab CI lint: PASS
  Environment variables: PASS
  Branch protection: PASS
  Result: PASS -- pipeline is production-ready
```

## Gates

These are hard stops enforced mechanically by `harness check-deployment`. A hard violation is an error, not a warning — it maps to a non-zero exit and the skill does not hand-wave past it.

- **`harness check-deployment` is the authority.** Each hard rule maps to exit `1`. If the gate returns `1`, the deploy does not proceed until the finding is fixed and the gate clears. There is no "explain it away" path around a `1`.
- **`DEPLOY-SEC001` (leaked/long-lived credential) is non-waivable.** A hardcoded secret or long-lived cloud credential in a pipeline file or a committed env file blocks the deploy and cannot be downgraded by `deployment.rules`. Rotate the credential and remove the literal.
- **`DEPLOY-RB001` (no rollback path) blocks.** A detected deploy target with no rollback path (no `rollback` config, no revert/rollback workflow or script, no runbook) exits `1`. Wire the path before this merges; on a block, hand off to `harness-rollback` for the post-ship half.
- **`DEPLOY-ENV001` (direct-to-prod, no promotion gate) blocks.** A production deploy reachable with no approval/promotion gate exits `1`.
- **Abstention (exit `3`) is not a pass.** When the gate detects no deployment configuration it examines nothing and abstains loudly — never green. Confirm the repo genuinely does not deploy before treating a `3` as clear.
- **Error (exit `2`) is not a pass.** A malformed or unreadable `harness.config.json` exits `2`; fix the tooling before shipping. The gate never guesses config.
- **Overrides are per-rule, explicit, and reviewable.** Downgrading `DEPLOY-RB001`/`DEPLOY-ENV001` requires a `deployment.rules` entry with a rationale comment. Disabling the gate (`deployment.enabled: false`) is a visible opt-out, not a remediation.

## Evidence Requirements

When this skill makes claims about existing code, architecture, or behavior,
it MUST cite evidence using one of:

1. **File reference:** `file:line` format (e.g., `src/auth.ts:42`)
2. **Code pattern reference:** `file` with description (e.g., `src/utils/hash.ts` —
   "existing bcrypt wrapper")
3. **Test/command output:** Inline or referenced output from a test run or CLI command
4. **Session evidence:** Write to the `evidence` session section via `manage_state`

**Uncited claims:** Technical assertions without citations MUST be prefixed with
`[UNVERIFIED]`. Example: `[UNVERIFIED] The auth middleware supports refresh tokens`.

## Red Flags

### Universal

These apply to ALL skills. If you catch yourself doing any of these, STOP.

- **"I believe the codebase does X"** — Stop. Read the code and cite a file:line
  reference. Belief is not evidence.
- **"Let me recommend [pattern] for this"** without checking existing patterns — Stop.
  Search the codebase first. The project may already have a convention.
- **"While we're here, we should also [unrelated improvement]"** — Stop. Flag the idea
  but do not expand scope beyond the stated task.

### Domain-Specific

- **"Deploying without a health check endpoint"** — Stop. Without health checks, the orchestrator cannot detect failed deployments. Add health checks before deploying.
- **"Skipping canary deployment, it's a small change"** — Stop. Small changes cause outages too. Follow the deployment policy regardless of change size.
- **"Rolling back manually if something goes wrong"** — Stop. Manual rollback under incident pressure fails. Automate rollback before deploying.
- **"We can update the runbook after the deploy"** — Stop. If the deployment changes operational behavior, update the runbook first. Stale runbooks during incidents cause escalations.

## Rationalizations to Reject

### Universal

These reasoning patterns sound plausible but lead to bad outcomes. Reject them.

- **"It's probably fine"** — "Probably" is not evidence. Verify before asserting.
- **"This is best practice"** — Best practice in what context? Cite the source and
  confirm it applies to this codebase.
- **"We can fix it later"** — If it is worth flagging, it is worth documenting now
  with a concrete follow-up plan.

### Domain-Specific

| Rationalization                                                                      | Reality                                                                                                                                                                                                 |
| ------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| "The secret is only in a workflow file, not application code"                        | A leaked credential in a pipeline is a live secret. `DEPLOY-SEC001` is non-waivable — rotate it and remove the literal; `deployment.rules` cannot downgrade it.                                         |
| "We'll add a rollback path before we actually deploy"                                | The gate verifies the rollback path exists **now**. "Later" means an incident finds you with no revert route. Wire `rollback` config, a rollback workflow/script, or a runbook before this merges.      |
| "This repo only has one environment, so the promotion gate does not apply"           | Then downgrade `DEPLOY-ENV001` explicitly via `deployment.rules` with a comment — do not disable the whole gate. An unconfigured direct-to-prod path is exactly what `DEPLOY-ENV001` exists to catch.   |
| "The gate abstained, so we're clear to ship"                                         | Abstention (exit `3`) means the gate examined **nothing** — it is not a pass. Either the repo genuinely does not deploy, or detection missed the config. Confirm which before treating it as green.     |
| "Just set `enabled: false` to get the pipeline green"                                | Disabling the gate is an explicit opt-out that shows up in config review, not a fix. If one hard rule is wrong for this repo, downgrade that single rule with a rationale; do not blind the whole gate. |
| "The pipeline references `${{ secrets.X }}`, so `DEPLOY-SEC001` is a false positive" | An env-var _reference_ is not a leak and does not trip the rule — a _hardcoded_ value does. If the gate flagged it, the literal is real; do not suppress it, remove it.                                 |
| "It's just a config change, not a code change"                                       | Config changes cause outages at the same rate as code changes. The gate applies the same rigor and rollback requirement to them.                                                                        |
| "We tested this in staging"                                                          | Staging is not production. Traffic patterns, data volume, and edge cases differ. Staging success does not clear a hard gate finding.                                                                    |

## Escalation

- **When the CI/CD platform is unsupported:** Report which platform was detected and that analysis is limited to general best practices. Recommend the user provide platform-specific documentation for deeper analysis.
- **When secrets are found hardcoded in pipeline files:** Immediately flag as a critical finding. Do not proceed with strategy recommendations until secrets are remediated. Recommend rotating the exposed credentials.
- **When multiple deployment strategies are mixed across environments:** This is valid (e.g., rolling for staging, canary for production). Analyze each independently and verify the promotion workflow handles the strategy transition.
- **When pipeline configuration is generated by a tool (Terraform, Pulumi):** Analyze the generated output but note that fixes must be applied to the generator configuration, not the output files.