Ci Test Setup

---
name: ci-test-setup
description: WAY 10 — make the generated playwright-bdd suites run in CI on every MR/PR, on the default branch, and on schedules, without Claude — GitLab CI, GitHub Actions, Azure Pipelines, or Bitbucket Pipelines. Use when the user says "ci setup", "add tests to ci", "run tests in pipeline", "set up qa pipeline". Copies the suite into the app repo and writes the CI config; never commits or pushes.
---

# WAY 10 — CI Test Setup

Tests are written to `./qa-artifacts/`, which is **gitignored**, so CI can't
see them. This WAY copies the runnable suite into the **application's**
repository and adds CI jobs for `QA_CODE_HOST` (see the `tracker` skill).

## Step 1: Gather inputs
- **App repo path**: a local clone, or `.` when running inside the app repo. Required
- CI system: from `QA_CODE_HOST` (`gitlab` → GitLab CI, `github` → GitHub Actions, `azure-devops` → Azure Pipelines, `bitbucket` → Bitbucket Pipelines). Confirm it with the user
- Projects to run: `e2e`, `api`, or both (default: whichever exist in `qa-artifacts/playwright.config.ts`)

## Phase 1: Export the suite → `<app-repo>/qa-tests/`
Copy from `qa-artifacts/`: `package.json`, `package-lock.json`, `playwright.config.ts`, `steps/`, `test-cases/`, `visual-baselines/`.

**Never copy:** `node_modules/`, `.features-gen/`, `runs/`, `api-responses/`, `.auth/` (saved sessions = credentials), `.env`.

Then check the exported config:
- `baseURL` must come from `process.env.BASE_URL` / `API_BASE_URL`. A hard-coded URL gets flagged
- `storageState: '.auth/…'` won't exist in CI. Flag it and propose a setup project (`auth.setup.ts`) that logs in with secret CI variables (`QA_USER`, `QA_PASSWORD`) and saves the state, so `e2e` gets `dependencies: ['setup']`. Write it only if the user agrees
- The runner template already writes `runs/<RUN_ID>/junit.xml` when `CI` is set. If the config came from elsewhere, add that reporter

## Phase 2: CI config
Read the exact `@playwright/test` version from `package-lock.json`. The Playwright image tag **must** match it (`mcr.microsoft.com/playwright:v<version>-noble`). Every template runs from `qa-tests/` with `RUN_ID=ci` and `CI=true`, and publishes `qa-tests/runs/` (JUnit + HTML + failure artifacts).

### GitLab CI → `.gitlab/ci/qa-tests.gitlab-ci.yml`
```yaml
# 🤖 Generated by DevOps QA Agent — review before relying on this test.
.qa-base:
  image: mcr.microsoft.com/playwright:v<version>-noble
  stage: test
  variables: { RUN_ID: ci }
  before_script: [cd qa-tests, npm ci, npx bddgen]
  artifacts:
    when: always
    expire_in: 14 days
    paths: [qa-tests/runs/]
    reports: { junit: qa-tests/runs/ci/junit.xml }
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
    - if: $CI_PIPELINE_SOURCE == "schedule"
qa-e2e: { extends: .qa-base, script: [npx playwright test --project=e2e] }
qa-api: { extends: .qa-base, script: [npx playwright test --project=api] }
```
Then add `include: [{ local: .gitlab/ci/qa-tests.gitlab-ci.yml }]` to `.gitlab-ci.yml` (and a `test` stage if it lists `stages` without one). Variables go in Settings → CI/CD → Variables, masked.

### GitHub Actions → `.github/workflows/qa-tests.yml`
```yaml
# 🤖 Generated by DevOps QA Agent — review before relying on this test.
name: qa-tests
on:
  pull_request:
  push: { branches: [main] }
  schedule: [{ cron: "0 5 * * 1-5" }]
  workflow_dispatch:
jobs:
  qa:
    runs-on: ubuntu-latest
    container: mcr.microsoft.com/playwright:v<version>-noble
    strategy: { fail-fast: false, matrix: { project: [e2e, api] } }
    defaults: { run: { working-directory: qa-tests } }
    env:
      RUN_ID: ci
      BASE_URL: ${{ vars.BASE_URL }}
      API_BASE_URL: ${{ vars.API_BASE_URL }}
      API_TEST_TOKEN: ${{ secrets.API_TEST_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npx bddgen
      - run: npx playwright test --project=${{ matrix.project }}
      - if: always()
        uses: actions/upload-artifact@v4
        with:
          name: qa-runs-${{ matrix.project }}
          path: qa-tests/runs/
          retention-days: 14
```
Set `main` to the real default branch. Values go in Settings → Secrets and variables → Actions (`BASE_URL`/`API_BASE_URL` as variables, tokens as secrets).

### Azure Pipelines → `qa-tests/azure-pipelines.qa.yml`
```yaml
# 🤖 Generated by DevOps QA Agent — review before relying on this test.
trigger: [main]
pr: [main]
schedules:
  - cron: "0 5 * * 1-5"
    branches: { include: [main] }
    always: true
pool: { vmImage: ubuntu-latest }
container: mcr.microsoft.com/playwright:v<version>-noble
strategy: { matrix: { e2e: { PROJECT: e2e }, api: { PROJECT: api } } }
variables: { RUN_ID: ci }
steps:
  - script: npm ci && npx bddgen && npx playwright test --project=$(PROJECT)
    workingDirectory: qa-tests
    env: { BASE_URL: $(BASE_URL), API_BASE_URL: $(API_BASE_URL), API_TEST_TOKEN: $(API_TEST_TOKEN) }
  - task: PublishTestResults@2
    condition: always()
    inputs: { testResultsFormat: JUnit, testResultsFiles: qa-tests/runs/ci/junit.xml }
  - publish: qa-tests/runs
    artifact: qa-runs-$(PROJECT)
    condition: always()
```
Create the pipeline from this file (Pipelines → New → Existing YAML). Set the variables there and mark `API_TEST_TOKEN` secret. Secret variables must be mapped through `env:`, as above.

### Bitbucket Pipelines → `bitbucket-pipelines.yml`
```yaml
# 🤖 Generated by DevOps QA Agent — review before relying on this test.
image: mcr.microsoft.com/playwright:v<version>-noble
definitions:
  steps:
    - step: &qa
        name: QA tests
        script:
          - cd qa-tests && npm ci && npx bddgen
          - RUN_ID=ci npx playwright test --project=e2e
          - RUN_ID=ci npx playwright test --project=api
        artifacts: [qa-tests/runs/**]
pipelines:
  pull-requests: { '**': [ { step: *qa } ] }
  branches: { main: [ { step: *qa } ] }
  custom: { qa-nightly: [ { step: *qa } ] }
```
If a `bitbucket-pipelines.yml` already exists, **merge** the `qa` step into it rather than overwriting. Bitbucket reads JUnit from `**/test-results/**`, so add `cp runs/ci/junit.xml test-results/` (after `mkdir -p test-results`) to surface results in the PR. Variables go in Repository settings → Repository variables (secured). Schedule `qa-nightly` there.

### Visual baselines (all CIs)
Screenshots rendered on Windows/macOS won't match the Linux image. Generate baselines in CI once (`npx playwright test --project=e2e --grep @visual --update-snapshots`, as a manual job), then commit the generated `qa-tests/visual-baselines/`.

## Phase 3: Validate
Use `ci.lint` from the code host's provider skill. GitLab has a real lint API. For the others, check the YAML parses and the keys match the template, and say that full lint was skipped (or run `actionlint` for GitHub if it's installed).

## Phase 4: Hand over (no commit, no push)
Print:
- the files written or changed (`git -C <app-repo> status --short`)
- the variables and secrets to create, with where they go for that CI: `BASE_URL`, `API_BASE_URL`, `API_TEST_TOKEN`, plus `QA_USER`/`QA_PASSWORD` if using a setup project
- the next steps: commit on a branch, open an MR/PR, and watch the QA job. `ci it` (WAY 5) explains failures, and the JUnit report names the `Test#<nn>`

Keeping the copy current: re-run this WAY after `write-tests`/`e2e-session` add tests. It overwrites `qa-tests/` from `qa-artifacts/`, after showing the diff.