Research

---
date: 2026-03-16T12:00:00Z
researcher: claude
branch: main
repository: ai-toolbox
topic: "QA Skill for cc-plugin/base — Design Research"
tags: [research, qa, skill, cc-plugin, desplega, testing, verification]
status: complete
autonomy: critical
last_updated: 2026-03-16
last_updated_by: claude
---

# QA Skill for cc-plugin/base — Design Research

## Research Question

How should a QA skill be designed for `cc-plugin/base/` that:
1. Fits into the existing skill chain (brainstorm → research → plan → implement → verify → **qa**)
2. Creates files in `thoughts/.../qa/YYYY-MM-DD-<name>.md`
3. Supports a clear template with references (images, links, videos)
4. Works standalone but also integrates naturally after planning/verification

## Summary

The base plugin (`cc-plugin/base/`) has 8 skills: brainstorming → researching → planning → tdd-planning → implementing → phase-running → verifying → reviewing. There is **no QA skill** today. The natural insertion point is as a **standalone skill** that can be invoked at any point but is most naturally offered after `verify-plan` or `review`, and can also be invoked independently. The `validate-thoughts.py` hook currently **blocks** writes to any subdirectory other than `research/`, `plans/`, and `brainstorms/` — it must be updated to allow `qa/`.

## Detailed Findings

### 1. Current Skill Chain & Flow

The existing skills chain via handoff prompts at the end of each skill:

```
brainstorm → research → plan → implement ──→ verify → review
                         ↓         ↑
                    tdd-plan    phase-running
                         ↓      (executes individual
                    implement    phases in background)
```

Note: `phase-running` is a sub-agent skill invoked by `implementing` to execute individual phases in background mode. It doesn't appear in the linear chain but is a distinct skill with its own `SKILL.md`.

Each skill ends with an AskUserQuestion offering the next logical step. The proposed QA skill would add:

```
... → verify → review → qa (standalone)
                ↓
plan → qa (pre-implementation verification)
```

**File**: `cc-plugin/base/skills/verifying/SKILL.md:142`
After verification, the skill offers `/review`. The QA skill could be offered here too, or from review's handoff.

**File**: `cc-plugin/base/skills/reviewing/SKILL.md:176-198`
The reviewing skill already has type-specific handoffs based on document type. A QA handoff could be added for plans and post-verification contexts.

### 2. Existing Skill Pattern (Consistent Across All Skills)

Every skill follows this structure in `SKILL.md`:

| Section | Purpose |
|---------|---------|
| Frontmatter | `name`, `description` |
| Working Agreement | AskUserQuestion primary, establish preferences upfront, autonomy modes |
| User Preferences | File-review preference + skill-specific preferences |
| When to Use | Trigger conditions |
| Autonomy Mode | Table of Autopilot/Critical/Verbose behaviors |
| Process Steps | Numbered steps with detailed instructions |
| Review Integration | Optional file-review at end |
| Workflow Handoff | AskUserQuestion with next-step options |

Every skill also has a `template.md` defining the output document structure with YAML frontmatter.

### 3. validate-thoughts.py Hook — Blocking Issue

**File**: `cc-plugin/base/hooks/validate-thoughts.py:29-61`

The PreToolUse hook validates Write/Edit to `thoughts/` directories against exactly three patterns:
- `thoughts/[^/]+/research/YYYY-MM-DD-*.md`
- `thoughts/[^/]+/plans/YYYY-MM-DD-*.md`
- `thoughts/[^/]+/brainstorms/YYYY-MM-DD-*.md`

Any other subdirectory (including `qa/`) gets blocked with: `"Invalid thoughts subdirectory."` (line 57).

**Required change**: Add a `qa_pattern` and corresponding `elif "/qa/" in file_path:` branch.

### 4. Thoughts Directory Structure

Current structure:
```
thoughts/
├── shared/
│   ├── plans/
│   └── research/
└── taras/
    ├── brainstorms/
    ├── plans/
    └── research/
```

No `qa/` subdirectory exists. It would be created as:
```
thoughts/
├── shared/
│   ├── qa/          ← new
│   └── ...
└── taras/
    ├── qa/          ← new
    └── ...
```

### 5. Cross-Referencing Patterns

Documents reference each other in two ways:

**Frontmatter links** (structured):
```yaml
source_research: thoughts/taras/research/2026-03-11-topic.md
source_brainstorm: thoughts/taras/brainstorms/2026-03-11-topic.md
source_plan: thoughts/taras/plans/2026-03-11-topic.md
```

**Body text links** (inline markdown):
```markdown
- Research: `thoughts/shared/research/2026-01-08-wts-cli-design.md`
```

A QA document should support both patterns, adding fields like `source_plan:` and `source_verification:` in frontmatter.

### 6. Reference Support in Existing Documents

Current documents use markdown links and code references but **no images, videos, or external media**. The QA template would be the first to formally support rich references:

- **Images**: Screenshots of UI, test results, error states → `![description](path-or-url)`
- **Links**: External URLs (Sentry, Grafana, Linear, PRs) → `[description](url)`
- **Videos**: Loom/screen recordings → `[Video: description](url)`
- **Files**: Related code, logs, configs → `` `path/to/file:line` ``

This is a natural extension — markdown already supports all these; the template just needs to formalize the pattern.

### 7. Reviewing Skill — Would Need QA Type Support

**File**: `cc-plugin/base/skills/reviewing/SKILL.md:63-70`

The reviewing skill classifies documents by path (`/research/`, `/plans/`, `/brainstorms/`) and applies type-specific quality checks. Adding QA would require:

1. Adding `/qa/` to the path classification table
2. Defining QA-specific structural requirements (required sections)
3. Defining QA-specific content quality criteria
4. Adding a QA handoff option in the workflow handoff section

### 8. Distinction: QA vs Verify vs Review

The three skills serve different purposes:

| Skill | Focus | When |
|-------|-------|------|
| **Verify** | Automated audit: checkboxes, git diff, success criteria re-run | Pre-QA (confirm plan items done) and post-QA (re-confirm after manual testing) |
| **Review** | Structural critique: gaps, quality, completeness | Against any document, including QA reports |
| **QA** (proposed) | Functional validation: does it actually work? Manual/exploratory testing checklist with evidence | After implementation or standalone |

Verify works **both pre and post QA** — before QA it confirms the automated checkboxes and plan alignment are in order; after QA it can re-run checks to confirm nothing regressed during manual testing. The updated flow:

```
implement → verify (pre-QA) → qa → verify (post-QA, optional) → review
```

Key differentiator: **QA captures test evidence** (screenshots, recordings, links to test runs) while verify/review are purely analytical. QA is the "prove it works" step.

## Architecture Decisions

### Decision 1: QA as Standalone Skill (not a verify extension)

**Rationale**: Verify is tightly coupled to plans (checkbox audit, git diff correlation, success criteria re-run). QA is broader — it can verify a feature, a bugfix, a deployment, or anything that needs functional validation. Making it standalone keeps both skills focused.

**Skill Shape** — The QA skill (`cc-plugin/base/skills/qa/SKILL.md`) would follow the standard pattern:

1. **Working Agreement**: AskUserQuestion primary, autonomy modes, establish preferences upfront
2. **User Preferences** (upfront):
   - **QA approach**: Manual-only / Browser automation (qa-use) / Mixed
   - **File review preference**: Use file-review for inline feedback on QA report?
3. **Process Steps**:
   - **Step 1: Context & Scope** — Read source plan/PR/issue, determine what to test, create QA document from template
   - **Step 2: Test Case Design** — Define test cases (happy path, edge cases, error scenarios). If browser automation selected, design qa-use test steps
   - **Step 3: Execute Tests** — For browser automation: use `qa-use` CLI (create session → navigate → snapshot → interact → screenshot). For manual: guide user through steps, collect evidence
   - **Step 4: Capture Evidence** — Screenshots via `qa-use browser screenshot`, videos via session recording URL, logs via `qa-use browser logs console/network`
   - **Step 5: Record Results** — Update QA document with actual results, evidence links, pass/fail per test case
   - **Step 6: Verdict** — Aggregate results into overall PASS/FAIL/BLOCKED verdict
   - **Step 7: Persist QA Knowledge** — If the project's `CLAUDE.md` or `agents.md` doesn't document how QA is done, offer to add a QA section describing the testing approach, tools used, and common test patterns. Also persist to memory if patterns emerge across QA sessions.
4. **Workflow Handoff**: Offer verify (post-QA) → review → done

**Browser Automation Default: qa-use** — The skill uses `qa-use` (from the `desplega-ai/qa-use` plugin) as the default browser automation tool. This can be overridden by project-specific instructions in `CLAUDE.md` (e.g., a project might prefer Playwright MCP, Cypress, or manual-only testing). The skill checks:
1. Is `qa-use` available? (check for `qa-use:*` commands in available skills)
2. Does `CLAUDE.md` specify a different QA/testing tool?
3. If neither, fall back to manual test case execution with user guidance

### Decision 2: Phase-Level QA Specs in Plans (Integrated QA)

**Rationale**: QA shouldn't only happen after all implementation is done. By embedding optional QA specs into each plan phase, issues are caught at phase boundaries rather than after Phase 5 breaks what Phase 1 built. This makes QA a planning concern, not an afterthought.

**How it works**:

The **planning skill** gains a new optional section per phase:

```markdown
### QA Spec (optional):

**Approach:** browser-automation | manual | cli-verification
**Test Scenarios:**
- [ ] TC-1: [Scenario name]
  - Steps: [1. Navigate to X, 2. Click Y, 3. Verify Z]
  - Expected: [What should happen]
- [ ] TC-2: [Scenario name]
  - Steps: [...]
  - Expected: [...]
```

**When to include a QA spec**: Phases that change user-facing behavior, add UI, modify API responses, or alter auth/permissions. **When to omit**: Internal refactors, type annotation additions, config changes, dependency updates.

**The implementing skill** gains awareness of QA specs:
1. Complete phase implementation
2. Run automated verification (existing Success Criteria)
3. If phase has a QA spec → execute it (via qa-use or manual guidance)
4. Record QA results in the QA document (not the plan — the plan stays clean)
5. Only proceed to next phase when QA passes

**Phase-running agents** report QA spec status:
- `QA: passed` — all test scenarios verified
- `QA: pending` — has QA spec but needs human/browser testing (escalate to orchestrator)
- `QA: n/a` — no QA spec for this phase

**The standalone QA skill** aggregates all phase QA specs from the plan into a full QA document, adds exploratory testing on top, and produces the final QA report.

**Distinction from Success Criteria**:

| Section | Answers | Example |
|---------|---------|---------|
| **Automated Verification** | Does the code compile/pass/lint? | `bun run typecheck` |
| **Manual Verification** | Quick human sanity check | "Verify the button appears" |
| **QA Spec** (new) | Does the feature actually work E2E? | "Open browser, login, navigate to /settings, change email, verify confirmation email" |

### Decision 3: Template with Rich References Section

**Rationale**: QA is uniquely suited for visual evidence. Unlike research/plans which are primarily text, QA documents benefit from screenshots, recordings, and links to external monitoring. The template should have a dedicated `## Evidence` section with subsections for different media types.

### Decision 4: Hook Update Required

**Rationale**: The `validate-thoughts.py` hook will block any write to `thoughts/*/qa/`. This must be updated before the skill can create files. Simple addition — one regex pattern and one `elif` branch.

## QA Template Design (Proposed)

Based on the patterns observed across all existing templates, here's the proposed structure:

```yaml
---
date: [ISO format]
author: [Name]
topic: "[Feature/Bug/Scenario being validated]"
tags: [qa, relevant-tags]
status: [in-progress|pass|fail|blocked]
source_plan: [optional — path to plan that motivated this QA]
source_verification: [optional — path to verification report]
related_pr: [optional — PR URL or number]
environment: [optional — local|staging|production]
last_updated: [YYYY-MM-DD]
last_updated_by: [Name]
---

# [Feature/Scenario] — QA Report

## Context
[What is being validated and why. Link to plan, PR, or issue.]

## Scope
### In Scope
- [What this QA covers]

### Out of Scope
- [What this QA does NOT cover]

## Test Cases

### TC-1: [Test Case Name]
**Steps:**
1. [Step 1]
2. [Step 2]

**Expected Result:** [What should happen]
**Actual Result:** [What actually happened]
**Status:** pass | fail | blocked | skipped

### TC-2: [Test Case Name]
[Same structure]

## Edge Cases & Exploratory Testing
- [Findings from exploratory testing]

## Evidence

### Screenshots
- ![Description](path-or-url)

### Videos
- [Video: Description](url)

### Logs & Output
```
[Relevant log output]
```

### External Links
- [Sentry Issue](url)
- [Grafana Dashboard](url)
- [CI/CD Run](url)
- [PR](url)

## Issues Found
- [ ] [Issue description — severity: critical|major|minor]

## Verdict
**Status**: PASS | FAIL | BLOCKED
**Summary**: [1-2 sentence summary of QA outcome]

## References
- Plan: `thoughts/.../plans/YYYY-MM-DD-topic.md`
- Verification: `thoughts/.../plans/YYYY-MM-DD-verification-report.md`
- Related documents: [links]
```

## Implementation Scope

### Updated Skill Flow

Before (current):
```
brainstorm → research → plan ──→ implement → verify → review
                         ↓
                    tdd-plan ──→ implement → verify → review
```

After (with QA):
```
brainstorm → research → plan ──→ implement ─────────→ verify(pre) → qa → verify(post) → review
                         ↓           │ (per phase)                   ↑
                    tdd-plan         ├─→ qa (optional) ──────────────┘
                         ↓           │
                    implement ───────┘

                    qa (standalone) ← can also be invoked independently
                         │              for any feature/bugfix/deployment
                         ↓
                    verify(post) → review
```

Handoff integration points:
- **implement** adds: "Run QA for this phase? → `/qa`" after each phase with a QA spec (optional, per-phase)
- **verify** adds: "Run QA? → `/qa`" after verification report
- **review** adds: "Run QA? → `/qa`" for plan documents
- **qa** offers: "Run verify? → `/verify-plan`" and "Run review? → `/review`" at completion

### Files to Create/Modify

To add the QA skill to `cc-plugin/base/`, the following files need to be created/modified:

| Action | File | Description |
|--------|------|-------------|
| **Create** | `cc-plugin/base/skills/qa/SKILL.md` | QA skill definition (standalone + aggregation from plan phases) |
| **Create** | `cc-plugin/base/skills/qa/template.md` | QA document template with rich references |
| **Create** | `cc-plugin/base/commands/qa.md` | `/qa` command wrapper |
| **Modify** | `cc-plugin/base/hooks/validate-thoughts.py` | Add `qa/` as valid subdirectory |
| **Modify** | `cc-plugin/base/skills/planning/SKILL.md` | Add optional QA Spec section per phase |
| **Modify** | `cc-plugin/base/skills/planning/template.md` | Add QA Spec template block to phase template |
| **Modify** | `cc-plugin/base/skills/implementing/SKILL.md` | Execute phase QA specs after success criteria |
| **Modify** | `cc-plugin/base/skills/phase-running/SKILL.md` | Report QA spec status (passed/pending/n/a) |
| **Modify** | `cc-plugin/base/skills/reviewing/SKILL.md` | Add QA document type support |
| **Modify** | `cc-plugin/base/skills/verifying/SKILL.md` | Add QA handoff option |
| **Modify** | `cc-plugin/base/.claude-plugin/plugin.json` | Bump version (1.5.1 → 1.6.0) |

## Code References

| File | Lines | Relevance |
|------|-------|-----------|
| `cc-plugin/base/hooks/validate-thoughts.py` | 29-61 | Blocks writes to non-allowed `thoughts/` subdirectories; must add `qa/` pattern |
| `cc-plugin/base/skills/verifying/SKILL.md` | 142 | Handoff point — offers `/review` after verification; add QA option here |
| `cc-plugin/base/skills/reviewing/SKILL.md` | 176-198 | Type-specific handoffs by document type; add QA document type |
| `cc-plugin/base/skills/reviewing/SKILL.md` | 63-70 | Document classification by path; add `/qa/` to path table |
| `cc-plugin/base/skills/implementing/SKILL.md` | 94 | Invokes `phase-running` sub-agent; add QA spec awareness |
| `cc-plugin/base/.claude-plugin/plugin.json` | 4 | Current version `1.5.1`; bump to `1.6.0` |

## Open Questions

None — all resolved. See Resolved Questions below.

## Resolved Questions

1. **Should QA support automated test execution?** — **Yes, with self-documenting behavior.** The skill defines how QA works (test case design, execution, evidence capture), but a key part of the QA skill is **persisting how QA is done** as it’s being used — updating `CLAUDE.md`, `agents.md`, or memory with the testing approach, tools, and patterns discovered during QA sessions. This way the QA process improves over time per-project.

2. **Should the reviewing skill learn to review QA documents?** — **Yes.** The reviewing skill will be extended with QA-specific structural requirements (test cases present, evidence provided, verdict stated) and content quality criteria.

3. **Status values**: Using `in-progress|pass|fail|blocked` — combines the standard `in-progress` for ongoing sessions with QA-specific verdicts for completed reports.

## Review Errata

_Reviewed: 2026-03-16 by Claude_

### Critical
_(none)_

### Important
_(all resolved)_

### Resolved
- [x] **Skill count corrected**: Updated from "7 skills" to "8 skills", added `phase-running` to chain diagram with note about its sub-agent role
- [x] **Decision numbering fixed**: Renumbered from 1,4,2,3 to sequential 1,2,3,4
- [x] **Open Questions section added**: Added with "None — all resolved" note pointing to Resolved Questions
- [x] **Code References table added**: Consolidated all inline `file:line` references into a summary table
- [x] **Implementing handoff added** (file-review comment): Added `implement` as handoff integration point with per-phase optional QA, updated flow diagram to show per-phase QA path
- [x] Minor: "Resolved Questions" item 3 (status values) reads more like a design decision than a resolved question — noted but left as-is since it's borderline