416 Skill Lifecycle Audit A555057e

# Skill Lifecycle Audit — Research Document

Deep semantic validation of how skills interconnect, what they load, what they produce, and whether the resulting call graph forms a coherent, non-contradictory workflow.

This document describes audit dimensions and detection strategies for future implementation. It is not an executable skill.

---

## What This Audit Does

Given a plugin (or a single skill within a plugin), trace every outbound reference — skills loaded, agents delegated to, commands invoked, data consumed, data produced — and evaluate whether the resulting graph is **coherent, complete, and non-redundant**.

This is NOT structural validation (that's what `NamespaceReferenceValidator` and `/plugin-creator:assessor` do). This is **semantic validation** — does the wiring make sense given what each component claims to do?

---

## Audit Dimensions

### 1. Call Chain Completeness

For each skill, answer:

- What does this skill claim to do? (from its description and body)
- What other skills/agents/commands does it invoke to accomplish that?
- For each invoked component: does that component's description and capability actually support what the caller needs from it?
- Are there steps in the skill's workflow that require capabilities not provided by any loaded component?

**Output**: A call graph per skill with annotations: `SUPPORTED` (target provides what caller needs), `PARTIAL` (target provides some but not all), `UNSUPPORTED` (target doesn't do what caller expects), `MISSING` (no target loaded for this step).

### 2. Bidirectional Lifecycle Coherence

For each skill that participates in a domain (e.g., "Python code quality"):

- Which other skills in the same domain reference it?
- Which other skills SHOULD reference it but don't?
- Is there a skill that produces output that this skill consumes, but the producer doesn't know about this consumer?

**Example**: `stinkysnake` identifies code smells and delegates remediation. Does the remediation agent (`python-code-reviewer`) know to invoke `stinkysnake` or `snakepolish` when doing code quality review? If not, the lifecycle has a gap — the reviewer can't trigger the smell detection workflow.

**Output**: Bidirectional reference matrix per domain. Flag one-directional references where bidirectional is expected.

### 3. Circular Loading Detection

Trace all `Skill(command:)` and `Skill(skill=)` references to build a directed graph. Identify:

- Direct cycles: A loads B, B loads A
- Indirect cycles: A loads B, B loads C, C loads A
- Self-references that could cause infinite recursion during skill activation

**Output**: List of cycles found, with the full chain.

### 4. Duplicated Datasets

Scan all skill bodies and reference files for:

- Identical or near-identical content blocks appearing in multiple skills (e.g., the same configuration template, the same list of rules, the same decision tree)
- Content that could be extracted into a shared reference file loaded by multiple skills
- Copy-paste indicators: identical code fences, identical tables, identical instruction blocks

**Threshold**: Content blocks >10 lines that appear in 2+ skills with >80% similarity.

**Output**: List of duplicated blocks with source locations and a recommendation for extraction into a shared reference.

### 5. Instruction Contradictions

Compare instructions across skills that operate in the same domain:

- Identify pairs of instructions that give opposing guidance
- Check whether contradictions are guarded by conditions (e.g., "for Python 3.9" vs "for Python 3.11+")
- Flag unguarded contradictions as errors

**Examples of guarded contradictions (acceptable)**:
- Skill A: "Use `List[str]` type hints" (within a Python 3.9 compatibility section)
- Skill B: "Use `list[str]` type hints" (within a Python 3.11+ section)

**Examples of unguarded contradictions (errors)**:
- Skill A: "Always use `Dict[str, Any]` for type hints"
- Skill B: "Use `dict[str, Any]` for type hints"
- Neither skill specifies a Python version condition

**Output**: List of contradiction pairs with file:line references and whether they are guarded or unguarded.

### 6. Scriptable Command Sequences

Identify patterns where a skill instructs the AI to:

1. Run a shell command
2. Parse the output
3. Make a decision based on the output
4. Run another command with arguments derived from step 3

These multi-step command sequences are candidates for wrapping in a Python script that:
- Handles argument validation
- Performs environment checks (tool availability, correct versions)
- Chains the steps with proper error handling
- Produces structured output the AI can consume directly

**Output**: List of scriptable sequences with the steps involved, the skill file:line where they appear, and the estimated complexity reduction from scripting.

### 7. Self-Referential Pattern Learning

As the audit discovers issues:

1. Classify each issue by the **detection pattern** that found it (e.g., "one-directional domain reference", "unguarded Python version contradiction", "duplicated pyproject.toml template")
2. Record the pattern with a description and example
3. Re-scan all already-audited skills using the newly discovered pattern
4. Repeat until no new patterns emerge

**Output**: A `patterns.md` file listing all discovered patterns, each with: pattern name, description, detection heuristic, and all instances found.

---

## Recursive Depth Strategy

Not all issues require the same depth of analysis. The audit uses a tiered approach:

### Tier 1: Structural Scan (fast, automated)

- Build call graph from regex extraction of Skill/Task/@ references
- Detect cycles, missing targets, one-directional references
- Identify duplicated content blocks via text similarity

### Tier 2: Semantic Analysis (AI reasoning required)

- Read skill descriptions and bodies to understand intent
- Evaluate whether invoked components actually provide needed capabilities
- Detect instruction contradictions with context awareness

### Tier 3: Specialist Delegation (for specific issue types)

When Tier 2 identifies a potential problem that requires domain expertise:

- **Contradictions in Python typing advice** → delegate to an agent that reads the relevant PEPs and Python version docs
- **Circular dependency that might be intentional** → delegate to an agent that evaluates whether the cycle serves a valid recursive workflow pattern
- **Duplicated content that might be intentionally different** → delegate to an agent that does fine-grained diff analysis

Each specialist delegation returns a verdict: `CONFIRMED_ISSUE`, `FALSE_POSITIVE`, or `NEEDS_HUMAN_REVIEW`.

---

## Output Artifacts

The audit produces:

1. **`audit-report-{plugin-slug}.md`** — Full findings organized by dimension
2. **`call-graph-{plugin-slug}.md`** — Visual call graph (mermaid or ASCII)
3. **`patterns.md`** — Self-referential pattern catalog with detection heuristics
4. **`recommendations.md`** — Prioritized list of actionable fixes

Each finding includes:
- File path and line number
- The specific text that triggered the finding
- The audit dimension that detected it
- Severity: `error` (broken workflow), `warning` (incomplete workflow), `info` (optimization opportunity)
- Whether the finding was confirmed by a specialist delegation

---

## What This Audit Does NOT Do

- Does NOT fix issues (that's a separate workflow)
- Does NOT validate namespace references resolve to files (that's `NamespaceReferenceValidator` in the pre-commit hook)
- Does NOT validate frontmatter schema (that's `plugin_validator.py`)
- Does NOT replace `/plugin-creator:assessor` (which does structural assessment for refactoring planning)

This audit answers: "Given that all the files exist and all the references resolve, does the resulting system actually work as a coherent whole?"