Audit Docs

---
name: audit-docs
description: Use when asked to audit documentation accuracy, coverage, or find documentation gaps.
disable-model-invocation: true
argument-hint: "[scope]"
allowed-tools:
  - Task
  - Read
  - Glob
  - Grep
  - Edit
  - Write
  - Bash(git:*)
arguments:
  - name: scope
    description: Audit scope (full|readme|file:<path>)
    required: false
  - name: fix
    description: Auto-apply fixable corrections without prompting
    required: false
metadata:
  short-description: Use when asked to audit documentation accuracy, coverage, or find documentation 
---

# Audit Docs

You are tasked with auditing project documentation for accuracy, completeness, and consistency with the codebase.

## Configuration

This command uses project configuration from `.ll/ll-config.json`:
- **Source directory**: `{{config.project.src_dir}}`

## Audit Scopes

- **full**: Audit all documentation files
- **readme**: Focus on README.md and linked docs
- **file:<path>**: Audit specific documentation file
- **dir:<path>** (or a bare directory path, e.g. `docs/guides/`): Audit every markdown file under that directory

## Context Budget (IMPORTANT)

This skill MUST NOT read documentation file bodies into the orchestrator
context. At `full`, `dir:`, or any multi-file scope, reading every file plus
every codebase verification search into one window overflows the context limit
(this is the failure this design exists to prevent).

Instead, the orchestrator **discovers** the file list and **fans out** one
audit subagent per file (batched, in parallel). Each subagent reads its file
and runs codebase verification *in its own context*, returning only a compact
structured findings list. The orchestrator aggregates findings — it never reads
a doc body or runs verification searches itself. This mirrors the wave-based
subagent architecture in `audit-claude-config`.

## Process

### 1. Find Documentation Files

Discover the file list **only** — do not read file bodies here.

`full` and `dir:`/bare-path scopes must exclude non-documentation
directories that happen to contain markdown — issue-tracker files
(`.issues/`), planning notes (`thoughts/`), loop/runtime artifacts
(`.loops/`, `.ll/`, `logs/`, `.pytest_cache/`, `.demo/`), and Claude Code
plugin components (`skills/`, `commands/`, `agents/`, `hooks/` — these are
commands and agent definitions, not documentation), plus vendored
dependencies (`node_modules/`, `.venv/`, `venv/`, anywhere in the tree, not
just at the repo root). Skipping this exclusion list is the single biggest
cause of this skill overflowing context or exhausting concurrent-agent
limits: in a repo with a large issue backlog, `full` scope can otherwise
discover thousands of files instead of dozens.

```bash
SCOPE="${scope:-readme}"

DOC_PRUNE=( -not -path "*/.git/*" -not -path "*/node_modules/*" \
    -not -path "*/.venv/*" -not -path "*/venv/*" \
    -not -path "./.issues/*" -not -path "./thoughts/*" \
    -not -path "./.loops/*" -not -path "./.ll/*" -not -path "./logs/*" \
    -not -path "./.pytest_cache/*" -not -path "./.demo/*" \
    -not -path "./skills/*" -not -path "./commands/*" \
    -not -path "./agents/*" -not -path "./hooks/*" )

case "$SCOPE" in
    full)
        # Find all documentation markdown files (excludes issues/thoughts/
        # loop artifacts/plugin components/vendored deps — see above)
        find . -name "*.md" "${DOC_PRUNE[@]}"
        ;;
    readme)
        # Start with README, follow links
        echo "README.md"
        ;;
    file:*)
        # Specific file
        echo "${SCOPE#file:}"
        ;;
    dir:*)
        # All markdown under a directory (same doc-only excludes as `full`)
        find "${SCOPE#dir:}" -name "*.md" "${DOC_PRUNE[@]}"
        ;;
    */|*/*)
        # Bare directory path (e.g. docs/guides/)
        find "${SCOPE%/}" -name "*.md" "${DOC_PRUNE[@]}"
        ;;
esac
```

**Large-scope guard**: if the discovered file count exceeds 30, do not
proceed directly to Phase 2. Show the user the count and use
`AskUserQuestion` (single-select) to confirm:
- Question: "Discovered N documentation files for this audit — auditing all
  of them will spawn many subagent batches. Proceed?"
- Options:
  - "Proceed with all N files"
  - "Narrow scope" — stop here and suggest the user re-run with a smaller
    `dir:<subpath>` scope
Only continue to Phase 2 after the user confirms.

### 2. Audit Each Document (Fan Out to Subagents)

**Do not read the files yourself.** For each discovered file, spawn a
`codebase-analyzer` subagent via the `Task` tool with `run_in_background: false`.
For a single-file scope (`readme`, `file:`), one subagent is fine.

For multi-file scopes, this is a **required sequential batch loop, not
optional guidance**:
1. Split the discovered file list into fixed batches of **at most 6 files**.
2. Send **one batch's worth of `Task` calls in a single message** (a message
   with multiple `Task` calls runs them concurrently — that's the
   parallelism). Never put more than 6 `Task` calls in one message.
3. **Wait for every result in that batch to return** before sending the
   next batch's `Task` calls. Never send a new batch while a previous
   batch's results are still outstanding.
4. Repeat until all discovered files have been audited.

This bounds concurrent-agent usage to 6 at a time regardless of total file
count, which is what keeps `full`/`dir:` scopes from exhausting the
concurrent-agent API limit.

Give each subagent this verbatim assignment (substitute `<FILE>`):

> Audit the documentation file `<FILE>` for accuracy against this codebase.
> Read the file, then verify its claims by reading/grepping the actual code,
> file paths, command syntax, config keys, and version numbers it references.
> Check the dimensions below. **Return ONLY a compact findings list** — for each
> finding: `file:line`, dimension, severity (high/med/low), a one-line
> description, and (when mechanically fixable) the exact `old → new` text.
> Do not return the file contents or your search transcript.
>
> Dimensions to check:
> - **Accuracy**: code examples run, file paths exist, API references match
>   actual code, version numbers current, command examples work.
> - **Completeness**: public APIs documented, install/usage/config/error
>   handling covered.
> - **Consistency**: terminology, formatting conventions, links resolve,
>   images accessible.
> - **Currency**: no deprecated info, reflects latest features, version
>   requirements accurate.
> - **Audience** (files under `docs/guides/`, `docs/reference/`, and
>   `README.md` only): the reader is a little-loops *end user* whose own
>   project consumes little-loops via `pip install little-loops` + `ll-init`;
>   their project is not little-loops. Flag any sentence that assumes the
>   reader edits the little-loops source, or that cites little-loops' own
>   test paths or lint/test commands as examples (see CONTRIBUTING.md
>   § Documentation Audience for the banned phrasings).

Where a subagent reports a runnable code block worth executing, it should test
it in its own context (`python -c "..."`, `bash -n`) rather than returning the
block for the orchestrator to run.

### 3. Collect Findings

Merge the structured findings returned by all subagents into a single list.
The orchestrator holds only these compact findings — never the file bodies — so
multi-file scopes stay within the context budget.

### 4. Output Report

Generate a comprehensive audit report using the format defined in [templates.md](templates.md) (see "Audit Report Format" section).

### 4.5. Direct Fix Option

After generating the report, classify each finding and offer direct fixes for auto-fixable items.

#### Finding Classification

Classify each finding from the report using the classification table in [templates.md](templates.md):

| Category | Criteria | Examples |
|----------|----------|----------|
| **Auto-fixable** | Specific old/new content known, mechanical replacement | Wrong counts, outdated paths, broken relative links, incorrect version numbers, wrong command syntax |
| **Needs issue** | Requires investigation, writing, or design decisions | Missing sections, incomplete docs, content rewrites, new examples needed |

#### Action Selection

**If `--fix` flag is set**: Skip the prompt. Auto-apply all auto-fixable corrections and output progress:

```
Applying auto-fixes...
Fix 1/N: [description] in [file:line]... done
Fix 2/N: [description] in [file:line]... done
...
Applied: X fixes
Remaining: Y findings (need issue tracking)
```

Then proceed to Phase 5 with only the non-fixable findings.

**Otherwise**: Present findings grouped by fixability using the format in [templates.md](templates.md) (see "Auto-Fixable Findings Format" section).

Use the AskUserQuestion tool with single-select:
- Question: "How would you like to handle the auto-fixable findings?"
- Header: "Doc fixes"
- Options:
  - label: "Fix all now"
    description: "Apply all N auto-fixable corrections directly to the documentation files"
  - label: "Create issues for all"
    description: "Skip direct fixes — create issues for all findings (auto-fixable and non-fixable)"
  - label: "Review each"
    description: "Decide per-finding whether to fix now, create issue, or skip"

If there are no auto-fixable findings, skip this phase and proceed directly to Phase 5 with all findings.

#### Fix All Now

For each auto-fixable finding:
1. Apply the edit using the Edit tool (old_string → new_string)
2. Report: `Fixed: [description] in [file:line]`

After all fixes applied:
```bash
git add [fixed files]
```

Output:
```
Direct fixes applied: N
- [file:line]: [description]
- [file:line]: [description]

Files staged. Run `/ll:commit` to commit, or continue to create issues for remaining findings.
```

Proceed to Phase 5 with only the non-fixable findings (skip issue management entirely if no non-fixable findings remain).

#### Create Issues for All

Skip direct fixes. Proceed to Phase 5 with all findings (both auto-fixable and non-fixable).

#### Review Each

For each auto-fixable finding, use the AskUserQuestion tool with single-select:
- Question: "Finding: [description] in [file:line]. Old: `[old]` → New: `[new]`"
- Header: "[file]:[line]"
- Options:
  - label: "Fix now"
    description: "Apply this correction directly"
  - label: "Create issue"
    description: "Create an issue for this finding instead"
  - label: "Skip"
    description: "Ignore this finding"

Apply fixes for "Fix now" selections, collect "Create issue" selections for Phase 5, discard "Skip" selections.

After review:
```bash
git add [fixed files]
```

Proceed to Phase 5 with findings marked "Create issue" plus all non-fixable findings.

### 5. Issue Management

**Note**: If findings were fixed directly in Phase 4.5, only the remaining unfixed findings are processed in this phase. If all findings were fixed directly, skip to Phase 8's summary output.

After generating the report, offer to create, update, or reopen issues for documentation problems.

#### Finding-to-Issue Mapping

Use the mapping table defined in [templates.md](templates.md) (see "Finding-to-Issue Mapping" section).

#### Deduplication

Before creating issues, search for existing issues that cover the same problem:

1. **Search active issues** by file path:
   ```bash
   # Search for issues mentioning the doc file
   grep -r "README.md" .issues/bugs/ .issues/enhancements/
   ```

2. **Search completed issues** for potential reopen:
   ```bash
   # Check if this was previously fixed and regressed (status: done, any type dir)
   ll-issues list --status done --json | python3 -c "
import json, sys
for i in json.load(sys.stdin):
    print(i['path'])
" | xargs grep -l "README.md"
   ```

3. **Match criteria**:
   - Same documentation file
   - Same type of issue (accuracy vs completeness)
   - Similar line numbers or sections

#### Deduplication Actions

| Match Found | Location | Action |
|-------------|----------|--------|
| High confidence match | Active issue | **Update** existing issue with new context |
| High confidence match | Completed | **Reopen** if problem recurred |
| Low/no match | - | **Create** new issue |

#### Issue File Format

Use the issue file template defined in [templates.md](templates.md) (see "Issue File Template" section).

### 6. Reopen Logic

If a completed issue matches a new finding:

1. **Verify it's the same problem**:
   - Same doc file
   - Same section or similar content
   - Problem has actually recurred (not just similar wording)

2. **Reopen it** (flip status back to `open` in place — issues never move directories):
   ```bash
   ll-issues set-status P2-BUG-XXX open
   ```

3. **Append Reopened section** using the template in [templates.md](templates.md) (see "Reopened Section Template").

### 7. User Approval

Present a summary before making any changes using the format in [templates.md](templates.md) (see "Proposed Issue Changes Format" section).

Use the AskUserQuestion tool with single-select:
- Question: "Proceed with issue changes?"
- Options:
  - "Create all" - Create/update/reopen all listed issues
  - "Skip" - Keep report only, no issue changes
  - "Select items" - Choose specific items to process

Wait for user selection before modifying any files.

### 8. Execute Issue Changes

After approval:

1. **Create new issues** in appropriate directories
2. **Update existing issues** by appending audit results section
3. **Reopen completed issues** by moving and appending Reopened section
4. **Stage changes**: stage only the issue files created, updated, or reopened above,
   by their explicit paths. Do **not** `git add .issues/` — a directory-level stage
   sweeps in unrelated untracked/modified files (BUG-1976).
   ```bash
   git add "<each created/updated/reopened issue-file-path>"
   ```
5. **Output summary**:
   ```
   Audit complete:
   - Fixed directly: N findings
   - Created: N issues (N BUG, N ENH)
   - Updated: N issues
   - Reopened: N issues
   - Skipped: N findings

   Run `/ll:commit` to commit these changes.
   ```

---

## Arguments

$ARGUMENTS

- **scope** (optional, default: `readme`): What to audit
  - `full` - All documentation
  - `readme` - README and linked docs
  - `file:<path>` - Specific file
  - `dir:<path>` or a bare directory path (e.g. `docs/guides/`) - All markdown under a directory

- **--fix** (optional, flag): Automatically apply all auto-fixable corrections without prompting. Skips the action selection prompt for fixable items and applies them directly. Non-fixable findings still flow to issue management.

---

## Examples

```bash
# Audit README and linked docs
/ll:audit-docs

# Full documentation audit
/ll:audit-docs full

# Audit specific file
/ll:audit-docs file:docs/api.md

# Audit every markdown file under a directory (fans out to subagents)
/ll:audit-docs docs/guides/

# Auto-fix documentation issues
/ll:audit-docs --fix

# Full audit with auto-fix
/ll:audit-docs full --fix
```

---

## Integration

After auditing:
1. Review the audit report
2. **Fix directly** auto-fixable issues (counts, paths, links) or create issues
3. **Manage issues** (create/update/reopen) for remaining findings with user approval
4. Use `/ll:commit` to save all changes (direct fixes + issue files)

Works well with:
- `/ll:scan-codebase` - May find related code issues
- `/ll:verify-issues` - Validate existing doc-related issues
- `/ll:manage-issue` - Process created documentation issues

---

## Output Evidence Contract (verbatim-output rule)

When this skill emits an audit finding, verdict, or scorecard, cite evidence
verbatim rather than re-summarizing — quoting is cheaper than paraphrasing and
keeps the audit auditable:

IMPORTANT: For each condition you evaluate:
1. State your verdict: Yes / No / Partial
2. Provide a VERBATIM quote from the output that supports your verdict (exact text, in quotes)
3. If you cannot quote specific text, your verdict is automatically No (or Partial if context suggests partial progress)

Do not assert a verdict without evidence. "The task appears complete" is not evidence.