Hallucination Guard

---
name: hallucination-guard
description: "Verify AI-generated imports, API calls, and packages actually exist. Use when finishing AI-generated code that introduces new imports or external API calls — auto-fires after fix/cook to catch phantom functions, non-existent packages, and slopsquatting attacks."
metadata:
  author: runedev
  version: "0.3.0"
  layer: L3
  model: sonnet
  group: validation
  tools: "Read, Bash, Glob, Grep"
---

# hallucination-guard

## Purpose

Post-generation validation that verifies AI-generated code references actually exist. Catches the 42% of AI code that contains hallucinated imports, non-existent packages, phantom functions, and incorrect API signatures. Also defends against "slopsquatting" — where attackers register package names that AI commonly hallucinates.

## Triggers

- Called by `cook` after code generation, before commit
- Called by `fix` after applying fixes
- Called by `preflight` as import verification sub-check
- Called by `review` during code review
- Auto-trigger: when new import statements are added to codebase

## Calls (outbound)

# Exception: L3→L3 coordination
- `research` (L3): verify package existence on npm/pypi

## Called By (inbound)

- `cook` (L1): after code generation, before commit
- `fix` (L2): after applying fixes
- `preflight` (L2): import verification sub-check
- `review` (L2): during code review
- `db` (L2): verify SQL syntax and ORM method calls are real
- `review-intake` (L2): verify imports in code submitted for review
- `skill-forge` (L2): verify imports in newly generated skill code
- `adversary` (L2): verify APIs/packages in plan actually exist
- `logic-guardian` (L2): verify referenced functions and imports after guarded logic edits

## Execution

### Step 1 — Extract imports

Use `Grep` to find all import/require/use statements in changed files:

```
Grep pattern: ^(import|require|use|from)\s
Files: changed files passed as input
Output mode: content
```

Collect every imported module name and file path. Separate into:
- Internal imports (start with `./`, `../`, `@/`, `~/`)
- External packages (bare module names)

### Step 2 — Verify internal imports

For each internal import path, use `Glob` to confirm the file exists in the codebase.

```
Glob pattern: <resolved import path>.*   (try .ts, .tsx, .js, .jsx, .py, .rs etc.)
```

If `Glob` returns no results → mark as **BLOCK** (file does not exist).

Also use `Grep` to verify that the specific exported name (function/class/const) exists in the resolved file:

```
Grep pattern: export (function|class|const|default) <name>
File: resolved file path
```

If export not found → mark as **WARN** (symbol may not be exported).

### Step 3 — Verify external packages (Dependency Check Before Import)

Before importing ANY third-party library, check the dependency manifest.

Use `Read` on the project's dependency manifest to confirm each external package is listed:

- JavaScript/TypeScript: `package.json` → check `dependencies` and `devDependencies`
- Python: `requirements.txt` or `pyproject.toml` → `[project.dependencies]` and `[project.optional-dependencies]`
- Rust: `Cargo.toml` → `[dependencies]` and `[dev-dependencies]`

**Pre-import gate** (BEFORE writing import statements, not just after):
1. If the agent is ABOUT to import a package → check manifest FIRST
2. If package is NOT in manifest → output install command before writing the import:
   ```
   ⚠ Package '<name>' not in dependencies. Install first:
     npm install <name>        # JS/TS
     pip install <name>        # Python
     cargo add <name>          # Rust
   ```
3. If package IS in manifest → proceed with import

**Post-import verification** (after code is written):
- If package is **not listed** in the manifest → mark as **BLOCK** (phantom dependency)
- If package is listed but not installed (no lockfile entry) → mark as **WARN** (not yet installed)

Also check for typosquatting: if package name has edit distance ≤ 2 from a known popular package (axios/axois, lodash/lodahs, react/recat), mark as **SUSPICIOUS**.

### Step 3.5 — Slopsquatting Registry Verification

<HARD-GATE>
Any NEW package added to the manifest (not previously in the lockfile) MUST be verified against the actual registry.
AI agents hallucinate package names at high rates. A package that doesn't exist on npm/PyPI/crates.io = supply chain risk.
</HARD-GATE>

For each NEW external package (present in manifest but absent from lockfile):

**3.5a. Registry existence check:**
```
JavaScript: Bash: npm view <package-name> version 2>/dev/null
Python:     Bash: pip index versions <package-name> 2>/dev/null
Rust:       Bash: cargo search <package-name> --limit 1 2>/dev/null
```

If command returns empty/error → **BLOCK** (package does not exist on registry — likely hallucinated name).

**3.5b. Popularity check (slopsquatting defense):**
```
JavaScript: Bash: npm view <package-name> 'dist-tags.latest' 'time.modified' 2>/dev/null
→ If last modified > 2 years ago AND weekly downloads < 100: SUSPICIOUS
Python:     Use rune:research to check PyPI page for download stats
```

Low-popularity packages with names similar to popular ones = **SUSPICIOUS** (potential slopsquatting attack).

**3.5c. Known slopsquatting patterns:**
```
Popular Package → Common AI Hallucination
axios           → axois, axio, axioss
lodash          → lodahs, loadash, lo-dash
express         → expresss, express-js
react-router    → react-routes, react-routing
python-dotenv   → dotenv (wrong package in Python context)
```

Flag any match with edit distance ≤ 2 from these known pairs.

### Step 4 — Verify API calls

For any API endpoint or SDK method call found in the diff, use `rune:docs-seeker` (Context7) to confirm:
- The method/function exists in the library's documented API
- The parameter signature matches usage in code

Mark unverifiable API calls as **WARN** (cannot confirm without docs).

### Step 5 — Report

Emit the report in the Output Format below. If any **BLOCK** items exist, return status `BLOCK` to the calling skill to halt commit/deploy.

## Check Types

```
INTERNAL    — file exists, function/class exists, signature matches
EXTERNAL    — package exists on registry, version is valid
API         — endpoint pattern valid, method correct
TYPE        — assertion matches actual type
SUSPICIOUS  — package name similar to popular package (slopsquatting)
```

## Output Format

```
## Hallucination Guard Report
- **Status**: PASS | WARN | BLOCK
- **References Checked**: [count]
- **Verified**: [count] | **Unverified**: [count] | **Suspicious**: [count]

### BLOCK (hallucination detected)
- `import { formatDate } from 'date-utils'` — Package 'date-utils' not found on npm. Did you mean 'date-fns'?
- `import { useAuth } from '@/hooks/useAuth'` — File '@/hooks/useAuth' does not exist

### WARN (verify manually)
- `import { newFunction } from 'popular-lib'` — Function 'newFunction' not found in popular-lib@3.2.0 exports

### SUSPICIOUS (potential slopsquatting)
- `import axios from 'axois'` — Typo? Similar to popular package 'axios'

### Verified
- 12/15 references verified successfully
```

## Constraints

1. MUST verify every import against actual installed packages — not just check if name looks reasonable
2. MUST verify API signatures against docs — not assume from function name
3. MUST report BLOCK verdict with specific evidence — never "looks suspicious"
4. MUST NOT say "no hallucinations found" without listing what was checked

## Sharp Edges

Known failure modes for this skill. Check these before declaring done.

| Failure Mode | Severity | Mitigation |
|---|---|---|
| Declaring "no hallucinations found" without listing what was checked | CRITICAL | Constraint 4 blocks this — always list verified count vs total |
| Marking phantom package (not in manifest) as WARN instead of BLOCK | HIGH | Unlisted package in manifest = BLOCK — not installed = won't run |
| Missing typosquatting check on external packages | MEDIUM | Edit distance ≤2 check is mandatory — check every external package name |
| Only checking package name, not the specific exported symbol | MEDIUM | Step 2: verify the specific function/class is exported, not just the file exists |
| Skipping registry verification for new packages | CRITICAL | Step 3.5 HARD-GATE: new packages MUST be verified against actual registry |
| AI-hallucinated package name passes because it "sounds right" | HIGH | Slopsquatting defense: check registry existence, not name plausibility |
| Low-popularity package with similar name to popular one not flagged | HIGH | Popularity check catches slopsquatting attacks on newly registered packages |

## Done When

- All imports extracted from changed files (internal + external separated)
- Internal imports: file existence AND symbol export verified
- External packages: manifest presence checked for every package
- Suspicious package names flagged (edit distance ≤2 from popular packages)
- API signatures checked via docs-seeker for new SDK/library calls
- Hallucination Guard Report emitted with PASS/WARN/BLOCK and verified count

## Cost Profile

~500-1500 tokens input, ~200-500 tokens output. Haiku for speed — this runs frequently as a sub-check.