Audit Ai Assistant

---
name: AI Shopping Assistant Audit
description: >
  Comprehensive audit of the SKAPARA AI shopping assistant — conversation quality,
  RAG retrieval, session memory, usage metering, tool execution, streaming UX,
  cost optimization, and security. Use when asked to audit the AI assistant, chatbot,
  conversation system, RAG pipeline, AI usage limits, or chat UX.
---

# AI Shopping Assistant Audit Skill

## Overview

The SKAPARA AI Shopping Assistant is a Gemini 2.5-flash powered conversational commerce agent with 28 tools, Vercel AI SDK 6 streaming, RAG (pgvector hybrid search), tier-based usage metering, anomaly detection, content safety, and conversation persistence. This skill defines a structured 8-phase audit workflow to evaluate every layer against industry best practices and surface actionable findings.

## Architecture Reference

| Layer | File | Lines | Description |
|---|---|---|---|
| Chat API | `frontend/src/app/api/chat/route.ts` | 2520 | 28 tools, system prompt, streaming, persistence |
| Chat UI | `frontend/src/components/storefront/ChatArea.tsx` | 830+ | useChat hook, message rendering, artifacts |
| Conversations API | `frontend/src/app/api/conversations/route.ts` | 57 | List user conversations with message counts |
| RAG Search | `frontend/src/app/api/rag/search/route.ts` | 227 | Hybrid vector + keyword search, Redis cache |
| RAG Index | `frontend/src/app/api/rag/index/route.ts` | -- | Document indexing pipeline |
| RAG Add Docs | `frontend/src/app/api/rag/add-documents/route.ts` | -- | Batch document ingestion |
| Usage Limiter | `frontend/src/lib/usage-limiter.ts` | 417 | Tier-based limits, Supabase RPC atomic ops |
| Rate Limiter | `frontend/src/lib/rate-limit.ts` | 129 | In-memory burst protection per instance |
| Anomaly Monitor | `frontend/src/lib/anomaly-monitor.ts` | 150 | Velocity block, rate-limit hit tracking |
| Content Safety | `frontend/src/lib/content-safety.ts` | 86 | Blocked terms filter (trademarks, NSFW, hate) |
| Artifact Registry | `frontend/src/components/artifacts/registry.tsx` | 80+ | 12 artifact types mapped to tool outputs |
| Transport | ChatArea.tsx:151-198 | -- | DefaultChatTransport with CSRF + conversation ID |
| DB Tables | `conversations`, `messages`, `user_usage`, `documents` | -- | RLS-protected persistence |

## Audit Workflow

Execute phases sequentially. Each phase produces findings in the standardized format (see Output Format below). Skip phases only if explicitly told to narrow scope.

---

### Phase 1: Conversation Quality & Flow

**Goal**: Evaluate how well the assistant guides users from intent to conversion.

#### 1.1 System Prompt Analysis

- **File**: `chat/route.ts:401-528`
- Read the full system prompt. Check for:
  - [ ] Clear persona definition (name, role, store context)
  - [ ] Locale-aware instructions (`route.ts:390-395` — en/es/de)
  - [ ] Tool routing instructions with numbered examples (`route.ts:432-507`)
  - [ ] Proactive no-results handling (`route.ts:514-520`)
  - [ ] Premium upsell awareness (`route.ts:522-527`)
  - [ ] Missing: proactive greeting strategy (industry benchmark: 45% higher engagement)
  - [ ] Missing: checkout hesitation detection cues
  - [ ] Missing: cart abandonment recovery prompts
  - [ ] Missing: cross-sell / upsell triggers after add-to-cart
  - [ ] Missing: sentiment detection instructions

**Industry Benchmarks**:
| Metric | Target | How to Measure |
|---|---|---|
| First recommendation | < 2 exchanges | Count messages before first product_search call |
| Conversion rate (chat-assisted) | > 8% | Track create_checkout calls / total conversations |
| Cart abandonment recovery | > 15% recovery | Track add_to_cart without confirm_checkout |
| Avg messages per session | 6-12 | Query messages table grouped by conversation_id |
| Fallback rate | < 10% | Count responses with no tool call and generic text |

#### 1.2 Tool Routing Accuracy

- **File**: `chat/route.ts:432-507` (WHEN TO USE EACH TOOL)
- Verify:
  - [ ] All 28 tools have routing instructions
  - [ ] No overlapping triggers between product_search and browse_catalog
  - [ ] Design tools have intent classification (`route.ts:451-459`)
  - [ ] Privacy classification for personal designs (`route.ts:461-464`)
  - [ ] Approval-required tools clearly marked (create_checkout, confirm_checkout, request_return, apply_design_to_product)

#### 1.3 Fallback & Error Handling

- **File**: `chat/route.ts:567-578` (product_search fallback), `route.ts:651-665` (browse_catalog fallback)
- Check:
  - [ ] Zero-results fallback provides suggestions + categories
  - [ ] Tool execution errors return structured error objects (not thrown exceptions)
  - [ ] `getSearchFallback()` (`route.ts:71-94`) returns top-rated + category counts
  - [ ] FAQs loaded via CAG pattern (`route.ts:101-162`) with 200K token guard

#### 1.4 Conversation Title Generation

- **File**: `chat/route.ts:2464-2469`
- Check:
  - [ ] Title set from first assistant response (first 100 chars)
  - [ ] Truncation at 100 chars — may cut mid-word (potential improvement)
  - [ ] No summarization model used — title is raw text prefix

---

### Phase 2: RAG & Product Retrieval

**Goal**: Evaluate retrieval accuracy, latency, and freshness.

#### 2.1 Hybrid Search Pipeline

- **File**: `rag/search/route.ts:105-117`
- Check:
  - [ ] Hybrid search weights: 70% vector + 30% keyword (`hybrid_search_documents` RPC)
  - [ ] Embedding model: `gemini-embedding-001` at 768 dimensions (`route.ts:66-79`)
  - [ ] Similarity threshold: 0.65 (`route.ts:154`)
  - [ ] Low-relevance flag set when no results above threshold (`route.ts:173-176`)
  - [ ] Fallback to direct query when RPC unavailable (`route.ts:122-151`)

#### 2.2 Caching

- **File**: `rag/search/route.ts:40-53`, `route.ts:178-195`
- Check:
  - [ ] Cache key: SHA-256 hash of normalized query + limit + locale (`route.ts:13-18`)
  - [ ] Cache TTL: 3600 seconds (1 hour) (`route.ts:189`)
  - [ ] Cache hit logged with response time (`route.ts:46`)
  - [ ] Cache miss logged with key (`route.ts:55`)
  - [ ] Only non-empty results cached (`route.ts:180`)

**Industry Benchmarks**:
| Metric | Target | Current |
|---|---|---|
| Query latency (cached) | < 50ms | Measure via responseTime field |
| Query latency (uncached) | < 200ms | Measure via responseTime field |
| Cache hit rate | > 60% | Monitor cache HIT vs MISS logs |
| Similarity threshold | 0.60-0.70 | Currently 0.65 |
| Embedding dimension | 768+ | 768 (Gemini) |
| Re-ranking | Yes | Not implemented (hybrid weights only) |

#### 2.3 RAG Integration in Chat

- **File**: `chat/route.ts:2374-2430`
- Check:
  - [ ] RAG search called with last user message (`route.ts:2379-2389`)
  - [ ] Top 3 results injected into system prompt (`route.ts:2405, 2415-2422`)
  - [ ] RAG failure is non-blocking (`route.ts:2427-2430`)
  - [ ] Auth cookies forwarded to RAG endpoint (`route.ts:2400`)
  - [ ] Missing: RAG is called on EVERY message — no intent filter to skip for greetings/cart ops
  - [ ] Missing: no re-ranking step after retrieval

#### 2.4 Document Freshness

- Check:
  - [ ] Product sync frequency (how often documents table updated)
  - [ ] Stale document detection (products deleted but documents remain)
  - [ ] Locale-filtered search available (`route.ts:115-116`)

---

### Phase 3: Session Memory & Persistence

**Goal**: Evaluate context retention, session lifecycle, and cross-session continuity.

#### 3.1 Working Memory (Context Window)

- **File**: `chat/route.ts:28, 339-341`
- Check:
  - [ ] Sliding window: MAX_CONTEXT_MESSAGES = 40 (`route.ts:28`)
  - [ ] Window preserves first message (system context) + last 39 (`route.ts:340`)
  - [ ] Missing: no summarization for conversations exceeding window
  - [ ] Missing: no priority retention (tool results may be dropped before user preferences)

#### 3.2 Session Persistence (Client-Side)

- **File**: `ChatArea.tsx:50-69` (serialization), `ChatArea.tsx:71-106` (TTL + restore)
- Check:
  - [ ] Messages serialized to sessionStorage (`ChatArea.tsx:316-326`)
  - [ ] TTL: 3 hours for anonymous users (`ChatArea.tsx:71`)
  - [ ] Expiry checked on mount, tab focus, visibility change, 10min interval (`ChatArea.tsx:280-308`)
  - [ ] Conversation ID persisted in sessionStorage (`ChatArea.tsx:140-142, 170-171`)
  - [ ] Tool parts serialized: only text + output-available tool results (`ChatArea.tsx:55`)
  - [ ] Missing: authenticated user sessions have no TTL (persist until tab close)

#### 3.3 Conversation Persistence (Server-Side)

- **File**: `chat/route.ts:344-387`
- Check:
  - [ ] Conversation upsert is fire-and-forget (`route.ts:349-362`)
  - [ ] User message saved fire-and-forget (`route.ts:364-387`)
  - [ ] Assistant message saved in onFinish callback (`route.ts:2448-2472`)
  - [ ] Write client uses user JWT for authenticated, service key for anonymous (`route.ts:207-215`)
  - [ ] Missing: no retry on persistence failure
  - [ ] Missing: no conversation list/history loading in chat UI (conversations API exists but unused in ChatArea)
  - [ ] Missing: no cross-session memory retrieval (user cannot reference old conversations)

#### 3.4 Conversation ID Tracking

- **File**: `ChatArea.tsx:140-142, 151-172`, `chat/route.ts:344, 2498-2501`
- Check:
  - [ ] Conversation ID sent via `x-conversation-id` header (`ChatArea.tsx:161-163`)
  - [ ] Server returns conversation ID in response header (`route.ts:2500`)
  - [ ] New ID generated if none provided (`route.ts:344`)

**Industry Benchmarks**:
| Metric | Target | Current |
|---|---|---|
| Context retention | Full session | 40-message sliding window |
| Long conversation handling | Summarization | Truncation only |
| Cross-session memory | Last 3 conversations | Not implemented |
| Session restore time | < 200ms | sessionStorage (sync, fast) |
| Persistence reliability | > 99.5% | Fire-and-forget (no retry) |

---

### Phase 4: Tool Execution & Reliability

**Goal**: Evaluate tool schema validation, error handling, execution patterns, and approval workflows.

#### 4.1 Tool Schema Validation

- **File**: `chat/route.ts:530-2372` (all tool definitions)
- Check:
  - [ ] All tools use Zod schema validation
  - [ ] @ts-expect-error workaround for AI SDK 6 type mismatch (`route.ts:539, 606, etc.`)
  - [ ] Optional parameters have `.optional()` and `.describe()`
  - [ ] product_search uses `sanitizeForPostgrest()` (`route.ts:553`)
  - [ ] get_product_detail uses `sanitizeForLike()` (`route.ts:715`)

#### 4.2 Approval Workflow

- Tools requiring approval:
  - [ ] `create_checkout` — shows cart summary, waits for user approval
  - [ ] `confirm_checkout` — creates Stripe session after approval
  - [ ] `request_return` — return/refund request needs approval
  - [ ] `apply_design_to_product` — `needsApproval: true` (`route.ts:2302`)
- **File**: `ChatArea.tsx:205` — `addToolApprovalResponse` from useChat
- Check:
  - [ ] Approval UI renders via `ApprovalCardArtifact` and `ReturnRequestArtifact`
  - [ ] User can approve/reject inline in chat

#### 4.3 Error Recovery

- Check across all tool execute functions:
  - [ ] Each tool has try/catch returning `{ success: false, error: string }`
  - [ ] No uncaught exceptions that would crash the stream
  - [ ] Database errors logged with console.error
  - [ ] Missing: no automatic retry on transient DB errors
  - [ ] Missing: no circuit breaker for repeated tool failures

#### 4.4 Tool Step Limits

- **File**: `chat/route.ts:2447`
- Check:
  - [ ] Step limit: 3 for free/anon, 5 for premium (`stepCountIs`)
  - [ ] Ensures tool loops terminate (prevents infinite agent loops)
  - [ ] Missing: no per-tool timeout (relies on global maxDuration = 60s at `route.ts:16`)

#### 4.5 Artifact Rendering

- **File**: `frontend/src/components/artifacts/registry.tsx:29-80+`
- Check:
  - [ ] 12 artifact types registered: ProductGrid, ProductDetail, ComparisonTable, SizeGuide, CartSummary, PricingTable, ApprovalCard, OrderTimeline, OrderList, ReturnRequest, DesignPreview
  - [ ] Each has Component + Skeleton (loading state)
  - [ ] Missing artifacts for: add_to_cart result, apply_coupon result, switch_language result

---

### Phase 5: Usage Metering & Rate Limiting

**Goal**: Evaluate quota enforcement accuracy, atomicity, and tier fairness.

#### 5.1 Tier Definitions

- **File**: `usage-limiter.ts:25-59`
- Verify tier limits match business requirements:

| Action | Anonymous | Free | Premium |
|---|---|---|---|
| chat (conversations/day) | 5 | 30 | 100 |
| chat:messages (messages/day) | 20 | 200 | unlimited (-1) |
| chat:tokens (tokens/day) | 50K | 500K | 2M |
| design:generate (designs/month) | 0 | 5 | 50 |
| design:ai-generate (AI designs/month) | 0 | 5 | 50 |

- Check:
  - [ ] Premium messages unlimited (`-1` at `route.ts:50`)
  - [ ] Anonymous blocked from all design actions (limit = 0)
  - [ ] Period types correct (daily for chat, monthly for designs)

#### 5.2 Atomic Enforcement

- **File**: `usage-limiter.ts:111-133` (supabaseIncrement), `usage-limiter.ts:217-298` (checkAndIncrementUsage)
- Check:
  - [ ] Atomic check+increment via Supabase RPC `increment_usage` (`route.ts:118`)
  - [ ] Fail-CLOSED: Supabase failure = deny request (`route.ts:284-298`)
  - [ ] Admin alert fired on Supabase failure (`route.ts:288-296`)
  - [ ] Token increment via separate `increment_usage_by` RPC (`route.ts:138-162`)
  - [ ] Premium credit overflow: `consume_credit_atomic` RPC (`route.ts:188-204`)
  - [ ] Decrement on rollback is best-effort (`route.ts:335-353`)

#### 5.3 Identifier Resolution

- **File**: `usage-limiter.ts:77-87`
- Check:
  - [ ] UUID user IDs pass through unchanged
  - [ ] Fingerprints (`fp:`) pass through unchanged
  - [ ] IPs hashed with daily salt for GDPR compliance (`SHA-256(ip:daySalt)`)
  - [ ] Missing: fingerprint rotation attack (same user, new fingerprint each time)

#### 5.4 Burst Rate Limiting

- **File**: `rate-limit.ts:17-56` (RateLimiter class), `rate-limit.ts:62-63` (chat limiters)
- Check:
  - [ ] In-memory per-instance (acceptable for Vercel, not shared across instances)
  - [ ] Chat: 20 msg/min with fingerprint, 5 msg/min without (`rate-limit.ts:62-63`)
  - [ ] E2E test bypass: `PLAYWRIGHT_TEST_BASE_URL` or `CI` env (`rate-limit.ts:29`)
  - [ ] Probabilistic cleanup (1% chance per check at `rate-limit.ts:36`)
  - [ ] Concurrent slot limiter: max 2 streaming requests per identifier (`rate-limit.ts:112-128`)

#### 5.5 Anomaly Detection

- **File**: `anomaly-monitor.ts`
- Check:
  - [ ] High usage warning at 80% of limit (`anomaly-monitor.ts:64`)
  - [ ] Auto-block after 5+ rate-limit hits in 5 min (`anomaly-monitor.ts:87-89`)
  - [ ] Velocity block: 5+ messages in <3 seconds (`anomaly-monitor.ts:113-148`)
  - [ ] Block duration: 30 minutes (`anomaly-monitor.ts:16`)
  - [ ] All in-memory (resets on deploy/restart)
  - [ ] Missing: no persistent block storage (attacker waits for deploy)
  - [ ] Missing: no alerting channel (console.warn only)

#### 5.6 Usage Headers

- **File**: `usage-limiter.ts:410-416`, `chat/route.ts:272`
- Check:
  - [ ] X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset headers set
  - [ ] Headers returned on 429 responses

**Industry Benchmarks**:
| Metric | Target | Current |
|---|---|---|
| Enforcement model | Fail-closed | Fail-closed (correct) |
| Quota accuracy | 100% atomic | Supabase RPC atomic (correct) |
| Block persistence | Survive deploys | In-memory only (gap) |
| Identifier evasion resistance | Fingerprint + IP | Fingerprint OR IP (gap) |
| Usage dashboard | Customer-visible | Not implemented |
| Grace period at limit | Soft warning at 80% | Warning logged, not shown to user |

---

### Phase 6: Chat UX & Accessibility

**Goal**: Evaluate message rendering, interaction patterns, mobile UX, and accessibility.

#### 6.1 Message Rendering

- **File**: `ChatArea.tsx:463-end` (JSX rendering)
- Check:
  - [ ] Text rendered via `SafeMarkdown` component (`ChatArea.tsx:26`)
  - [ ] Tool results rendered via artifact registry (`ChatArea.tsx:27`)
  - [ ] Loading skeletons shown during tool execution
  - [ ] Auto-scroll to bottom on new messages (`ChatArea.tsx:311-313`)
  - [ ] Missing: message chunking for long responses (industry best practice)

#### 6.2 Welcome Screen

- **File**: `ChatArea.tsx:471-478+`
- Check:
  - [ ] Shown when messages.length === 0
  - [ ] Brand mark + personalized greeting (user name if available)
  - [ ] Quick-reply prompt buttons available
  - [ ] Missing: proactive greeting with user context (active orders, recent favorites)

#### 6.3 Input Area

- **File**: `ChatArea.tsx:336-366`
- Check:
  - [ ] Enter key sends message (`ChatArea.tsx:337-341`)
  - [ ] Image upload via file input + drag-and-drop (`ChatArea.tsx:387-461`)
  - [ ] Image validation: type check + 5MB limit (`ChatArea.tsx:392-401`)
  - [ ] Voice input via `useSpeechToText` hook (`ChatArea.tsx:214-232`)
  - [ ] Limit-reached state disables input (`ChatArea.tsx:137`)
  - [ ] Missing: typing indicator ("SKAPARA is thinking...")
  - [ ] Missing: message editing/retry

#### 6.4 Engagement Modals

- **File**: `ChatArea.tsx:134-137, 174-195`
- Check:
  - [ ] AuthWallModal shown when anonymous user hits limit (`ChatArea.tsx:182`)
  - [ ] UpgradeModal shown when free user hits limit (`ChatArea.tsx:184`)
  - [ ] 429 responses intercepted and converted to engagement flow (`ChatArea.tsx:175-195`)
  - [ ] Fake empty SSE stream returned to prevent raw error display (`ChatArea.tsx:187-190`)

#### 6.5 Mobile UX

- Check:
  - [ ] Responsive padding: `px-3 py-4 sm:px-4 md:px-6 md:py-6` (`ChatArea.tsx:470`)
  - [ ] Touch targets for buttons >= 44px
  - [ ] Product cards in artifacts are mobile-responsive
  - [ ] Voice input button accessible on mobile
  - [ ] Missing: haptic feedback on send

#### 6.6 Accessibility

- Check:
  - [ ] ARIA labels on input field, send button, voice button, attach button
  - [ ] Screen reader announcements for new messages
  - [ ] Keyboard navigation through quick-reply buttons
  - [ ] Color contrast for message bubbles (semantic tokens)
  - [ ] Focus management after tool approval/rejection

**Industry Benchmarks**:
| Metric | Target | Current |
|---|---|---|
| Time to first response | < 1.5s | Streaming (good), no measurement |
| Product card render | Image + price + rating | Via artifacts (good) |
| Quick-reply buttons | 3-5 contextual options | Static prompts on welcome only |
| Brand voice consistency | 100% on-brand | System prompt only, no validation |
| Mobile touch targets | >= 44px | Needs verification |
| Approval workflow clarity | 1-click approve/reject | Via artifact cards (good) |

---

### Phase 7: Cost & Performance Optimization

**Goal**: Evaluate token costs, model selection, caching strategy, and latency.

#### 7.1 Model Configuration

- **File**: `chat/route.ts:46-48, 2442`
- Check:
  - [ ] Model: `gemini-2.5-flash` (cost-optimized)
  - [ ] API key: `GOOGLE_GENERATIVE_AI_API_KEY` or `GEMINI_API_KEY` fallback
  - [ ] Missing: no context caching for system prompt (Gemini supports cached context)
  - [ ] Missing: no model fallback on 429/5xx from Gemini

#### 7.2 Token Budget

- **File**: `chat/route.ts:18-23, 2446`
- Check:
  - [ ] Per-response output caps: anon=2048, free=4096, premium=8192 (`route.ts:19-23`)
  - [ ] Daily token budgets enforced via `checkTokenBudget` (`route.ts:300-310`)
  - [ ] Token tracking in onFinish callback (`route.ts:2474-2493`)
  - [ ] Cost alert threshold: $0.05 per response (`route.ts:2483`)
  - [ ] Gemini Flash pricing used: $0.30/M input, $1.25/M output (`route.ts:2482`)

#### 7.3 Input Cost Control

- **File**: `chat/route.ts:26-28, 339-341`
- Check:
  - [ ] MAX_MESSAGE_CHARS = 4000 (`route.ts:26`)
  - [ ] MAX_CONTEXT_MESSAGES = 40 sliding window (`route.ts:28`)
  - [ ] System prompt size: ~3K tokens (measured from route.ts:401-528)
  - [ ] FAQ context loaded conditionally with 200K token guard (`route.ts:102-103`)
  - [ ] RAG context adds ~200-500 tokens per query (`route.ts:2415-2422`)
  - [ ] Missing: input token counting before sending to model

#### 7.4 RAG Call Optimization

- **File**: `chat/route.ts:2374-2430`
- Check:
  - [ ] RAG called on every message (even greetings, cart ops, confirmations)
  - [ ] RAG adds latency: embedding generation + vector search + network round-trip
  - [ ] Recommendation: skip RAG for non-product queries (cart, checkout, order, language, design operations)
  - [ ] Redis cache mitigates repeated queries (`rag/search/route.ts:44-53`)

#### 7.5 Streaming Performance

- **File**: `chat/route.ts:2441-2502`
- Check:
  - [ ] `streamText` with SSE (`toUIMessageStreamResponse`)
  - [ ] maxDuration = 60s at route level (`route.ts:16`)
  - [ ] Concurrent slot limit prevents resource exhaustion (`route.ts:253-258`)
  - [ ] Slot released in finally block (`route.ts:2503-2506`)

**Industry Benchmarks**:
| Metric | Target | Current |
|---|---|---|
| Cost per conversation | < $0.02 | ~$0.005-0.02 (Flash pricing, good) |
| Cost alert threshold | > $0.05/response | $0.05 (correct) |
| System prompt caching | Reduce 50% input cost | Not implemented |
| Unnecessary RAG calls | 0% | ~40% (cart/order/design ops) |
| Time to first token | < 500ms | Not measured |
| End-to-end latency | < 3s for simple queries | Not measured |

---

### Phase 8: Security & Safety

**Goal**: Evaluate input validation, injection prevention, content safety, auth boundaries, and data protection.

#### 8.1 Input Validation

- **File**: `chat/route.ts:315-336`
- Check:
  - [ ] Message array required (`route.ts:315-320`)
  - [ ] Last message length validated: max 4000 chars (`route.ts:322-336`)
  - [ ] Text extracted from both string content and parts array (`route.ts:325-329`)
  - [ ] Missing: no validation of message role (user could send role=system)
  - [ ] Missing: no validation of parts types (could send arbitrary part types)

#### 8.2 SQL Injection Prevention

- **File**: `chat/route.ts:14, 553, 715`
- Check:
  - [ ] `sanitizeForPostgrest()` used in product_search (`route.ts:553-554`)
  - [ ] `sanitizeForLike()` used in get_product_detail name search (`route.ts:715`)
  - [ ] Import from `@/lib/query-sanitizer` (`route.ts:14`)
  - [ ] Other tools using `.eq()` or `.in()` (parameterized, safe)

#### 8.3 Content Safety

- **File**: `content-safety.ts`
- Check:
  - [ ] `checkPromptSafety()` filters design prompts before generation
  - [ ] 45 blocked terms across 6 categories: Trademark, NSFW, Hate, Violence, Illegal (`content-safety.ts:8-45`)
  - [ ] Word-boundary regex matching (`content-safety.ts:74`)
  - [ ] Input normalized: lowercase, special chars removed (`content-safety.ts:51-57`)
  - [ ] Missing: no content safety check on chat messages (only design prompts)
  - [ ] Missing: no output safety check on model responses
  - [ ] Missing: no image content moderation for uploaded images
  - [ ] Missing: prompt injection detection (user could instruct model to ignore system prompt)

#### 8.4 Authentication & Authorization

- **File**: `chat/route.ts:175-231, 204-231`
- Check:
  - [ ] User resolved from `sb-access-token` cookie (`route.ts:197, 210-211`)
  - [ ] Tier resolved from users table with subscription validation (`route.ts:216-230`)
  - [ ] Expired premium treated as free (`route.ts:223-229`)
  - [ ] User-scoped write client for authenticated users (`route.ts:215`)
  - [ ] Anonymous sessions use service key for writes (`route.ts:209`)
  - [ ] Conversations API requires auth (`conversations/route.ts:11`)
  - [ ] RAG search requires auth (`rag/search/route.ts:29`)

#### 8.5 CSRF Protection

- **File**: `ChatArea.tsx:155-158`
- Check:
  - [ ] CSRF token read from cookie (`ChatArea.tsx:155`)
  - [ ] Token sent via `x-csrf-token` header (`ChatArea.tsx:157`)
  - [ ] Missing: CSRF token rotation strategy

#### 8.6 Rate Limit Evasion

- Check:
  - [ ] IP-based rate limiting can be bypassed with VPN/proxy rotation
  - [ ] Fingerprint-based limiting can be bypassed with browser fingerprint spoofing
  - [ ] In-memory anomaly blocks reset on deploy (`anomaly-monitor.ts` — all Maps)
  - [ ] No persistent block list (Redis or DB-backed)
  - [ ] E2E test bypass in production code (`rate-limit.ts:29` — check env var security)

#### 8.7 Data Exposure

- Check:
  - [ ] Service key used for admin reads in chat route (`route.ts:52-55`) — bypasses RLS
  - [ ] Conversation writes use user-scoped client (correct, `route.ts:215`)
  - [ ] Product data exposed: id, title, description, price, images, rating (safe)
  - [ ] User profile data: only tier + subscription status read (`route.ts:217-221`)
  - [ ] Token counts logged with truncated identifier (`anomaly-monitor.ts:44`)

**Industry Benchmarks**:
| Metric | Target | Current |
|---|---|---|
| Prompt injection resistance | Model-level + input filter | No filter (gap) |
| Output content moderation | Automated check | Not implemented (gap) |
| Image upload moderation | Automated NSFW check | Not implemented (gap) |
| SQL injection prevention | Parameterized + sanitized | Sanitized (good) |
| Auth boundary enforcement | RLS + JWT scoping | Hybrid (mostly good) |
| GDPR compliance (identifiers) | Hashed/anonymized | IP hashed with daily salt (good) |
| Rate limit persistence | Survive restarts | In-memory only (gap) |

---

## Output Format

Generate a structured Markdown report with the following sections:

### 1. Executive Summary

3-5 sentences summarizing overall health, top strengths, and critical gaps.

### 2. Scorecard

| Phase | Score (1-10) | Critical Issues | Notes |
|---|---|---|---|
| 1. Conversation Quality | X | N | ... |
| 2. RAG & Retrieval | X | N | ... |
| 3. Session Memory | X | N | ... |
| 4. Tool Execution | X | N | ... |
| 5. Usage Metering | X | N | ... |
| 6. Chat UX | X | N | ... |
| 7. Cost & Performance | X | N | ... |
| 8. Security & Safety | X | N | ... |
| **Overall** | **X/10** | **N** | |

### 3. Findings Table

| ID | Finding | Severity | Phase | File:Line | Recommendation |
|---|---|---|---|---|---|
| F-001 | ... | Critical/High/Medium/Low | 1-8 | path:line | ... |

Severity definitions:
- **Critical**: Security vulnerability, data loss risk, or compliance violation
- **High**: Significant UX degradation, revenue impact, or reliability gap
- **Medium**: Suboptimal behavior that should be addressed in next sprint
- **Low**: Nice-to-have improvement, polish item

### 4. Priority Action Items

Top 10 findings ranked by impact, with effort estimate (S/M/L) and suggested implementation approach.

### 5. Metrics Dashboard Proposal

Recommended observability metrics to track ongoing health:

| Metric | Source | Alert Threshold |
|---|---|---|
| Chat API p95 latency | Vercel analytics | > 3s |
| RAG cache hit rate | Redis logs | < 50% |
| Tool execution error rate | console.error logs | > 5% |
| Usage limiter denial rate | Supabase user_usage | > 20% of requests |
| Cost per conversation | onFinish token tracking | > $0.05 |
| Conversation completion rate | conversations + messages | < 40% with 3+ messages |
| Anomaly blocks per day | anomaly-monitor logs | > 50 |

---

## Checklist Before Reporting

- [ ] All 8 phases executed with file:line references verified
- [ ] Every finding has a specific file path and line number
- [ ] Severity levels applied consistently
- [ ] Recommendations are actionable (not just "improve X")
- [ ] Industry benchmarks cited for each phase
- [ ] No assumptions — every claim verified against source code
- [ ] Report saved to `frontend/docs/audit-ai-assistant-YYYY-MM-DD.md`