Common Llm Security

Assumption: the agent can read arbitrary files and delete files based on model decisions, with no human confirmation or equivalent host-enforced policy.

**Overall: 🔴 P0 — LLM06 Excessive Agency.** Autonomous deletion is a confirmed high-impact capability. Security score is capped at **40/100** until fixed.

| OWASP ID | Status | Review |
|---|---|---|
| LLM01 Prompt Injection | ⚠️ Needs review | Check whether file contents or user input are concatenated into system prompts. Keep untrusted content in a separate user/data message; do not treat file text as instructions. |
| LLM02 Sensitive Information Disclosure | ⚠️ Needs review | File reads may expose credentials or PII through prompts, responses, or logs. Minimize access and redact sensitive data before logging or persistence. |
| LLM03 Supply Chain | ⚠️ Needs review | Review model, plugins, and packages. Pin source revisions and hashes; hashes provide integrity, not trusted authorship. |
| LLM04 Data & Model Poisoning | ⚠️ Needs review | Treat files, logs, retrieved documents, and proposed learning entries as untrusted. Validate and sanitize data before persistence or indexing. |
| LLM05 Improper Output Handling | ⚠️ Needs review | Never use raw model output directly as a filesystem path or delete command. Parse against a strict schema, canonicalize paths, validate authorization, and reject traversal or ambiguous paths. |
| LLM06 Excessive Agency | 🔴 Confirmed | Read/delete tools operate without confirmation. Require human-in-the-loop approval for every deletion, or restrict deletion to an explicitly authorized, host-enforced sandbox with deny-by-default permissions. |
| LLM07 System Prompt Leakage | ⚠️ Needs review | Ensure tool errors, file contents, and API responses cannot reveal system prompts, policies, credentials, or hidden tool instructions. |
| LLM08 Vector & Embedding Weaknesses | ⚠️ Needs review | If file contents are embedded, sanitize untrusted text and enforce tenant/user namespace isolation. |
| LLM09 Misinformation | ⚠️ Needs review | Do not let an unverified model judgment trigger irreversible deletion, especially for legal, financial, medical, or compliance records. |
| LLM10 Unbounded Consumption | ⚠️ Needs review | Enforce `max_tokens`, invocation rate limits, file-size/read limits, timeout limits, and a maximum agent iteration/depth cap. |

Minimum remediation:

1. Default all file operations to read-only.
2. Require explicit confirmation containing the exact resolved path, reason, and irreversible effect before deletion.
3. Enforce permissions in the host/tool layer; skill text or `allowed-tools` declarations are not sufficient.
4. Use an allowlist of directories, deny symlinks and traversal, canonicalize paths, and restrict deletion to a sandbox.
5. Add dry-run mode, immutable audit logs, rollback/recycle-bin behavior where possible, and emergency disablement.
6. Validate structured tool arguments and sanitize model-generated text before filesystem use.
7. Add tests for prompt injection through file contents, path traversal, symlink escapes, sensitive-file access, repeated deletion loops, and confirmation bypasses.