Pentest Red Team Ai And Reporting

---
name: pentest-red-team-ai-and-reporting
description: "Use when running the adversary-simulation and communication side of an authorized engagement: social engineering and physical testing (within an agreed scope and rules of engagement), red teaming vs purple teaming, post-exploitation reasoning (goals, not tooling), testing AI/LLM systems (OWASP LLM Top 10, prompt injection, the model and its surrounding system) and using AI in offensive work, and reporting — the report as the product, writing a finding that is actually usable (evidence, reproduction, business risk, CVSS), and what happens after the report (retest, remediation support). Authorized, defensive-context work only."
---

# Penetration Testing: Social Engineering, Red/Purple Teaming, Post-Exploitation, AI Systems, and Reporting

> **Part 3 of 4** of the *Penetration Testing and Offensive Security* reference (plugin `penetration-testing`), covering §10–§13. Sibling skills: `pentest-authorization-scoping-and-methodology` (§0–§3), `pentest-technical-domains` (§4–§9), `pentest-disclosure-career-and-reference` (§14–§20). Section numbers are shared across the set; a reference written as §N → `skill` points into that sibling skill.
>
> **Currency:** Verified August 2026. See §17 → `pentest-disclosure-career-and-reference` for the currency snapshot and what goes stale first.

> **⚠️ THE PREREQUISITE FOR EVERYTHING BELOW.** The entire difference between this
> profession and a felony is **documented, informed authorization from someone with the
> authority to grant it, for a defined scope, in a defined window.** Not implied consent.
> Not a verbal go-ahead. Not "the client seemed fine with it." Not a bug bounty scope you
> read quickly. **§1 → `pentest-authorization-scoping-and-methodology` is the section that matters most, and it is first for that reason.**
>
> This document is written for people doing authorized work — in-house security teams,
> consultancies, bug bounty participants operating inside a published scope, and the
> people who hire and manage them. **It is a methodology, process, and professional
> practice reference.** It deliberately does not contain exploitation recipes, working
> exploit code, or techniques for evading specific defensive products; those live in
> engagement-specific research, and the value of a reference like this is in the parts
> practitioners actually get wrong — scoping, method, evidence, and communication.
>
> **This is not legal advice.** §1 → `pentest-authorization-scoping-and-methodology` tells you what to ask your counsel, not what your
> obligations are in your jurisdiction.

> **How to read this.** Three markers:
> - **[DURABLE]** — professional practice, methodology, or a lesson the field has settled.
> - **[VERSIONED]** — regulation, tooling, market state. Verify.
> - **[CONTESTED]** — practitioners genuinely disagree.
>
> **The three framings that organize everything below:**
> 1. **The report is the product.** The client is not buying access to their systems —
>    they already have that. **They are buying a prioritized, reproducible, actionable
>    understanding of their risk.** A brilliant compromise written up badly is a failed
>    engagement (§13).
> 2. **Your job is to model a threat, not to win.** "I got domain admin" is not the goal;
>    **"here is how an attacker with these capabilities would reach these assets, here is
>    what detected them and what didn't, and here is what to fix first"** is.
> 3. **You are operating in someone's production environment and you can break it.**
>    Professional restraint — knowing what *not* to run — is a larger part of competence
>    than the ability to run anything.

---

## §10. Social Engineering and Physical

**[DURABLE] The human layer is consistently the most effective initial access vector**, and
consistently the most sensitive to test.

**Techniques**: phishing (and its voice and SMS variants), pretexting, **MFA fatigue and
consent phishing**, and physical entry — tailgating, badge cloning, and testing what
happens when someone walks in wearing a hi-vis vest.

**⚠️ The ethical and legal handling is different in kind here**, and it is where
engagements most often cause real harm:
- **Requires specific, explicit written authorization** naming the techniques.
- **Physical assessments require the authorization letter on your person**, plus a named
  contact reachable at any hour who can vouch for you. **Assume you will be detained** and
  plan for it; also assume law enforcement may be called, which is a serious situation
  regardless of your paperwork.
- **Never impersonate law enforcement or government officials.**
- **Report on the process, not the person.** "Three of forty staff entered credentials" is
  a finding. Naming and shaming individuals is unprofessional, destroys future
  cooperation, and can be a serious HR and legal matter.
- **⚠️ Avoid pretexts that cause genuine distress** — fake layoffs, fake bonus
  announcements, fake emergencies involving family. These have caused real harm, and
  "the attacker would do it" is not sufficient justification when *you* are the one doing
  it to real employees.
- Consider works councils, union agreements, and employment law — **in several
  jurisdictions testing employees has consultation requirements.**

---

## §11. Red Teaming, Purple Teaming, Post-Exploitation

### 11.1 Post-exploitation, conceptually

**[DURABLE] The point of post-exploitation is to establish business impact, not to
maximize access.** The questions: what does this foothold reach? What credentials or
tokens are available here? What would an attacker do next, and what does that reach? **And
critically — what did the defenders see?**

**Map everything to MITRE ATT&CK.** It is the shared vocabulary that lets a defender take
your report and build a detection, and it turns "we moved laterally" into a specific
technique ID they can hunt for.

**⚠️ Restraint applies most here.** Persistence mechanisms must be documented and removed;
**leave-behinds are how testers cause incidents months later.** Keep a written cleanup
inventory — every account created, tool dropped, configuration changed, and persistence
established — and **verify removal with the client rather than asserting it.**

### 11.2 Red vs. purple

**Red team** tests **detection and response**, not vulnerabilities. It has an objective
(reach this data, execute this transaction), a long timeline, and a **blue team that
doesn't know** — which is the whole point and also the source of its cost.

**Purple team** puts testers and defenders in the same room: run a technique, check whether
it alerted, tune the detection, run it again. **[CONTESTED but with a strong majority
view]: for most organizations, purple teaming delivers more defensive improvement per dollar
than red teaming**, because red teaming tells you that you failed while purple teaming
fixes it. The counter-argument is real: **only an unannounced exercise tests the actual
human response under genuine uncertainty**, and organizations that only purple team never
learn whether their process works when nobody is expecting it.

**[DURABLE] The maturity ladder is real: do not buy a red team engagement if you have not
done the basic testing.** An organization that hasn't patched its perimeter and has a flat
network doesn't need an adversary simulation to know it will fail. **Red teaming is
appropriate when you have a functioning security program and want to test it** — buying one
earlier is paying premium rates for findings a vulnerability assessment would have given
you.

---

## §12. AI Systems

**[DURABLE] Two distinct things share a name, and conflating them causes confusion:
*testing AI systems* and *using AI to test*.**

### 12.1 Testing AI systems

**[VERSIONED] A genuinely new attack surface with new vulnerability classes** that don't
map cleanly onto existing methodology. Scoping guidance now explicitly holds that **once AI
features are part of the application boundary, the test has to ask different questions**
about providers, self-hosting, output handling, keys, logging, and worst-case impact.

**What you're testing for**: **prompt injection** (direct and indirect — and note that
this is **not solved**, so any system giving a model tools plus untrusted input has real
exposure), **insecure output handling** (model output reaching an interpreter, a browser,
or a shell), **excessive agency** (what can the agent's tools actually do, and what happens
when it's manipulated into doing it), **data leakage** through retrieval systems and
context, **training data and model supply chain**, and **denial-of-wallet**. **OWASP's LLM
and Agentic Applications material** is the reference framework.

### 12.2 AI in offensive work

**[VERSIONED] This moved from novelty to material capability faster than almost anything
else in the field, and the honest picture cuts both ways.**

**What the evidence shows:**
- **Adoption is near-universal.** Bugcrowd's 2026 research reports **82% of hackers already
  use AI in their workflows**, mainly for automation, code analysis, and getting unstuck.
- **The capability is real.** **XBOW's autonomous agent took #1 on HackerOne in June 2025**
  and later published 1,060+ valid submissions. **ARTEMIS (December 2025) reportedly beat
  9 of 10 human pentesters on a live 8,000-host enterprise network at $18/hour.** A
  February 2026 benchmark of an LLM agent on a realistic five-host Active Directory
  engagement **compromised four of five hosts for $28.50 in API fees**, running multiple
  exploitation paths concurrently.
- **Adversaries use it too.** Google's threat intelligence confirms attackers integrating
  AI for reconnaissance, social engineering, and exploit work.
- **The productivity gain is measurable and mundane**: reported ~**35% reduction in
  time-to-report**, which makes sense given how much of a report is templated (§13).

**What the evidence also shows:**
- **The gap is judgment, not execution.** Synack's CTO frames it well: AI performs broad
  attack-surface reconnaissance well, but **"where AI often falls short is in contextual
  understanding and creativity. Humans infer business logic that AI will sometimes miss"** —
  which is precisely §5.1 → `pentest-technical-domains`'s highest-value finding category.
- **Practitioners can't drive it well yet.** A HackerOne survey found **54% of respondents
  lacked the skills to effectively prompt, evaluate, and course-correct AI tools** during
  offensive work.
- **⚠️ It introduces new operational risk.** Handing an autonomous agent offensive tooling
  against live systems raises real safety and scope questions — **an agent that adapts is
  an agent that can wander outside where you meant it to go.** In an engagement governed by
  §1 → `pentest-authorization-scoping-and-methodology`'s rules of engagement, that is not a minor concern; it is a potential authorization
  violation with your name on it. Platforms addressing this use graduated trust levels and
  hard-blocked destructive commands.
- **The vulnerability-report ecosystem is straining.** AI-generated reports at volume are a
  live problem for maintainers and triage teams.

**⚠️ The marketing substantially outruns the evidence**, and this is a domain where vendor
material dominates the search results. **"Manual pentesting will be a boutique service by
2027" is a vendor claim, not a finding.** The defensible position: **treat AI agents as
augmented operators, not replacements**, keep a human accountable for scope and for the
report, and note that the parts of the job with the highest client value — scoping,
business-logic reasoning, impact judgment, and communication (§13) — are exactly the parts
the tooling is weakest at.

---

## §13. Reporting

**[DURABLE] This is the product. Everything before it is raw material.**

### 13.1 The structure

```
EXECUTIVE SUMMARY      1–2 pages, no jargon. What was tested, what the risk is,
                       what to do first. ⚠️ Assume this is ALL an executive reads
SCOPE & METHODOLOGY    exactly what was and wasn't tested, when, by what method,
                       and the LIMITATIONS (time-boxed? staging? credentialed?)
FINDINGS SUMMARY       table: ID, title, severity, affected asset, status
DETAILED FINDINGS      one per finding — the core of the document (§13.2)
STRATEGIC OBSERVATIONS the patterns behind the findings. Often the most valuable page
APPENDICES             tooling, source IPs, full output, timeline
```

### 13.2 A finding that is actually usable

Every finding needs: **a title that names the issue**, **severity with the reasoning shown**
(CVSS if you use it — **and state the vector string**, because a bare number is
unfalsifiable), **affected assets enumerated**, **a plain-language description of what it
is**, **business impact in the client's terms** ("an unauthenticated attacker can read any
customer's records" — not "IDOR in the user endpoint"), **reproduction steps a competent
engineer can follow without you**, **evidence** (redacted screenshots, requests/responses),
**specific remediation** — not "validate input" but what to change, where, and to what —
plus references (CWE, vendor docs), and **retest status**.

> **⚠️ GOTCHA — the reporting failures that waste the engagement:**
> - **Severity inflation.** Rating everything high destroys your credibility and the
>   client's ability to prioritize. **If everything is critical, nothing is.**
> - **CVSS base score used as the priority.** Base score ignores your client's environment
>   entirely. **Use environmental/temporal metrics, or say plainly that the score is a
>   starting point.** Consider **EPSS** for likelihood-of-exploitation context.
> - **Scanner output pasted in as findings.** Unverified findings are the fastest way to be
>   dismissed. **Verify everything you report, or label it explicitly as unverified.**
> - **No reproduction steps**, so the fix can't be validated.
> - **Generic remediation** copied from a template.
> - **Burying the important finding** among forty informational ones.
> - **Missing the pattern.** Fifteen instances of missing access control is one systemic
>   finding about how the team builds authorization — **say that**, because fixing fifteen
>   endpoints doesn't fix the sixteenth they ship next month.
> - **No negative findings.** "We tested X and found it well-implemented" is genuinely
>   valuable and almost never written.

**[DURABLE] Write the executive summary for someone who will make a budget decision from it
and read nothing else.** That is not a compromise; that is the job.

### 13.3 After the report

**Debrief both audiences** — technical walkthrough for engineers, risk framing for
leadership. **Support remediation** (the questions afterwards are where a lot of real value
transfers). **Retest and issue a delta.** And **handle the artifacts**: destroy engagement
data per the agreed schedule, confirm in writing, and remove every persistence mechanism
and account from §11.1's inventory.