Performing Live Cluster Penetration Testing

---
name: performing-live-cluster-penetration-testing
description: >-
  Run a structured, authorized internal penetration test from INSIDE a live
  Kubernetes cluster (Azure AKS focus). Covers pod recon, Azure workload-identity
  and Key Vault credential escalation, internal-only endpoint testing, IMDS
  reachability, RBAC and privilege-escalation checks, network-policy gaps, and
  source-code-to-runtime auth correlation — the classes of finding that are only
  discoverable from inside the perimeter and invisible to the passive/static
  skills in this pack. Read-only and exec-only: never writes to the cluster,
  never runs against production. Use when: an engagement has an authorized
  dev/test AKS cluster in scope and the user wants internal cluster testing,
  lateral-movement validation, or shared-identity / internal-endpoint checks.
  Trigger with: "test the cluster from inside", "internal AKS pentest",
  "workload identity escalation", "can pods read each other's secrets",
  "internal endpoint test", "live cluster pentest".
---

# Performing Live Cluster Penetration Testing

Internal, authenticated penetration test executed from within a running
Kubernetes cluster. The passive skills in this pack (headers, TLS, CORS, SAST,
secrets scanning) stop at the perimeter. The most severe real findings —
shared workload identity granting every pod the same Key Vault reach, internal
endpoints gated only by a shared secret, plaintext credentials in pod env —
are only observable from a shell inside a pod. This skill covers that gap.

> This skill performs ACTIVE testing against live infrastructure. It runs only
> after `confirming-pentest-authorization` has passed AND the operator has
> explicitly confirmed the target cluster is a dev/test cluster. See **Safety
> rules** before any other step. Stop and refuse if either gate is unmet.

## Prerequisites

- A passing ROE attestation from `confirming-pentest-authorization`, with the
  target cluster listed as an in-scope target and the time window current.
- `kubectl` on PATH, and a kubeconfig whose current context points at the
  authorized cluster.
- For the Azure-specific phases (B, D): the target is Azure AKS using workload
  identity federation (the common AKS pattern).
- For Phase G: the service source repositories available locally (read-only).

## Step 0 — Authorization & environment gate (mandatory, do first)

Do not run any command from later phases until all of the following pass. If any
fails, STOP and report a CRITICAL process finding instead of proceeding.

1. **ROE check.** Confirm `confirming-pentest-authorization` passed and the
   cluster is named in the in-scope list. If not, refuse.
2. **Cluster identity check — never prod.** Print the context and cluster, and
   require the operator to confirm it is dev/test:
   ```bash
   kubectl config current-context
   kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}'; echo
   ```
   Refuse if the context, server host, or any namespace carries a prod marker
   (`prod`, `production`, `prd`, `live`). When in doubt, treat it as prod and stop.
3. **TLS flag for IP-based AKS dev endpoints.** AKS dev API servers are often
   reached by IP with a self-signed cert. When (and only when) the server is an
   IP, append `--insecure-skip-tls-verify` to kubectl calls. Do NOT use this
   flag against a named/FQDN endpoint.
4. **Record the plan.** State which namespaces and pods are in scope. Only touch those.

## Step 1 — Phase A: Pod reconnaissance (read-only)

```bash
kubectl get pods -A -o wide
kubectl get ns
kubectl get serviceaccounts -A
```

For each in-scope service pod, read its declared environment, mounted secrets, and volumes:

```bash
kubectl get pod <pod> -n <ns> -o yaml      # env, envFrom, volumeMounts, serviceAccountName
kubectl exec <pod> -n <ns> -- env           # live env as the process sees it
```

Flag credential patterns: `DATABASE_URL`, `*_SECRET`, `*_KEY`, `*_TOKEN`, `*_PASSWORD`.
Check for Azure workload-identity markers: `AZURE_FEDERATED_TOKEN_FILE`, `AZURE_CLIENT_ID`, `AZURE_TENANT_ID`.

## Step 2 — Phase B: Workload identity & cloud credential escalation

Goal: prove whether a pod's identity can read secrets beyond its own service's need.

1. If `AZURE_FEDERATED_TOKEN_FILE` present, exchange the federated token for an Azure AD access token scoped to Key Vault using the pod's own `AZURE_CLIENT_ID`/`AZURE_TENANT_ID`.
2. Enumerate which Key Vault secrets that token can list/read.
3. **Cross-service test:** from pod A's identity, attempt to read a secret belonging to pod B's service. Success = CRITICAL (shared identity, lateral movement).
4. Record per-pod the full set of readable secrets.

## Step 3 — Phase C: Internal endpoint testing

```bash
kubectl get svc -A        # ClusterIP = internal-only surface
```

For each internal service not exposed via ingress, from a neighbouring pod:
- **Shared-secret-gated endpoints:** replay a secret obtained in Phase B against another service's `/internal/*` endpoint. Cross-service secret acceptance = broken trust boundary.
- **Unauthenticated operational endpoints:** probe `/health`, `/metrics`, `/debug`, admin endpoints on internal ports.

## Step 4 — Phase D: IMDS & instance metadata

```bash
kubectl exec <pod> -n <ns> -- curl -s -H "Metadata:true" \
  "http://169.254.169.254/metadata/instance?api-version=2021-02-01"
```

Flag if IMDS is reachable from pods (NetworkPolicy should block 169.254.169.254). If reachable, test managed-identity token retrieval.

## Step 5 — Phase E: RBAC & privilege escalation

```bash
kubectl auth can-i --list
kubectl auth can-i get secrets -n <other-ns>
kubectl auth can-i create pods -n <ns>
kubectl auth can-i '*' '*' -A
```

Any `create pods`, `pods/exec`, cross-namespace `get secrets`, or wildcard verb on a workload SA = privilege-escalation finding. Confirm with `auth can-i`; do NOT actually exploit write capabilities.

## Step 6 — Phase F: Network policy gaps

Test TCP connectivity across pods and namespaces. Test egress to public internet (`curl https://example.com`). Unrestricted egress = exfiltration path.

## Step 7 — Phase G: Source code + runtime correlation (recommended model: Fable)

Cross-reference live routes (OpenAPI/openapi.json) against source auth guards. Flag any route present at runtime with no auth middleware in source. Flag `include_in_schema=False` routes. Fable is the recommended model for this phase — the correlation is large-context and detail-sensitive.

## Safety rules (non-negotiable)

1. **Never run against production.** Step 0 gate must pass.
2. **Read-only + exec-only.** Allowed: `get`, `describe`, `logs`, `auth can-i`, `exec` for read/probe commands. FORBIDDEN: `apply`, `create`, `patch`, `delete`, `edit`, `scale`, any cluster mutation.
3. **Stay in scope.** Only namespaces/pods named in Step 0.
4. **`--insecure-skip-tls-verify` only for IP-based dev endpoints.**
5. **Redact secrets in report.** Full values go only to the encrypted engagement archive.

## Output format

```
## Live Cluster Pentest Report

### Environment
- Cluster: {name}
- Context: {kube-context}  (dev/test confirmed: yes)
- Namespaces tested: {list}
- Pods exec'd into: {list}

### Findings

#### [SEVERITY] Finding title
- **Phase**: A/B/C/D/E/F/G
- **Pod**: {pod-name} ({namespace})
- **Evidence**: exact command + output snippet (secrets redacted)
- **Exploit chain**: step-by-step
- **Remediation**: one line

### Phase Results Summary
| Phase | Tests Run | Findings | Pass |
|-------|-----------|----------|------|
| A Pod recon             |  |  |  |
| B Workload identity     |  |  |  |
| C Internal endpoints    |  |  |  |
| D IMDS                  |  |  |  |
| E RBAC / privesc        |  |  |  |
| F Network policy        |  |  |  |
| G Source correlation    |  |  |  |
```

## Success criteria

- Step 0 gate passed before any active command ran.
- All seven phases executed or explicitly marked N/A with reason.
- Every finding carries exact command, output evidence, exploit chain, one-line remediation.
- No write/mutation command issued against the cluster.
- Cross-service identity reach (Phase B) and internal-endpoint trust boundaries (Phase C) explicitly tested.