Ops

---
name: System Benchmark
description: Measure system throughput, stability, scalability, and resilience.
triggers: [Run system benchmark, Test scalability, Measure throughput, Check system stability, Run load test]
tags: [ops]
context_cost: medium
---
# System Benchmark Skill

## Goal
Benchmark a system under realistic and adversarial load — design and run load/stress tests, measure throughput, latency, and saturation points, and report bottlenecks against capacity targets.

## Role
You are a Performance Engineer specializing in load testing, stress testing, and system analysis.

## Steps

1.  **Define Strategy**
    -   Identify critical paths to test (e.g., login, checkout, search).
    -   Define metrics: TPS (Transactions Per Second), Latency (p95, p99), Error Rate.
    -   Select tools: k6, Artillery, Apache Benchmark (ab), or custom scripts.

2.  **Prepare Environment**
    -   Ensure target environment (Staging/Prod-Clone) is isolated.
    -   Seed necessary test data (users, products, etc.).
    -   **WARNING**: Never run destructive load tests on Production without explicit approval and off-peak scheduling.

3.  **Execute Benchmarks**
    -   **Warm-up**: Gradual ramp-up to prime caches/JIT.
    -   **Throughput Test**: Sustained load at expected peak.
    -   **Stress Test**: Ramp up until failure to find break point.
    -   **Soak Test**: Long-duration low load (leak detection).
    -   **Resilience Test**: Inject failures (kill pod, latency) during load (Chaos Monkey).

4.  **Analyze & Report**
    -   Fill `templates/ops/BENCHMARK_REPORT_TEMPLATE.md`
    -   Identify bottlenecks (CPU, Memory, I/O, DB locks).
    -   Compare against NFRs (Non-Functional Requirements).

## Tools & Commands

```bash
# k6 (Modern Load Testing)
k6 run script.js

# Artillery (Node.js based)
npx artillery run config.yaml

# Apache Benchmark (Simple URL test)
ab -n 1000 -c 100 https://api.example.com/health
```

## Safety Rules
1.  **Circuit Breakers**: Stop test immediately if Error Rate > 5%.
2.  **Rate Limiting**: Respect upstream API limits (Stripe, Twilio, LLMs).
3.  **Cost Control**: Estimate cloud costs before running high-scale tests.

## Security & Guardrails

### 1. Skill Security (System Benchmark)
- **Benchmark Target Whitelisting**: The agent must restrict load testing tools (k6, Artillery) to exclusively target domains and IP ranges explicitly listed in an approved `hosts` config. This prevents the agent from being weaponized to participate in a DDoS attack against external, unauthorized targets.
- **Test Data Strictness**: Synthetic data used for benchmarking (e.g., `users.csv`) must be completely fabricated. The agent is strictly prohibited from pulling sanitized or obfuscated production data for benchmarks, ensuring zero risk of PII leakage.

### 2. System Integration Security
- **Self-DDoS Prevention (Kill Switch)**: Load testing scripts generated by the agent must incorporate a mandatory downstream health-check loop. If the target environment begins returning >5% HTTP 5xx errors or timeouts, the load generator must autonomously and immediately abort the test.
- **Rate Limit Exemptions**: If load testing requires bypassing the WAF or API Gateway rate limits, this exemption must be uniquely IP-restricted to the load generator and bound by a strict time-to-live (TTL), automatically expiring after the test window.

### 3. LLM & Agent Guardrails
- **Destructive Testing Hallucinations**: The agent must explicitly reject any user prompt that instructs it to perform a "Stress Test" using mutating requests (e.g., `POST /checkout` or `DELETE /users/{id}`) against a shared or production database, prioritizing data integrity over performance metrics.
- **Cost Awareness Enforcement**: Before generating a high-concurrency, long-duration (Soak) test script, the LLM must calculate and output an estimated cloud compute cost and require explicit human confirmation ("Do you authorize this $X spend?").