Artillery Load

---
name: Artillery Load Testing
description: Write and run Artillery load tests with YAML phases and scenarios, CSV data payloads, the expect plugin for functional checks, and ensure thresholds that fail CI when latency or error budgets are breached.
version: 1.0.0
author: thetestingacademy
license: MIT
tags: [artillery, load-testing, performance, stress-testing, yaml, thresholds, ci, api, latency]
testingTypes: [load, performance]
frameworks: [k6]
languages: [javascript]
domains: [api, web]
agents: [claude-code, cursor, github-copilot, windsurf, codex, aider, continue, cline, zed, bolt, gemini-cli, amp]
---

# Artillery Load Testing

This skill makes an AI agent author Artillery 2.x load tests as declarative YAML: realistic traffic phases, multi-step scenarios with captured variables, CSV-driven virtual user data, functional `expect` checks inside load flows, and `ensure` thresholds that turn latency regressions into red CI builds. Trigger it when a project contains `artillery.yml`, `artillery` in package.json, or when the user asks for load, stress, soak, or spike testing of an HTTP API or website in a Node.js stack.

## Core Principles

1. **Model traffic in phases, never one flat rate.** Real load has a warm-up, a ramp, and a sustained plateau. A single `arrivalRate` hides cold-start effects and autoscaling lag. Always define at least warm-up, ramp, and sustain phases.
2. **`arrivalRate` is new virtual users per second, not concurrency.** Each arriving VU runs the whole scenario. If your scenario takes 10 seconds and you arrive 50/sec, you have roughly 500 concurrent users. Calculate this before picking numbers.
3. **Thresholds belong in the test file, not in a wiki.** Use the `ensure` plugin so `artillery run` exits non-zero when p95/p99 or error rate budgets are blown. A load test that cannot fail is a demo, not a test.
4. **Assert correctness under load with `expect`.** A server returning 200 with an empty body at p99 latency is still broken. Check `statusCode`, `contentType`, and `hasProperty` inside the flow.
5. **Parameterize virtual users from CSV payloads.** Hammering one account exercises one cache line and one DB row. Use `payload` with hundreds of distinct credentials and SKUs to defeat caches realistically.
6. **Never load test production without a plan; never load test third parties at all.** Point Artillery at a staging environment sized like production, announce test windows, and rate-limit anything that crosses a vendor boundary.

## Setup

```bash
npm install --save-dev artillery@latest
npx artillery version

# Run a test locally
npx artillery run load/checkout.yml

# Save raw metrics for trend analysis
npx artillery run load/checkout.yml --output artillery-report.json
```

## Patterns

### 1. Phases and a multi-step scenario with capture

```yaml
# load/checkout.yml
config:
  target: https://staging-api.example.com
  http:
    timeout: 10
  phases:
    - duration: 60
      arrivalRate: 2
      name: warm-up
    - duration: 120
      arrivalRate: 5
      rampTo: 40
      name: ramp-to-peak
    - duration: 300
      arrivalRate: 40
      name: sustained-peak
  payload:
    path: users.csv
    fields:
      - email
      - password
    order: random
    skipHeader: true
  plugins:
    expect: {}
    ensure: {}
  ensure:
    thresholds:
      - http.response_time.p95: 250
      - http.response_time.p99: 500
    conditions:
      - expression: http.codes.200 > 0
        strict: true
    maxErrorRate: 1

scenarios:
  - name: login-browse-order
    flow:
      - post:
          url: /auth/login
          json:
            email: '{{ email }}'
            password: '{{ password }}'
          capture:
            - json: $.token
              as: authToken
          expect:
            - statusCode: 200
            - contentType: json
            - hasProperty: token
      - get:
          url: /products?category=audio
          headers:
            Authorization: 'Bearer {{ authToken }}'
          capture:
            - json: $[0].id
              as: productId
          expect:
            - statusCode: 200
      - post:
          url: /orders
          headers:
            Authorization: 'Bearer {{ authToken }}'
          json:
            productId: '{{ productId }}'
            quantity: 1
          expect:
            - statusCode: 201
            - hasProperty: orderId
      - think: 2
```

### 2. CSV payload file

```csv
email,password
loadtest-001@example.com,Str0ngPass!001
loadtest-002@example.com,Str0ngPass!002
loadtest-003@example.com,Str0ngPass!003
loadtest-004@example.com,Str0ngPass!004
```

Generate hundreds of rows with a one-liner instead of writing them by hand:

```bash
seq -w 1 500 | awk -F, 'BEGIN{print "email,password"} {printf "loadtest-%s@example.com,Str0ngPass!%s\n", $1, $1}' > users.csv
```

### 3. Custom logic with a JavaScript processor

```yaml
# In config:
config:
  target: https://staging-api.example.com
  processor: ./processor.js

scenarios:
  - name: create-order-with-dynamic-payload
    flow:
      - function: generateOrderPayload
      - post:
          url: /orders
          json:
            sku: '{{ sku }}'
            quantity: '{{ quantity }}'
          afterResponse: logSlowResponse
```

```js
// processor.js
module.exports = { generateOrderPayload, logSlowResponse };

function generateOrderPayload(context, events, done) {
  // Runs before the request; sets template variables on the VU context
  context.vars.sku = `SKU-${1000 + Math.floor(Math.random() * 9000)}`;
  context.vars.quantity = 1 + Math.floor(Math.random() * 4);
  return done();
}

function logSlowResponse(requestParams, response, context, events, done) {
  const tookMs = response.timings ? response.timings.phases.total : 0;
  if (tookMs > 1000) {
    console.warn(`SLOW ${requestParams.url} -> ${response.statusCode} in ${tookMs}ms`);
    events.emit('counter', 'custom.slow_responses', 1);
  }
  return done();
}
```

### 4. CI gate in GitHub Actions

The `ensure` plugin makes Artillery exit with code 1 on any threshold breach, so the job fails without extra scripting.

```yaml
# .github/workflows/load-test.yml
name: load-test
on:
  workflow_dispatch:
  schedule:
    - cron: '0 2 * * 1'

jobs:
  artillery:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
      - run: npm ci
      - name: Run load test against staging
        run: npx artillery run load/checkout.yml --output artillery-report.json
        env:
          ARTILLERY_DISABLE_TELEMETRY: 'true'
      - name: Upload raw metrics
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: artillery-report
          path: artillery-report.json
```

## Best Practices

- Name every phase; phase names appear in the metrics output and make reports readable.
- Use `think` steps (1-3 seconds) between requests so VUs pace like humans instead of a retry storm.
- Keep one scenario per user journey and weight them with `weight:` to mirror real traffic mix (for example 70 percent browse, 25 percent search, 5 percent checkout).
- Pin the p95 and p99 thresholds to your SLOs, not to whatever the system currently does.
- Run a 1-VU smoke (`--overrides '{"config":{"phases":[{"duration":10,"arrivalRate":1}]}}'`) before any big run to catch broken auth and 4xx noise cheaply.
- Store `--output` JSON artifacts from every CI run so you can diff p95 across releases.
- Set `http.timeout` explicitly; the 120-second default hides hangs as slow successes.

## Anti-Patterns

- A single phase with a huge `arrivalRate` and no ramp: you are testing the load balancer's SYN queue, not your application.
- Asserting nothing: runs that "pass" while every response is a 500 error page, because no `expect` or `ensure` block exists.
- Reusing one hardcoded user for every VU: session caches and row locks make results meaninglessly optimistic or pessimistic.
- Load testing through localhost against a dev-mode server with hot reload: numbers are fiction; test a production build on production-like hardware.
- Comparing runs executed from a laptop on Wi-Fi against runs from CI: generator location and capacity are part of the experiment, keep them constant.
- Cranking `arrivalRate` past what one generator machine can produce: watch for Artillery's own CPU warnings, and split load across workers (or Fargate via `artillery run-fargate`) instead.

## When to Trigger This Skill

- The repository contains `artillery.yml`, files under `load/` or `perf/` with Artillery config, or `artillery` in `package.json` devDependencies.
- The user asks to "load test", "stress test", "soak test", or "spike test" an HTTP API, GraphQL endpoint, or website in a JavaScript or Node.js project.
- The user wants latency SLO enforcement (p95/p99 budgets, max error rate) wired into CI.
- An existing Artillery suite needs CSV data, captured variables, custom processor functions, or threshold gating added.
- Choose Artillery over k6 when the team prefers declarative YAML scenarios and npm-native tooling; recommend k6 instead when tests need rich scripted logic in JavaScript with custom metrics math.