Nginx

---
name: nginx
description: "Expert coverage of NGINX OSS and NGINX Plus across all versions: master/worker event-driven architecture, reverse proxy and upstream configuration, load balancing algorithms, SSL/TLS termination, rate limiting, caching, NGINX Plus active health checks and session persistence, NGINX Ingress Controller for Kubernetes, and VirtualServer CRDs. Use for \"NGINX\", \"nginx.conf\", \"upstream\", \"proxy_pass\", \"NGINX Plus\", \"NGINX Ingress\", \"Ingress Controller\", \"VirtualServer CRD\". Do NOT use for cross-vendor comparison, platform selection, or category-wide architecture -- use the `load-balancing` skill. Do NOT use for Kubernetes-native service mesh sidecar/mTLS questions -- that's the `istio` skill (in the `containers` plugin)."
license: MIT
---

# NGINX

This skill covers NGINX OSS and NGINX Plus across all supported versions (NGINX OSS 1.x, NGINX Plus R30 through R35). Areas of expertise include:

- Master/worker event-driven architecture (epoll/kqueue)
- Reverse proxy and upstream configuration (proxy_pass, upstream blocks)
- Load balancing algorithms (round robin, least_conn, ip_hash, hash, random)
- SSL/TLS termination and certificate management
- Rate limiting (limit_req_zone, limit_conn_zone)
- Proxy caching (proxy_cache_path, cache keys, stale content)
- NGINX Plus features: active health checks, session persistence (sticky), live API, key-value store, JWT auth
- NGINX Ingress Controller for Kubernetes (VirtualServer CRD, annotations)
- Configuration patterns for microservices, API gateways, and web applications
- NGINX App Protect (WAF) for Plus deployments

When a question is version-specific, consult the matching `references/versions/` file. When the version is unknown, provide general guidance and note OSS vs Plus differences.

## How to Approach Tasks

1. **Classify** the request:
   - **Troubleshooting** -- Check error logs, configuration validation, upstream status
   - **Configuration** -- Load `references/best-practices.md` for upstream, SSL, rate limiting, caching patterns
   - **Architecture** -- Load `references/architecture.md` for master/worker model, event loop, Plus features
   - **Kubernetes** -- Apply Ingress Controller guidance below
   - **Performance** -- Worker tuning, connection limits, buffer sizing

2. **Determine OSS vs Plus** -- Many features (active health checks, sticky sessions, live API, JWT auth) require NGINX Plus. Always clarify which edition is in use.

3. **Load context** -- Read the relevant reference file for deep knowledge.

4. **Analyze** -- Apply NGINX-specific reasoning, not generic web server advice.

5. **Recommend** -- Provide actionable configuration snippets with explanations.

6. **Verify** -- Suggest validation steps (`nginx -t`, `curl` tests, access/error log analysis).

## Core Architecture

### Master/Worker Process Model
```
Master Process (root)
  +-- Worker Process 1 (handles thousands of connections via epoll/kqueue)
  +-- Worker Process 2
  +-- Worker Process N (typically: worker_processes auto = 1 per CPU core)
  +-- Cache Manager / Cache Loader (if caching enabled)
```

**Master Process**: Reads/validates configuration, creates/destroys workers, performs hot reload (`nginx -s reload` spawns new workers, drains old gracefully).

**Worker Process**: Handles all network I/O using non-blocking system calls. Single-threaded per worker; no shared memory between workers (except shared zones). Processes HTTP, TCP/UDP proxying, SSL, FastCGI, gzip, cache.

### Event-Driven Model
NGINX is fully asynchronous: one worker handles thousands of simultaneous connections using an event loop. This differs from thread-per-connection models and is highly efficient for high-concurrency, I/O-bound workloads.

### Hot Reload
`nginx -s reload` performs zero-downtime configuration updates:
1. Master reads and validates new configuration
2. Master spawns new worker processes with new config
3. Old workers stop accepting new connections
4. Old workers drain existing connections
5. Old workers exit after all connections complete (or timeout)

## Reverse Proxy and Load Balancing

### Upstream Configuration
```nginx
upstream app_backend {
    least_conn;
    
    server 192.168.10.11:8080 weight=3;
    server 192.168.10.12:8080 weight=2;
    server 192.168.10.13:8080 backup;
    
    keepalive 32;    # persistent connections to upstream
}

server {
    listen 80;
    server_name app.example.com;
    
    location / {
        proxy_pass http://app_backend;
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_connect_timeout 5s;
        proxy_read_timeout    60s;
        proxy_send_timeout    60s;
    }
}
```

### Load Balancing Algorithms (OSS)

| Method | Directive | Description |
|---|---|---|
| Round Robin | (default) | Sequential distribution |
| Least Connections | `least_conn` | Fewest active connections |
| IP Hash | `ip_hash` | Client IP-based affinity |
| Generic Hash | `hash $key` | Hash of any variable (URI, cookie) |
| Random | `random` | Random; `random two least_conn` for P2C |

### Server Parameters

| Parameter | Description |
|---|---|
| `weight=N` | Relative weight for round robin/least_conn |
| `backup` | Only used when all primary servers are down |
| `down` | Permanently marks server as unavailable |
| `max_fails=N` | Failures before temporary unavailability (passive health) |
| `fail_timeout=Ns` | Time to consider server unavailable after max_fails |
| `max_conns=N` | Limit concurrent connections to server |
| `slow_start=Ns` | Gradually ramp traffic to recovered server (Plus only) |

### Passive Health Checks (OSS)
NGINX OSS uses passive health detection based on actual traffic:
- `max_fails` (default 1): Number of failed requests before marking server down
- `fail_timeout` (default 10s): Window for counting failures AND duration of unavailability
- Limitation: Only detects failures after real user requests fail

## SSL/TLS Termination

```nginx
server {
    listen 443 ssl;
    server_name secure.example.com;
    
    ssl_certificate      /etc/ssl/certs/example.crt;
    ssl_certificate_key  /etc/ssl/private/example.key;
    ssl_protocols        TLSv1.2 TLSv1.3;
    ssl_ciphers          HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;
    ssl_session_cache    shared:SSL:10m;
    ssl_session_timeout  10m;
    ssl_stapling         on;
    ssl_stapling_verify  on;
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
}
```

### SSL Best Practices
- Minimum TLS 1.2 (TLS 1.3 preferred)
- Enable OCSP stapling for validation performance
- Use `ssl_session_cache shared:SSL:10m` for session resumption across workers
- Enable HSTS header
- Redirect HTTP to HTTPS at the server block level

## Rate Limiting

```nginx
# Define rate limit zone (10 requests/second per client IP)
limit_req_zone $binary_remote_addr zone=api_limit:10m rate=10r/s;

server {
    location /api/ {
        limit_req zone=api_limit burst=20 nodelay;
        limit_req_status 429;
        proxy_pass http://api_backend;
    }
}
```

- `rate=10r/s`: Sustained rate of 10 requests per second per key
- `burst=20`: Allow 20 requests to burst above the rate
- `nodelay`: Process burst immediately rather than queuing
- Without `nodelay`: Excess requests are queued (delayed) up to burst limit

### Connection Limiting
```nginx
limit_conn_zone $binary_remote_addr zone=conn_limit:10m;

server {
    location / {
        limit_conn conn_limit 10;    # max 10 simultaneous connections per IP
    }
}
```

## Proxy Caching

```nginx
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=app_cache:10m max_size=1g inactive=60m;

server {
    location / {
        proxy_cache app_cache;
        proxy_cache_valid 200 10m;
        proxy_cache_valid 404 1m;
        proxy_cache_use_stale error timeout updating http_500 http_502 http_503;
        add_header X-Cache-Status $upstream_cache_status;
        proxy_pass http://app_backend;
    }
}
```

- `proxy_cache_use_stale`: Serve stale content when backend is unavailable (resilience)
- `$upstream_cache_status`: Reports HIT/MISS/BYPASS/STALE in response header

## NGINX Plus Features

### Active Health Checks
```nginx
upstream app_backend {
    zone backend 64k;
    server 192.168.10.11:8080;
    server 192.168.10.12:8080;
}

server {
    location / {
        proxy_pass http://app_backend;
        health_check interval=5s fails=3 passes=2 uri=/health;
    }
}
```

- Proactively checks backend health independent of client traffic
- Detects failures before any user is affected
- Requires `zone` directive in upstream block (shared memory for state)

### Session Persistence (Sticky Sessions)
```nginx
upstream app_backend {
    zone backend 64k;
    server 192.168.10.11:8080;
    server 192.168.10.12:8080;
    
    sticky cookie srv_id expires=1h domain=.example.com path=/;
}
```

Three sticky methods:
- `sticky cookie`: LB inserts a cookie identifying the server
- `sticky learn`: LB learns from application-set cookies
- `sticky route`: Route based on cookie or URI value

### Live Activity Monitoring API
```nginx
server {
    listen 8080;
    location /api/ {
        api write=on;
        allow 10.0.0.0/8;
        deny all;
    }
}
```

API endpoints: `/api/8/http/upstreams`, `/api/8/stream/upstreams`, `/api/8/connections`, `/api/8/ssl`

Enables dynamic upstream management without configuration reload.

### Key-Value Store
```nginx
keyval_zone zone=blocklist:1m;
keyval $remote_addr $blocked zone=blocklist;

server {
    if ($blocked) { return 403; }
}
```
Keys set/updated via REST API dynamically without reload.

### JWT Authentication
```nginx
server {
    auth_jwt "API Access";
    auth_jwt_key_file /etc/nginx/jwk.json;
    
    location /api/v1/ {
        auth_jwt_claim_set $user sub;
        proxy_set_header X-User $user;
    }
}
```

## NGINX Ingress Controller for Kubernetes

### Context
The official F5 NGINX Ingress Controller (nginx/kubernetes-ingress) is the recommended Kubernetes ingress solution. The community ingress-nginx project was retired in November 2025.

### Standard Ingress Resource
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-ingress
  annotations:
    nginx.org/proxy-connect-timeout: "10s"
    nginx.org/proxy-read-timeout: "60s"
spec:
  ingressClassName: nginx
  tls:
  - hosts: [app.example.com]
    secretName: app-tls
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 80
```

### VirtualServer CRD (Advanced Routing)
```yaml
apiVersion: k8s.nginx.org/v1
kind: VirtualServer
metadata:
  name: app-vs
spec:
  host: app.example.com
  upstreams:
  - name: app-v1
    service: app-v1-service
    port: 80
  - name: app-v2
    service: app-v2-service
    port: 80
  routes:
  - path: /
    splits:
    - weight: 90
      action:
        pass: app-v1
    - weight: 10
      action:
        pass: app-v2     # canary
```

VirtualServer CRD advantages over standard Ingress:
- Traffic splitting (canary deployments)
- Custom error pages
- Per-route rate limiting
- Advanced health checks
- Circuit breaker patterns

### ConfigMap Global Configuration
```yaml
kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-config
  namespace: nginx-ingress
data:
  proxy-connect-timeout: "10s"
  worker-processes: "auto"
  ssl-protocols: "TLSv1.2 TLSv1.3"
```

## Stream (L4) Proxying

NGINX supports TCP/UDP proxying via the `stream` module:
```nginx
stream {
    upstream db_backend {
        server 192.168.10.20:5432;
        server 192.168.10.21:5432;
    }
    
    server {
        listen 5432;
        proxy_pass db_backend;
        proxy_connect_timeout 5s;
    }
}
```

Use for: Database load balancing, MQTT, custom TCP protocols, DNS (UDP).

## Common Pitfalls

1. **Forgetting `proxy_set_header Host`** -- Without it, the upstream receives the upstream group name as the Host header, not the original client hostname. Always set `Host $host`.

2. **OSS without health checks** -- NGINX OSS has only passive health detection. Dead servers receive traffic until a real request fails. Use NGINX Plus for active health checks in production.

3. **`keepalive` misunderstanding** -- The `keepalive` directive in upstream sets the maximum number of idle keepalive connections PER WORKER, not total. With 4 workers and `keepalive 32`, up to 128 idle connections are maintained.

4. **Configuration reload is not instant** -- `nginx -s reload` spawns new workers but old workers drain connections. During drain, both old and new configs are active. Long-lived connections (WebSocket) may delay old worker shutdown.

5. **`ip_hash` behind NAT/CDN** -- When clients share IPs (corporate NAT, CDN), `ip_hash` sends all their traffic to one server. Use `hash $cookie_session consistent` or Plus sticky cookies instead.

6. **Rate limiting without `nodelay`** -- Without `nodelay`, requests exceeding the rate are delayed (queued) up to the burst limit. This can cause unexpected latency spikes. Use `nodelay` for API rate limiting.

7. **Missing `zone` for Plus features** -- Active health checks, session persistence, and the live API all require a `zone` directive in the upstream block. Without it, these features silently fail.

8. **Using community ingress-nginx in new deployments** -- The community ingress-nginx project was retired in November 2025. Use the official F5 NGINX Ingress Controller for new Kubernetes deployments.

## Version-Specific Guidance

| Version | Reference | What's version-specific |
|---|---|---|
| plus-r35 | `references/versions/plus-r35.md` | NGINX Plus R35, latest features, API version updates |

## Reference Files

Load these when you need deep knowledge:

- `references/architecture.md` -- Master/worker model, event-driven I/O, Plus feature architecture, Ingress Controller internals. Read for "how does X work" questions.
- `references/best-practices.md` -- Upstream configuration, SSL setup, rate limiting patterns, caching design, Kubernetes patterns. Read for design and configuration questions.