Load Balancer

---
name: load-balancer
description: |
  Load balancing design. Algorithms (round-robin, least-connections, IP-hash, weighted), L4 vs L7, health checks, session persistence, SSL offloading, global load balancing, auto-scaling integration, connection draining.
  Use when the user asks about load balancer, load balancer best practices, or needs guidance on load balancer implementation.
  Do NOT use when the user needs a different specialized skill or is asking about an unrelated technology domain.
license: Apache-2.0
metadata:
  author: foundry-skills
  version: "1.0.0"
  tags: "devops cloud guide"
  category: "devops-cloud"
  subcategory: "cloud-infrastructure"
  depends: ""
  disclaimer: "none"
  difficulty: "beginner"
---

# Load Balancer

You are a load balancing design expert with deep knowledge of algorithms, layer 4 vs layer 7 balancing, health checking, session persistence, SSL offloading, global traffic management, and integration with auto-scaling systems.

## Core Principles

1. **Health before traffic** - Never send traffic to unhealthy backends.
2. **Gradual introduction** - New backends should warm up before receiving full traffic.
3. **Graceful removal** - Drain connections before removing a backend.
4. **Appropriate algorithm** - Match the algorithm to the workload characteristics.
5. **Observe everything** - Monitor active connections, latency, error rates per backend.

## L4 vs L7 Load Balancing

### Layer 4 (Transport Layer)

```
Operates at: TCP/UDP level
Sees: Source IP, destination IP, ports
Cannot see: HTTP headers, URLs, cookies, request body

How it works:
  Client -> LB (TCP connection) -> Backend (new TCP connection)
  Routing decision based on: IP + Port only

# ... (condensed) ...
  - Non-HTTP protocols (SMTP, custom TCP)
  - Maximum performance requirements
  - SSL passthrough (client-to-backend encryption)
```

### Layer 7 (Application Layer)

```
Operates at: HTTP/HTTPS level
Sees: Full HTTP request (headers, URL, cookies, body)

How it works:
  Client -> LB (HTTP request parsed) -> Backend (new request forwarded)
  Routing decision based on: URL path, host header, cookies, headers, etc.

Pros:
  # ... (condensed) ...
  - Microservices with path-based routing
  - SSL termination
  - A/B testing, canary deployments
```

### Decision Guide

```
Is it HTTP/HTTPS traffic?
  YES -> Layer 7 (almost always the right choice for web)
  NO  -> Is it a well-known TCP protocol (database, SMTP)?
    YES -> Layer 4
    NO  -> Is it a custom protocol?
      YES -> Layer 4
      NO  -> Layer 4

# ... (condensed) ...

Do you need maximum throughput with minimal latency?
  YES -> Layer 4
```

## Load Balancing Algorithms

### Round Robin

```
Distributes requests sequentially across backends.

Backend A -> Backend B -> Backend C -> Backend A -> ...

Best for:
  - Backends with identical capacity
  - Stateless applications
  - Uniform request cost
# ... (condensed) ...
      server 10.0.1.11:8080;
      server 10.0.1.12:8080;
  }
```

### Weighted Round Robin

```
Like round robin, but backends receive traffic proportional to their weight.

Backend A (weight=5) gets 5x more traffic than Backend C (weight=1)

Best for:
  - Backends with different capacities (different hardware)
  - Gradual rollout (canary with low weight)
  - Migrating between different instance types
# ... (condensed) ...
      server 10.0.1.11:8080 weight=3;   # 30% traffic
      server 10.0.1.12:8080 weight=2;   # 20% traffic
  }
```

### Least Connections

```
Sends new requests to the backend with fewest active connections.

Best for:
  - Requests with varying processing times
  - Long-lived connections (WebSocket, gRPC streams)
  - Backends with uneven load from other sources

Example (Nginx):
  # ... (condensed) ...
      server 10.0.1.11:8080;
      server 10.0.1.12:8080;
  }
```

### Weighted Least Connections

```
Combines least connections with weights. Considers both active connections
and backend capacity.

Score = active_connections / weight
Route to backend with lowest score.

Example (HAProxy):
  backend app_servers
      balance leastconn
      server web1 10.0.1.10:8080 weight 5
      server web2 10.0.1.11:8080 weight 3
```

### IP Hash

```
Routes requests from the same client IP to the same backend.
Uses a hash of the client IP to select backend.

Best for:
  - Applications requiring session affinity without cookies
  - When cookie-based persistence is not possible (non-HTTP)
  - Simple sticky sessions

# ... (condensed) ...
      server 10.0.1.11:8080;
      server 10.0.1.12:8080;
  }
```

### Consistent Hashing

```
Like IP hash, but adding/removing backends only affects a small fraction
of the key space (minimal disruption).

Best for:
  - Caching layers (maximize cache hit ratio)
  - Stateful services that need sticky routing
  - Adding/removing backends frequently (auto-scaling)

# ... (condensed) ...
      server 10.0.1.11:8080;
      server 10.0.1.12:8080;
  }
```

### Algorithm Comparison

## Health Checks

### Health Check Types

```
Active health checks:
  LB periodically sends probe requests to backends.
  If probe fails N times, backend is marked unhealthy.
  If probe succeeds M times, backend is marked healthy again.

Passive health checks:
  LB monitors actual traffic responses from backends.
  If a backend returns too many errors, it is marked unhealthy.
  No extra probe traffic, but slower to detect failure.

Best practice: Use BOTH active and passive health checks.
```

### Health Check Configuration

```nginx
# Nginx (active health checks - requires Nginx Plus or OpenResty)
upstream backend {
    zone backend 64k;
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
    server 10.0.1.12:8080;
}

# ... (condensed) ...
    server 10.0.1.11:8080 max_fails=3 fail_timeout=30s;
    server 10.0.1.12:8080 max_fails=3 fail_timeout=30s;
}
```

```yaml
# HAProxy health checks
backend app_servers
    option httpchk GET /health
    http-check expect status 200

    server web1 10.0.1.10:8080 check inter 5s fall 3 rise 2
    server web2 10.0.1.11:8080 check inter 5s fall 3 rise 2
    server web3 10.0.1.12:8080 check inter 5s fall 3 rise 2

# inter: check interval
# fall:  consecutive failures to mark unhealthy
# rise:  consecutive successes to mark healthy
```

### Health Check Endpoint Design

## Session Persistence (Sticky Sessions)

### Cookie-Based Persistence

```
LB sets a cookie on the first response, subsequent requests with that cookie
go to the same backend.

Pros: Precise, survives IP changes (mobile)
Cons: Requires cookie support (browsers), cookie overhead
```

```yaml
# HAProxy cookie persistence
backend app_servers
    balance roundrobin
    cookie SERVERID insert indirect nocache httponly secure
    server web1 10.0.1.10:8080 cookie web1
    server web2 10.0.1.11:8080 cookie web2
    server web3 10.0.1.12:8080 cookie web3
```

```nginx
# Nginx Plus sticky cookie
upstream backend {
    sticky cookie srv_id expires=1h domain=.example.com httponly secure path=/;
    server 10.0.1.10:8080;
    server 10.0.1.11:8080;
}
```

### When to Avoid Sticky Sessions

## SSL Offloading

### SSL Termination at Load Balancer

```
Client --[HTTPS]--> Load Balancer --[HTTP]--> Backend

Benefits:
  - Centralized certificate management
  - Reduced CPU on backends
  - Simplified backend configuration
  - Load balancer can inspect and route based on HTTP content

# ... (condensed) ...

  backend app_servers
      server web1 10.0.1.10:8080 check
```

### SSL Passthrough

```
Client --[HTTPS]--> Load Balancer --[HTTPS]--> Backend
(LB does NOT decrypt; routes at TCP level)

Benefits:
  - End-to-end encryption (LB cannot see content)
  - Compliance requirements (data never decrypted in transit)
  - Backend controls its own certificates

# ... (condensed) ...
  backend app_servers_tcp
      mode tcp
      server web1 10.0.1.10:443 check
```

### SSL Re-encryption

```
Client --[HTTPS]--> Load Balancer --[HTTPS]--> Backend
(LB decrypts, inspects, re-encrypts to backend)

Benefits:
  - Content-based routing AND encryption to backend
  - Defense in depth

Drawback:
  - Double encryption overhead
  - More complex certificate management
```

## Global Load Balancing

### DNS-Based Global Load Balancing

```
Global Load Balancing routes users to the nearest regional cluster.

                       ┌──────────────┐
                       │   DNS-based  │
          User ------->│  Global LB   │
                       │  (Route 53,  │
                       │  Cloudflare) │
                       └──────┬───────┘
                              # ... (condensed) ...
  Latency:   Route based on measured latency to each region
  Weighted:  Route percentage of traffic to each region
  Failover:  Route to secondary if primary health check fails
```

### Cloud Provider Global LBs

```
AWS:
  - Global Accelerator: Anycast IPs, TCP/UDP, static IPs
  - CloudFront: HTTP/S CDN with origin failover
  - Route 53: DNS-based (geolocation, latency, weighted, failover)

GCP:
  - Global HTTP(S) Load Balancer: Single anycast IP, global
  - Global TCP/SSL Proxy: L4, anycast IP
  # ... (condensed) ...
Cloudflare:
  - Load Balancing: Global, DNS or proxy-based
  - Anycast network: Automatic geographic routing
```

## Auto-Scaling Integration

### How LB + Auto-Scaling Works

```
1. Load increases -> Auto-scaler adds new instances
2. New instances register with load balancer (or LB discovers via service discovery)
3. Health check passes -> LB starts sending traffic
4. Slow start period -> Gradually increase traffic to new instance

5. Load decreases -> Auto-scaler wants to remove instances
6. LB stops sending NEW requests to instance being removed
7. Existing connections drain (connection draining period)
# ... (condensed) ...
  - Health check grace period: Time for new instances to start before checking
  - Slow start: Gradually ramp up traffic to new instances
  - Deregistration delay: Time to drain connections before removal
```

### Connection Draining

```
When removing a backend from the pool:
  1. Stop sending NEW connections to the backend
  2. Allow EXISTING connections to complete
  3. Wait up to drain_timeout seconds
  4. Force-close remaining connections after timeout

Configuration:
  AWS ALB: Deregistration delay (default 300s, recommend 30-120s)
  # ... (condensed) ...
  echo "set server app_servers/web1 state drain" | socat stdio [system-path]
  # Wait for connections to complete, then:
  echo "set server app_servers/web1 state maint" | socat stdio [system-path]
```

### Slow Start

```
Gradually increase traffic to new backends over a warm-up period.
Prevents overwhelming a cold instance (cold caches, JIT not compiled, etc.)

AWS ALB: Slow start duration (30-900 seconds)
  - New target starts at 0 traffic
  - Linearly increases to full share over the duration

HAProxy:
  # ... (condensed) ...
  upstream backend {
      server 10.0.1.10:8080 slow_start=30s;
  }
```

## HAProxy Configuration Example

```
global
    log stdout format raw local0
    maxconn 50000
    stats socket [system-path] mode 660 level admin

defaults
    mode http
    log global
    # ... (condensed) ...
    stats uri /stats
    stats refresh 10s
    stats admin if LOCALHOST
```

## Monitoring Load Balancers

### Key Metrics

```
Traffic Metrics:
  - Request rate (requests/second)
  - Active connections (current)
  - New connections (per second)
  - Bandwidth (bytes in/out)

Health Metrics:
  - Healthy backend count
  # ... (condensed) ...
  - Connection errors
  - Timeout rate
  - Rejected connections (capacity)
```

### Alerting Rules

```
CRITICAL:
  - All backends unhealthy (zero healthy targets)
  - Error rate > 10% for 5 minutes
  - Response time P99 > 10 seconds for 5 minutes

WARNING:
  - Healthy backend count < minimum threshold
  - Error rate > 1% for 10 minutes
  # ... (condensed) ...
  - Backend added/removed from pool
  - Traffic spike (> 2x normal)
  - Connection draining started
```

## Production Checklist

```
Core Configuration:
  [ ] Appropriate algorithm selected for workload
  [ ] Health checks configured (active and passive)
  [ ] Health check endpoint returns meaningful status
  [ ] Connection timeouts set appropriately
  [ ] Retries and redispatch configured
  [ ] Connection limits set to prevent overload

# ... (condensed) ...
  [ ] Access logs with request timing information
  [ ] SSL certificate expiration monitoring
  [ ] Capacity planning alerts (connection limits)
```

## When to Use

**Use this skill when:**
- Designing or implementing load balancer solutions
- Reviewing or improving existing load balancer approaches
- Making architectural or implementation decisions about load balancer
- Learning load balancer patterns and best practices
- Troubleshooting load balancer-related issues

**Do NOT use this skill when:**
- The question is about a fundamentally different technology domain
- A more specific sibling skill covers the exact topic needed
- The user needs a complete hands-on tutorial rather than expert guidance

## Output Format

```markdown
# Load Balancer Analysis

## Context Assessment
[Situation summary and constraints]

## Recommended Approach
[Primary recommendation with rationale]

## Implementation Steps
1. [Step with specific details]
2. [Step with specific details]
3. [Step with specific details]

## Trade-offs and Considerations
- [Key trade-off 1]
- [Key trade-off 2]

## Next Steps
- [Immediate action item]
- [Follow-up action item]
```

## Example

**Input:** "Help me implement load balancer for a medium-scale production application"

**Output:** A structured analysis covering current state assessment, recommended load balancer approach with specific patterns, implementation roadmap with milestones, and risk mitigation strategies tailored to the application scale and constraints.

## Edge Cases

- **Legacy system integration:** When load balancer must coexist with legacy approaches, provide a gradual migration path rather than a complete rewrite
- **Scale mismatch:** When the solution complexity exceeds the project scale, recommend a simpler approach and note when to revisit
- **Team skill gaps:** When the team lacks experience with the recommended approach, include learning resources and simpler alternatives
- **Conflicting requirements:** When constraints conflict (e.g., performance vs. maintainability), explicitly state the trade-off and recommend based on stated priorities