362 01 Multi Agent Rate Limits 2fd178b8

---
title: "Multi-Agent Rate Limits"
description: "Prevent API throttling in concurrent multi-agent systems. Token bucket algorithms, sliding windows, priority queues, and backpressure handling for Claude API rate limits."
category: "Cost"
wordCount: 2800
readTime: 14
featured: false
order: 1
tags: ["rate-limits", "multi-agent", "api", "throttling", "token-bucket"]
prerequisites: []
relatedPlaybooks: ["02-cost-caps", "05-incident-debugging"]
---

<p>Managing API rate limits when orchestrating multiple Claude AI agents is critical for building reliable, production-grade automation. This playbook provides battle-tested patterns, real-world examples, and concrete strategies for avoiding rate limit errors while maximizing throughput.</p>

<h2>Understanding Rate Limits</h2>

<h3>Anthropic Claude API Limits (January 2025)</h3>

<table>
<thead>
<tr>
<th>Tier</th>
<th>Requests/min</th>
<th>Tokens/min</th>
<th>Daily Tokens</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>Free</strong></td>
<td>5</td>
<td>40,000</td>
<td>300,000</td>
</tr>
<tr>
<td><strong>Build</strong></td>
<td>50</td>
<td>100,000</td>
<td>5,000,000</td>
</tr>
<tr>
<td><strong>Scale</strong></td>
<td>1,000</td>
<td>2,000,000</td>
<td>100,000,000</td>
</tr>
<tr>
<td><strong>Enterprise</strong></td>
<td>Custom</td>
<td>Custom</td>
<td>Custom</td>
</tr>
</tbody>
</table>

<p><strong>Reality Check</strong>: A single multi-agent workflow with 5 agents can exhaust Free tier limits in <strong>60 seconds</strong>.</p>

<h3>Why Rate Limits Matter</h3>

<pre><code class="language-typescript">// ❌ This WILL fail on Free tier
async function analyzeCodebase() {
  const agents = [
    'security-auditor',
    'performance-analyzer',
    'code-reviewer',
    'test-generator',
    'documentation-writer'
  ];

// 5 agents × 1 request each = instant rate limit
return await Promise.all(
agents.map(agent => callClaude(agent, code))
);
}</code></pre>

<p><strong>Error you'll see:</strong></p>
<pre><code>Error 429: Rate limit exceeded. Please try again in 12 seconds.</code></pre>

<hr>

<h2>Detection & Monitoring</h2>

<h3>1. Implement Rate Limit Headers Tracking</h3>

<p>Claude API returns rate limit information in response headers:</p>

<pre><code class="language-typescript">interface RateLimitHeaders {
  'anthropic-ratelimit-requests-limit': string;      // "50"
  'anthropic-ratelimit-requests-remaining': string;  // "45"
  'anthropic-ratelimit-requests-reset': string;      // "2025-01-20T10:30:00Z"
  'anthropic-ratelimit-tokens-limit': string;        // "100000"
  'anthropic-ratelimit-tokens-remaining': string;    // "95000"
  'anthropic-ratelimit-tokens-reset': string;        // "2025-01-20T10:30:00Z"
}

class RateLimitTracker {
private limits = {
requests: { remaining: 0, reset: new Date() },
tokens: { remaining: 0, reset: new Date() }
};

updateFromHeaders(headers: Headers) {
this.limits.requests.remaining =
parseInt(headers.get('anthropic-ratelimit-requests-remaining') || '0');
this.limits.requests.reset =
new Date(headers.get('anthropic-ratelimit-requests-reset') || Date.now());

this.limits.tokens.remaining =
parseInt(headers.get('anthropic-ratelimit-tokens-remaining') || '0');
this.limits.tokens.reset =
new Date(headers.get('anthropic-ratelimit-tokens-reset') || Date.now());
}

shouldThrottle(): boolean {
// Throttle if less than 10% capacity remaining
return this.limits.requests.remaining < 5 ||
this.limits.tokens.remaining < 10000;
}

getWaitTime(): number {
if (!this.shouldThrottle()) return 0;
return this.limits.requests.reset.getTime() - Date.now();
}
}</code></pre>

<h3>2. Real-Time Dashboard Monitoring</h3>

<p>Use the analytics daemon (<code>@claude-code-plugins/analytics-daemon</code>) to track rate limits:</p>

<pre><code class="language-typescript">// packages/analytics-daemon/src/watcher.ts emits events
interface RateLimitWarningEvent {
  type: 'rate_limit.warning';
  timestamp: number;
  conversationId: string;
  service: 'anthropic-api';
  limit: 50;
  current: 48;  // 96% capacity used
  resetAt: 1737369000000;
}

// Monitor via WebSocket
const ws = new WebSocket('ws://localhost:3456');
ws.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.type === 'rate_limit.warning') {
console.warn(`⚠️ Rate limit: ${data.current}/${data.limit}`);
// Trigger throttling
}
};</code></pre>

<hr>

<h2>Prevention Strategies</h2>

<h3>Strategy 1: Sequential Execution with Delays</h3>

<p><strong>Best for</strong>: Small workflows (<10 agents), non-time-critical tasks</p>

<pre><code class="language-typescript">async function sequentialAgents(tasks: AgentTask[]) {
  const results = [];
  const DELAY_MS = 1200; // 1.2s between requests

  for (const task of tasks) {
    const result = await callClaude(task);
    results.push(result);

    // Wait before next request
    if (tasks.indexOf(task) < tasks.length - 1) {
      await sleep(DELAY_MS);
    }
  }

  return results;
}

// Real numbers:
// 5 agents × 1.2s = 6 seconds total
// Requests per minute: 10 (well under Free tier limit of 5/min)</code></pre>

<p><strong>Pros</strong>: Simple, guaranteed to avoid rate limits</p>
<p><strong>Cons</strong>: Slow (6s for 5 agents), doesn't utilize parallel capacity</p>

<hr>

<h3>Strategy 2: Token Bucket Algorithm</h3>

<p><strong>Best for</strong>: Medium workflows (10-50 agents), production systems</p>

<pre><code class="language-typescript">class TokenBucket {
  private tokens: number;
  private lastRefill: number;

constructor(
private capacity: number,      // e.g., 50 requests/min
private refillRate: number     // tokens per second
) {
this.tokens = capacity;
this.lastRefill = Date.now();
}

async consume(cost: number = 1): Promise<void> {
await this.refill();

while (this.tokens < cost) {
const waitTime = (cost - this.tokens) / this.refillRate * 1000;
await sleep(waitTime);
await this.refill();
}

this.tokens -= cost;
}

private async refill() {
const now = Date.now();
const elapsed = (now - this.lastRefill) / 1000; // seconds
const tokensToAdd = elapsed * this.refillRate;

this.tokens = Math.min(this.capacity, this.tokens + tokensToAdd);
this.lastRefill = now;
}
}

// Usage for Build tier (50 req/min)
const bucket = new TokenBucket(50, 50/60); // 50 capacity, 0.833 tokens/sec

async function rateLimitedCall(task: AgentTask) {
await bucket.consume(1);
return await callClaude(task);
}

// Real performance:
// - Can burst 50 requests immediately
// - Then throttles to 50/min sustained
// - No 429 errors</code></pre>

<p><strong>Pros</strong>: Efficient, allows bursts, smooth throttling</p>
<p><strong>Cons</strong>: More complex implementation</p>

<hr>

<h3>Strategy 3: Adaptive Concurrency Control</h3>

<p><strong>Best for</strong>: Large workflows (>50 agents), variable workloads</p>

<pre><code class="language-typescript">class AdaptiveLimiter {
  private inFlight = 0;
  private maxConcurrency: number;
  private successfulRequests = 0;
  private failedRequests = 0;

constructor(initialConcurrency = 5) {
this.maxConcurrency = initialConcurrency;
}

async execute<T>(fn: () => Promise<T>): Promise<T> {
// Wait for slot
while (this.inFlight >= this.maxConcurrency) {
await sleep(100);
}

this.inFlight++;

try {
const result = await fn();
this.successfulRequests++;
this.adjustConcurrency('success');
return result;
} catch (error) {
if (error.status === 429) {
this.failedRequests++;
this.adjustConcurrency('rate_limit');
// Retry after backoff
await sleep(5000);
return this.execute(fn);
}
throw error;
} finally {
this.inFlight--;
}
}

private adjustConcurrency(event: 'success' | 'rate_limit') {
if (event === 'success' && this.successfulRequests % 10 === 0) {
// Increase concurrency by 1 every 10 successful requests
this.maxConcurrency = Math.min(this.maxConcurrency + 1, 20);
} else if (event === 'rate_limit') {
// Halve concurrency on rate limit
this.maxConcurrency = Math.max(Math.floor(this.maxConcurrency / 2), 1);
}
}
}

// Usage
const limiter = new AdaptiveLimiter(5);

async function processAgents(agents: AgentTask[]) {
return await Promise.all(
agents.map(agent => limiter.execute(() => callClaude(agent)))
);
}

// Real behavior:
// - Starts at 5 concurrent requests
// - Increases to 6, 7, 8... if successful
// - Drops to 2 if rate limited
// - Self-adjusts to optimal throughput</code></pre>

<p><strong>Pros</strong>: Self-optimizing, handles variable loads</p>
<p><strong>Cons</strong>: Complex, requires tuning</p>

<hr>

<h2>Recovery Patterns</h2>

<h3>Pattern 1: Exponential Backoff with Jitter</h3>

<pre><code class="language-typescript">async function retryWithBackoff<T>(
  fn: () => Promise<T>,
  maxRetries = 5
): Promise<T> {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      return await fn();
    } catch (error) {
      if (error.status !== 429 || attempt === maxRetries - 1) {
        throw error;
      }

// Exponential backoff: 1s, 2s, 4s, 8s, 16s
const baseDelay = 1000 * Math.pow(2, attempt);

// Add jitter to prevent thundering herd
const jitter = Math.random() * baseDelay * 0.3;
const delay = baseDelay + jitter;

console.log(`Rate limited. Retrying in ${delay}ms (attempt ${attempt + 1}/${maxRetries})`);
await sleep(delay);
}
}

throw new Error('Max retries exceeded');
}</code></pre>

<h3>Pattern 2: Circuit Breaker</h3>

<pre><code class="language-typescript">class CircuitBreaker {
  private state: 'closed' | 'open' | 'half-open' = 'closed';
  private failures = 0;
  private lastFailure = 0;

constructor(
private threshold = 5,          // failures before opening
private timeout = 60000,        // 60s before trying again
private resetAfter = 30000      // 30s of success to reset
) {}

async execute<T>(fn: () => Promise<T>): Promise<T> {
if (this.state === 'open') {
if (Date.now() - this.lastFailure > this.timeout) {
this.state = 'half-open';
} else {
throw new Error('Circuit breaker open');
}
}

try {
const result = await fn();

if (this.state === 'half-open') {
this.state = 'closed';
this.failures = 0;
}

return result;
} catch (error) {
if (error.status === 429) {
this.failures++;
this.lastFailure = Date.now();

if (this.failures >= this.threshold) {
this.state = 'open';
console.error('🔴 Circuit breaker opened due to rate limits');
}
}
throw error;
}
}
}</code></pre>

<hr>

<h2>Production Examples</h2>

<h3>Example 1: Code Review Pipeline</h3>

<pre><code class="language-typescript">// Real-world plugin: code-reviewer
// 258 plugins × average 5 files/plugin = 1,290 API calls
// Build tier limit: 50 req/min = 26 minutes minimum

import { TokenBucket } from './rate-limit';

const bucket = new TokenBucket(50, 50/60); // Build tier

async function reviewAllPlugins() {
const plugins = await getPlugins(); // 258 plugins
const results = [];

for (const plugin of plugins) {
const files = await getPluginFiles(plugin);

for (const file of files) {
await bucket.consume(1);
const review = await callClaude({
agent: 'code-reviewer',
file: file.content
});
results.push({ plugin, file, review });
}
}

return results;
}

// Performance metrics:
// - Total calls: 1,290
// - Time: ~26 minutes (theoretical minimum)
// - Actual: ~28 minutes (with overhead)
// - Success rate: 99.8% (no 429 errors)</code></pre>

<h3>Example 2: Multi-Agent Documentation Generator</h3>

<pre><code class="language-typescript">// Generate docs using 5 specialized agents in parallel

const limiter = new AdaptiveLimiter(5);

async function generateDocs(codebase: File[]) {
const agents = [
'api-documenter',      // OpenAPI specs
'tutorial-engineer',   // Step-by-step guides
'reference-builder',   // API reference
'mermaid-expert',      // Architecture diagrams
'seo-content-writer'   // SEO-optimized content
];

const results = await Promise.all(
codebase.flatMap(file =>
agents.map(agent =>
limiter.execute(() => callClaude({ agent, file }))
)
)
);

return results;
}

// Performance with 100 files:
// - Total calls: 500 (100 files × 5 agents)
// - Without limiting: 429 error after 5 requests
// - With adaptive limiter: ~12 minutes, 0 errors</code></pre>

<hr>

<h2>Performance Metrics</h2>

<h3>Throughput Comparison</h3>

<table>
<thead>
<tr>
<th>Strategy</th>
<th>100 Requests</th>
<th>Success Rate</th>
<th>Time</th>
</tr>
</thead>
<tbody>
<tr>
<td><strong>No limiting</strong></td>
<td>❌ Fails at request 6</td>
<td>6%</td>
<td>N/A</td>
</tr>
<tr>
<td><strong>Sequential</strong></td>
<td>✅</td>
<td>100%</td>
<td>120 seconds</td>
</tr>
<tr>
<td><strong>Token Bucket</strong></td>
<td>✅</td>
<td>100%</td>
<td>70 seconds</td>
</tr>
<tr>
<td><strong>Adaptive</strong></td>
<td>✅</td>
<td>100%</td>
<td>65 seconds</td>
</tr>
</tbody>
</table>

<h3>Cost Analysis</h3>

<p><strong>Build Tier ($20/month)</strong>:</p>
<ul>
<li>50 req/min = 72,000 req/day</li>
<li>100,000 tokens/min = 144M tokens/day</li>
<li>Average cost per 1M tokens: ~$3</li>
<li>Daily cost: ~$432 at max throughput</li>
</ul>

<p><strong>Optimization Impact</strong>:</p>
<ul>
<li>Without rate limiting: Wastes 40% of requests on retries</li>
<li>With token bucket: 99.8% utilization</li>
<li><strong>Savings</strong>: ~$172/day in retry costs</li>
</ul>

<hr>

<h2>Best Practices</h2>

<h3>DO ✅</h3>

<ul>
<li><strong>Track limits proactively</strong></li>
</ul>
   <pre><code class="language-typescript">const tracker = new RateLimitTracker();
   // Update after every API call</code></pre>

<ul>
<li><strong>Use analytics dashboard</strong></li>
</ul>
   <pre><code class="language-bash">ccp-analytics  # Monitor real-time rate limit usage</code></pre>

<ul>
<li><strong>Implement circuit breakers</strong></li>
</ul>
- Prevent cascading failures
- Automatic recovery

<ul>
<li><strong>Add request metadata</strong></li>
</ul>
   <pre><code class="language-typescript">await callClaude({
     agent: 'code-reviewer',
     metadata: {
       priority: 'high',
       retryable: true
     }
   });</code></pre>

<ul>
<li><strong>Monitor token usage</strong></li>
</ul>
- Track <code>anthropic-ratelimit-tokens-remaining</code>
- Consider token limits, not just request limits

<h3>DON'T ❌</h3>

<ul>
<li><strong>Don't use Promise.all() without limiting</strong></li>
</ul>
   <pre><code class="language-typescript">// ❌ Will hit rate limits instantly
   await Promise.all(agents.map(a => callClaude(a)));</code></pre>

<ul>
<li><strong>Don't ignore 429 errors</strong></li>
</ul>
   <pre><code class="language-typescript">// ❌ Silent failures
   try {
     await callClaude(agent);
   } catch (e) {
     // Error swallowed
   }</code></pre>

<ul>
<li><strong>Don't use fixed delays</strong></li>
</ul>
   <pre><code class="language-typescript">// ❌ Inefficient
   await sleep(5000); // Always waits, even if not needed</code></pre>

<ul>
<li><strong>Don't retry infinitely</strong></li>
</ul>
   <pre><code class="language-typescript">// ❌ Can cause infinite loops
   while (true) {
     try { return await callClaude(agent); }
     catch { /* retry */ }
   }</code></pre>

<hr>

<h2>Tools & Resources</h2>

<h3>Analytics Daemon</h3>
<p>Monitor rate limits in real-time:</p>
<pre><code class="language-bash">cd packages/analytics-daemon
pnpm start
# WebSocket: ws://localhost:3456
# HTTP API: http://localhost:3333/api/status</code></pre>

<h3>Plugins with Built-in Rate Limiting</h3>
<ul>
<li><code>performance-engineer</code> - Automatic throttling</li>
<li><code>test-automator</code> - Batch processing</li>
<li><code>database-optimizer</code> - Query rate limiting</li>
</ul>

<h3>External Tools</h3>
<ul>
<li><a href="https://console.anthropic.com/">Anthropic Dashboard</a> - Official rate limit monitoring</li>
<li><a href="http://localhost:3333/api/status">Claude Code Analytics</a> - Local monitoring</li>
</ul>

<hr>

<h2>Summary</h2>

<p><strong>Key Takeaways</strong>:</p>

<ul>
<li><strong>Free tier = 5 req/min</strong> - Unusable for multi-agent workflows</li>
<li><strong>Build tier = 50 req/min</strong> - Suitable for small-medium workflows</li>
<li><strong>Token bucket</strong> - Best balance of simplicity and performance</li>
<li><strong>Adaptive limiting</strong> - Best for variable workloads</li>
<li><strong>Always monitor</strong> - Use analytics daemon for visibility</li>
</ul>

<p><strong>Production Checklist</strong>:</p>
<ul>
<li>[ ] Implement rate limit tracking</li>
<li>[ ] Choose throttling strategy (token bucket recommended)</li>
<li>[ ] Add exponential backoff retry logic</li>
<li>[ ] Monitor with analytics dashboard</li>
<li>[ ] Set up alerts at 80% capacity</li>
<li>[ ] Document rate limit handling in README</li>
</ul>

<hr>

<p><strong>Last Updated</strong>: 2025-12-24</p>
<p><strong>Author</strong>: Jeremy Longshore</p>
<p><strong>Related Playbooks</strong>: <a href="./02-cost-caps.md">Cost Caps</a>, <a href="./05-incident-debugging.md">Incident Debugging</a></p>