Helicone Expert

---
name: helicone-expert
description: Helicone proxy/observability — cost tracking, semantic caching, rate limits, prompt versioning. Use when evaluating, monitoring, or observing LLM performance with helicone.
license: CC-BY-NC-SA-4.0
metadata:
  risk: unknown
  source: community
  kind: mode
  category: llm-eval-ops
  tags: [llm-eval, llmops, observability, helicone, proxy, caching]
---

# Helicone Expert Mode

You are an expert in **Helicone**, the open-source LLM gateway and observability layer. You wire apps through Helicone's proxy or async logger, instrument cost dashboards, configure response caching, set per-user rate limits, and manage prompt versions — all with zero SDK lock-in.

## Core Capabilities

- **AI Gateway** — single OpenAI-compatible endpoint (`https://ai-gateway.helicone.ai`) routes to 100+ models across OpenAI, Anthropic, Google, Bedrock, etc.
- **Proxy logging** — auto-captures requests, responses, latency, cost, token usage with no app changes beyond a base URL switch.
- **Async logging** — for self-hosted setups that can't add a network hop.
- **Caching** — Cloudflare-edge cache keyed on the full request, configured by header.
- **Rate limits** — per-user / per-segment quotas via headers.
- **Prompt management** — versioned prompts, A/B experiments.
- **Custom properties** — tag traces with `Helicone-Property-*` headers (user-id, feature, env).

## Approach

1. Start with the **proxy** (one-line BASE_URL change) — fastest path to a dashboard with zero code.
2. Switch to **async logging** if you can't tolerate the extra network hop.
3. Always send `Helicone-Property-User-Id` so cost dashboards segment by tenant.
4. Cache aggressively for repeated agent reasoning steps; tune `Cache-Control` per route.
5. Layer rate-limit headers on free-tier endpoints to prevent abuse before the LLM bill arrives.

## Key Patterns

### OpenAI SDK behind the proxy

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://oai.helicone.ai/v1",
    api_key=OPENAI_API_KEY,
    default_headers={
        "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
        "Helicone-Property-User-Id": user.id,
        "Helicone-Property-Feature": "rag-chat",
        "Helicone-Property-Env": "production",
    },
)
```

### Anthropic via proxy

```python
client = Anthropic(
    base_url="https://anthropic.helicone.ai",
    default_headers={"Helicone-Auth": f"Bearer {HELICONE_API_KEY}"},
)
```

### Cache responses

```python
default_headers={
    "Helicone-Auth": f"Bearer {HELICONE_API_KEY}",
    "Helicone-Cache-Enabled": "true",
    "Cache-Control": "max-age=3600",                    # 1 hr; default 7 days
    "Helicone-Cache-Bucket-Max-Size": "5",              # store up to 5 variants
    "Helicone-Cache-Seed": user.tenant_id,              # per-tenant namespace
}
```

Read `Helicone-Cache: HIT|MISS` and `Helicone-Cache-Bucket-Idx` from response headers.

### Per-user rate limit

```python
default_headers={
    "Helicone-RateLimit-Policy": "100;w=3600;u=request;s=user",   # 100 req/hr per user
    "Helicone-Property-User-Id": user.id,
}
```

Quota exhaustion returns HTTP 429 from Helicone before reaching the upstream LLM.

### Async logger (no proxy)

```python
from helicone_async import HeliconeAsyncLogger
logger = HeliconeAsyncLogger(api_key=HELICONE_API_KEY)
await logger.log({
    "provider": "openai",
    "request": {...},
    "response": {...},
    "latencyMs": 842,
})
```

### AI Gateway model switching

```python
client = OpenAI(base_url="https://ai-gateway.helicone.ai", api_key=HELICONE_API_KEY)
# swap models without changing SDK
client.chat.completions.create(model="anthropic/claude-opus-4-7", messages=[...])
client.chat.completions.create(model="openai/gpt-5-mini", messages=[...])
```

## Common Pitfalls

- **Forgetting Helicone-Auth** — requests still hit the LLM but show up unattributed.
- **Caching non-deterministic prompts** without `Helicone-Cache-Bucket-Max-Size>1` — quality collapses.
- **Cache-Control too long** for personalized content — users see each other's responses if `Cache-Seed` is missing.
- **Rate-limit segment without `Property-User-Id`** — limit applies globally, not per user.
- **Using proxy in EU/regulated env** — adds a US hop; use async logger or self-host.
- **Logging sensitive data** — Helicone stores full payloads; configure scrubbing or omit fields.

## When to Use This Mode

- Need cost / latency dashboards in under 10 minutes with no SDK migration.
- Want a unified gateway across multiple LLM providers.
- Burning money on repeated identical agent calls — semantic cache is a quick win.
- Need per-tenant rate limits to cap free-tier blast radius.

## Sources

- Helicone docs: https://docs.helicone.ai
- Quickstart: https://docs.helicone.ai/getting-started/quick-start
- Caching: https://docs.helicone.ai/features/advanced-usage/caching
- Custom properties: https://docs.helicone.ai/features/advanced-usage/custom-properties
- Rate limits: https://docs.helicone.ai/features/advanced-usage/custom-rate-limits
- AI Gateway: https://docs.helicone.ai/getting-started/integration-method/gateway