Ai

---
name: ai
description: Use when calling the app's AI gateway from agent tools — chat completions, embeddings, listing models, configuring defaults or BYOK, reading token/cost usage
---

# Butterbase AI Gateway

Every app has an LLM gateway with chat, embeddings, model listing, configuration, and usage reporting. One umbrella tool: **`manage_ai`**.

| Action | What it does | Returns |
|---|---|---|
| `chat` | Synchronous chat completion (no streaming) | OpenAI-shaped `{ choices: [...] }` |
| `embed` | Vector embeddings for string or string[] | OpenAI-shaped `{ data: [{ embedding: [...] }] }` |
| `list_models` | Available models with capabilities | `{ models: AiModel[] }` |
| `get_config` | Current AI config (default model, BYOK key flag, etc.) | `AiConfig` |
| `update_config` | Set defaults, allowed models, max tokens, BYOK | `AiConfig` |
| `get_usage` | Token + cost aggregate over a window | usage record |

---

## 1. Chat

```
manage_ai({
  action: "chat",
  app_id,
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user",   content: "What's RAG?" }
  ],
  model: "openai/gpt-4o-mini",     // optional — falls back to app's default
  temperature: 0.2,                // optional
  max_tokens: 500                  // optional
})
```

This action sets `stream: false` deliberately — agent tools don't stream. If you need partial-token deltas, drive the SDK's `ai.chatStream(…)` from inside a function or DO instead.

`messages[].content` can be a string or an array of content parts (`{ type: "text", text }`, `{ type: "image_url", image_url: {...} }`, `{ type: "video_url", video_url: {...} }`).

---

## 2. Embed

```
manage_ai({
  action: "embed",
  app_id,
  input: "hello world",            // or ["a", "b", "c"]
  model: "openai/text-embedding-3-small",   // optional
  encoding_format: "float"          // or "base64"
})
```

---

## 3. List models

```
manage_ai({ action: "list_models", app_id })
// → { models: [{ id, provider, capabilities: ["chat", "embed", ...], context_window, pricing }, ...] }
```

Use this to discover what the app can call — capabilities + context window matter when picking a model.

---

## 4. Configure

```
manage_ai({
  action: "update_config",
  app_id,
  config: {
    defaultModel: "openai/gpt-4o-mini",
    allowedModels: ["openai/gpt-4o-mini", "anthropic/claude-haiku-4-5"],
    maxTokensPerRequest: 4000,
    byokKey: "..." // optional — rotates the customer-supplied OpenRouter / Anthropic key
  }
})
```

- `maxTokensPerRequest` is server-clamped to 1–100000.
- `allowedModels` is a whitelist — empty means all models the provider exposes.
- Setting `byokKey` switches the app to route through that customer key. Clear it by passing `byokKey: ""` (returns to platform pool).

---

## 5. Usage

```
manage_ai({
  action: "get_usage",
  app_id,
  startDate: "2026-05-01",
  endDate:   "2026-05-31"
})
```

Returns aggregate token counts + cost. Useful for billing reconciliation, spending-cap diagnostics, and showing dashboards.

---

## 6. Common pitfalls

- **Trying to stream from a tool** — `manage_ai` is synchronous. Use the SDK inside a function for streamed deltas.
- **Sending `stream: true` in the body** — the tool ignores it; always wired to `false`.
- **Hardcoding `model`** — better to omit, let the app's `defaultModel` win, and surface that knob via `update_config`.
- **Skipping `list_models` before suggesting one** — model availability shifts; verify before recommending.

---

## 7. What this skill does NOT cover

- Streaming chat — use the SDK (`ai.chatStream`) inside a function or DO.
- Vector storage / retrieval — see `butterbase-skills:rag-dev` (RAG collections wrap embeddings + search together).
- AI in deployed functions — they import `@butterbase/sdk` and call `client.ai.*` directly; no MCP needed at runtime.

---

## 8. Decisions (typed classification)

For routing, classification, moderation or scoring, where code needs a choice / yes-no / score with probabilities, use `manage_ai` `action: "decide"` (SDK `ai.decide`) with a decision model (default `typesafe/jev-1.13`) instead of asking a chat model for JSON. It is cheaper, faster and returns typed probabilities. Full reference: `butterbase_docs` topic `ai`, "Decision models".

---

If a `docs/butterbase/00-state.md` exists in the working directory, prefer invoking via `/butterbase-skills:journey-ai` so the journey orchestrator stays in sync.