Add Model

---
name: add-model
description: Add a model or provider mapping to the catalogue, or verify one that was already written — pricing, capability and reasoning metadata, scoped e2e, and playground options for image/video models. Use when the user asks to add a named model on a provider, create a provider mapping, add a provider, check model pricing, verify a model, or change packages/models/src/models.
---

# Add a model

Prove every declared value against the live provider. A wrong price mis-bills
every request until someone audits it; a wrong capability flag routes traffic to
a deployment that 400s. Values handed to you in a prompt or PR are unverified —
re-derive them.

## New providers

Third-party inference providers onboard only through
[Airside](https://airside.llmgateway.io), the self-serve provider portal. Never
add a new provider or its mappings here on someone's behalf; if asked to, stop
and point the user to https://airside.llmgateway.io.

## 1. Scope

List every `(model, provider, region)` triple that is new or changed. Regions in
a `regions: []` array are separate mappings with their own prices and e2e cases.
Report results per mapping at the end.

```bash
gh pr checkout <n>   # someone else's PR; sets the upstream too
git diff origin/main...HEAD -- packages/models/src/models/
```

## 2. Where things live

| What                                        | Where                                                                                                                                |
| ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| Definitions, field docs                     | `packages/models/src/models/<family>.ts`; types in `packages/models/src/models.ts`                                                   |
| Providers, env vars, regions, service tiers | `packages/models/src/providers.ts`                                                                                                   |
| Catalogue invariants                        | `packages/models/src/model-metadata.spec.ts`, `packages/models/src/providers.spec.ts`, `packages/models/src/realtime-models.spec.ts` |
| Cost engine                                 | `packages/actions/src/costs.ts`                                                                                                      |
| Token extraction                            | `apps/gateway/src/chat/tools/extract-token-usage.ts`, `apps/gateway/src/chat/tools/parse-provider-response.ts`                       |
| Request shaping                             | `packages/actions/src/prepare-request-body.ts`                                                                                       |
| New-provider endpoint wiring                | `packages/actions/src/get-provider-endpoint.ts`                                                                                      |
| e2e                                         | `apps/gateway/src/*.e2e.ts`, split by chat behavior and endpoint capability                                                          |
| Playground options                          | `apps/playground/src/lib/image-gen.ts`, `apps/playground/src/lib/video-gen.ts`                                                       |

## Catalogue rules

Apply these to every change in `packages/models`:

- Write each model and provider mapping in full as a plain object literal in the `models` array. Duplicated fields are preferred here; small shared `const` values are fine, but no helper (e.g. `makeModel(...)`) builds definition objects.
- A mapping on `origin/main` is never removed: historical usage and analytics reference it. Retire it with `deactivatedAt: new Date("YYYY-MM-DD")`. Mappings added only on the current branch may be removed. Exception: Iceberg and Granite may be removed with a one-time database cleanup after the source removal deploys, keeping shared models, logs, usage aggregates, and history. Also exempt: a mapping whose pair every environment serves from an active Airside listing (`source = "airside"`) that carries its catalogue metadata. Remove it, and mark a provider left with no static mappings `managedInAirside: true`.
- Set `deactivatedAt` at least 7 days out for a mapping that still serves live requests, or on the provider's announced retirement date. A same-day or retroactive date requires the mapping to be unable to serve any live request today. A rejected parameter, a missing capability, or flaky quality is a metadata fix (`jsonOutput`, `vision`, `reasoningEfforts`, `supportedToolChoices`, `stability: "unstable"`, `test: "skip"`) or a gateway fix; leave the mapping active and say so in the PR.
- Write per-token prices (`inputPrice`, `outputPrice`, `cachedInputPrice`, …) in `e-6` notation so the coefficient reads as USD per million tokens (`"1.4e-6"`). `requestPrice` (flat USD per request) and `perSecondPrice` are exempt.
- One model definition has at most one mapping per `providerId`; regional variants go in that mapping's `regions` array. Lookups key on `(providerId, region)`, so a second same-provider mapping silently resolves to the first and bills at its prices. A distinct upstream deployment (e.g. a priority router with its own `externalId` and pricing) gets its own model entry and `id`.
- A mapping with both `peakPricing` and `regions` gives every region with its own rates its own full `peakPricing` block; flat overrides alone inherit the base peak tiers.
- When the provider prices each context-length band by time of day, give every `pricingTiers` entry its own `peakPricing` (`peak`/`offPeak` rates); the schedule stays on the mapping or region. Don't model limited-time promotional discounts.
- Identify models by `model.id`, scoped by `providerId` and optional `:region`. `externalId` is only the upstream API identifier: keep it out of URLs, selectors, billing and analytics keys, and lookups, including fallbacks.
- Let metadata fields speak for themselves. Comment only behavior the metadata cannot express that a maintainer needs, such as the operational cause of `stability: "unstable"` or `test: "skip"`. Sources, verification, and pricing choices go in the PR body.

## 3. Pricing

Billing is tokens: `calculateCosts` multiplies token counts by the mapping's
per-token prices, and no code path reads a provider-reported cost. So research
the per-token rate card, then verify the token semantics — the more common
failure.

**Rates.** Use the provider's current first-party pricing page or metadata
endpoint and record its units. Use aggregators only as a cross-check. Verify
cached rates, context-length bands, and regional rates independently; do not
derive or round them when the provider publishes exact values.

**Token semantics** — establish each from a live `usage` block:

- Reasoning inside `completion_tokens`? `normalizeCompletionTokens` in
  `extract-token-usage.ts` adds reasoning only when the reported total equals
  prompt + completion + reasoning; otherwise completion is billed as reported.
  Verify both streaming and non-streaming extraction against the provider's
  `usage`.
- Cached tokens inside `prompt_tokens`? `costs.ts` assumes yes and subtracts
  them.
- Cache writes priced (`cacheWriteInputPrice`, `cacheWriteInputPrice1h`)? A
  missing 1h rate silently bills 1h writes at the 5m rate.

**Reconcile.** Pin the provider and vary the prompt (Redis caches on the body):

```bash
curl -N "${GATEWAY_URL:-http://localhost:4001}/v1/chat/completions" \
  -H "Authorization: Bearer test-token" -H "x-no-fallback: true" \
  -H "Content-Type: application/json" \
  -d '{"model":"<provider>/<model>","messages":[{"role":"user","content":"hi"}]}'
```

For one small and one large request (crossing a tier boundary where there is
one): token counts must match between the upstream `usage` and the `log` row,
and a hand-computed cost must match `log.cost`. Where a provider echoes its own
cost, compare that too as a cross-check; it is not a billing input. Investigate
every mismatch instead of inferring a discount from its shape.

If a handed-in price, first-party rate card, and measurement disagree, reconcile
the units and token semantics before changing the catalogue. Do not choose one
without explaining the mismatch.

## 4. Metadata

Probe the deployment. The same model differs between providers, and an
`externalId` bump can silently drop a capability.

- `contextSize` / `maxOutput` — probe with an oversized prompt and read the
  rejection; listings under- and overstate.
- `vision` / `audio` / `document` — routing skips mappings lacking them. A
  deployment that 400s on images gets `vision: false`, never an error-text
  classification rule. `audio.e2e.ts` covers audio input.
- `supportedToolChoices` — probe `auto`, `none`, `required`, named function.
  Unlisted modes are downgraded to `auto`. Some deployments accept `required`
  only with thinking off.
- Capability combinations — probe tool calls and structured output with
  reasoning both enabled and disabled. If a capability only fails while
  reasoning is on, do not immediately flatten the mapping to `tools: false` or
  `jsonOutput: false`: first check `ProviderModelMapping` and request shaping
  for an existing conditional compatibility mechanism. If none can express the
  result, call out the gap instead of misrepresenting the standalone capability.
- `jsonOutput` / `jsonOutputSchema` — probe `json_object` and `json_schema`
  separately.
- `supportedParameters` — probe before declaring; omission elsewhere is not a
  claim of non-support.
- `supportsDeveloperRole` / `supportsAssistantPrefill` — default true, flip only
  on a 400.
- `serviceTiers` — declaring one narrows routing pre-flight.
- `stability: "unstable"` / `test: "skip"` for flaky, paid-per-call or
  region-locked mappings.
- `releasedAt`, plus an `output` entry per capability flag — both enforced by
  `model-metadata.spec.ts`.

Follow the comment rule in [Catalogue rules](#catalogue-rules).

A new provider also needs a `providers.ts` entry, endpoint wiring in
`get-provider-endpoint.ts`. Set `encryptedReasoning: true` when its replayed
reasoning payloads only verify on that provider; routing then never moves such
models to another provider (exploration, low-uptime fallback, error retry).
Probed 2026-09: OpenAI and Azure reject each other's `encrypted_content`, AI
Studio and Vertex reject each other's Gemini 3 signatures, but signatures
replay across different keys of the same Google platform.

## 5. Reasoning efforts

Efforts are forwarded as-is, so a declared tier the upstream rejects is a 4xx for
users, and each declared tier becomes an e2e case. Probe `none | minimal | low |
medium | high | xhigh | max` individually and declare the accepted subset in
ascending order — docs and reality diverge often, including between chat
completions and the Responses API.

| Toggle                                     | Field                                |
| ------------------------------------------ | ------------------------------------ |
| Standard effort tiers                      | `reasoningEfforts` alone             |
| Thinks by default, vLLM chat-template flag | `chatTemplateThinkingKey: "<key>"`   |
| Off by default, needs an enable flag       | `requiresEnableThinking: true`       |
| Off only via `thinking: {type:"disabled"}` | `requiresDisableThinkingParam: true` |
| Explicit token budget                      | `reasoningMaxTokens: true`           |
| Anthropic adaptive thinking                | `reasoningMode: "adaptive"`          |
| Reasons but returns no reasoning content   | `reasoningOutput: "omit"`            |

Verify the toggle takes effect rather than just returning 200 — a provider can
accept `enable_thinking: false` and still return `reasoning_content`. Declare it
only on mappings where it works.

Gemini 3+ efforts become `thinkingLevel`; an undeclared tier rises to the next
declared one, so declare `reasoningEfforts` from a `thinkingLevel` probe
(`minimal` 400s on several models). Google is retiring `thinkingBudget` and
sampling params: omit `reasoningMaxTokens` where a budget 400s, and give 3.6+
mappings a `supportedParameters` list without `temperature`/`top_p`.

## 6. Image, video, and other endpoints

Probe the size/quality/duration grid; rate cards list tiers deployments refuse.
Leave rejected values off the mapping and record the accepted grid in the PR.

**Video** — `supportedVideoSizes`, `supportedVideoDurationsSeconds`,
`supportedVideoDurationsSecondsImageToVideo`, `supportsVideoAudio`,
`supportsVideoWithoutAudio`, `perSecondPrice`, model-level
`maxVideoDurationSeconds`. The studio derives its menus from those, but
frame/reference input is gated by hardcoded allowlists in
`mappingSupportsVideoRequest` and the `supportsVideoFrameInput` /
`supportsVideoReference*Input` helpers — add the model there, matched on root
model id, never `externalId`. Cover it in `video-gen.spec.ts`.

**Image** — `perImagePrice` (keyed by resolution, or `"<quality>/<resolution>"`
when both are priced), or `imageOutputPrice` + `imageOutputTokensByResolution`;
`imageInputPrice` / `imageInputTokensByResolution` for edit models;
`imageInputRequired` when it cannot run text-only. A `perImagePrice` map without
a `"default"` key falls back to its most expensive tier, so a wrong key
overcharges silently. `getModelImageConfig` contains hardcoded model-id logic;
inspect and update it when its defaults do not match the new deployment. If a
newly priced knob isn't forwarded by
`prepare-request-body.ts`, you price tiers you never serve. Cover it in
`image-gen.spec.ts` and `costs.spec.ts`.

| Flag                | Prices                                                       | e2e                     |
| ------------------- | ------------------------------------------------------------ | ----------------------- |
| `imageGenerations`  | above                                                        | `images.e2e.ts`         |
| `embeddings`        | input tokens only                                            | `embeddings.e2e.ts`     |
| `speechGenerations` | `outputAudioPrice`, `inputCharacterPrice`, `supportedVoices` | `speech.e2e.ts`         |
| `transcriptions`    | `inputAudioHourPrice`                                        | `transcriptions.e2e.ts` |
| `ocr`               | `ocrPagePrice`                                               | `ocr.e2e.ts`            |
| `rerank`            | token prices                                                 | `rerank.e2e.ts`         |

`realtime` has no e2e file — it is covered by `apps/gateway/src/realtime/*.spec.ts`
plus `realtime-models.spec.ts`, which fails any realtime mapping missing its
per-modality token prices. `realtimeTranscription` only on token-metered ASR
mappings.

Image and video are the one docs exception to never enumerating models: update
`apps/docs/content/(gateway)/features/{image,video}-generation.mdx`.

## 7. Verify

Use the `verify` skill to launch an isolated stack so another worktree cannot
corrupt the run.

The gateway resolves models from the catalogue, so a rebuild is enough for API
requests. The playground and dashboard read the DB via `/internal/models`, so
run `pnpm seed` before expecting a new model in any UI.

```bash
pnpm build:core
pnpm exec vitest run --no-file-parallelism \
  packages/models/src/model-metadata.spec.ts \
  packages/models/src/providers.spec.ts \
  packages/models/src/realtime-models.spec.ts \
  packages/actions/src/costs.spec.ts

TEST_MODELS="<provider>/<model>" FULL_MODE=true pnpm test:e2e
```

Add the relevant action and Playground specs when those files changed. Run all
tests against the isolated database required by `AGENTS.md`.

- For mapping-only changes, the scoped local run is the only relevant e2e
  result. Never run e2e without `TEST_MODELS` or trigger the GitHub e2e
  workflow. Failures outside the selected mappings, including failures from an
  accidentally triggered full run, do not affect acceptance or auto-merge.
  Once every selected mapping passes, e2e has passed for the change.
- Scope e2e with `TEST_MODELS`, and don't invoke the `*.e2e.ts` files one by
  one. It overrides `test: "skip"`, takes regions as `provider/model:region`,
  and fails loudly when an entry matches no mapping.
- `FULL_MODE=true` expands the per-effort cases and includes free models.
- `pnpm test:e2e` loads `.env` automatically. Use credentials already
  configured in `.envrc` or `.env`; never inspect or invoke password managers
  or other credential stores, and never retrieve provider secrets manually.
- API key env names aren't derivable from the provider id — read
  `env.required` from `providers.ts`. Vertex provider variants use different
  credentials, and keys can be region-scoped; do not infer either from the
  provider display name.
- e2e asserts shape, not cost — the §3 reconciliation is the only pricing check.

For image/video models, open the studio and confirm the offered
sizes/qualities/durations match exactly what the deployment accepted in §6.

## 8. When something fails

| Symptom                                       | Cause                                                                                                      |
| --------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |
| JSON output cases fail                        | deployment rejects `response_format`; set `jsonOutput: false` and drop it from `supportedParameters`       |
| Reasoning-effort case 400s                    | trim the tier from `reasoningEfforts`                                                                      |
| Forced tool_choice 400s                       | narrow `supportedToolChoices`                                                                              |
| Vision case 400s                              | `vision: false` on that mapping                                                                            |
| Cost ~2x the provider's on reasoning requests | reasoning double-counted — check `normalizeCompletionTokens` against the provider's `usage` totals         |
| Cost far below on reasoning requests          | reasoning tokens never extracted (nested `completion_tokens_details`)                                      |
| Cost mismatch only on long prompts            | wrong or missing `pricingTiers` band                                                                       |
| Manual curl hits the wrong provider           | missing `x-no-fallback: true`                                                                              |
| `max_tokens` case returns no content          | reasoning spends the whole budget; declare the tiers the deployment actually honours in `reasoningEfforts` |

If a failure predates the change, fix what's in scope and say in the PR that it
also fails on `main`.

A failing case is a metadata or gateway fix, never a reason to retire a mapping
that still serves live requests. Before writing `deactivatedAt`, apply the
7-day notice rule in [Catalogue rules](#catalogue-rules).

## 9. Finish

`pnpm format`, `pnpm build`, conventional title, then the `pull-request` skill.

Never merge — and never enable auto-merge on — a mapping that does not actually
work: it could not serve a live request, or its scoped e2e fails with no fix
available from our side. `test: "skip"` removes a mapping from default e2e
selection; `stability: "unstable"` also removes it from normal routing. Neither
makes the model work. Say plainly that it does not work and why, leave the PR
open, and hand the merge decision to the user. Same for prices nobody could
verify.

The PR body carries the evidence:

- Per mapping: external id, prices, context, max output, capabilities.
- How each price was verified — rates, token counts, hand-computed cost,
  `log.cost`.
- Reasoning tiers accepted vs rejected; for image/video, the accepted grid.
- The scoped e2e command and its result per mapping.
- Anything corrected from the handed-in values, with both numbers.
- Anything not verified, and why.

Follow the `pull-request` skill for screenshots. Catalogue and Playground-only changes do
not create a dashboard screenshot requirement; scoped e2e and the accepted
image/video grid are the evidence.