Token Factory

---
name: token-factory
description: Use for zero-GPU hosted inference through Nebius Token Factory — captioning, batch text generation, and Cosmos physical-AI reasoning — including key setup, model selection, and the npa.workflow toolRefs that need no cluster.
---

# Token Factory (zero-GPU hosted inference)

## Robot demonstration SDG

For robot training data, use `npa workbench token-factory robot-sdg`, not the
text-only `sdg` command. It routes declarative scene planning through hosted
Lightning/MiniMax, executes real MuJoCo Fetch pick-and-place, records workspace
and wrist RGB with aligned joint state/actions, and exports accepted episodes
through the LeRobot v3 adapter. Physics state and contact checks judge success;
model text cannot declare an episode successful.

Install `npa[robot-sdg]` and `ffmpeg`. The public CLI uses S3 input/output paths;
the shared SDK `robot_sdg(RobotSdgRequest(...))` also accepts local files. Outputs
must be new. Open `index.html` for real videos, and validate the dataset with the
native LeRobot reader. The simulator uses a scripted teacher with privileged
state; do not describe the output as physical robot capture, learned-policy
performance, or photorealistic video. See `docs/workbench/token-factory-robot-sdg.md`
for the supported scene contract, live test, and workflow prerequisites.

Nebius Token Factory is an OpenAI-compatible hosted-inference API for open text
and vision models. It is the cheapest tier in the workbench that produces a real
artifact: **no cluster, no GPU, no provisioning**. Reach for it before standing up
anything, both for real work and to prove a toolchain end to end.

Full reference: `docs/workbench/token-factory.md`. Key setup only:
`docs/workbench/token-factory-key.md`.

## The credential is not a Nebius IAM token

This is the single most common failure. A Token Factory key is a separate
credential minted in the separate Token Factory console
(<https://tokenfactory.nebius.com/>). It is a long opaque token starting with
`v1.`, read from `NEBIUS_TOKEN_FACTORY_KEY` or `~/.npa/credentials.yaml`. Your
`nebius` CLI IAM token returns `403` here — having Nebius CLI access is not
enough, and no amount of re-authenticating the CLI will help.

Keys are shown once at creation. A project with no balance returns `402`/`403` on
inference even with a valid key.

```bash
npa workbench token-factory status    # connection settings, no network call
npa workbench token-factory verify    # live models call; non-zero on auth failure
npa workbench token-factory models    # what this key can actually reach
```

Run `verify` before a batch job and `models` before pinning a model name — model
availability is per-key, so a model in the docs may not be in your project.

Defaults: base URL `https://api.tokenfactory.nebius.com/v1/`, overridable with
`NEBIUS_TOKEN_FACTORY_BASE_URL`. Requests retry on 429 and 5xx.

## August 2026 migration

The [official notice](https://docs.tokenfactory.nebius.com/august-2026-deprecation-notice)
retires the old public text, vision, and reasoning defaults. The replacements
are Nemotron-3.5-Lightning for text and MiniMax-M3 for vision/reasoning.
See `docs/workbench/token-factory-deprecation-verification.md` for exact IDs,
live observations, API differences, vendor terms, and verification commands.
MiniMax-M3 is under the MiniMax Community License; hosted API access does not
establish the operator's commercial entitlement. No vendor weights are bundled.

Direct-output client calls disable thinking with model-specific template keys:
Lightning `enable_thinking=false`, MiniMax `thinking_mode=disabled`. Explicit
client `extra` values win. Agent reasoning turns enable thinking deliberately.

## Commands

### Automatically routed synthetic instruction data

`npa workbench token-factory sdg` runs seed classification, generation, model
review, exact-pair deduplication, and JSONL export through the shared SDK pipeline.
Its default router runs on Lightning through Token Factory and maps
transformation tasks to Lightning and reasoning tasks to MiniMax. `--router jev`
uses the optional TypeSafe adapter and additionally requires `TYPESAFE_API_KEY`.
Generation and review use hosted open-weight models in either mode.

This command uses S3 handoff paths; the SDK accepts local files for contributor
development. Its `--dry-run` reads and validates inputs without inference or
writes. Every input seed is processed once; no record-count or token cap is
added. Outputs are `dataset.jsonl`, `rejected.jsonl`, `provenance.jsonl`, and
`report.json`. Review is a fallible model judgment, so inspect samples before
training. See `docs/workbench/token-factory-sdg.md` for the runnable SDK example,
live evidence, and the `workbench.token_factory.sdg` workflow.

### Current pricing and usage

When the task requires a price check, fetch the official public catalog at
<https://tokenfactory.nebius.com/api/public/models_info>. The field meanings
are documented at <https://tokenfactory.nebius.com/model-catalog.md>; the site
also exposes <https://tokenfactory.nebius.com/llms.txt> for discovery. An empty
rendered pricing page is not evidence that pricing is unavailable.

Match pricing to the exact catalog model/flavor. Record the retrieval date and
applicable input/output rates, including the serving mode. A public-price lookup
does not require credentials. When inference is requested, also verify that the
selected model is in the key-scoped model list. Public catalog presence does not
prove account access or account-specific billing.
If required pricing remains unresolved, stop before paid inference.

Report inference usage and finish reason only when the response exposes them.
Keep the coding agent's token usage separate. Requested output tokens are a
limit, not measured usage; missing provider usage means cost is unmeasured.

### Inference commands

Every command takes local paths or `s3://` URIs for both input and output, and
supports `--dry-run` (compute without writing the artifact) and
`--output text|json`.

**Caption images** — default model `MiniMaxAI/MiniMax-M3`:

```bash
npa workbench token-factory caption \
  --input-path s3://<bucket>/frames/ \
  --output-path s3://<bucket>/captions.json \
  --max-images 50 --max-tokens 512 --temperature 0.2 \
  --instruction "Describe the scene, objects, and any action."
```

Caption requests append `If you cannot see the image pixels, respond exactly:
NO IMAGE RECEIVED.` after the operator instruction. The closed
case-insensitive whole-answer match also accepts an omitted final period or one
or more nested matching pairs of Markdown asterisk or underscore emphasis,
ASCII quotes, or smart quotes. It does not accept longer answers, mismatched
wrappers, code wrappers, punctuation outside the wrappers, other punctuation,
or paraphrases, and it retains the exact stripped answer. Exact-answer
induction can still false-fail, including when an image's only salient text is
the sentinel. Earlier retained hosted evidence observed the exact punctuated
sentinel; the composed thinking panel also observed MiniMax's periodless
sentinel for a valid blank PNG. These false unavailability indications do not
prove missing image delivery. Wrapped, nested, and emphasis variants remain
deterministic local controls, not observed hosted outputs. An unavailable item
fails the aggregate, is written before the CLI exits 1, and does not trigger a silent
retry. Later selected images are still attempted once. `--dry-run` emits the
same failed result and exits 1 without writing it.

The partial-result guarantee applies to cooperative sentinel responses, not
provider/configuration exceptions or storage-write failures; those can abort
before an artifact is saved.

Caption preserves existing model defaults when neither thinking flag is present.
If an explicitly selected reasoning-capable vision model returns no visible
caption, retry the command with `--no-thinking`. `--thinking` deliberately
reverses the built-in suppression for
Lightning/MiniMax and may consume the output allowance without yielding visible
text. Explicit overrides use verified model-specific fields for Lightning and
MiniMax and the generic `thinking` field for other model IDs, except known
`reasoning_effort` profiles such as Kimi-K3. Those reject either boolean override
before inference instead of sending a known unsupported template field; omitted
thinking options retain their defaults. Direct client `extra` can set
`reasoning_effort` explicitly. Other unsupported
providers may reject or ignore that control. Rejected requests and empty or
reasoning-only answers fail, but silently ignored controls with visible text
cannot be detected; control compliance is not validated.
The override is direct CLI/workbench/SDK only; the caption workflow toolRef does
not expose it.

**Batch text generation** over a JSONL/text prompt file — default model
`nvidia/Nemotron-3_5-Lightning`:

```bash
npa workbench token-factory generate \
  --input-path prompts.jsonl \
  --output-path s3://<bucket>/generations.jsonl \
  --max-prompts 0 --max-tokens 512 --temperature 0.7 \
  --system-prompt "<applied to every request>"
```

`--max-prompts 0` means all of them. Set a small non-zero value first: this is
the command that turns a typo into a large token bill.

Prompt parsing depends on the file extension. Keep plain prompt lines in a
`.txt` file. Each nonempty `.jsonl` line must be a JSON string or an object with
a `prompt`, `text`, or `instruction` field, not bare text. Use a JSON serializer
when converting prompts, including prompts written by a workflow shell step.
For a local, no-inference check, the installed
`npa.workbench.token_factory._load_prompts(Path(...))` reader returns
`(id, prompt)` pairs. Compare the prompt values and count with the requested
input. This is an internal Python helper, not a CLI validation command;
confirm it exists in the installed version before using it. It does not verify
remote staging or model access.

**Batch text generation** — same prompt file, same `generations.jsonl`, batch
token rates, default model `openai/gpt-oss-120b`:

```bash
npa workbench token-factory batch-generate \
  --input-path prompts.jsonl \
  --output-path s3://<bucket>/generations.jsonl \
  --model openai/gpt-oss-120b --completion-window 24h

# or submit now, collect later
npa workbench token-factory batch-generate ... --no-wait
npa workbench token-factory batch-status --operation-id <id> --output-path <same> --wait
```

Reach for `batch-generate` over `generate` whenever nothing is waiting on the
answer, which is most bulk stages. Three properties are unique to it, and each
one has already cost real debugging time:

- **Batch routing is a per-model entitlement, unrelated to real-time chat.** Most
  models that serve `generate` are rejected for batch. Historical measurements across eight
  text models on one key, exactly one — `openai/gpt-oss-120b` — was batch
  routable; `meta-llama/Llama-3.3-70B-Instruct`, `Qwen/Qwen3-32B`,
  `Qwen/Qwen3-30B-A3B-Instruct-2507`, `Qwen/Qwen3-235B-A22B-Instruct-2507`,
  `google/gemma-3-27b-it`, `deepseek-ai/DeepSeek-V4-Flash`,
  `nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B`, and `zai-org/GLM-5.1` were not. That
  is why `DEFAULT_BATCH_MODEL` is `openai/gpt-oss-120b` and not
  `DEFAULT_TEXT_MODEL`. Treat the routable set as per-key and verify on a couple
  of prompts before pointing a large run at a new model.
- **Batch is text-to-text only.** A vision model is rejected at submit with
  `Batch inference is only supported for text2text models`, so there is no batch
  captioning path; use `caption`, which is real-time.
- **The completion window is a deadline, not a latency.** Observed live: batches
  of one and three prompts sat `in_progress` with `completed: 0` for over an hour
  against a 24h window. Do not read a slow batch as a hung one, and never put
  `--wait` on a path that has its own timeout.

Where the failure reason actually lives matters. `GET /operations/{id}/errors`
returns a single empty string for a failed batch — useless. The real per-row
reason is in the batch record's error file
(`GET /batches/{id}` → `error_file_id` → `GET /files/{id}/content`, which
redirects, so redirects must be followed). `batch-generate` reads that file and
reports it, and also surfaces `request_counts` (`total`, `completed`, `failed`,
`invalid`) as the only genuine progress signal a pending batch offers.

**Distinguish a degraded platform from your own bug.** A batch that is accepted,
reports `in_progress` with rows validated (`total: 2, invalid: 0`), and holds
`completed: 0` is usually not your job's fault. Batch execution has been observed
unavailable while submissions were still accepted through the datasets/operations
route. The cheapest tell is `POST /v1/batches`, the OpenAI-compatible submit,
returning `403 Creating new batch job is temporarily unavailable`. Confirm it is a
server-side switch rather than your request by checking where the 403 lands: an
empty body returns `422` naming the missing fields, but a *valid* payload with a
genuinely uploaded `input_file_id` still returns `403`, so the gate sits ahead of
resource validation. Meanwhile the rest of the key stays healthy — real-time chat
on the same model, `POST /v1/files` with `purpose=batch`, `GET /v1/batches`, and
dataset create/delete all succeed — which rules out the key, the balance, the
model, and the payload. When you see this, stop debugging your spec, cancel what
you queued (`POST /batches/{id}/cancel`), and use `generate` until batch recovers.
Do not wait it out: the same 403 was still being returned eight days after it was
first seen, so "temporarily" can outlast any plausible stage timeout. Plan the
run on `generate` and re-probe later rather than leaving a stage parked.

**That 403 is not the quota rejection**, and conflating the two sends you down the
wrong path. The documented limits are 10 active batches per customer and 100
submissions per hour, a batch counts as active only until its processing
finishes, and rate limiting surfaces as `429`. So before blaming a limit, list
your batches (`GET /v1/batches?limit=100` — the default page is 10, which makes a
long history look artificially short) and count the non-terminal ones. All
terminal plus a 403 with no `x-ratelimit-*` headers means availability, not quota.

**Physical-AI reasoning over a scene** — default model
`MiniMaxAI/MiniMax-M3`. Point it at scene images and ask what a robot
should do:

```bash
npa workbench token-factory reason \
  --input-path s3://<bucket>/scene/ \
  --output-path s3://<bucket>/plan.json \
  --task "Describe this scene and give a step-by-step plan of action." \
  --max-images 8 --max-tokens 1024 --temperature 0.2
```

## In workflows

These run as CPU-only `npa.workflow` steps with no accelerator request. The
renderer injects `NEBIUS_TOKEN_FACTORY_KEY` for `workbench.token_factory.*`
steps, so pass it as a secret at submit time and never in the YAML:

```bash
npa workbench workflow submit <spec.yaml> --secret-env NEBIUS_TOKEN_FACTORY_KEY
```

toolRefs: `workbench.token_factory.caption`, `.generate`, `.batch_generate`,
`.reason`, `.triage` (digest a run's textual artifacts into a triage report).

`npa workbench token-factory workflow` prints six:
`token-factory-caption.yaml`, `token-factory-generate.yaml`,
`token-factory-cosmos-reason.yaml`, `token-factory-sdg.yaml`,
`token-factory-robot-sdg.yaml`, and `vlm-eval-token-factory.yaml`. Several
more are checked in but not listed by that command, so do not treat its output as
the full inventory:

- `token-factory-batch-generate.yaml` — the batch-inference twin of
  `token-factory-generate.yaml`.
- `token-factory-parallel-fanout.yaml` — parallel batches.
- `token-factory-gate-loop.yaml`, `tokenfactory-cosmos-gate.yaml` — a hosted
  model as a gate that decides whether the pipeline continues.
- `tokenfactory-rollout-judge.yaml`, `tokenfactory-scene-to-rollout-judge.yaml` —
  reason about a scene, then judge a rollout against that plan.
- `tokenfactory-train-triage.yaml` — triage a training run's artifacts.

All live under `workflows/testing/`.

## Choosing between Token Factory and VLM eval

They overlap and are easy to confuse. `token-factory reason` **produces** an
analysis or plan. `vlm-eval` **scores** a rollout against a task and emits a
pass/fail gate with a threshold. When you want a judged number for a gate, use
`skills/tools/vlm-eval/SKILL.md` — and note it can consume a Token Factory
reasoning artifact directly through `--task-from`, so a judge scores against a
plan an earlier stage wrote rather than a hardcoded string.

## Gotchas

- **Canonical Sim2Real is scoring, not planning.** Stage 8 uses
  `MiniMaxAI/MiniMax-M3` by default as its single Stage 8 evaluator, on CPU with no
  self-hosted evaluator image. It sends a bounded, deterministic rollout-wide
  frame sample and requires event-local structured scores. Stage 9 compares the
  single evaluator result with the authoritative Stage 7 rollout set and rejects
  missing, duplicate, or extra evaluations before PPO. Preserve request
  IDs, token usage, latency, retries, and an authoritative returned cost or
  explicit null separately from model-agent tokens.
  Hosted rollout requests use a strict JSON response schema for required event
  fields, selected camera filenames, action indices, and nonempty error-tag
  arrays. The parser still rejects incomplete coverage, invalid values, and
  substituted model identities; an endpoint that rejects the schema must fail
  visibly instead of retrying without it.
  The v4 hosted contract binds events to recorded primary-frame `sim_step`
  values. A sampled frame may support only an action at that exact time. For
  unsampled actions, the model must return a null camera, zero confidence,
  neutral tags, and an explicit insufficient-evidence critique. Those events
  retain simulator truth but cannot shape visual corrections or PPO tag counts.
  Preserve selected frame metadata and generated bindings in the artifact;
  Stage 9 reconstructs them and rejects older unbound evaluator artifacts.
- **Sim2Real preflight is stronger than model listing.** Its submit and prepared
  action paths declare `NEBIUS_TOKEN_FACTORY_KEY` by name only, then require
  both key-scoped model availability and a minimal inference before provisioning.
- **Model availability is per-key.** Confirm with `models` before pinning a name
  in a spec; a spec that names an unavailable model fails at run time, not at
  validation.
- **`--max-images` and `--max-prompts` are cost controls, not correctness knobs.**
  Defaults are 50 images and unlimited prompts. Always bound the first run.
- **`--dry-run` still calls the model.** It skips writing the artifact, so it is
  not a free syntax check. For a free check, validate the spec instead.
- **Hosted inference is not a rendering or simulation path.** It has no access to
  your cluster, your PVCs, or a GPU; give it S3 or local inputs it can read.
- **The key belongs in credentials, not in a spec or a shell history.** Persist it
  with `npa configure --save-env-credentials` (atomic `0600` write, never
  printed).

## Verify

```bash
npa/.venv/bin/python -m pytest npa/tests/guardrails/test_skills_index.py -q
```