Openai Serving

---
name: openai-serving
description: "Use for vLLM OpenAI-compatible serving workflows: vllm
  serve/chat/complete/run-batch, OpenAI clients, server lifecycle, API request
  validation, batch JSONL, and client/server troubleshooting."
disable-model-invocation: true
metadata:
  disco-role: operating
license: Apache 2.0
---

# openai-serving

Use this sub-skill when the task is about running or consuming vLLM as an OpenAI-compatible HTTP server rather than using the offline Python `LLM` API.

## Route here

- Build or explain `vllm serve` commands, including host/port, API key, CORS, frontend, logging, model identity, dtype, tensor parallelism, memory, and max context flags.
- Use `vllm chat`, `vllm complete`, OpenAI Python clients, `curl`, or batch JSONL against `/v1` endpoints.
- Diagnose client errors such as connection refused, wrong `/v1` base URL, auth mismatch, 404 model mismatch, request-schema validation, streaming expectations, and port conflicts.
- Validate server readiness with `/health`, discover model names with `/v1/models`, inspect logs, and separate server lifecycle issues from request payload issues.

## Route elsewhere

- Offline in-process inference with `vllm.LLM`, `SamplingParams`, or `LLM.chat`: use `../offline-inference/SKILL.md`.
- Deep structured outputs, tool calling, tool parsers, or reasoning schemas: use `../structured-tool-reasoning/SKILL.md`.
- Multimodal, embeddings, pooling, rerank, score, LoRA, and adapter-specific endpoint behavior: use `../modalities-adapters-pooling/SKILL.md`.
- GPU capacity planning, throughput tuning, multi-node scaling, and deployment architectures: use `../deployment-performance/SKILL.md`.

## References

- Start with `references/cli-reference.md` for `serve`, `chat`, `complete`, and `run-batch` command patterns.
- Use `references/openai-api-workflows.md` for OpenAI-compatible client payloads, endpoints, model naming, and validation checks.
- Use `references/troubleshooting.md` for symptoms, likely causes, commands to run, and routing for hardware/performance failures.

## Bundled helpers

- `scripts/serve_command_builder.py` prints a safe `vllm serve` command from common frontend and engine options.
- `scripts/openai_client_smoke.py` checks `/v1/models` and can issue a minimal chat or completion request when a user-provided server is running.

Both scripts are self-contained and require only normal Python plus the `openai` package for the smoke client. They do not start model downloads or a server unless the user runs the printed command.