Ascend Inference

---
name: ascend-inference
description: "Route Huawei Ascend vllm-ascend and XLLM inference for VLM-R1 OVD
  checkpoints."
disable-model-invocation: true
metadata:
  disco-role: operating
license: Apache 2.0
---

# ascend-inference

Use this sub-skill when a user needs to deploy or exercise a VLM-R1 OVD checkpoint on Huawei Ascend hardware through `vllm-ascend` or XLLM. It is a recipe and command-template skill: it does not claim that the current host has an Ascend NPU.

## Route first

- **Use here** for Ascend Atlas 800T A2 / 910B or Atlas 300I Duo online service, offline request scaffolds, OpenAI-compatible chat requests, XLLM command rendering, and Ascend performance notes.
- **Route away** for CUDA GRPO training, LoRA/freeze/multi-node training, JSONL/reward design, and REC/OVD CUDA evaluation. Those belong to sibling VLM-R1 sub-skills.
- **Do not ask future agents to reopen the source repository.** The operating facts needed for Ascend workflows are distilled in `references/ascend-inference-workflows.md` and `references/troubleshooting.md`.

## Minimum safe workflow

1. Identify the target hardware and engine:
   - Atlas 800T A2 / 910B: `vllm-ascend` and XLLM recipes are available.
   - Atlas 300I Duo: `vllm-ascend` recipe is available; use float16-oriented settings.
2. Confirm the deployment host has Ascend device visibility (`npu-smi` and required device nodes) before rendering runnable commands.
3. Make the model checkpoint available locally. The examples use `omlab/VLM-R1-Qwen2.5VL-3B-OVD-0321` as the model id and `VLM-R1-Qwen2.5VL-3B-OVD-0321` as a placeholder local directory.
4. Use the bundled renderers instead of copying hard-coded commands:
   - `scripts/ascend_offline_request_template.py --help`
   - `scripts/ascend_server_client_templates.sh --help`
5. If the user requests performance testing, preserve the evalscope multimodal caveat from `references/ascend-inference-workflows.md`; do not silently run text-only stress tests against XLLM's VLM backend.

## Key operating notes

- `vllm-ascend` uses OpenAI-compatible `/v1/chat/completions` requests for online serving and a vLLM Python `LLM.generate` pattern for offline inference.
- Atlas 300I Duo evidence uses the `vllm-ascend` 310P image tag and float16 settings; if a downloaded model metadata declares bfloat16, adjust it to float16 for that deployment target before serving.
- XLLM evidence covers Atlas 800T A2 / 910B only. Its server command needs a built XLLM executable, `--backend=vlm`, `--model`, `--port`, `--max_memory_utilization`, and `--model_id`.
- All bundled scripts are renderers/templates. They should print or write request scaffolds and commands; they should not start a server, call a live service, download a model, or probe a real NPU by default.

## Internal references

- `references/ascend-inference-workflows.md` — hardware matrix, vllm-ascend Docker/server/offline/client flows, XLLM build/server/client flow, prompt schema, evalscope caveat.
- `references/troubleshooting.md` — device initialization, container mounts, dtype, XLLM build, service/client, prompt JSON, and performance triage.
- `scripts/ascend_offline_request_template.py` — offline vLLM request/Python scaffold renderer.
- `scripts/ascend_server_client_templates.sh` — vllm-ascend/XLLM server and client command renderer.