Installs into .claude/skills of the current project.
Are you the author of Vllm Speculative Decoding?
Add the live security badge to your README. It updates with every re-scan.
[](https://www.skillsdirectory.com/skills/air-gapped-vllm-speculative-decoding)
---
name: vllm-speculative-decoding
allowed-tools: Bash, Read, Write, Edit, Grep, Glob
description: |-
Pick, configure, tune, monitor vLLM speculative decoding in production. Thirteen SpeculativeMethod options (ngram, ngram_gpu, medusa, mlp_speculator, draft_model, suffix, custom_class, eagle, eagle3, dflash, dspark, mtp, extract_hidden_states), `--speculative-config` JSON schema, which methods pair with which target model family, Prometheus acceptance metric surface, version gates (v0.11.1 EAGLE-3 preamble fix, v0.16 parallel drafting, v0.18 ngram_gpu, v0.19 dflash and zero-bubble), composability with chunked prefill / PP / LoRA / FP8 / structured outputs, Arctic Inference plugin, where spec-dec stops paying at high batch.
when_to_use: |-
Trigger on speculative decoding, draft model, EAGLE/EAGLE-2/EAGLE-3, MTP/multi-token prediction, deepseek_mtp, glm4_moe_mtp, qwen3_next_mtp, Medusa, MLPSpeculator, DFlash, suffix decoding, ngram speculation, ngram_gpu, acceptance rate / AL, --speculative-config, -sc, --num-speculative-tokens, prompt_lookup_min/max, draft_tensor_parallel_size, parallel_drafting, vllm:spec_decode_num_drafts, vllm:spec_decode_num_accepted_tokens, Arctic Inference, SuffixDecodingCache, SpeculativeMethod, P-EAGLE, PARD, dynamic-K, rejection_sample_method. Low-acceptance troubleshooting, BS>=32 regressions, method-selection decisions, air-gapped draft-checkpoint sourcing, integration with chunked prefill/disagg serving/PP/LoRA/FP8 KV/structured outputs. Also implicit — "speed up decode", "draft model for X", "audit spec dec", "deploy-memo speculative", "can we get more tokens/sec".
---
# vLLM speculative decoding — operator skill
For production vLLM operators deciding which speculative method fits a given
model + workload, configuring it correctly, wiring the acceptance metrics into
their dashboards, and diagnosing why a deployment isn't seeing the expected
speedup.
## When spec-dec wins, when it loses
Spec-dec amortises memory-bandwidth-bound decode by letting a cheap proposer
guess k tokens that a single target-model forward can verify in parallel.
- **Wins at low concurrency (BS 1–8)**: decode is bandwidth-bound, k verified
tokens per target step → 1.5–3× throughput on a well-matched target+drafter.
EAGLE-3 on Llama-3.1-8B: +32% TPOT over EAGLE-1 at BS=4 (vLLM v0.11.1+, PR
#25916). DFlash on Qwen3-8B: 3.5× at BS=1, 1.6× at BS=32 (PR #36847).
- **Hurts at high concurrency (BS ≥ 32)**: target becomes compute-bound, draft
latency is no longer hidden, rejections waste GPU time. Red Hat, Snowflake
and the P-EAGLE author all report this. Gate spec-dec to the low-concurrency
tier of a disagg or multi-tenant deployment, or disable above a threshold.
- **Domain mismatch sinks acceptance**: stock EAGLE-3 checkpoints are chat-tuned.
Code / agentic / RL-rollout workloads see AL drop from ~3 to ~2. Measure on
actual traffic before trusting vendor numbers.
## Method selection
Pick once by target-model family and workload shape. Full per-method detail in
`references/methods.md`; MTP in `references/mtp.md`; EAGLE-3 specifics including
P-EAGLE in `references/eagle3.md`; DFlash in `references/dflash.md`; Arctic
plugin and suffix in `references/arctic-inference.md`. **If training your own
EAGLE-3 / DFlash drafter (vs. picking a published one), see
`references/training-data-recipes.md` for the five recipe families surveyed
across 466 published checkpoints.**
| Situation | Pick | Why |
|---|---|---|
| Target ships MTP heads (DeepSeek V3/R1/V3.2, GLM-4.5/4.6 MoE, Qwen3-Next, Qwen3.5, Nemotron-H, MiMo, ERNIE 4.5, EXAONE-MoE, LongCat-Flash, Pangu-Ultra-MoE, Step-3.5, **Kimi K3**, **Inkling**, MiniMax-M3, Bailing-hybrid, Gemma 4) | `mtp` | Heads trained during pretraining, no second checkpoint, best AL. Authoritative list: `MTPModelTypes` in `vllm/config/speculative.py` — 27 aliases at v0.30.0 |
| Qwen3 / Llama / DeepSeek / gpt-oss / Kimi K2 / Minimax M2 / Gemma 4 / Nemotron-H target on B200 class | `dflash` | Block-diffusion parallel drafter, 2.5–4.6× at BS=1, v0.19+ |
| Same list above, want mature / pre-trained head | `eagle3` | Current SOTA model-based method for listed families (v0.11.1+) |
| Agentic / code-editing / RL-rollout workload with repetition | `suffix` | Model-free suffix trees, 1.8–4.5× on SWE-Bench. Requires `pip install arctic-inference` |
| No pre-trained head, have a good tiny model (≤2B) in same family | `draft_model` | Runs full LM as drafter; TP must match target |
| Quick win, no drafter of any kind | `ngram_gpu` (v0.18+) or `ngram` | Prefix-matching only; fine for repetitive prompts, skip for open chat |
| Locked into vendor checkpoint | `medusa` / `mlp_speculator` | Legacy; still works, do not adopt for new deployments |
**EAGLE-3/DFlash aux-hidden-state support — no longer a hardcoded list.**
Through ~v0.21 this was a 14-name allowlist in `vllm/config/speculative.py`
(llama, qwen, minicpm, gpt_oss, hunyuan_vl, hunyuan_v1_dense, afmoe,
nemotron_h, deepseek_v2, deepseek_v3, kimi_k2, kimi_k25, minimax_m2, gemma4).
**At v0.27.0 that list is still gone from the file** (re-verified 2026-08-11). Support is now a *capability
interface*: `SupportsEagle3` in `vllm/model_executor/models/interfaces.py`,
checked by `supports_eagle3(model)` in
`vllm/v1/worker/gpu/spec_decode/eagle/eagle3_utils.py`, which raises
`RuntimeError("Model does not support EAGLE3 interface")` when absent. Models
declare their own layers via `get_eagle3_aux_hidden_state_layers()` /
`set_aux_hidden_state_layers()`.
So the question "is my model supported?" changed shape: **grep the model class
for the `SupportsEagle3` interface**, don't look for a name in a config list.
`speculative.py` now only records which *methods* consume aux hidden states —
`eagle3`, `extract_hidden_states`, `dflash`, **`dspark`**.
## Canonical `--speculative-config` shapes
Single source of truth: **`--speculative-config` JSON** (or `-sc` alias, v0.19+).
Legacy `--speculative-model` / `--num-speculative-tokens` CLI is deprecated.
```bash
# EAGLE-3 (the default "give me more tokens" choice in 2026)
vllm serve meta-llama/Llama-3.1-8B-Instruct \
--speculative-config '{"method":"eagle3","model":"yuhuili/EAGLE3-LLaMA3.1-Instruct-8B","num_speculative_tokens":3}'
# MTP (target model has native heads — model field not set)
vllm serve deepseek-ai/DeepSeek-V3 \
--speculative-config '{"method":"mtp","num_speculative_tokens":1}'
# DFlash (Qwen3 on B200)
vllm serve Qwen/Qwen3-8B \
--attention-backend flash_attn \
--speculative-config '{"method":"dflash","model":"<dflash-checkpoint>","num_speculative_tokens":15,"parallel_drafting":true}'
# Suffix decoding (agentic / code workloads)
# Requires: pip install arctic-inference
vllm serve <target> \
--speculative-config '{"method":"suffix","num_speculative_tokens":32}'
# N-gram GPU (v0.18+, model-free, repetitive prompts)
vllm serve <target> \
--speculative-config '{"method":"ngram_gpu","num_speculative_tokens":5,"prompt_lookup_min":2,"prompt_lookup_max":5}'
# Draft model (own tiny same-family LM, TP must match target)
vllm serve <target> \
--speculative-config '{"method":"draft_model","model":"<tiny-model>","num_speculative_tokens":4,"parallel_drafting":true}'
```
`num_speculative_tokens` tuning guidance in `references/methods.md`. `n_predict`
interaction for MTP (multiples-of-N rule) in `references/mtp.md`.
## Acceptance-rate metric surface
vLLM V1 emits four spec-dec metrics on `/metrics`
(`vllm/v1/spec_decode/metrics.py:154-198`). Every production spec-dec deployment
should scrape and dashboard all four:
| Metric | Type | Purpose |
|---|---|---|
| `vllm:spec_decode_num_drafts` | Counter | Spec-dec invocations |
| `vllm:spec_decode_num_draft_tokens` | Counter | Tokens proposed by drafter |
| `vllm:spec_decode_num_accepted_tokens` | Counter | Tokens accepted after verification |
| `vllm:spec_decode_num_accepted_tokens_per_pos` | Counter (label: position) | Per-position acceptance (position 0 to num_speculative_tokens-1) |
Counters export with `_total` suffix (prometheus_client convention).
**Per-request acceptance, v0.29.0+.** Those counters are server-wide, so they
cannot tell one bad workload from a uniformly bad drafter — which is the first
question low acceptance raises. From **v0.29.0**
([#48915](https://github.com/vllm-project/vllm/pull/48915), not in v0.28.0), start the server with
`--per-request-spec-decode-metrics summary` (or `detailed`) and each response
carries its own stats in the top-level `metrics` object under
`speculative_decoding`, including `mean_acceptance_length`. **Off by default and
the response is byte-identical when off**, so turning it on is safe to trial and
has to be turned on deliberately — it is a server-start flag, not per-request.
Use it to attribute acceptance before tuning anything global.
**PromQL recipes (from source comments in `metrics.py:122-139`):**
```promql
# Acceptance rate
rate(vllm:spec_decode_num_accepted_tokens_total[5m]) /
rate(vllm:spec_decode_num_draft_tokens_total[5m])
# Mean acceptance length (+1 for the bonus target token)
1 + (
rate(vllm:spec_decode_num_accepted_tokens_total[5m]) /
rate(vllm:spec_decode_num_drafts_total[5m])
)
# Per-position acceptance — watch the tail falloff
rate(vllm:spec_decode_num_accepted_tokens_per_pos_total[5m]) /
rate(vllm:spec_decode_num_drafts_total[5m])
```
**Expected steady-state acceptance** (rough bands — measure actuals):
- EAGLE-3 / MTP: 0.75–0.92
- DFlash: 0.80–0.90
- Draft model (well-matched): 0.70–0.85
- ngram / ngram_gpu: 0.30–0.60 (higher on repetitive prompts)
- Medusa: 0.60–0.75
- **Alert if below 0.50** — drafter divergence, tokenizer mismatch, or temperature drift.
Alertmanager templates, Grafana-panel layout, and cross-metric diagnostics
(AL-falling-while-KV-growing, acceptance-stable-but-throughput-flat, etc.) in
`references/metrics.md`.
Smoke-check a live endpoint with `${CLAUDE_SKILL_DIR}/scripts/check-spec-decode.sh <base-url>`.
## Critical version gates
Spec-dec shipped many fixes in 2025-2026 that affect *correctness*, not just
perf. If operating off a build older than these, upgrade before benchmarking.
| Fix / feature | Min version | Impact |
|---|---|---|
| EAGLE-3 MTBench +32% (preamble dedup fix, PR #25916) | **v0.11.1** | Older builds show only +5% — benchmark numbers are wrong |
| Unsupported sampling params now **hard-fail** (PR #31982) | **v0.14.0** | Prior versions silently ignored them |
| Async scheduling default ON with spec-dec (PR #27614, #31998) | **v0.14.0** | Big throughput win; assume on |
| Spec-dec + structured outputs (PR #33374) | **v0.16.0** | Prior versions: mutually exclusive |
| Unified parallel drafting (PR #32887) — enables P-EAGLE | **v0.16.0** | Required for `parallel_drafting: true` on EAGLE/draft_model/dflash |
| Pipeline parallel + spec-dec on MRV2 (PR #33960) | **v0.17.0** | Docs still say PP-incompatible as of the public page |
| Spec-dec + disaggregated serving (PR #34529) | **v0.17.0** | Required for NixlConnector + EAGLE-3 |
| ngram on GPU + async-scheduler compatible (PR #29184) | **v0.18.0** | `ngram_gpu` method enum |
| Zero-bubble async scheduling + spec-dec (PR #32951) | **v0.19.0** | ~small % throughput recovery |
| `dflash` method + Qwen3.5 / Kimi K2.5 / Mistral Large 3 EAGLE3 | **v0.19.0** | New method, new targets |
| `--speculative-config` / `-sc` alias (PR #38380) | **v0.19.0** | Short form; flag names stabilised |
| Per-draft-model MoE backend (PR #37880) | **v0.19.0** | `moe_backend` field inside `--speculative-config` |
| Configurable acceptance rate for synthetic rejection (PR #38045) | **v0.19.0** | Testing only; not for prod |
| Spec-dec respects reasoning/**thinking budgets** (PR #34668) | **v0.21.0** | Correct spec-dec on reasoning models — before this, budgets were not honoured |
| Independent **drafter attention-backend** selection (PR #39930) | **v0.21.0** | Draft and target no longer forced onto one backend |
| **`custom_class`** method — callable proposer backend (PR #39487) | **v0.22.0** | New base method; bring your own proposer |
| **peagle** speculators (#41826) and **post-norm EAGLE-3** speculators (#42764) | **v0.22.0** | Wider checkpoint compatibility |
| **Dynamic SD** (PR #32374) | **v0.24.0** | Adapts speculation depth at runtime; made full-CUDA-graph compatible in v0.25.0 (#45953) |
| ⚠ **DoS fix: invalid recovered-token reinjection in spec-dec** (PR #44744) | **v0.24.0** | **Remote denial of service.** Listed under Security in the v0.24.0 notes. Anyone running spec-dec on an internet-reachable endpoint should be ≥ v0.24.0 |
| **TLI — universal spec-dec for heterogeneous vocabularies** (PR #38174) | **v0.25.0** | Target and drafter may have different but *overlapping* vocabularies. **Opt-in:** `"use_heterogeneous_vocab": true`, `method: "draft_model"` only, `draft_sample_method: "greedy"` only |
| **`kv_cache_dtype` inside `--speculative-config`** (PR #48787) | **v0.26.0** | Drafter KV dtype set independently of the target's `--kv-cache-dtype`. Unset = inherit. Lets a small drafter keep BF16 KV while the target runs FP8 |
| **Multi-layer MTP speculator on Model Runner V2** (PR #48892) | **v0.27.0** | MTP heads with more than one layer now run under MRV2 |
| **Quantized DSpark Markov heads** (PR #50424) | **v0.27.1** | `DSparkMarkovHead.markov_w2` accepts `quant_config`; W4A16 (incl. `weight_scale_2`) now loads through normal quantization dispatch. Before this, the Markov head stayed unquantized whatever the checkpoint declared |
| Native-MTP target list gains **Kimi K3** (`kimi_k3_mtp`) and **Inkling** (`inkling_mtp`) | **v0.26–v0.27** | `MTPModelTypes` 20 → 22 entries |
| **`dspark`** drafter (#46995, #47093) | **v0.25.0** | New base method + speculators checkpoint support |
| **Block verification** for rejection sampling (#46781) | **v0.25.0** | `RejectionSampleMethod` gains `"block"` alongside `standard` / `synthetic` |
| DFlash: **CPU support** (#44029), backend selection (#46770), FlashInfer (#43081), per-layer RMSNorm fusion (#46761) | **v0.24–v0.25** | DFlash matured well past its v0.19 debut |
| EAGLE-3 for Qwen3 (#43132); reduced TP comms for large-vocab drafts (#39419) | **v0.24.0** | |
| **EAGLE3 + pipeline parallelism** on MRV2 (#50514); MTP under PP for sparse-MLA targets (#46994) | **v0.30.0** | Before this, `eagle3` + `pipeline_parallel_size > 1` was a hard-rejected combination (`vllm/config/vllm.py`). The draft's own parallel config is now always `pipeline_parallel_size=1` regardless of the target's PP degree — the drafter itself is never split across pipeline stages |
| Draft's own `moe_backend` (#54788) and `attention_backend` (#54826) **actually honored on MRV2** | **v0.30.0** | The v0.19.0/v0.21.0 rows above only ever took effect on the legacy V1 speculator (`vllm/v1/spec_decode/`); Model Runner V2 (`vllm/v1/worker/gpu/spec_decode/eagle/utils.py`) silently forced the draft to inherit the target's backend until this fix |
| **Adaptive verification** opened to every draft-model speculator via an online acceptance estimator (#52228) | **v0.30.0** | `enable_adaptive_verification: true` in `--speculative-config` previously required a DSpark checkpoint with a trained confidence head; now any MRV2 draft-model-based method (eagle3, dflash, draft_model, or dspark without a head) gets per-position acceptance predicted online from the draft logits. Incompatible with `use_local_argmax_reduction: true` |
| Dual-key Gumbel-max watermarking compatible with spec-dec (#56122) | **v0.30.0** | `--watermark-config '{"algorithm":"dual_key_gumbel",...}'` works with probabilistic-sampling, standard-rejection `eagle`/`eagle3`/`mtp`/`dspark`. Other algorithms need `"allow_target_only_watermarking": true` (dilutes the signal) |
## Critical pitfalls
The "wins/loses" section above covers BS regime and domain mismatch. The
method-selection matrix covers TP constraints and the Arctic plugin
requirement. The items below are the silent-behaviour and version-specific
traps not captured by those.
1. **Draft-model TP must equal target TP** — hard error at
`vllm/config/speculative.py:46-51`, root cause is torch.compile cache
corruption. Medusa and `mlp_speculator` instead silently force TP=1
(validation at line 731), so a TP=8 target with a Medusa drafter runs the
drafter serially. Undocumented.
2. **MTP `num_speculative_tokens` > model's native `n_predict`** must be a
*multiple* of `n_predict`; vLLM re-runs the MTP layer. Logged warning at
speculative.py:531-536 ("may result in lower acceptance rate"). DeepSeek-V3
ships `n_predict=1` so asking for 5 runs the layer 5× sequentially.
3. **DeepSeek-V3.2 MTP forces `enforce_eager=True`** (speculative.py:397-398).
No CUDA graphs → ~10-20% throughput hit. Marked FIXME; recheck on upgrade.
4. **DFlash requires `--attention-backend flash_attn`** — Triton and
FlashInfer-TRTLLM don't support the non-causal cross-attention path.
5. **MTP model-specific method names are deprecated.** `deepseek_mtp`,
`glm4_moe_mtp`, `qwen3_next_mtp`, etc. all unified under `method: "mtp"`
(PR #25232). Old names still work but log deprecation.
6. **Logprob stability is not guaranteed** with spec-dec. If downstream eval
requires reproducible logprobs, disable spec-dec. Lossless *token*
guarantee holds "up to hardware numerics."
7. **Tokenizer mismatch between draft and target destroys acceptance.** The
`draft_model` validator does a vocab-size check, but not token-id
alignment. Verify with a shared-tokenizer family — **unless you are on
v0.25.0+ and opt in to TLI**. TLI is *not* automatic: set
`"use_heterogeneous_vocab": true` in `--speculative-config`, or the
equal-vocab check still rejects the pair. It requires
`method: "draft_model"` and `draft_sample_method: "greedy"` (the default);
both are hard `ValueError`s. See `references/methods.md`.
8. **Quantization mismatch**: drafter in FP16 + target in FP8 is fine;
drafter in INT4 + target in FP8 causes AL collapse. Align where possible,
or measure before adopting. **DSpark exception (v0.27.1+):** the Markov head
is a separate quantizable surface — `DSparkMarkovHead.markov_w2` only honours
the checkpoint's quantization from [#50424](https://github.com/vllm-project/vllm/pull/50424)
onward. On ≤ v0.27.0 it silently loads unquantized, so a W4A16 DSpark
checkpoint uses more drafter VRAM than its card implies.
## Composability (current, verified)
`chunked_prefill + spec-dec` → supported since v0.11.1 (PR #26263 fix). No mutual
exclusivity in code.
`async_scheduling + spec-dec` → default since v0.14.0. Zero-bubble variant in
v0.19.0 (#32951).
`LoRA + spec-dec` → EAGLE + LoRA CUDA graph specialisation in v0.11.1 (#28318).
Nemotron-H MTP LoRA in v0.16.0 (#32265).
`Pipeline parallel + spec-dec` → MRV2 only, v0.17.0+ (#33960). **Not V1 on
current engine runner**; if pinning PP on non-MRV2, spec-dec is off. `eagle3`
specifically stayed hard-blocked with PP until **v0.30.0** (#50514); MTP under
PP for sparse-MLA targets fixed the same release (#46994).
`Watermarking + spec-dec` → `dual_key_gumbel` supports spec-dec natively from
**v0.30.0** (#56122); other algorithms need `allow_target_only_watermarking:
true` in `--watermark-config`, and only with probabilistic draft sampling,
standard rejection sampling, and `eagle`/`eagle3`/`mtp`/`dspark`.
`FP8 KV + spec-dec` → supported. Sparse MLA + MTP full CUDA graphs v0.17.0
(#34457). FP8 MLA KV specific fix #37054. From **v0.26.0** the drafter can carry
its own `kv_cache_dtype` inside `--speculative-config` (#48787); unset means it
inherits the target's `--kv-cache-dtype`.
`Structured outputs + spec-dec` → v0.16.0+ (#33374). Prior: incompatible.
`Disaggregated serving + spec-dec` → v0.17.0+ (#34529). KV-transfer fix #35158
in v0.18.0.
`Multimodal + spec-dec` → v0.11.1 EAGLE/EAGLE3 on Qwen2.5-VL (#22872); text-only
drafters auto-disabled for MM when mismatch (#25667); v0.19 multimodal
embeddings for spec decode (#36097). Broad but spotty; check target family.
## Air-gapped operation
Spec-dec checkpoints (EAGLE-3 heads, MLP speculators, LSTM speculators, DFlash
adapters) live on Hugging Face. Treat them like any other HF model per the
`vllm-configuration` skill's air-gap patterns (`HF_ENDPOINT` mirror,
`HF_HUB_OFFLINE=1`, or `VLLM_USE_MODELSCOPE=True`). Specific gotchas:
- EAGLE-3 auto-detection fires on `"eagle3"` substring in `model` field —
renaming a locally-cached dir can break detection; keep the upstream name.
- Arctic Inference speculator checkpoints (Llama-3.1-8B/70B, Llama-3.3-70B,
Qwen2.5-32B) also live on HF — mirror before cutover.
- MTP heads ship *inside* the target model checkpoint — no separate artefact.
Air-gap story is the same as the base model.
## When spec-dec is the wrong lever
- Workload is embedding / classification / reward-model scoring — no autoregressive
decode, spec-dec does nothing.
- Single-token generation (tool-call decision, yes/no classification) — overhead
without benefit.
- Prefill-dominated traffic (TTFT SLO, long prompts, short outputs) — spec-dec
helps decode only. Chunked prefill and prefix cache matter more.
- Already-fast model (small <1B, or quantised to where decode is compute-bound)
— headroom for spec-dec is minimal.
See `references/troubleshooting.md` for diagnostic flow on low acceptance,
silent regressions after upgrade, and metric-interpretation pitfalls.
## External references
Last verified: **2026-08-11**. Source claims are probed against **v0.27.0**
(2026-08-10) except the DSpark Markov head, probed against **v0.27.1**
(2026-08-11, current stable). vLLM source and PRs probed via `gh`;
HF-checkpoint and Arctic-Inference rows carry older stamps — see
`references/sources.md` for the per-row provenance table, classifications, and
re-verification recipe. The **v0.27.1 container images** were pushed 2026-08-11 10:24-10:42Z, before the GitHub release at 10:47Z;
pull `vllm/vllm-openai:v0.27.1` rather than waiting on a PyPI wheel.
- Main docs: <https://docs.vllm.ai/en/latest/features/speculative_decoding/>
- Per-method docs: `/features/speculative_decoding/{eagle,mtp,draft_model,mlp,n_gram,suffix}/`
- P-EAGLE blog (2026-03-13): <https://vllm.ai/blog/p-eagle>
- Speculators v0.3 (2025-12-13): <https://vllm.ai/blog/speculators-v030>
- Red Hat EAGLE-3 (Jul 2025): <https://developers.redhat.com/articles/2025/07/01/fly-eagle3-fly-faster-inference-vllm-speculative-decoding>
- Snowflake Arctic Inference: <https://www.snowflake.com/en/engineering-blog/fast-speculative-decoding-vllm-arctic/>
- SuffixDecoding (NeurIPS 2025): <https://arxiv.org/abs/2411.04975>
- EAGLE-3 paper: <https://arxiv.org/abs/2503.01840>
- DeepSeek V3 (MTP §2.2): <https://arxiv.org/abs/2412.19437>
- Spec-Bench: <https://github.com/hemingkx/Spec-Bench>
- SPEED-Bench (NVIDIA): <https://huggingface.co/blog/nvidia/speed-bench>
