Adadecode Eval

---
name: adadecode-eval
description: Evaluates the inference speedup and output consistency of adaptive layer parallelism for LLM decoding compared to standard autoregressive generation. Use when the user wants to benchmark on HumanEval, or asks about evaluating this task. Reports speedup.
metadata:
  skill_kind: dataset_eval
  source_arxiv: 2506.03700
  bibtex_key: wei2025adadecode
  confidence: high
---

# adadecode-eval

> AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism — Wei et al. (2025) (arXiv:2506.03700, 2025)

## What this evaluates

Evaluates the inference speedup and output consistency of adaptive layer parallelism for LLM decoding compared to standard autoregressive generation.

## Datasets

- **HumanEval** — total ?; splits: test (-1)

## Metrics

- `speedup` **(primary)** — range: other
  - Ratio of inference time of standard autoregressive decoding to AdaDecode inference time.
- `output consistency ratio` — range: percent
  - Percentage of generated outputs that exactly match the output of standard autoregressive decoding.
- `early prediction rate` — range: percent
  - Percentage of decoding steps where early prediction is triggered based on the confidence threshold γ.
- `verification rejection rate` — range: percent
  - Percentage of early-predicted tokens that fail verification and are rejected during decoding.

## Input / output format

**Input**: Text prompts or code generation tasks fed to the LLM backbone (e.g., CodeLlama-34B-Instruct, Llama3.1-8B-Instruct).

**Output**: Autoregressive token sequences generated by the model.

## Scoring recipe

```python
speedup = time_vanilla_ar / time_adadecode
consistent = sum(1 for p, g in zip(predictions, gold) if p == g)
consistency_ratio = (consistent / len(predictions)) * 100
early_prediction_rate = (steps_with_early_pred / total_steps) * 100
verification_rejection_rate = (rejected_tokens / total_early_pred_tokens) * 100
```

## Common pitfalls

- FP16 numerical precision can cause minor deviations from theoretical output parity, affecting consistency ratio measurements.
- Hardware FLOP bottlenecks can artificially lower measured speedup for methods requiring parallel n-gram generation.
- Hyperparameter γ significantly impacts early prediction rate and rejection rate; results are sensitive to its exact value.

## Evidence (verbatim from paper)

> As presented in[Table 2], our method consistently delivers superior speedup compared to all baseline approaches regardless of the backbone model size, achieving up to $1.73	imes$ speedup compared to standard autoregressive decoding. As shown in[Figure 4], we empirically evaluate the output consistency ratio of all baseline methods against the vanilla decoding and report their corresponding speedup ratios on the HumanEval benchmark with CodeLlama-34B-Instruct as the backbone.

## Citation

```bibtex
@misc{wei2025adadecode,
  title={AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism},
  author={Wei et al. (2025)},
  year={2025},
  note={arXiv:2506.03700}
}
```

- arXiv: 2506.03700