--- license: mit pipeline_tag: text-generation library_name: transformers --- PARD

PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation

## Introduction PARD is a high-performance speculative decoding method that also enables low-cost adaptation of autoregressive draft models into parallel draft models. It offers the following advantages: - **Low-Cost Training**: PARD adapts AR (autoregressive) draft models into parallel draft models with minimal overhead. Compared to pure AR draft models, PARD achieves an average inference speedup of 1.78×. By introducing a conditional drop-token strategy, PARD improves training efficiency by up to 3× while maintaining the same level of accuracy. - **Generalizability**: Thanks to its target-independent design, a single PARD draft model can accelerate an entire family of target models. This contrasts with target-dependent approaches such as Medusa and EAGLE, which require retraining or tuning for each new target. As a result, PARD significantly reduces both deployment complexity and adaptation cost. - **High Performance**: When integrated into an optimized inference framework called Transformers+ PARD delivers up to a 4.08× speedup, with LLaMA3.1 8B reaches a state-of-the-art 311.5 tokens per second. When integrated into vLLM, PARD delivers up to 3.06× speedup, outperforming other speculative decoding methods in vLLM by 1.51×.

AR and AR+ represent baseline auto-regressive generation using Transformers and Transformers+, respectively. VSD denotes vanilla speculative decoding. PARD refers to the proposed method in this work.

## Model Weights | Model Series | Model Name | Download | |--------------|---------------------------------------|---------------| | llama3 | PARD-Llama-3.2-1B | [🤗 HuggingFace](https://huggingface.co/amd/PARD-Llama-3.2-1B) | | DSR Qwen | PARD-DeepSeek-R1-Distill-Qwen-1.5B | [🤗 HuggingFace](https://huggingface.co/amd/PARD-DeepSeek-R1-Distill-Qwen-1.5B) | | Qwen | PARD-Qwen2.5-0.5B | [🤗 HuggingFace](https://huggingface.co/amd/PARD-Qwen2.5-0.5B) | ## How To Use Please visit [PARD](https://github.com/AMD-AIG-AIMA/PARD) repo for more information ## Citation ``` @article{an2025pard, title={PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation}, author={An, Zihao and Bai, Huajun and Liu, Ziqiong and Li, Dong and Barsoum, Emad}, journal={arXiv preprint arXiv:2504.18583}, year={2025} } ```