amd/PARD-Qwen3-0.6B

PARD-Qwen3-0.6B: Accelerating LLM Inference

The amd/PARD-Qwen3-0.6B is a Qwen-based parallel draft model developed by AMD as part of the PARD (PARallel Draft Model Adaptation) framework. PARD is a high-performance speculative decoding method designed to significantly accelerate LLM inference by adapting autoregressive (AR) draft models into parallel draft models with minimal overhead.

Key Capabilities & Features

• Low-Cost Training: PARD adapts AR draft models efficiently, achieving an average inference speedup of 1.78x compared to pure AR draft models. It incorporates a conditional drop-token strategy that improves training efficiency by up to 3x while maintaining accuracy.
• Generalizability: Unlike target-dependent approaches (e.g., Medusa, EAGLE), PARD's design allows a single draft model to accelerate an entire family of target models. This reduces deployment complexity and adaptation costs by eliminating the need for retraining or tuning for each new target model.
• High Performance: When integrated into optimized inference frameworks, PARD delivers substantial speedups. For instance, with Transformers+, it achieves up to a 4.08x speedup, enabling LLaMA3.1 8B to reach 311.5 tokens per second. When integrated into vLLM, PARD provides up to a 3.06x speedup, outperforming other speculative decoding methods in vLLM by 1.51x.

Good For

• Accelerating LLM Inference: Ideal for developers looking to significantly boost the inference speed of various large language models.
• Reducing Deployment Complexity: Suitable for environments where a single draft model needs to accelerate multiple target models without extensive re-tuning.
• Cost-Effective Adaptation: Beneficial for projects requiring efficient adaptation of existing autoregressive models into parallel draft models with minimal training overhead.