PARD-Llama-3.2-1B: Accelerating LLM Inference

PARD-Llama-3.2-1B is a 1 billion parameter model developed by AMD as part of the PARD (PARallel Draft Model Adaptation) framework. PARD is a high-performance speculative decoding method that significantly accelerates Large Language Model (LLM) inference by adapting autoregressive (AR) draft models into parallel draft models with minimal overhead.

Key Capabilities and Differentiators

Low-Cost Training: PARD adapts AR draft models efficiently, achieving an average inference speedup of 1.78x compared to pure AR draft models. A conditional drop-token strategy further improves training efficiency by up to 3x.
Generalizability: Unlike target-dependent approaches (e.g., Medusa, EAGLE), PARD's design allows a single PARD draft model to accelerate an entire family of target models. This significantly reduces deployment complexity and adaptation costs.
High Performance: When integrated into optimized inference frameworks, PARD delivers substantial speedups. With Transformers+, it achieves up to 4.08x speedup, reaching 311.5 tokens per second with LLaMA3.1 8B. In vLLM, it provides up to 3.06x speedup, outperforming other speculative decoding methods by 1.51x.

When to Use This Model

This model is particularly well-suited for scenarios requiring:

Accelerated LLM Inference: For applications where high throughput and low latency are critical.
Cost-Efficient Deployment: Its generalizability reduces the need for retraining or tuning for each new target model, lowering operational costs.
Broad Model Compatibility: Ideal for accelerating various Llama3-based target models without specific draft model adaptation for each.

For more detailed information and usage instructions, refer to the PARD GitHub repository.