ProCreations/grug-27b-mtp
ProCreations/grug-27b-mtp is a 27 billion parameter language model, a variant of grug-27b (v2.1), specifically enhanced with a retrained Multi-Token Prediction (MTP) head. This model is optimized for speculative decoding, allowing it to predict multiple tokens simultaneously for faster inference. It maintains the terse output style of the original grug-27b while significantly improving draft quality for speedup. The model is designed for applications requiring high-speed text generation with a focus on efficiency.
Loading preview...
ProCreations/grug-27b-mtp: Accelerated Inference with Multi-Token Prediction
ProCreations/grug-27b-mtp is a 27 billion parameter language model, building upon the grug-27b (v2.1) base. Its primary differentiator is the integration of a trained Multi-Token Prediction (MTP) head, specifically designed for speculative decoding. This enhancement allows the model to predict multiple tokens ahead, significantly boosting inference speed without compromising the model's core output style.
Key Capabilities & Features
- Speculative Decoding: Utilizes a retrained MTP head to enable faster text generation by predicting future tokens.
- Optimized Draft Quality: The MTP head was retrained on 3 million tokens of
grug v2.1data, achieving an 89.3% agreement rate for draft quality, a notable improvement over an untrained head (81.7%). This ensures that the predicted drafts closely match the model's intended output. - Consistent Core Performance: The backbone of the model remains identical to
grug-27b, ensuring the same quality and terse output style. - Efficient Deployment: Supports
vLLM(version >= 0.19) with specific configuration for speculative decoding, and is available in GGUF format forllama.cppbuilds with Qwen3.5 MTP speculative support.
When to Use This Model
This model is ideal for use cases where inference speed and efficiency are critical, such as real-time conversational AI, rapid content generation, or applications requiring quick responses. Developers seeking to leverage speculative decoding for performance gains while retaining the distinct output characteristics of the grug-27b series will find this model particularly beneficial. For scenarios where MTP is not required, the plain grug-27b offers the same core intelligence in a smaller checkpoint.