ProCreations/grug-v1.1-qwen-3.8-27b-mtp
ProCreations/grug-v1.1-qwen-3.8-27b-mtp is a 27 billion parameter Qwen3.8-based language model developed by ProCreations, specifically enhanced for speculative decoding. It features a retuned multi-token prediction (MTP) draft head, achieving 95.37% agreement with the verifier, significantly improving over the native Qwen3.8 head's 90.04%. This model is optimized for faster inference through speculative decoding while maintaining the original grug-v1.1-qwen-3.8-27b's output quality, making it suitable for applications requiring high throughput.
Loading preview...
Overview
ProCreations/grug-v1.1-qwen-3.8-27b-mtp is an enhanced version of the grug-v1.1-qwen-3.8-27b model, featuring a specialized multi-token prediction (MTP) draft head. This enhancement is designed to significantly improve inference speed through speculative decoding without altering the model's core output quality or verified weights. The model's backbone remains frozen, with only the 425M-parameter mtp.* module being retuned.
Key Enhancements
- Improved Speculative Decoding: The draft head's agreement with the verifier (t+2 agreement) has been boosted from 90.04% (native Qwen3.8 head) to 95.37% after grug's specific tuning. This reduces the disagreement rate from 1-in-10 to 1-in-21, enabling more efficient speculative execution.
- Better Gold Top-1 Accuracy: The tuning process also improved the gold top-1 accuracy from 85.50% to 88.20%, indicating a genuine improvement in predicting real tokens, not just mimicking verifier behavior.
- Self-Distillation Training: The MTP head was trained using self-distillation, targeting acceptance directly by agreeing with the
grug-v1.1-qwen-3.8-27b's own greedy next-token choices. Training data included agent trajectories, tool calls, and code from theProCreations/grug-27b-v2-correctivedataset.
Usage and Compatibility
Engines that support Qwen3.8 MTP, such as vLLM and sglang, can leverage this model's draft head for accelerated inference. Standard transformers will ignore the mtp.* module and run the normal model without speedup, ensuring compatibility. The verified model output remains identical to grug-v1.1-qwen-3.8-27b, as the draft head only impacts speed, not the generated answer.