SC117/QwenPaw-Flash-9B-heretic
SC117/QwenPaw-Flash-9B-heretic is a 9 billion parameter dense language model fine-tuned by SC117 using the Heretic methodology on the Qwen3.5-9B base model. This model is provided in F32 safetensors format and includes a Multi-Token Prediction (MTP) head, enabling faster inference through speculative decoding with an acceptance rate of approximately 43% and a 1.5-1.9x decode throughput speedup. It is designed for efficient text generation, particularly benefiting from the MTP feature for improved speed.
Loading preview...
QwenPaw-Flash-9B-heretic Overview
QwenPaw-Flash-9B-heretic is a 9 billion parameter dense language model developed by SC117, fine-tuned from the Qwen3.5-9B base model using the "Heretic" methodology. This model is distributed in F32 (float32) safetensors format, ensuring high precision.
Key Capabilities
- Enhanced Inference Speed: A primary differentiator is the inclusion of a Multi-Token Prediction (MTP) head, extracted from Qwen3.5-9B. This feature allows the model to predict multiple future tokens in a single forward pass, significantly improving generation speed through speculative decoding.
- Efficient Decoding: The MTP head achieves an acceptance rate of approximately 43%, leading to a notable 1.5-1.9x increase in decode throughput.
- High Precision: Provided in F32 safetensors, maintaining full float32 precision for model weights.
Good For
- Applications Requiring Fast Text Generation: Ideal for scenarios where rapid response times are crucial, such as interactive chatbots, real-time content creation, or any application benefiting from accelerated token generation.
- Developers Utilizing GGUF: Optimized GGUF versions are available, including MTP-enabled GGUF, for efficient inference with tools like llama.cpp, Ollama, and LM Studio.
- Research and Development: Offers a fine-tuned Qwen3.5-9B variant with a focus on inference optimization, suitable for exploring advanced decoding techniques.