Hugging Face Overview
OpenPipe/Qwen3-14B-Instruct Overview
OpenPipe/Qwen3-14B-Instruct is an instruction-tuned variant of the Qwen3-14B causal language model, developed to enhance finetuning compatibility. While the original Qwen3 release did not include a 14B Instruct model, this fork introduces an updated chat template that makes Qwen3-14B non-thinking by default and highly compatible with finetuning frameworks like OpenPipe.
Key Features and Improvements
- Finetuning-Friendly Chat Template: The primary differentiator is its updated chat template, which resolves inconsistencies found in the default Qwen3 template regarding the rendering of
<think></think>tags. This version ensures that all assistant prompts and generation templates include these tags, providing consistent message formatting during both training and inference. - Model Architecture: Based on the Qwen3-14B model, it features 14.8 billion parameters (13.2 billion non-embedding parameters), 40 layers, and 40 attention heads for queries with 8 for key/value (GQA).
- Context Length: It supports a native context length of 32,768 tokens, which can be extended to 131,072 tokens using YaRN.
- Retained Capabilities: The model maintains the robust general capabilities inherent to the Qwen3-14B base model.
Ideal Use Cases
This model is particularly well-suited for developers and researchers looking to:
- Finetune Qwen3-14B: Its primary design goal is to provide a stable and consistent base for further instruction finetuning.
- Consistent Training and Inference: Benefit from a chat template that ensures uniform message formatting across training and inference stages, reducing potential discrepancies.
- Applications Requiring Strong General Language Understanding: Leverage the base Qwen3-14B's general capabilities in a finetuning-optimized package.