empero-ai/Qwen3.8-9B-Distill
Qwen3.8-9B is a 9-billion parameter causal language model developed by Empero, distilled from a larger Qwen3.8 2.4T A95B teacher model. It leverages the Qwen3.5-9B architecture and is specifically optimized for complex reasoning, mathematics, and code generation. The model features a native 262,144-token context length and incorporates distilled chain-of-thought reasoning, making it suitable for tasks requiring deep analytical processing on a single GPU.
Loading preview...
Empero Qwen3.8-9B: Distilled Reasoning Power
Empero's Qwen3.8-9B is a 9-billion parameter language model engineered for advanced reasoning, mathematics, and code. It is a full-parameter distillation of an internal frontier-scale teacher, Qwen3.8 2.4T A95B, into the Qwen3.5-9B architecture. The model was trained on approximately 70,000 curated teacher traces, focusing on dense chain-of-thought examples across mathematics, code, general reasoning, instruction following, and tool use.
Key Capabilities & Features
- Distilled Chain-of-Thought: Every response begins with a
<think>block, directly learned from the high-capacity Qwen3.8 teacher, providing robust reasoning steps. - Mathematics and Code Emphasis: The training data heavily weights hard math and competitive programming tasks, where distillation significantly enhances performance at this scale.
- Native Function Calling: Supports Qwen3.5's function calling specification without requiring additional wrappers or fine-tuning.
- Extended Context Window: Inherits a substantial 262,144-token native context length from its Qwen3.5 base.
- Full Fine-tune: All parameters were updated during training, not just adapters, ensuring comprehensive knowledge transfer.
Performance Highlights
Benchmarks show significant improvements in reasoning tasks compared to its Qwen3.5-9B base. On the MMLU (CoT, 57 subjects) benchmark, Qwen3.8-9B achieves an acc (flexible-extract) of 0.751, a +0.205 improvement over the base model's 0.546. For acc (strict-match) on MMLU, it reaches 0.511, a +0.260 gain. While gsm8k_cot scores are slightly lower than the base, the substantial MMLU gains highlight its enhanced general reasoning capabilities.
Best Practices for Usage
For optimal results, Empero recommends specific sampling parameters (temperature=0.6, top_p=0.95, top_k=20) and allowing generous max_new_tokens (16,384) due to the initial <think> block in responses. Users should parse and strip this block for end-user presentation. The model's reasoning style, including occasional lengthy deliberations, is inherited from its teacher.