empero-ai/Qwen3.8-2B-Distill
empero-ai/Qwen3.8-2B is a 2.3 billion parameter causal language model developed by Empero, distilled from the larger Qwen3.8 2.4T A95B teacher model. It utilizes the Qwen3.5-2B architecture and is specifically optimized for reasoning, mathematics, and instruction following, inheriting a 262,144-token native context length. This model is designed for edge deployments, offering strong chain-of-thought capabilities in a compact size.
Loading preview...
Model Overview
Empero's Qwen3.8-2B is a 2.3 billion parameter causal language model, a full-parameter distillation of the powerful Qwen3.8 2.4T A95B teacher model into the Qwen3.5-2B architecture. It was trained on approximately 30,000 curated teacher traces, focusing on dense chain-of-thought reasoning across mathematics, general reasoning, and instruction following. This model aims to bring advanced reasoning capabilities to edge devices, running efficiently on phones, single-board computers, and CPU-only machines.
Key Capabilities and Features
- Distilled Chain-of-Thought: Every response begins with a
<think>block, directly learned from high-quality Qwen3.8 teacher traces, ensuring robust reasoning. This is not synthetically generated reasoning. - Consistent Curriculum: It shares the exact same quality-filtered training curriculum as its larger siblings, Qwen3.8-4B and Qwen3.8-9B, with the primary difference being the student model's capacity.
- Edge-Optimized: With 2 billion parameters, it fits into approximately 4 GB in bf16, making it suitable for resource-constrained environments.
- Native Function Calling: Supports function calling as per Qwen3.5 specifications without requiring additional wrappers or fine-tuning.
- Extended Context Window: Inherits a substantial 262,144-token native context length from its Qwen3.5 base.
- Full Fine-tune: All parameters were updated during training, not just an adapter.
Performance Highlights
Qwen3.8-2B demonstrates significant improvements over its Qwen3.5-2B base model in reasoning benchmarks:
- GSM8K (CoT): Achieves 0.640 exact match (flexible), a +0.310 improvement over the base.
- MMLU (CoT, 57 subjects): Reaches 0.548 accuracy (flexible-extract), a +0.265 improvement.
Recommended Use Cases
This model is ideal for applications requiring strong mathematical reasoning, general problem-solving, and instruction following in a compact footprint. It's particularly well-suited for deployment on devices with limited computational resources. For code-heavy workloads, the larger Qwen3.8-9B is recommended, as this model's training emphasized mathematics and reasoning.