khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled
The khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled model is a 4 billion parameter reasoning-distilled variant of Qwen3-4B-Thinking, fine-tuned by khazarai using QLoRA via Unsloth. It is designed to replicate the advanced reasoning capabilities of the larger Qwen3.6-plus teacher model. This model excels at providing concise, structured, and actionable solution paths for complex tasks, reducing verbosity and uncertainty. Its primary strength lies in transforming raw potential into an engineering-grade tool for structured problem-solving.
Loading preview...
Model Overview
This model, khazarai/Qwen3-4B-Qwen3.6-plus-Reasoning-Distilled, is a 4 billion parameter language model developed by khazarai. It is a reasoning-distilled variant of the Qwen3-4B-Thinking base model, fine-tuned using QLoRA via Unsloth. The core innovation is its ability to replicate the advanced reasoning capabilities of the larger Qwen3.6-plus teacher model.
Key Capabilities & Differentiators
- Enhanced Reasoning Structure: Significantly improves upon the base model by providing structured, professional, and report-oriented problem analysis.
- Concise Solutions: Reduces "rambling" and "uncertainty" often found in smaller models, delivering clear and actionable solution paths.
- Problem Analysis: Capable of immediately analyzing problems, separating concerns (Input, Output, Constraints), and formulating concrete algorithmic plans.
- High-Quality Output: Generates clean breakdowns including Problem Analysis, Intuition, Algorithm, Complexity Analysis, and Pseudocode, particularly for complex tasks like graph problems.
- Distillation Process: Achieved through QLoRA fine-tuning on a specialized dataset (
khazarai/qwen3.6-plus-high-reasoning-500x) with a max sequence length of 6,500 tokens.
Performance
On the khazarai/Multi-Domain-Reasoning-Benchmark (100 questions), this distilled model achieved a score of 75.64, outperforming its base model, Qwen/Qwen3-4B-Thinking-2507, which scored 73.73.
Ideal Use Cases
This model is particularly well-suited for applications requiring:
- Structured Problem Solving: Where clear, step-by-step reasoning and algorithmic planning are crucial.
- Engineering-Grade Solutions: For tasks demanding precise and professional output rather than exploratory thought processes.
- Complex Task Analysis: Especially beneficial for problems involving intricate constraints and multi-faceted solutions, such as advanced graph theory or optimization problems.