khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled
khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled is a 4 billion parameter language model developed by khazarai, fine-tuned from the Qwen3-4b-Thinking-2507 base model. Optimized for structured, long-form reasoning, it bridges the gap between small, efficient models and complex reasoning capabilities. This model excels at problem decomposition, self-correction, and providing detailed analytical answers, making it suitable for tasks requiring advanced logical processing.
Loading preview...
Model Overview
khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled is a 4 billion parameter language model developed by khazarai, specifically optimized for structured, long-form reasoning tasks. It is fine-tuned from the Qwen3-4b-Thinking-2507 base model using a specialized distillation dataset generated by Kimi-2.5-thinking.
Key Capabilities
- Enhanced Reasoning: Designed to bridge the gap between smaller, efficient models and the complex reasoning typically found in much larger models.
- Problem Decomposition: Excels at breaking down complex problems into manageable parts.
- Self-Correction: Capable of self-correcting its reasoning process to improve accuracy.
- Detailed Analysis: Provides comprehensive and analytical answers.
- Performance: Achieves a score of 76.09 on the
khazarai/Multi-Domain-Reasoning-Benchmark, outperforming its base modelQwen/Qwen3-4B-Thinking-2507(73.73).
Training Details
The model was fine-tuned using Unsloth + QLoRa on the khazarai/kimi-2.5-high-reasoning-250x dataset. This dataset comprises 250 samples and 1,114,407 tokens, with Kimi-2.5-Thinking serving as the teacher model.
Good For
- Applications requiring advanced logical reasoning and problem-solving.
- Generating detailed analytical responses.
- Use cases where efficient, smaller models need to perform complex reasoning tasks.