Model Overview

khazarai/Qwen3-4B-Kimi2.5-Reasoning-Distilled is a 4 billion parameter language model developed by khazarai, fine-tuned from the Qwen3-4b-Thinking-2507 base. Its primary objective is to deliver advanced, long-form reasoning capabilities within a smaller, more efficient model size. The model was trained using Unsloth and QLoRa techniques.

Key Capabilities

• Structured Reasoning: Optimized to break down complex problems and provide detailed, analytical answers.
• Self-Correction: Designed to exhibit self-correction mechanisms during reasoning processes.
• Distilled Intelligence: Leverages a specialized distillation dataset from Kimi-2.5-thinking to transfer sophisticated reasoning patterns.
• Efficient Performance: Aims to bridge the performance gap between small (0.6B-4B) and much larger models for reasoning tasks.

Training Details

The model was fine-tuned on the khazarai/kimi-2.5-high-reasoning-250x dataset. This dataset comprises 250 samples and 1,114,407 tokens, generated by the Kimi-2.5-Thinking teacher model.

Good For

• Applications requiring detailed, step-by-step reasoning.
• Scenarios where complex problem-solving is needed from a resource-efficient model.
• Tasks involving strategic analysis, decision frameworks, and contingency planning, as demonstrated in the example.