reaperdoesntknow/Qwen3-1.7B-Thinking-Distil
The reaperdoesntknow/Qwen3-1.7B-Thinking-Distil is a 2 billion parameter Qwen3-based causal language model developed by Convergent Intelligence LLC. It is specifically fine-tuned to capture extended deliberation and reasoning patterns from a larger Qwen3-30B-A3B-Thinking teacher model, compressing complex internal monologues into a smaller, efficient 1.7B student. This model excels at generating long-form reasoning chains, making it suitable for tasks requiring detailed thought processes and problem-solving before arriving at a conclusion. It features a 40,960 token context length and was trained using supervised fine-tuning on the longwriter-6k dataset.
Loading preview...
Model Overview
The reaperdoesntknow/Qwen3-1.7B-Thinking-Distil is a 2 billion parameter Qwen3-based language model developed by Convergent Intelligence LLC. It is a distilled version of the Qwen3-30B-A3B-Thinking teacher model, specifically designed to capture and reproduce extended deliberation patterns and long-form reasoning chains. Unlike models focused on direct instruction following or hierarchical code decomposition, this variant emphasizes the internal monologue of a larger model, including reasoning through uncertainty, backtracking, and re-evaluation.
Key Capabilities & Features
- Extended Reasoning: Specializes in generating detailed thought processes and deliberative responses, mirroring the internal monologue of a larger teacher model.
- Efficient Size: Compresses complex reasoning capabilities into a compact 1.7B effective parameter model, making it efficient for deployment.
- Qwen3 Architecture: Built on the Qwen3ForCausalLM architecture with a 40,960 token maximum context length.
- Supervised Fine-Tuning (SFT): Trained on the
longwriter-6kdataset to directly transfer the teacher's extended thinking traces. - BF16 Precision: Utilizes BF16 precision for training and inference.
Ideal Use Cases
- Complex Problem Solving: Generating detailed explanations or step-by-step reasoning for intricate problems.
- Content Generation: Creating long-form content that requires a deep, deliberative approach.
- Educational Tools: Simulating a thought process for learning or instructional purposes.
- Research & Development: Exploring how larger models arrive at conclusions by observing their distilled reasoning patterns.