reaperdoesntknow/Qwen3-1.7B-Thinking-Distil
Qwen3-1.7B-Thinking-Distil is a 1.7 billion parameter causal language model developed by Convergent Intelligence LLC, distilled from the 30B Qwen3-30B-A3B-Thinking teacher. It specializes in capturing and reproducing extended reasoning chains and internal monologue patterns, trained via supervised fine-tuning on the longwriter-6k dataset. With a context length of 40,960 tokens, this model is optimized for tasks requiring deliberative thought processes and detailed explanations.
Loading preview...
Overview
Qwen3-1.7B-Thinking-Distil is a 1.7 billion parameter language model from Convergent Intelligence LLC, designed to emulate the extended reasoning capabilities of its larger 30B Qwen3-30B-A3B-Thinking teacher. This model captures the teacher's internal monologue, including reasoning through uncertainty, backtracking, and re-evaluation, through supervised fine-tuning on the longwriter-6k dataset. It features a Qwen3ForCausalLM architecture with approximately 2.03B parameters (1.7B effective) and supports a substantial context length of 40,960 tokens.
Key Capabilities
- Extended Reasoning: Excels at generating long-form reasoning chains and deliberative thought processes, mirroring the "Thinking" teacher's approach to problem-solving.
- Efficient Distillation: Compresses complex reasoning patterns from a 30B parameter model into a compact 1.7B student model, making advanced reasoning more accessible.
- Long Context Handling: Capable of processing and generating long sequences, trained with a max sequence length of 4,096 tokens.
- Robust Generation: Optimized for reasoning tasks with recommended generation parameters (temperature 0.6–0.8, repetition penalty 1.1–1.2) to encourage coherent and extended outputs.
Good For
- Applications requiring detailed explanations and step-by-step reasoning.
- Tasks where the process of arriving at an answer, including exploration and reconsideration, is as important as the final result.
- Scenarios benefiting from a smaller, faster model that retains sophisticated deliberative intelligence.