reaperdoesntknow/Qwen3-1.7B-Thinking-Distil
reaperdoesntknow/Qwen3-1.7B-Thinking-Distil is a 1.7 billion parameter Qwen3ForCausalLM model developed by Convergent Intelligence LLC: Research Division. It is a distilled version of the Qwen3-30B-A3B-Thinking teacher model, specifically fine-tuned on the longwriter-6k dataset to capture and reproduce extended reasoning chains and deliberative thought patterns. With a context length of 40,960 tokens, this model excels at generating long-form content that includes detailed internal monologues, uncertainty handling, and re-evaluation before reaching a conclusion, making it ideal for tasks requiring deep reasoning.
Loading preview...
Overview
reaperdoesntknow/Qwen3-1.7B-Thinking-Distil is a 1.7 billion parameter language model from Convergent Intelligence LLC: Research Division, distilled from the larger Qwen3-30B-A3B-Thinking teacher model. This model is specifically designed to emulate the teacher's extended deliberation patterns, capturing how it reasons through uncertainty, backtracks, and re-evaluates before arriving at a final answer. It achieves this through supervised fine-tuning on the longwriter-6k dataset, which contains long-form generation samples preserving these detailed reasoning chains.
Key Capabilities
- Extended Reasoning: Captures and reproduces the internal monologue and deliberative thought processes of a 30B parameter teacher model.
- Long-Form Generation: Trained to generate extensive responses, supporting a maximum sequence length of 4,096 tokens during training and capable of generating up to 2048 new tokens.
- Efficient Distillation: Compresses complex reasoning capabilities into a compact 1.7B parameter model, making it efficient for deployment while retaining rich deliberative depth.
- Qwen3 Architecture: Built on the Qwen3ForCausalLM architecture with Grouped-Query Attention (GQA) and a large vocabulary of 151,936 tokens.
Good For
- Complex Problem Solving: Ideal for applications requiring models to show their 'work' or reasoning process, rather than just providing a final answer.
- Content Generation: Suitable for generating detailed explanations, analyses, or creative writing that benefits from a visible thought process.
- Educational Tools: Can be used to demonstrate reasoning steps in educational contexts.
- Research & Development: Provides a compact model for exploring and experimenting with advanced reasoning distillation techniques, particularly those grounded in Discrepancy Calculus.