MMOPD/Qwen3-1.7B-OT3-1ep
MMOPD/Qwen3-1.7B-OT3-1ep is a 1.7 billion parameter language model developed by MMOPD, fine-tuned from Qwen3-1.7B-Base on the OpenThoughts3-1.2M dataset. This model is specifically designed as a "thinking" model, where every response begins with a dedicated block, emphasizing long-chain-of-thought reasoning in math, code, and science. It serves as a general reasoning student checkpoint, intended as a starting point for further domain-specific teachers and distillation experiments within the MMOPD project. With a context length of 32768 tokens, it is optimized for complex reasoning tasks requiring explicit thought processes.
Loading preview...
MMOPD/Qwen3-1.7B-OT3-1ep: A Thinking Model for Reasoning
This model, developed by MMOPD, is a 1.7 billion parameter variant of Qwen3-1.7B-Base, fine-tuned for one epoch on the extensive OpenThoughts3-1.2M dataset. OpenThoughts3-1.2M comprises 1.2 million examples of long-chain-of-thought reasoning traces across mathematics, coding, and scientific domains. The core differentiator of this model is its "thinking" architecture: every generated answer explicitly starts with a <think> block, making its reasoning process transparent.
Key Capabilities and Design
- Explicit Reasoning: Designed to articulate its thought process before providing a final answer, using a
<think> ... </think>format. - Foundation for Specialization: Serves as a general reasoning student, intended as a base for creating domain-specific teachers and for on-policy distillation experiments within the MMOPD project.
- Training Data: Fine-tuned on the full OpenThoughts3-1.2M dataset, which focuses on complex, multi-step reasoning.
- Context Length: Supports a sequence length of 18,432 tokens during training, with a generation budget of up to 32,768 tokens.
Intended Use Cases
This model is particularly well-suited for applications requiring:
- Complex Problem Solving: Excels in tasks demanding multi-step reasoning in math, code, and science.
- Explainable AI: Its explicit thinking blocks can provide insights into how the model arrives at its conclusions.
- Foundation for Customization: Ideal as a base model for further fine-tuning into specialized reasoning agents or for knowledge distillation research.
Users should enable enable_thinking=True and use sampling (not greedy decoding) for optimal performance, leveraging its unique reasoning capabilities.