Iambackup/DeepSeek-R1-Distill-Qwen-32B
The Iambackup/DeepSeek-R1-Distill-Qwen-32B is a 32.8 billion parameter language model developed by DeepSeek-AI, distilled from the larger DeepSeek-R1 reasoning model and based on the Qwen2.5 architecture. It is specifically fine-tuned using reasoning patterns generated by DeepSeek-R1, enabling it to achieve strong performance in math, code, and general reasoning tasks. With a context length of 32768 tokens, this model is optimized for complex problem-solving and outperforms OpenAI-o1-mini on various benchmarks for dense models.
Loading preview...
DeepSeek-R1-Distill-Qwen-32B Overview
This model is a 32.8 billion parameter language model from DeepSeek-AI, part of the DeepSeek-R1-Distill series. It is a distilled version of the larger DeepSeek-R1 reasoning model, built upon the Qwen2.5 architecture. The core innovation lies in its training methodology: it leverages reasoning patterns generated by the powerful DeepSeek-R1 model, which was developed using large-scale reinforcement learning (RL) without initial supervised fine-tuning (SFT) to foster complex chain-of-thought capabilities.
Key Capabilities & Differentiators
- Reasoning Distillation: Demonstrates that complex reasoning patterns from larger models can be effectively transferred to smaller, dense models, leading to superior performance compared to reasoning patterns discovered directly on small models.
- Strong Performance: Achieves competitive results across math, code, and general reasoning benchmarks. For instance, it scores 72.6 on AIME 2024 pass@1 and 94.3 on MATH-500 pass@1, outperforming GPT-4o-0513 and Claude-3.5-Sonnet-1022 in these specific metrics.
- Optimized for Complex Tasks: Designed to excel in scenarios requiring detailed step-by-step reasoning, such as mathematical problem-solving and code generation.
- Qwen2.5 Base: Built on the robust Qwen2.5 series, ensuring a solid foundation for its language understanding and generation capabilities.
- Long Context: Supports a context length of 32,768 tokens, suitable for handling extensive inputs and generating detailed responses.
Usage Recommendations
- Temperature Setting: Recommended to set temperature between 0.5-0.7 (0.6 ideal) to prevent repetitive or incoherent outputs.
- Prompting: Avoid system prompts; include all instructions within the user prompt.
- Reasoning Directive: For mathematical problems, include a directive like "Please reason step by step, and put your final answer within \boxed{}" to encourage detailed thought processes.
- Enforce Thinking: To ensure thorough reasoning, it's recommended to enforce the model to start its response with "\n".