NostraEmpire/mirror-deepseek-r1-distill-qwen-32b
NostraEmpire/mirror-deepseek-r1-distill-qwen-32b is a 32.8 billion parameter language model developed by DeepSeek-AI, distilled from the larger DeepSeek-R1 model. It is fine-tuned using reasoning data generated by DeepSeek-R1, a model that leverages large-scale reinforcement learning for enhanced reasoning capabilities. This Qwen-based model excels in complex reasoning tasks across math, code, and general English and Chinese benchmarks, offering strong performance in a smaller, dense architecture.
Loading preview...
Overview
NostraEmpire/mirror-deepseek-r1-distill-qwen-32b is a 32.8 billion parameter model from DeepSeek-AI, part of the DeepSeek-R1-Distill series. This model is a distillation of the larger DeepSeek-R1, which was developed using a novel large-scale reinforcement learning (RL) approach without initial supervised fine-tuning (SFT) to foster strong reasoning abilities. The distillation process transfers these advanced reasoning patterns into smaller, dense models like this Qwen-based variant.
Key Capabilities
- Enhanced Reasoning: Inherits and distills the reasoning capabilities of the DeepSeek-R1 model, which demonstrated self-verification, reflection, and long chain-of-thought generation.
- Strong Benchmark Performance: Outperforms OpenAI-o1-mini across various benchmarks, including AIME 2024 (72.6% pass@1), MATH-500 (94.3% pass@1), GPQA Diamond (62.1% pass@1), and LiveCodeBench (57.2% pass@1).
- Efficient Architecture: Provides high reasoning performance in a 32.8B parameter dense model, making it more accessible than larger MoE models.
- Multilingual Support: Shows strong performance in both English and Chinese benchmarks, including C-Eval and CLUEWSC.
Good For
- Applications requiring robust reasoning in mathematics and coding.
- Scenarios where a powerful yet more efficient model than very large MoE architectures is needed.
- Research into distillation techniques for transferring complex reasoning from larger RL-trained models to smaller ones.