HBkira/DeepSeek-R1-Distill-Qwen-1.5B
HBkira/DeepSeek-R1-Distill-Qwen-1.5B is a 1.5 billion parameter language model developed by DeepSeek-AI, distilled from the larger DeepSeek-R1 model and based on Qwen2.5-Math-1.5B. This model specializes in reasoning tasks, leveraging reasoning patterns learned from its larger counterpart. It is designed to offer strong performance in math, code, and general reasoning benchmarks within a smaller, more efficient footprint, making it suitable for applications requiring robust reasoning capabilities with reduced computational overhead.
Loading preview...
DeepSeek-R1-Distill-Qwen-1.5B Overview
This model is a 1.5 billion parameter distilled version of DeepSeek-R1, developed by DeepSeek-AI. It is built upon the Qwen2.5-Math-1.5B base model and has been fine-tuned using reasoning data generated by the larger DeepSeek-R1 model. The core innovation behind the DeepSeek-R1 series is its focus on reinforcement learning (RL) for reasoning capabilities, demonstrating that complex reasoning can be incentivized without initial supervised fine-tuning (SFT) in its larger counterparts.
Key Capabilities and Features
- Reasoning Distillation: Leverages reasoning patterns from the powerful DeepSeek-R1 model, enabling smaller models to achieve strong performance in complex tasks.
- Optimized for Reasoning: Shows competitive performance on benchmarks like AIME 2024 (28.9 pass@1), MATH-500 (83.9 pass@1), and CodeForces (954 rating) for its size class.
- Efficient Size: At 1.5 billion parameters, it offers a more efficient solution for deploying reasoning-focused applications compared to much larger models.
- High Context Length: Supports a context length of 32,768 tokens, allowing for processing longer inputs and generating detailed reasoning chains.
When to Use This Model
This model is particularly well-suited for use cases where:
- Resource Efficiency is Key: When deploying reasoning capabilities on hardware with limited resources or when faster inference is required.
- Mathematical and Coding Tasks: Its base in Qwen2.5-Math and distillation from a reasoning-focused model make it strong for math and code-related problem-solving.
- Complex Reasoning in Smaller Footprints: Ideal for applications that need to perform multi-step reasoning, self-verification, or generate chain-of-thought responses without the overhead of very large models.
DeepSeek-AI emphasizes specific usage recommendations, such as setting temperature between 0.5-0.7, avoiding system prompts, and enforcing a <think> token to ensure thorough reasoning.