Iambackup/DeepSeek-R1-Distill-Qwen-7B

TEXT GENERATIONConcurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jul 7, 2026License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

DeepSeek-R1-Distill-Qwen-7B is a 7.6 billion parameter language model developed by DeepSeek-AI, distilled from the larger DeepSeek-R1 model and based on Qwen2.5-Math-7B. It is specifically fine-tuned using reasoning data generated by DeepSeek-R1, aiming to transfer advanced reasoning patterns to a smaller, dense model. This model excels in mathematical, code, and general reasoning tasks, offering strong performance for applications requiring robust analytical capabilities within a 32K context length.

Loading preview...

Model Overview

DeepSeek-R1-Distill-Qwen-7B is a 7.6 billion parameter model developed by DeepSeek-AI, part of their DeepSeek-R1-Distill series. This model is a distillation of the larger DeepSeek-R1, which itself was trained using large-scale reinforcement learning (RL) to develop strong reasoning capabilities without initial supervised fine-tuning (SFT). The distillation process transfers these advanced reasoning patterns from the 671B parameter DeepSeek-R1 into smaller, more efficient dense models like this 7B variant, based on Qwen2.5-Math-7B.

Key Capabilities

  • Enhanced Reasoning: Inherits sophisticated reasoning patterns from the DeepSeek-R1 parent model, which demonstrated capabilities like self-verification and reflection.
  • Strong Performance: Achieves competitive results across various benchmarks, particularly in math, code, and general reasoning tasks, outperforming several larger models in specific metrics.
  • Efficient Distillation: Demonstrates that complex reasoning abilities can be effectively transferred to smaller models, making high-performance reasoning more accessible.
  • Context Length: Supports a substantial context window of 32,768 tokens.

Usage Recommendations

  • Optimal Temperature: Recommended temperature setting between 0.5-0.7 (0.6 is ideal) to ensure coherent and non-repetitive outputs.
  • Prompting: Avoid system prompts; integrate all instructions directly into the user prompt.
  • Mathematical Tasks: For math problems, include a directive like "Please reason step by step, and put your final answer within \boxed{}" to encourage detailed reasoning.
  • Enforced Reasoning: To ensure the model engages in thorough reasoning, enforce it to begin its response with "\n".

Finetunes
1 models