pixas/DECS_7B
DECS_7B is a 7.6 billion parameter causal language model developed by pixas, based on deepseek-ai/DeepSeek-R1-Distill-Qwen-7B. This model is specifically trained with the DECS algorithm to reduce token usage by 50% for reasoning-required problems. It excels in long-form reasoning, mathematical problem-solving, and general problem-solving tasks, offering efficient output generation for complex queries. The model has a context length of 32768 tokens.
Loading preview...
Model Overview
DECS_7B is a 7.6 billion parameter causal language model developed by pixas, presented as the official model for an ICLR 2026 Oral paper. It is built upon the deepseek-ai/DeepSeek-R1-Distill-Qwen-7B base model and has been further fine-tuned using the novel DECS (Decoupled Rewards and Curriculum Data Scheduling) algorithm. The primary innovation of DECS_7B is its ability to significantly reduce token generation, aiming for 50% fewer tokens when addressing problems that require complex reasoning.
Key Capabilities
- Efficient Reasoning: Optimized to provide concise yet comprehensive answers for reasoning-intensive tasks, reducing output length without sacrificing accuracy.
- Mathematical Problem Solving: Demonstrates strong performance in mathematical and general problem-solving scenarios.
- Long-Form Generation: Suitable for tasks requiring detailed, multi-step reasoning processes.
- Base Model Heritage: Leverages the robust capabilities of the DeepSeek-R1-Distill-Qwen-7B architecture.
Use Cases and Considerations
DECS_7B is recommended for applications where efficient, reasoning-focused text generation is crucial, such as automated problem solvers, educational tools, or systems requiring concise analytical outputs. Developers should note that while optimized for reasoning, outputs should always be validated in high-stakes environments due to the potential for incorrect or unverifiable reasoning. Performance can also be influenced by prompt engineering and decoding parameters. Further technical details are available in the associated research paper and project page.