fenglui/DeepCoder-1.5B-Preview-16bit
DeepCoder-1.5B-Preview-16bit by fenglui is a 1.5 billion parameter code reasoning LLM, fine-tuned from DeepSeek-R1-Distilled-Qwen-1.5B. It utilizes distributed reinforcement learning (RL) with an improved GRPO+ algorithm and iterative context lengthening to excel in code generation and problem-solving. The model demonstrates strong performance on coding benchmarks like HumanEval+ (73.0%) and LiveCodeBench (25.1%), and is optimized for long context reasoning up to 32K tokens.
Loading preview...
DeepCoder-1.5B-Preview: Code Reasoning LLM
DeepCoder-1.5B-Preview is a 1.5 billion parameter language model developed by fenglui, specifically designed for code reasoning tasks. It is fine-tuned from DeepSeek-R1-Distilled-Qwen-1.5B using a novel distributed reinforcement learning (RL) approach.
Key Differentiators & Capabilities
- Advanced RL Training: Employs an enhanced GRPO+ algorithm, incorporating insights from DAPO for more stable training, including offline difficulty filtering, no entropy or KL loss, overlong filtering, and clip high techniques.
- Iterative Context Lengthening: Trained with an iterative context lengthening strategy, enabling strong generalization to long contexts. It successfully generalizes to 64K context inference despite being trained with 32K context.
- Superior Code Performance: Achieves significantly higher scores compared to its base model, with 73.0% on HumanEval+, 25.1% on LiveCodeBench (v5), and a Codeforces rating of 963.
- Long Context Reasoning: Demonstrates improved long-context reasoning capabilities, outperforming the base model in 32K and 64K context evaluations.
Ideal Use Cases
- Code Generation: Generating code solutions for various programming problems.
- Code Problem Solving: Excelling in competitive programming and algorithmic challenges.
- Long Context Code Analysis: Handling and reasoning over large codebases or complex problem descriptions due to its optimized long-context capabilities.