Elliott/LUFFY-Qwen-Math-1.5B-Zero
LUFFY-Qwen-Math-1.5B-Zero is a 1.5 billion parameter model developed by Elliott, built upon the Qwen2.5-Math-1.5B-Base architecture. It utilizes the LUFFY reinforcement learning framework, which integrates off-policy reasoning traces and policy shaping to enhance learning. This model is specifically optimized for mathematical reasoning tasks, demonstrating superior performance on various math benchmarks like AIME, AMC, and MATH-500 compared to its base and instruct counterparts.
Loading preview...
LUFFY-Qwen-Math-1.5B-Zero Overview
LUFFY-Qwen-Math-1.5B-Zero is a 1.5 billion parameter model developed by Elliott, leveraging the LUFFY reinforcement learning framework. This framework uniquely bridges zero-RL and imitation learning by incorporating off-policy reasoning traces and introducing policy shaping via regularized importance sampling. This approach allows the model to emphasize crucial, low-probability actions often overlooked in standard policy gradients, leading to improved generalization and performance in complex reasoning tasks.
Key Capabilities
- Off-Policy Guidance: Integrates external reasoning traces from stronger models to accelerate and bootstrap the learning process.
- Dynamic Balance: Adapts its learning strategy over time, balancing between imitating demonstrations and exploring new solutions.
- Policy Shaping: Focuses on important actions, enhancing the model's ability to generalize and solve challenging problems.
- Strong Mathematical Reasoning: Achieves competitive results across various mathematical benchmarks.
Good For
- Mathematical Problem Solving: Excels in complex math reasoning tasks, as evidenced by its performance on AIME, AMC, and MATH-500.
- Research in Reinforcement Learning: Demonstrates an innovative approach to combining on-policy and off-policy learning with policy shaping.
- Applications Requiring Robust Reasoning: Suitable for scenarios where accurate and generalized reasoning is critical, especially in quantitative domains.