zbeeb/Qwen2.5-3B-GRPO-Staleness-4
The zbeeb/Qwen2.5-3B-GRPO-Staleness-4 is a 3.1 billion parameter language model, a GRPO-fine-tuned checkpoint of Qwen/Qwen2.5-3B. It was specifically trained on the 17,005-row DAPO math dataset using a Grouped Reinforcement Learning from Policy Optimization (GRPO) method with a staleness cap of 4. This model is optimized for mathematical problem-solving, demonstrating its primary strength in accurately computing and explaining solutions to complex math problems.
Loading preview...
Overview
zbeeb/Qwen2.5-3B-GRPO-Staleness-4 is a 3.1 billion parameter model derived from Qwen/Qwen2.5-3B, specifically fine-tuned using a Grouped Reinforcement Learning from Policy Optimization (GRPO) approach. This model's training focused on mathematical problem-solving, leveraging the 17,005-row DAPO math dataset. A key aspect of its training was a 'staleness cap' of 4, which limits the age of the rollout policy during the 1,000 updates.
Key Capabilities
- Advanced Mathematical Reasoning: Optimized for solving complex math problems, including those from benchmarks like MATH500, AMC, and Minerva.
- Detailed Explanations: Capable of providing reasoning alongside final answers, formatted for clarity (e.g.,
\boxed{...}orFinal answer: ...). - Robust Training Methodology: Utilizes PrimeRL v0.9.0 with specific parameters like AdamW learning rate 1e-6 and PPO clip 0.2, ensuring a focused and effective training process.
Good For
- Mathematical Problem Solving: Ideal for applications requiring accurate and explained solutions to various math challenges.
- Educational Tools: Can be integrated into platforms for tutoring or generating step-by-step math solutions.
- Research in RLHF for Math: Provides a strong baseline for further research into reinforcement learning from human feedback (RLHF) applied to mathematical domains.