liangzhidanta/Qwen3-4B-GRPO-119
liangzhidanta/Qwen3-4B-GRPO-119 is a 4 billion parameter Qwen3-based language model fine-tuned using GRPO reinforcement learning for mathematical reasoning. Developed by liangzhidanta, this model significantly improves accuracy and conciseness on math problems, achieving a 65.0% pass@1 on AIME 2024, a 7.5 percentage point increase over its base model. It is specifically optimized for solving complex mathematical reasoning tasks in English, demonstrating reduced response length and truncation rates.
Loading preview...
Overview
liangzhidanta/Qwen3-4B-GRPO-119 is a 4 billion parameter model based on Qwen3, specifically fine-tuned for mathematical reasoning using the GRPO (Group-Relative Advantage Policy Optimization) reinforcement learning algorithm. This model was trained end-to-end on 8xRTX 4090 GPUs using the slime framework, adapting an H100 recipe for consumer hardware.
Key Capabilities
- Enhanced Math Reasoning: Achieves a 65.0% pass@1 on AIME 2024, a substantial +7.5 percentage point improvement over the base Qwen3-4B model (57.5%).
- Increased Conciseness: Simultaneously became more accurate and concise, reducing median response length by 49% and truncation ratio by 54% without reward hacking.
- Efficient Training: Utilizes a rule-based verifier for rewards and was trained on the zhuzilin/dapo-math-17k dataset.
Good for
- Solving complex mathematical problems, particularly those requiring step-by-step reasoning.
- Applications where both accuracy and concise, non-redundant outputs are valued in mathematical contexts.
Limitations
- Trained and evaluated exclusively on English math reasoning data.
- Some training rollouts still hit the 8192-token cap, indicating potential for further optimization in handling very long reasoning chains.