SpyRL/SpyRL-Qwen3-4B-Math
SpyRL-Qwen3-4B-Math is a 4 billion parameter language model developed by SpyRL, fine-tuned from Qwen3-4B-Instruct-2507. It is specifically optimized for mathematical reasoning tasks using the novel SpyRL self-play training algorithm, which requires no human annotation, reward models, or LLM judges. This model demonstrates significant performance gains across various math benchmarks, making it suitable for complex quantitative problem-solving.
Loading preview...
SpyRL-Qwen3-4B-Math: Self-Improved Mathematical Reasoning
SpyRL-Qwen3-4B-Math is a 4 billion parameter model based on Qwen/Qwen3-4B-Instruct-2507, specifically enhanced for mathematical reasoning. Its key differentiator is the training methodology: it utilizes SpyRL, the reference implementation of RLSVR (Reinforcement Learning with Self-Verifiable Rewards).
Key Capabilities and Training
- Self-Supervised Improvement: Trained without any human annotation, reward models, or LLM judges. Rewards are generated intrinsically through a multi-agent self-play game inspired by Who Is the Spy?.
- Mathematical Task Focus: Optimized for designing and solving math problems grounded in web documents, using the
nvidia/Nemotron-CC-Math-v1corpus. - Algorithm: Employs GRPO with alternating performing and detection stages over 100 iterations.
Performance Highlights
The model shows substantial improvements over its base model across several mathematical and reasoning benchmarks:
- GSM8K: 84.5 (Base) → 93.4 (+8.9)
- Math500: 68.2 (Base) → 79.5 (+11.3)
- AIME 25: 6.7 (Base) → 20.0 (+13.3)
- Minerva: 42.3 (Base) → 47.8 (+5.5)
- GPQA-Diamond: 26.3 (Base) → 41.3 (+15.0)
On average, SpyRL-Qwen3-4B-Math achieves an +8.97% gain across seven benchmarks compared to the base Qwen3-4B model.
Use Cases
This model is ideal for applications requiring robust mathematical problem-solving, complex reasoning, and quantitative analysis where high accuracy is critical. Its unique training approach makes it a notable example of self-improving LLMs for specialized tasks.