Elliott/LUFFY-Qwen-Math-7B-Zero-On-Policy

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Apr 23, 2025License:mitArchitecture:Transformer Open Weights Featherless Exclusive Cold

Elliott/LUFFY-Qwen-Math-7B-Zero-On-Policy is a 7.6 billion parameter language model developed by Elliott, trained with the OpenR1-Math-220k dataset. This model utilizes the LUFFY reinforcement learning framework, which integrates off-policy reasoning traces and policy shaping to enhance learning. It is specifically optimized for mathematical reasoning tasks, achieving state-of-the-art results among zero-RL methods on competition-level benchmarks.

Loading preview...

LUFFY-Qwen-Math-7B-Zero-On-Policy: Enhanced Mathematical Reasoning

This model, developed by Elliott, is a 7.6 billion parameter language model built upon the Qwen architecture and trained with the OpenR1-Math-220k dataset. It leverages the novel LUFFY reinforcement learning framework, which bridges zero-RL and imitation learning by incorporating off-policy reasoning traces into the training process. LUFFY combines on-policy rollouts with off-policy demonstrations during advantage estimation and introduces policy shaping via regularized importance sampling to emphasize crucial actions.

Key Capabilities:

  • Off-Policy Guidance: Integrates external reasoning traces from stronger models to bootstrap learning.
  • Dynamic Balance: Adapts between imitation and exploration throughout training.
  • Policy Shaping: Focuses on low-probability but important actions for better generalization.
  • Mathematical Reasoning: Achieves state-of-the-art results among zero-RL methods on six competition-level mathematical benchmarks, including AIME, AMC, MATH-500, Minerva, and Olympiad.

Good for:

  • Complex Mathematical Problem Solving: Excels in competition-level math tasks, surpassing traditional on-policy RL and SFT methods, particularly in generalization.
  • Research in Reinforcement Learning: Demonstrates an innovative approach to combining on-policy and off-policy learning with policy shaping.
  • Applications requiring robust reasoning: Its ability to learn from external traces and dynamically balance exploration makes it suitable for tasks demanding strong logical inference.

Finetunes
1 models