BayesRL/Qwen-B3PO-7B
BayesRL/Qwen-B3PO-7B is a 7 billion parameter language model based on Qwen2.5-Math-7B, fine-tuned using the B3PO (Bayesian Perturbed Parameter Policy Optimization) algorithm. This model specializes in mathematical reasoning and problem-solving by employing parameter-space exploration strategies for Reinforcement Learning with Verifiable Rewards (RLVR). It is designed to enhance exploration during training by sampling model weights from an approximate posterior, making it suitable for complex mathematical tasks.
Loading preview...
BayesRL/Qwen-B3PO-7B: Enhanced Mathematical Reasoning
BayesRL/Qwen-B3PO-7B is a 7 billion parameter model built upon the Qwen/Qwen2.5-Math-7B foundation, specifically fine-tuned for advanced mathematical problem-solving. Its core innovation lies in the application of B3PO (Bayesian Perturbed Parameter Policy Optimization), a novel parameter-space exploration strategy for Reinforcement Learning with Verifiable Rewards (RLVR).
Key Capabilities & Innovations
- B3PO Algorithm: Unlike traditional methods that rely on action-space heuristics, B3PO samples model weights from an approximate posterior learned via the IVON variational optimizer. This introduces controlled weight noise, acting as an additional lever for exploration during training.
- Targeted Mathematical Performance: The model is fine-tuned on the
DAPO-Math-17kdataset, indicating a strong focus on mathematical reasoning and problem-solving. - Variational Learning: Utilizes IVON (Variational Optimizer) for learning the approximate posterior, with a single weight perturbation drawn per gradient step and synced to the rollout engine.
Evaluation & Use Cases
Evaluated on challenging mathematical benchmarks such as AIME 2024ā2026, MATH-500, AMC 2023, and Minerva, this model is designed for tasks requiring robust mathematical understanding and problem-solving. Developers can integrate it using the Hugging Face transformers library for applications demanding high-accuracy mathematical outputs.