BayesRL/Qwen-M3PO-7B
BayesRL/Qwen-M3PO-7B is a 7.6 billion parameter language model developed by BayesRL, fine-tuned from Qwen2.5-Math-7B using the M3PO algorithm. This model specializes in mathematical reasoning tasks by employing parameter-space exploration strategies for Reinforcement Learning with Verifiable Rewards (RLVR). It utilizes Monte-Carlo weight perturbations from an IVON posterior to enhance exploration, making it distinct for complex problem-solving in mathematics.
Loading preview...
Model Overview
BayesRL/Qwen-M3PO-7B is a 7.6 billion parameter language model built upon the Qwen/Qwen2.5-Math-7B foundation model. It was fine-tuned by BayesRL using the M3PO algorithm, a novel approach to Reinforcement Learning with Verifiable Rewards (RLVR) detailed in the paper "Parameter Exploration for RLVR via Variational Learning".
Key Capabilities & Differentiators
- Advanced Exploration: M3PO introduces parameter-space exploration by sampling model weights from an approximate posterior learned with the IVON variational optimizer. This provides an additional control lever for exploration beyond traditional action-space heuristics.
- Mathematical Reasoning: The model is specifically fine-tuned for mathematical tasks, leveraging the
Qwen2.5-Math-7Bbase and training on the DAPO-Math-17k dataset. - Variational Learning: Utilizes the IVON optimizer with specific learning rate and ESS parameters, trained on 8 NVIDIA H200 GPUs.
- Evaluated Performance: Performance is evaluated on challenging mathematical benchmarks including AIME 2024ā2026, MATH-500, AMC 2023, and Minerva, with full results available in the associated research paper.
Ideal Use Cases
- Mathematical Problem Solving: Excellent for applications requiring robust mathematical reasoning and problem-solving capabilities.
- Research in RLVR: Suitable for researchers exploring advanced reinforcement learning techniques, particularly those involving parameter-space exploration and variational inference.
- Complex Reasoning Tasks: Can be applied to other domains demanding precise and verifiable reasoning, benefiting from its unique exploration strategy.