BayesRL/Qwen-C3PO-7B

TEXT GENERATIONPricing:Input $0.4 / Cached $0.08 / Output $0.8Concurrent Unit Cost:1Model Size:7.6BQuant:FP8Context Size:32kTool Calling:SupportedPublished:Jun 13, 2026License:apache-2.0Architecture:Transformer Open Weights Featherless Exclusive Cold

BayesRL/Qwen-C3PO-7B is a 7 billion parameter language model fine-tuned from Qwen2.5-Math-7B using the C3PO (Causal 3PO) algorithm. Developed by BayesRL, this model specializes in mathematical reasoning tasks by employing parameter-space exploration strategies for Reinforcement Learning with Verifiable Rewards (RLVR). It utilizes variational optimization with IVON and sequence-level multiple-importance-sampling to enhance exploration and performance in complex mathematical problem-solving.

Loading preview...

Model Overview

BayesRL/Qwen-C3PO-7B is a 7 billion parameter language model derived from Qwen/Qwen2.5-Math-7B, specifically fine-tuned with the C3PO algorithm. C3PO, detailed in the paper "Parameter Exploration for RLVR via Variational Learning," introduces a novel approach to Reinforcement Learning with Verifiable Rewards (RLVR) by exploring the parameter-space rather than solely relying on action-space heuristics.

Key Capabilities & Innovations

  • Parameter-Space Exploration: C3PO samples model weights from an approximate posterior learned with the IVON variational optimizer, using weight noise as an additional control lever for exploration.
  • Enhanced Diversity: It splits GRPO (Generalized Policy Optimization) rollouts across multiple independent weight perturbations, leveraging sequence-level multiple-importance-sampling (Seq-MIS) to account for differing perturbations and achieve greater parameter-space diversity within the same rollout budget.
  • Mathematical Reasoning: The model is warm-started from BayesRL/Qwen2.5Math-IVON-SFT-7B and trained on the DAPO-Math-17k dataset, indicating a strong focus on mathematical problem-solving.

Evaluation & Use Cases

This model has been evaluated on challenging mathematical benchmarks including AIME 2024ā€“2026, MATH-500, AMC 2023, and Minerva, with full results available in the associated research paper. Developers can integrate this model using the Hugging Face transformers library for tasks requiring advanced mathematical reasoning and robust exploration strategies.

Finetunes
1 models