agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21groot16
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21groot16 is a 4 billion parameter Qwen3-based model fine-tuned using OpenRLHF GRPO. This reinforcement learning checkpoint is specifically optimized for code generation tasks, achieving a pass@8 score of 9.8652 on the cobalt-train frontier. It excels at solving programming problems by generating correct code, making it suitable for applications requiring robust code completion or problem-solving capabilities.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21groot16, is a 4 billion parameter Qwen3-based language model that has undergone reinforcement learning (RL) using the OpenRLHF GRPO algorithm. It is an RL checkpoint derived from Qwen/Qwen3-4B-Instruct-2507, with RL applied directly to the base Qwen3-4B model without an initial Supervised Fine-Tuning (SFT) seed.
Key Capabilities and Performance
This checkpoint is specifically optimized for code generation and problem-solving, demonstrated by its performance on the cobalt-train frontier dataset. It was trained and validated on problems that the base model struggled with, focusing on improving code correctness. The reward signal used during RL was binary code-correctness, meaning it was rewarded for generating programs that passed problem-specific tests.
Key evaluation metrics on a held-out validation set (with 8 samples per problem) include:
- pass@1: 6.3831 (mean per-problem fraction correct)
- pass@8: 9.8652 (problem counted solved if any sample is correct)
This model represents the best checkpoint by pass@8 in its training run, indicating its strong ability to produce at least one correct solution among multiple attempts.
Training Details
The model was trained using the GRPO algorithm, which incorporates group-normalized advantages and no KL penalty. It includes specific penalties to discourage truncated or overly long responses, such as a -1.0 reward for truncated samples and an additive penalty ramping to -0.25 for responses in the last 1024 tokens before the context cap. The training involved 2 episodes with a maximum of 4096 new tokens per rollout.