agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21iid16
agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21iid16 is a 4 billion parameter Qwen3-based model fine-tuned using OpenRLHF GRPO. This reinforcement learning checkpoint is optimized for code generation, specifically achieving the best pass@8 score in its training run. It was trained on a challenging subset of code problems with a binary code-correctness reward signal. The model is designed for tasks requiring robust code output and problem-solving capabilities.
Loading preview...
Model Overview
agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21iid16 is a 4 billion parameter language model based on Qwen/Qwen3-4B-Instruct-2507. This model is a reinforcement learning (RL) checkpoint, specifically trained using the OpenRLHF GRPO algorithm. It stands out as the best performing checkpoint by pass@8 metric within its training run, indicating strong code generation capabilities.
Key Training Details
- Base Model: Qwen3-4B (RL applied directly without an SFT seed).
- Training Data: Focused on the "cobalt-train \u22642/64 frontier," comprising 1833 training and 112 validation problems that the base model struggled with.
- Reward Signal: Binary code-correctness (1.0 for passing tests, 0.0 otherwise).
- RL Algorithm: GRPO (Group-normalized advantages, no KL penalty).
- Penalties: Includes a stop-properly penalty (-1.0 for truncated samples) and a DAPO overlong penalty for responses nearing the token cap.
- Max New Tokens: Rollouts allow up to 4096 new tokens.
Performance
At global step 72, this checkpoint achieved notable performance on its held-out validation set (8 samples per problem):
- pass@1: 1.1036
- pass@8: 2.9324
Use Cases
This model is particularly suited for applications requiring robust and accurate code generation, especially for problems where the base model initially struggled. Its optimization for pass@8 suggests it can produce multiple correct solutions, increasing the likelihood of success in coding challenges.