agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21vs16
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21vs16 is a 4 billion parameter Qwen3-based model fine-tuned using OpenRLHF GRPO for code generation tasks. This reinforcement learning checkpoint, developed by agurung, is optimized for code correctness, achieving a pass@8 score of 8.5543 on the cobalt-train frontier. It specializes in solving programming problems by generating correct code, making it suitable for automated code generation and competitive programming assistance.
Loading preview...
Model Overview
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-nb21vs16 is a 4 billion parameter language model based on Qwen/Qwen3-4B-Instruct-2507. This model is a reinforcement learning (RL) checkpoint, specifically trained using the OpenRLHF GRPO algorithm. It was seeded directly from the base Qwen3-4B model without an initial Supervised Fine-Tuning (SFT) step, applying RL directly to enhance its capabilities.
Key Capabilities and Optimization
This model is primarily optimized for code generation and correctness. Its training focused on solving programming problems, with the reward signal being binary code-correctness (1.0 for passing tests, 0.0 otherwise). It was trained and validated on the cobalt-train <=2/64 frontier, a dataset of problems that the base model struggled with.
Key performance metrics on the held-out validation set (8 samples per problem) include:
- pass@1: 3.2805
- pass@8: 8.5543
This checkpoint is noted as the best by pass@8 in its development run, indicating its proficiency in generating at least one correct solution among multiple attempts for a given coding problem.
Training Recipe Highlights
The model utilized the GRPO algorithm, which incorporates group-normalized advantages without a KL penalty. Specific training techniques included:
- A stop-properly penalty where truncated samples received a -1.0 reward to discourage early stopping.
- A DAPO overlong penalty applied to responses nearing the token cap, ramping up to -0.25.
- Training involved 8 samples per prompt, with rollout and train batch sizes of 128.
- The model was trained for 2 episodes with a maximum of 4096 new tokens per rollout.
Use Cases
This model is particularly well-suited for applications requiring accurate code generation for programming challenges. Its optimization for code correctness makes it a strong candidate for tasks such as:
- Automated problem-solving in competitive programming contexts.
- Generating functional code snippets based on problem descriptions.
- Assisting developers by providing correct code solutions.