agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-q4v3-iid16
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-q4v3-iid16 is a 4 billion parameter language model, based on Qwen3-4B-Instruct-2507, fine-tuned using OpenRLHF GRPO for code generation. This model excels at producing correct code, achieving a pass@8 score of 0.5564 on a held-out validation set. Its primary use case is generating functional code, particularly for problems where the base model struggled.
Loading preview...
Overview
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-q4v3-iid16 is a 4 billion parameter language model, derived from Qwen/Qwen3-4B-Instruct-2507. It has been fine-tuned using the OpenRLHF GRPO (Group-Normalized Advantages, no KL penalty) algorithm, with Reinforcement Learning (RL) applied directly to the base model without an initial Supervised Fine-Tuning (SFT) seed. This checkpoint was saved at global step 16 of its RL training run and is specifically optimized for code generation tasks.
Key Capabilities
- Enhanced Code Correctness: This model is the best checkpoint by
pass@8in its training run, achieving a score of 0.5564 on a held-out validation set. This indicates its strong ability to produce at least one correct solution among multiple samples. - Targeted Problem Solving: Trained and validated on the "cobalt-train \u22642/64 frontier," focusing on problems that the base model solved on at most 2 of 64 samples, making it effective for challenging coding tasks.
- Binary Code-Correctness Reward: The RL training utilized a binary reward signal (1.0 for passing tests, 0.0 otherwise), directly optimizing for functional code.
- Anti-Truncation Shaping: Incorporates a stop-properly penalty where truncated samples receive a -1.0 reward, and an overlong penalty for responses in the last 1024 tokens before the cap, ramping to -0.25.
Good for
- Code Generation: Ideal for applications requiring the generation of functionally correct code, especially for problems where initial models might struggle.
- Improving Base Model Performance: Suitable for developers looking to enhance the code generation capabilities of the Qwen3-4B-Instruct-2507 base model through RL fine-tuning.
- Research in RLHF for Code: Provides a practical example of GRPO application with specific penalty mechanisms for code generation.