agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-iid16
agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-iid16 is a 4 billion parameter OpenRLHF GRPO reinforcement-learning checkpoint based on Qwen3-4B-Instruct-2507. This model is specifically fine-tuned for code generation, excelling at solving programming problems by optimizing for binary code-correctness. It was trained using a GRPO algorithm with specific penalties for truncated and overlong responses, making it particularly robust for generating complete and correct code solutions.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-iid16, is a 4 billion parameter reinforcement-learning checkpoint derived from the Qwen/Qwen3-4B-Instruct-2507 base model. Developed by agurung, it utilizes the OpenRLHF GRPO algorithm and represents the best performing checkpoint by pass@8 in its training run.
Key Capabilities & Training
- Code Generation Focus: The model is specifically optimized for generating correct code, with its reward signal based on binary code-correctness (1.0 if the generated program passes problem tests, 0.0 otherwise).
- Reinforcement Learning: It was trained using the GRPO (group-normalized advantages, no KL penalty) algorithm, applied directly to the base Qwen3-4B model without an initial SFT seed.
- Robustness: Incorporates specific penalties to enhance output quality:
- Truncated samples receive a -1.0 reward (ProRL-style anti-truncation shaping).
- Responses in the last 1024 tokens before the cap incur an additive penalty ramping to -0.25.
- Training Data: Validated on the "cobalt-train ≤2/64 frontier" using 1833 training and 112 held-out validation problems, focusing on problems the base model struggled with.
When to Use This Model
This model is particularly well-suited for tasks requiring:
- High-quality code generation: Especially for competitive programming or scenarios where functional correctness is paramount.
- Robust output: Its training methodology helps mitigate issues like truncated or overly long responses, leading to more complete code solutions.