agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-base
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-base model is a 4 billion parameter Qwen3-based language model, fine-tuned using OpenRLHF GRPO with a 32K context length. This model is specifically optimized for code generation and problem-solving, achieving a pass@8 score of 0.2261 on held-out validation problems. It excels at generating correct code by leveraging a binary code-correctness reward signal during reinforcement learning.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp20-base, is a 4 billion parameter Qwen3-based language model fine-tuned using the OpenRLHF GRPO (Group-normalized advantages, no KL penalty) reinforcement learning algorithm. It is seeded directly from the base Qwen/Qwen3-4B-Instruct-2507 model, meaning RL was applied without an initial Supervised Fine-Tuning (SFT) step.
Key Capabilities & Training
- Code Generation Optimization: The model is specifically trained and validated on the "cobalt-train ≤2/64 frontier," focusing on problems where the base model struggled. Its primary reward signal during training is binary code-correctness, meaning it is rewarded for generating programs that pass problem-specific tests.
- Performance: At global step 24 of its RL run, this checkpoint achieved a pass@8 score of 0.2261 and a pass@1 score of 0.0810 on held-out validation problems, making it the best checkpoint by pass@8 in its run.
- Anti-Truncation Shaping: The training incorporates a "stop-properly penalty" where truncated samples receive a -1.0 reward, and an "overlong penalty" for responses in the last 1024 tokens before the cap, ramping to -0.25.
When to Use This Model
This model is particularly well-suited for tasks requiring accurate code generation and problem-solving, especially in competitive programming or similar environments where functional correctness is paramount. Its direct RL application to a base model, coupled with a strong focus on code-correctness, makes it a specialized tool for developers seeking to generate robust and verifiable code.