agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-biid8
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-biid8 is a 4 billion parameter OpenRLHF GRPO reinforcement-learning checkpoint based on Qwen3-4B-Instruct-2507. This model was seeded directly from the base Qwen3-4B and trained using a binary code-correctness reward signal. It is specifically optimized for code generation tasks, demonstrating strong performance on the cobalt-train frontier for solving programming problems. This checkpoint is noted as the best by pass@8 in its training run, making it suitable for applications requiring robust code generation capabilities.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-biid8, is a 4 billion parameter reinforcement-learning checkpoint derived from the Qwen/Qwen3-4B-Instruct-2507 base model. It was developed using the OpenRLHF GRPO algorithm, applying RL directly to the base Qwen3-4B without an intermediate SFT seed. The model was saved at global step 60 of its RL training run.
Key Capabilities & Training
- Code Generation Optimization: The primary strength of this model lies in its ability to generate correct code. It was trained and validated on the cobalt-train \u22642/64 frontier, a dataset of programming problems where the base model initially struggled.
- Reward Signal: Training utilized a binary code-correctness reward, meaning the model was rewarded 1.0 for generating code that passed problem tests and 0.0 otherwise. This direct feedback mechanism is crucial for its code generation proficiency.
- GRPO Algorithm: The model employs the GRPO (group-normalized advantages, no KL penalty) algorithm for reinforcement learning.
- Anti-Truncation & Overlong Penalties: It incorporates a stop-properly penalty (-1.0 for truncated samples) and a DAPO overlong penalty (ramping to -0.25 for responses in the last 1024 tokens) to encourage complete and concise outputs.
When to Use This Model
This checkpoint is particularly well-suited for:
- Code generation: Its training methodology and reward signal make it highly effective for tasks requiring the generation of functional and correct code.
- Programming problem-solving: Ideal for applications that involve solving coding challenges or generating solutions to defined programming problems.
- Research in RL for code: Useful for researchers exploring the application of GRPO and similar RL techniques to code generation with specific penalty mechanisms.