agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-iid8
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-iid8 model is a 4 billion parameter OpenRLHF GRPO reinforcement-learning checkpoint for Qwen3-4B, fine-tuned for code generation. It was seeded directly from the base Qwen3-4B model without prior Supervised Fine-Tuning (SFT). This model is optimized for solving coding problems, particularly those where the base model struggled, and is distinguished as the best checkpoint by pass@8 in its training run. It excels at generating correct code based on a binary code-correctness reward signal.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-iid8, is a 4 billion parameter reinforcement-learning checkpoint based on the Qwen/Qwen3-4B-Instruct-2507 architecture. It was developed using the OpenRLHF GRPO algorithm, applying RL directly to the base Qwen3-4B model without an initial Supervised Fine-Tuning (SFT) phase.
Key Capabilities & Training
- Code Generation: The model is specifically fine-tuned for generating correct code, with a reward signal based on binary code-correctness (1.0 for passing tests, 0.0 otherwise).
- Targeted Improvement: Training focused on the "cobalt-train ≤2/64 frontier," addressing 1833 training problems and 112 validation problems that the base model solved on at most 2 of 64 samples.
- RL Algorithm: Utilizes GRPO (group-normalized advantages, no KL penalty) with specific anti-truncation and overlong penalties to guide generation.
- Performance: Identified as the best checkpoint by pass@8 in its training run, indicating strong performance in generating correct code solutions.
Use Cases
- Code Problem Solving: Ideal for tasks requiring the generation of functional code that passes predefined tests.
- Reinforcement Learning Research: Useful for researchers exploring GRPO and RL-based fine-tuning strategies for code generation, particularly without an SFT seed.
- Benchmarking: Can serve as a strong baseline for evaluating code generation capabilities on challenging problem sets.