agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-base-b20m
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-base-b20m model is a 4 billion parameter Qwen3-4B-Instruct-based reinforcement learning checkpoint, fine-tuned using OpenRLHF GRPO. This model is specifically optimized for code generation tasks, leveraging a binary code-correctness reward signal. It excels at solving programming problems, particularly those where the base model struggled, and is identified as the best checkpoint by pass@8 in its development run.
Loading preview...
Model Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp10-base-b20m, is a 4 billion parameter reinforcement learning (RL) checkpoint based on Qwen/Qwen3-4B-Instruct-2507. It was developed using the OpenRLHF GRPO algorithm, directly applying RL to the base Qwen3-4B model without an initial Supervised Fine-Tuning (SFT) seed.
Key Capabilities
- Code Generation Optimization: Specifically trained to improve performance on coding problems, using a binary code-correctness reward signal (1.0 for passing tests, 0.0 otherwise).
- Targeted Problem Solving: Validated and trained on a frontier of problems that the base model could solve on at most 2 of 64 samples, indicating its focus on improving challenging cases.
- RL Algorithm: Utilizes GRPO (group-normalized advantages, no KL penalty) for training.
- Anti-Truncation Shaping: Incorporates a stop-properly penalty where truncated samples receive a -1.0 reward, and an overlong penalty for responses nearing the token cap.
Best Use Cases
- Code Generation: Ideal for applications requiring accurate and robust code generation, especially for problems where initial models might struggle.
- Research in RL for Code: Useful for researchers exploring the application of GRPO and similar RL techniques to improve code generation capabilities of large language models.