agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-q3c-vs16
The agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-q3c-vs16 model is a 4 billion parameter OpenRLHF GRPO reinforcement-learning checkpoint based on Qwen3-4B-Instruct-2507, with a context length of 32768 tokens. It was seeded directly from the base Qwen3-4B model without SFT and trained using binary code-correctness as the reward signal. This model is specifically optimized for code generation tasks, particularly for solving programming problems where the generated code must pass tests, and is noted as the best checkpoint by pass@8 in its development run.
Loading preview...
Overview
This model, agurung/cobalt-seeded-rl-base-ramp25-stoppen-gen4k-ep2-ncp5-q3c-vs16, is a 4 billion parameter reinforcement-learning (RL) checkpoint derived from the Qwen/Qwen3-4B-Instruct-2507 base model. It was developed using the OpenRLHF GRPO algorithm, applying RL directly to the base Qwen3-4B without an initial Supervised Fine-Tuning (SFT) seed. The model is saved at global step 24 of its RL training run and is specifically optimized for code generation tasks.
Key Capabilities
- Code Generation: Primarily designed for generating correct code that passes problem-specific tests.
- Reinforcement Learning Optimization: Utilizes the GRPO algorithm with group-normalized advantages and no KL penalty for efficient learning.
- Anti-Truncation Shaping: Incorporates a "stop-properly" penalty where truncated samples receive a -1.0 reward, and a DAPO overlong penalty for responses nearing the token cap.
- Performance in Code Challenges: Identified as the "best checkpoint by pass@8" in its development run, indicating strong performance in solving programming problems.
Good for
- Automated Code Problem Solving: Ideal for applications requiring the generation of functional code that can pass unit tests.
- Research in RL for Code Generation: Useful for researchers exploring GRPO and similar RL techniques for improving code synthesis.
- Benchmarking Code Models: Can serve as a strong baseline or comparison point for other 4B parameter models in code-centric tasks.
- Environments requiring specific RL training methodologies: Suitable for scenarios where direct RL application to a base model without SFT is desired.